Commit Graph

864 Commits

Author SHA1 Message Date
Jez Ng 3e951808d5 [lld-macho][nfc] Comments and style fixes
Added some comments (particularly around finalize() and
finalizeContents()) as well as doing some rephrasing / grammar fixes for
existing comments.

Also did some minor style fixups, such as by putting methods together in
a class definition and having fields of similar types next to each
other.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D118714
2022-02-01 13:45:59 -05:00
Keith Smiley a6298fb160 [lld-macho] Add support for -add_empty_section
This is a ld64 option equivalent to `-sectcreate seg sect /dev/null`
that's useful for creating sections like the RESTRICT section.

Differential Revision: https://reviews.llvm.org/D117749
2022-01-30 10:03:41 -08:00
Roger Kim 422084332a [lld][Macho] Include dead-stripped symbols in mapfile
ld64 outputs dead stripped symbols when using the -dead-strip flag. This change mimics that behavior for lld.

ld64's -dead_strip flag outputs:
```
$ ld -map map basics.o -o out -dead_strip -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -lSystem
$ cat map
# Path: out
# Arch: x86_64
# Object files:
[  0] linker synthesized
[  1] basics.o
# Sections:
# Address       Size            Segment Section
0x100003F97     0x00000021      __TEXT  __text
0x100003FB8     0x00000048      __TEXT  __unwind_info
0x100004000     0x00000008      __DATA_CONST    __got
0x100008000     0x00000010      __DATA  __ref_section
0x100008010     0x00000001      __DATA  __common
# Symbols:
# Address       Size            File  Name
0x100003F97     0x00000006      [  1] _ref_local
0x100003F9D     0x00000001      [  1] _ref_private_extern
0x100003F9E     0x0000000C      [  1] _main
0x100003FAA     0x00000006      [  1] _no_dead_strip_globl
0x100003FB0     0x00000001      [  1] _ref_from_no_dead_strip_globl
0x100003FB1     0x00000006      [  1] _no_dead_strip_local
0x100003FB7     0x00000001      [  1] _ref_from_no_dead_strip_local
0x100003FB8     0x00000048      [  0] compact unwind info
0x100004000     0x00000008      [  0] non-lazy-pointer-to-local: _ref_com
0x100008000     0x00000008      [  1] _ref_data
0x100008008     0x00000008      [  1] l_ref_data
0x100008010     0x00000001      [  1] _ref_com

# Dead Stripped Symbols:
#               Size            File  Name
<<dead>>        0x00000006      [  1] _unref_extern
<<dead>>        0x00000001      [  1] _unref_local
<<dead>>        0x00000007      [  1] _unref_private_extern
<<dead>>        0x00000001      [  1] _ref_private_extern_u
<<dead>>        0x00000008      [  1] _unref_data
<<dead>>        0x00000008      [  1] l_unref_data
<<dead>>        0x00000001      [  1] _unref_com
```

Reviewed By: int3, #lld-macho, thevinster

Differential Revision: https://reviews.llvm.org/D114737
2022-01-28 10:51:27 -08:00
Leonard Grey a5c9d71780 [lld-macho] Move order file and call graph sorting into SectionPriorities
See https://reviews.llvm.org/D117354 for context and discussion.
2022-01-25 12:18:15 -05:00
Leonard Grey f23d57a632 [lld-macho] Rename CallGraphSort.{h,cpp} to SectionPriorities
This is in preparation for moving the code that parses and processes
order files into this file.

See https://reviews.llvm.org/D117354 for context and discussion.
2022-01-25 12:15:14 -05:00
Roger Kim f84023a812 [lld][macho] Stop grouping symbols by sections in mapfile.
As per [Bug 50689](https://bugs.llvm.org/show_bug.cgi?id=50689),

```
2. getSectionSyms() puts all the symbols into a map of section -> symbols, but this seems unnecessary. This was likely copied from the ELF port, which prints a section header before the list of symbols it contains. But the Mach-O map file doesn't print these headers.
```

This diff removes `getSectionSyms()` and keeps all symbols in a flat vector.

What does ld64's mapfile look like?
```
$ llvm-mc -filetype=obj -triple=x86_64-apple-darwin test.s -o test.o
$ llvm-mc -filetype=obj -triple=x86_64-apple-darwin foo.s -o foo.o
$ ld -map map test.o foo.o -o out -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -lSystem
```

```
[  0] linker synthesized
[  1] test.o
[  2] foo.o
0x100003FB7     0x00000001      __TEXT  __text
0x100003FB8     0x00000000      __TEXT  obj
0x100003FB8     0x00000048      __TEXT  __unwind_info
0x100004000     0x00000001      __DATA  __common
0x100003FB7     0x00000001      [  1] _main
0x100003FB8     0x00000000      [  2] _foo
0x100003FB8     0x00000048      [  0] compact unwind info
0x100004000     0x00000001      [  1] _number
```

Perf numbers when linking chromium framework on a 16-Core Intel Xeon W Mac Pro:
```
base           diff           difference (95% CI)
sys_time   1.406 ± 0.020  1.388 ± 0.019  [  -1.9% ..   -0.6%]
user_time  5.557 ± 0.023  5.914 ± 0.020  [  +6.2% ..   +6.6%]
wall_time  4.455 ± 0.041  4.436 ± 0.035  [  -0.8% ..   -0.0%]
samples    35             35
```

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D114735
2022-01-20 12:16:37 -08:00
Alexandre Ganea 83d59e05b2 Re-land [LLD] Remove global state in lldCommon
Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe3622 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac.

Differential Revision: https://reviews.llvm.org/D108850
2022-01-20 14:53:26 -05:00
Jez Ng 8f811effac [lld-macho] Fix grammar in doc 2022-01-19 23:59:35 -08:00
Jez Ng ef95d45138 [lld-macho] Mention string literal deduplication as a difference from ld64
Reviewed By: keith

Differential Revision: https://reviews.llvm.org/D117250
2022-01-19 16:30:52 -08:00
Keith Smiley 3f38dc5c04 [lld-macho] Silence XAR deprecation warning
If you're building this on macOS 12.x+ this produces a deprecation
warning. I'm not sure what this means for the bitcode format going
forward, but it seems safe to silence for now.

Do we need to worry about GCC for this?

Differential Revision: https://reviews.llvm.org/D117718
2022-01-19 13:51:55 -08:00
Keith Smiley 67090e3446 [lld-macho] Implement -noall_load
This flag is the default, so in ld64 it is not implemented, but it can
be useful to negate previous -all_load arguments. Specifically if your
build system has some global linker flags, that you may want to negate
for specific links. We use something like this today to make sure some
C++ symbols are automatically discovered for all links, which passing
-all_load hides.

Differential Revision: https://reviews.llvm.org/D117629
2022-01-19 13:12:18 -08:00
Fangrui Song 0aae2bf373 [lld-macho] Add --start-lib --end-lib
In ld.lld, when an ObjFile/BitcodeFile is read in --start-lib state, the file is
given archive semantics. --end-lib closes the previous --start-lib. A build
system can use this feature as an alternative to archives. This patch ports
the feature to lld-macho.

--start-lib and --end-lib are positional, unlike usual ld64 options.
I think the slight drawback does not matter as (a) reusing option names
make build systems convenient (b) `--start-lib a.o b.o --end-lib` conveys more
information than an alternative design: `-objlib a.o -objlib b.o` because
--start-lib makes it clear which objects are in the same conceptual archive.
This provides flexibility (c) `-objlib`/`-filelist` interaction may be weird.

Close https://github.com/llvm/llvm-project/issues/52931

Reviewed By: #lld-macho, Jez Ng, oontvoo

Differential Revision: https://reviews.llvm.org/D116913
2022-01-19 10:14:49 -08:00
Vincent Lee e5347f2556 [lld-macho] Allow deduplicate-literals to be overridden
It's still uncertain but whether we want to have `deduplicate-literals` be the
default flag for LLD out of the box or not. If `deduplicate-literals` is the default
behavior, then we will need a way override it and not deduplicate. Luckily, we
have `no_deduplicate` to fill this gap. For now, I've set the default to be false
which aligns with the existing behavior. That can only always be changed after
discussions on D117250.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D117387
2022-01-18 15:42:59 -08:00
Alexandre Ganea e6b153947d Revert [LLD] Remove global state in lldCommon
It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383
2022-01-16 11:03:06 -05:00
Alexandre Ganea f860fe3622 [LLD] Remove global state in lldCommon
Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

Differential Revision: https://reviews.llvm.org/D108850
2022-01-16 08:57:57 -05:00
Fangrui Song 7c269db779 [lld-macho] Simplify DeduplicatedCStringSection::finalizeContents. NFC
Tail merge is slow and of low value. With regular string deduplication, we can
just use the return value of StringTableBuilder::add.

There is no noticeable performance increase because without deduplication
`__cstring` is quite small (7.6MiB for chromium_framework).

Reviewed By: #lld-macho, Jez Ng

Differential Revision: https://reviews.llvm.org/D117273
2022-01-14 13:12:57 -08:00
Juergen Ributzka 3025c3eded Replace PlatformKind with PlatformType.
The PlatformKind/PlatformType enums contain the same information, which requires
them to be kept in-sync. This commit changes over to PlatformType as the sole
source of truth, which allows the removal of the redundant PlatformKind.

The majority of the changes were in LLD and TextAPI.

Reviewed By: cishida

Differential Revision: https://reviews.llvm.org/D117163
2022-01-13 09:23:49 -08:00
Leonard Grey 6db04b97e6 [lld-macho] Port CallGraphSort from COFF/ELF
Depends on D112160

This adds the new options `--call-graph-profile-sort` (default),
`--no-call-graph-profile-sort` and `--print-symbol-order=`. If call graph
profile sorting is enabled, reads `__LLVM,__cg_profile` sections from object
files and uses the resulting graph to put callees and callers close to each
other in the final binary via the C3 clustering heuristic.

Differential Revision: https://reviews.llvm.org/D112164
2022-01-12 10:47:04 -05:00
Jez Ng e976c457c5 [lld-macho] Initialize separate time trace profiler for mapfile worker
After {D115416}, the "Write map file" event no longer shows up
in the time trace. Each time trace profiler instance is thread-local,
but we had neglected to initialize a separate instance for the mapfile
worker thread.

Reviewed By: keith

Differential Revision: https://reviews.llvm.org/D117069
2022-01-11 17:45:18 -08:00
Fangrui Song 97a5dccb7d [lld-macho] Rename LazySymbol to LazyArchive. NFC
D116913 will add LazyObject. Rename LazySymbol to LazyArchive to avoid confusion
and mirror ELF.

Reviewed By: #lld-macho, Jez Ng

Differential Revision: https://reviews.llvm.org/D116914
2022-01-11 16:49:06 -08:00
Fangrui Song 477bc36d3b [lld-macho] Change some global pointers to unique_ptr
Similar to D116143. My x86-64 `lld` is ~8KiB smaller.

Reviewed By: keith

Differential Revision: https://reviews.llvm.org/D116902
2022-01-10 19:39:14 -08:00
Vincent Lee 7a161eb43b [lld-macho] Fix shadowed variable
This fixes a windows build failure from D115416.
2022-01-10 00:20:35 -08:00
Kazu Hirata 8afcfbfb8f Use true/false instead of 1/0 (NFC)
Identified by modernize-use-bool-literals.
2022-01-09 12:21:06 -08:00
Kazu Hirata b12fd13812 Fix bugprone argument comments.
Identified by bugprone-argument-comment.
2022-01-09 12:21:02 -08:00
Vincent Lee a963bc490d [lld-macho] Increase slops to prevent thunk out of range
One of our internal arm64 apps hit a thunk out of range error when building
with LLD. Per the comment, I'm arbitrarily increasing slop size to 256.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D116705
2022-01-06 12:29:12 -08:00
Luís Ferreira 10e40a4ea3 [lld] Add support for other demanglers other than Itanium
LLVM core library supports demangling other mangled symbols other than itanium,
such as D and Rust. LLD should use those demanglers in order to output pretty
demangled symbols on error messages.

Reviewed By: MaskRay, #lld-macho

Differential Revision: https://reviews.llvm.org/D116279
2022-01-05 03:25:41 +00:00
Luís Ferreira 8792cd75d0 Revert "[lld] Add support for other demanglers other than Itanium"
This reverts commit e60d6dfd5a.

clang-ppc64le-rhel buildbot failed (https://lab.llvm.org/buildbot#builders/57/builds/13424):

    tools/lld/MachO/CMakeFiles/lldMachO.dir/Symbols.cpp.o: In function `lld::demangle(llvm::StringRef, bool)':
    Symbols.cpp:(.text._ZN3lld8demangleEN4llvm9StringRefEb[_ZN3lld8demangleEN4llvm9StringRefEb]+0x90): undefined reference to `llvm::demangle(std::string const&)'
2021-12-30 18:04:21 +00:00
Luís Ferreira e60d6dfd5a [lld] Add support for other demanglers other than Itanium
LLVM core library supports demangling other mangled symbols other than itanium,
such as D and Rust. LLD should use those demanglers in order to output pretty
demangled symbols on error messages.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D116279
2021-12-30 17:52:38 +00:00
Mike Hommey 319181f767 [lld-macho] Fix alignment of TLV data sections
References from thread-local variable sections are treated as offsets
relative to the start of the thread-local data memory area, which is
initialized via copying all the TLV data sections (which are all
contiguous). If later data sections require a greater alignment than
earlier ones, the offsets of data within those sections won't be
guaranteed to aligned unless we normalize alignments. We therefore use
the largest alignment for all TLV data sections.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D116263
2021-12-28 19:03:13 -05:00
Vy Nguyen 4f90e67e2f [lld-macho] Handle $ld$hide[$os] symbols.
PR/52708

Differential Revision: https://reviews.llvm.org/D115775
2021-12-17 16:40:07 -05:00
Nico Weber c4b45eeb44 [lld/mac] Don't lose "weak ref" bit when doing LTO
Fixes .

Probably fixes Chromium crashing on startup on macOS 10.15 (and older) systems
when building with LTO, but I haven't verified that yet.

Differential Revision: https://reviews.llvm.org/D115949
2021-12-17 15:26:35 -05:00
Vincent Lee d17b092fe6 [lld-macho] Make writing map file asynchronous
For large applications that write to map files, writing map files can take quite
a bit of time. Sorting the biggest contributors to link times, writing map files
ranks in at 2nd place, with load input files being the biggest contributor of
link times. Avoiding writing map files on the critical path (and having its own
thread) saves ~2-3 seconds when linking chromium framework on a 16-Core
Intel Xeon W.

```
           base            diff            difference (95% CI)
sys_time   1.617 ± 0.034   1.657 ± 0.026   [  +1.5% ..   +3.5%]
user_time  28.536 ± 0.245  28.609 ± 0.180  [  -0.1% ..   +0.7%]
wall_time  23.833 ± 0.271  21.684 ± 0.194  [  -9.5% ..   -8.5%]
samples    31              24
```

Reviewed By: #lld-macho, oontvoo, int3

Differential Revision: https://reviews.llvm.org/D115416
2021-12-15 16:37:04 -08:00
Jez Ng 098430cd25 [lld-macho][nfc] Simplify LC_DATA_IN_CODE generation
1. After D113241, we have the section address easily accessible and no
   longer need to iterate across the LC_SEGMENT commands to emit
   LC_DATA_IN_CODE.

2. There's no need to store a pointer to the data in code entries during
   the parse step; we can just look it up as part of the output step.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D115556
2021-12-11 01:01:57 -05:00
Jez Ng 40bcbe48e8 [lld-macho][nfc] InputSections don't need to track their total # of callsites
... only whether they have more than zero. This simplifies the code slightly.

I've also moved the field into the ConcatInputSection subclass since it doesn't
actually get used by the other InputSections.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D115539
2021-12-11 01:01:57 -05:00
Jez Ng 8a1f2d6580 [lld-macho] Include archive name in bitcode files
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D115281
2021-12-07 19:11:23 -05:00
Jez Ng 1b44364714 [lld-macho] Unreferenced weak dylib symbols shouldn't fetch archive symbols
We were fetching archive symbols too eagerly, bloating binary size as well as
just screwing up binaries that expected to look up certain symbols only at
runtime.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D115092
2021-12-05 15:11:44 -05:00
Vy Nguyen 74cbd71072 [lld-macho] Mark dylib symbols coming from -weak_framework as weak-ref.
PR:52564

Differential Revision: https://reviews.llvm.org/D114397
2021-11-30 09:54:59 -05:00
Shoaib Meenai 2f5d6a0ea5 [MachO] Fix struct size assertion
std::vector can have different sizes depending on the STL's debug level,
so account for its size separately. (You could argue that we should be
accounting for all the other members separately as well, but that would
be very unergonomic, and std::vector is the only one that's caused
problems so far.)
2021-11-22 15:02:30 -08:00
Vy Nguyen 944071eca2 [lld-macho] Don't replace local personality symbol with LazySymbol
Follup-up to D107533, where we replaced local syms with non-local.
It doesn't make sense to replace local symbol with lazy.

Differential Revision: https://reviews.llvm.org/D110040
2021-11-22 14:09:54 -05:00
Nico Weber bc20bcb39e [lld/mac] Crash even less on undefined symbols with --icf=all
Follow-up to https://reviews.llvm.org/D112643. Even after that change, we were
still asserting if two separate functions that are eligible for ICF (same size,
same data, same number of relocs, same reloc types, ...) referred to
Undefineds. This fixes that oversight.

Differential Revision: https://reviews.llvm.org/D114195
2021-11-19 09:23:19 -05:00
Vincent Lee adfbb5411b [lld-macho] Add warn flags to enable/disable warnings on -install_name
ld64 doesn't warn on builds using `-install_name` if it's a bundle. But, the
current warning is nice to have because `install_name` only works with dylib.
To prevent an overflow of warnings in build logs and have parity with ld64,
create a `--warn-dylib-install-name` and `--warn-no-dylib-install-name` flag
that enables this LLD specific warning.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D113534
2021-11-17 16:18:14 -08:00
Greg McGary 9cc489a4b2 [lld-macho][nfc] Factor-out NFC changes from main __eh_frame diff
In order to keep signal:noise high for the `__eh_frame` diff, I have teased-out the NFC changes and put them here.

Differential Revision: https://reviews.llvm.org/D114017
2021-11-17 15:16:44 -07:00
Shoaib Meenai 01510ac084 [MachO] Move type size asserts to source files. NFC
As discussed in https://reviews.llvm.org/D113809#3128636. It's a bit
unfortunate to move the asserts away from the structs whose sizes
they're checking, but it's a far better developer experience when one of
the asserts is violated, because you get a single error instead of every
single source file including the header erroring out.
2021-11-16 17:14:16 -08:00
Vy Nguyen 34d15eaced [lld-macho][nfc] Sanity check on template type
Differential Revision: https://reviews.llvm.org/D114044
2021-11-16 20:04:49 -05:00
Shoaib Meenai 93bf271f27 [MachO] Shrink reloc from 32 bytes to 24 bytes
The `r_address` field of `relocation_info` is only 4 bytes, so our
offset field (which is the `r_address` field adjusted for subsection
splitting) also only needs to be 4 bytes. This reduces the structure
size from 32 bytes to 24 bytes.

Combined with https://reviews.llvm.org/D113813, this is a minor perf
improvement for linking an internal app, tested on two machines:

```
           smol-relocs     baseline        difference (95% CI)
sys_time   7.367 ± 0.138   7.543 ± 0.157   [  +0.9% ..   +3.8%]
user_time  21.843 ± 0.351  21.861 ± 0.450  [  -1.3% ..   +1.4%]
wall_time  20.301 ± 0.307  20.556 ± 0.324  [  +0.1% ..   +2.4%]
samples    16              16

           smol-relocs     baseline        difference (95% CI)
sys_time   2.923 ± 0.050   2.992 ± 0.018   [  +1.4% ..   +3.4%]
user_time  10.345 ± 0.039  10.448 ± 0.023  [  +0.8% ..   +1.2%]
wall_time  12.068 ± 0.071  12.229 ± 0.021  [  +1.0% ..   +1.7%]
samples    15              12
```

More importantly though, this change by itself reduces our maximum
resident set size by 220 MB (2.75%, from 7.85 GB to 7.64 GB) on the
first machine. On the second machine, it reduces it by 125 MB (1.94%,
from 6.31 GB to 6.19 GB).

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D113818
2021-11-16 16:30:34 -08:00
Shoaib Meenai 3195297897 [MachO] Reduce size of Symbol and Defined
We can lay out Symbol more optimally to reduce its size from 56 bytes to
48 bytes by eliminating unnecessary padding, and we can lay out Defined
such that its bitfield members are placed in the tail padding of Symbol
(on ABIs which support this), to reduce it from 96 bytes to 80 bytes (8
bytes from the Symbol reduction, and 8 bytes from the tail padding
reuse).

This is perf-neutral for an internal app (results from two different
machines):

```
           smol-syms       baseline        difference (95% CI)
sys_time   7.430 ± 0.202   7.440 ± 0.193   [  -2.6% ..   +2.9%]
user_time  21.443 ± 0.513  21.206 ± 0.396  [  -3.3% ..   +1.1%]
wall_time  20.453 ± 0.534  20.222 ± 0.488  [  -3.7% ..   +1.5%]
samples    9               8

           smol-syms       baseline        difference (95% CI)
sys_time   3.011 ± 0.050   3.040 ± 0.052   [  -0.4% ..   +2.3%]
user_time  10.416 ± 0.075  10.496 ± 0.091  [  +0.1% ..   +1.4%]
wall_time  12.229 ± 0.144  12.354 ± 0.192  [  -0.1% ..   +2.1%]
samples    14              13
```

However, on the first machine, it reduces maximum resident set size by
65.9 MB (0.8%, from 7.92 GB to 7.85 GB). On the second machine, it
reduces it by 92 MB (1.4%, from 6.40 GB to 6.31 GB).

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D113813
2021-11-16 16:30:33 -08:00
Shoaib Meenai 637a3396b3 [MachO] Fix struct size assertion
It was checking for 64-bit builds incorrectly. Unfortunately,
ConcatInputSection has grown a bit in the meantime, and I don't see any
obvious way to shrink it. Perhaps icfEqClass could use 32-bit hashes
instead of 64-bit ones, but xxHash64 is supposed to be much faster than
xxHash32 (https://github.com/Cyan4973/xxHash#benchmarks), so that sounds
like a loss. (Unrelatedly, we should really look at using XXH3 instead
of xxHash64 now.)

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D113809
2021-11-16 16:30:31 -08:00
Greg McGary 3a1b3c9afe [lld-macho][nfc] rename parsed-section types & variables
This is an NFC diff that prepares for pruning & relocating `__eh_frame`.

Along the way, I made the following changes to ...
* clarify usage of `section` vs. `subsection`
* remove `map` & `vec` from type names
* disambiguate class `Section` from template parameter `SectionHeader`.

Differential Revision: https://reviews.llvm.org/D113241
2021-11-16 07:06:41 -07:00
Shao-Ce SUN 0c660256eb [NFC] Trim trailing whitespace in *.rst 2021-11-15 09:17:08 +08:00
Keith Smiley 51715fbd96 [lld-macho] Fix warning
```
/Users/ksmiley/dev/llvm-project/lld/MachO/Symbols.cpp:43:27: warning: field 'external' will be initialized after field 'weakDefCanBeHidden' [-Wreorder-ctor]
      weakDef(isWeakDef), external(isExternal),
                          ^
1 warning generated.
```

Differential Revision: https://reviews.llvm.org/D113823
2021-11-12 19:36:51 -08:00
Vy Nguyen 9b29dae3ca [lld-macho] Allow exporting weak_def_can_be_hidden(AKA "autohide") symbols
autohide symbols behaves similarly to private_extern symbols.
However, LD64 allows exporting autohide symbols. LLD currently does not.
This patch allows LLD to export them.

Differential Revision: https://reviews.llvm.org/D113167
2021-11-12 21:57:30 -05:00
Vy Nguyen ad932320d8 [lld-macho] Parallelize scanning the symbol tables in export/unexport-ing.
(Split from D113167)
Benchmarking on one of our large apps which exports a few thousands symbols,
this showed an improvement of ~17%.

x ./LLD_no_parallel.txt
+ ./LLD_with_parallel.txt

    N           Min           Max        Median           Avg        Stddev
x  10         84.01         89.41         88.64        87.693     1.7424061
+  10          71.9         74.29         72.63        72.753    0.77734663
Difference at 95.0% confidence
	-14.94 +/- 1.26763
	-17.0367% +/- 1.44553%
	(Student's t, pooled s = 1.34912)

(wallclock)

Differential Revision: https://reviews.llvm.org/D113820
2021-11-12 20:57:24 -05:00
Jez Ng 9d0b237c51 [lld-macho] Fix symbol relocs handling for LSDAs
Similar to D113702, but for the LSDAs. Clang seems to emit all LSDA
relocs as section relocs, but ld -r can turn those relocs into symbol
ones.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D113721
2021-11-12 16:02:49 -05:00
Jez Ng d9b6f7e312 [lld-macho] Teach ICF to dedup functions with identical unwind info
Dedup'ing unwind info is tricky because each CUE contains a different
function address, if ICF operated naively and compared the entire
contents of each CUE, entries with identical unwind info but belonging
to different functions would never be considered identical. To work
around this problem, we slice away the function address before
performing ICF. We rely on `relocateCompactUnwind()` to correctly handle
these truncated input sections.

Here are the numbers before and after D109944, D109945, and this diff
were applied, as tested on my 3.2 GHz 16-Core Intel Xeon W:

Without any optimizations:

             base           diff           difference (95% CI)
  sys_time   0.849 ± 0.015  0.896 ± 0.012  [  +4.8% ..   +6.2%]
  user_time  3.357 ± 0.030  3.512 ± 0.023  [  +4.3% ..   +5.0%]
  wall_time  3.944 ± 0.039  4.032 ± 0.031  [  +1.8% ..   +2.6%]
  samples    40             38

With `-dead_strip`:

             base           diff           difference (95% CI)
  sys_time   0.847 ± 0.010  0.896 ± 0.012  [  +5.2% ..   +6.5%]
  user_time  3.377 ± 0.014  3.532 ± 0.015  [  +4.4% ..   +4.8%]
  wall_time  3.962 ± 0.024  4.060 ± 0.030  [  +2.1% ..   +2.8%]
  samples    47             30

With `-dead_strip` and `--icf=all`:

             base           diff           difference (95% CI)
  sys_time   0.935 ± 0.013  0.957 ± 0.018  [  +1.5% ..   +3.2%]
  user_time  3.472 ± 0.022  6.531 ± 0.046  [ +87.6% ..  +88.7%]
  wall_time  4.080 ± 0.040  5.329 ± 0.060  [ +30.0% ..  +31.2%]
  samples    37             30

Unsurprisingly, ICF is now a lot slower, likely due to the much larger
number of input sections it needs to process. But the rest of the
linker only suffers a mild slowdown.

Note that the compact-unwind-bad-reloc.s test was expanded because we
now handle the relocation for CUE's function address in a separate code
path from the rest of the CUE relocations. The extended test covers both
code paths.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D109946
2021-11-12 16:02:49 -05:00
Jez Ng ad8df21db2 [reland][lld-macho] Fix symbol relocs handling for compact unwind's functionAddress
Clang seems to emit all functionAddress relocs as section relocs, but
`ld -r` can turn those relocs into symbol ones. It turns out that we
weren't handling that case correctly when the symbol was a weak def
whose definition did not prevail.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D113702
2021-11-12 15:01:51 -05:00
Keith Smiley eb6f9f3123 [lld-macho] Fix trailing slash in oso_prefix
Previously if you passed `-oso_prefix path/to/foo/` with a trailing
slash at the end, using `real_path` would remove that slash, but that
slash is necessary to make sure OSO prefix paths end up as valid
relative paths instead of starting with `/`.

Differential Revision: https://reviews.llvm.org/D113541
2021-11-12 11:29:08 -08:00
Kazu Hirata 835135a8ae Revert "[lld-macho] Fix symbol relocs handling for compact unwind's functionAddress"
This reverts commit e941fe5061.

The commit in question causes:

  lld/MachO/InputFiles.cpp:916:13: error: use of undeclared identifier
  'it'
2021-11-11 20:29:48 -08:00
Jez Ng e941fe5061 [lld-macho] Fix symbol relocs handling for compact unwind's functionAddress
Clang seems to emit all functionAddress relocs as section relocs, but
`ld -r` can turn those relocs into symbol ones. It turns out that we
weren't handling that case correctly when the symbol was a weak def
whose definition did not prevail.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D113702
2021-11-11 22:53:35 -05:00
Jez Ng a2404f11c7 [lld-macho] Support renaming of LSDA section
Previously, our unwind info finalization logic assumed that the LSDA
section referenced by `__compact_unwind` was already finalized before
`__TEXT,__unwind_info` itself. However, that assumption could be broken
by the use of `-rename_section` -- it could be (and is) used to move
`__gcc_except_tab` it into a different segment later in the file.
(__TEXT is always the first non-zerofill segment, so any rename
basically guarantees that the section will be ordered after
`__unwind_info`.)

To handle this case, we compare LSDA relocations instead of their final
values in `UnwindInfoSection::finalize()`, and we actually relocate
those LSDAs in `UnwindInfoSection::writeTo()`. In order to do this, we
need an easy way to track which Symbol a given CUE corresponds to. My
solution was to change our `cuPtrVector` into a vector of indices, with
each index used for both the symbols vector (`symbolsVec`) as well as
the CUE vector (`cuVector`).

This change seems perf neutral. Numbers for linking chromium_framework
on my 16 core Mac Pro:

             base           diff           difference (95% CI)
  sys_time   1.248 ± 0.025  1.245 ± 0.026  [  -1.3% ..   +0.8%]
  user_time  3.588 ± 0.045  3.587 ± 0.037  [  -0.6% ..   +0.5%]
  wall_time  4.605 ± 0.069  4.595 ± 0.069  [  -1.0% ..   +0.5%]
  samples    42             26

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D113582
2021-11-10 19:31:54 -05:00
Vy Nguyen 2e1be96df6 Reland "[lld-macho] Fix assertion failure in registerCompactUnwind""
PR/52372

  Differential Revision: https://reviews.llvm.org/D112977

New changes:
- use llvm-otool instead of `otool` which doesn't in exist on non-OSX platforms
- add llvm-otool to the set of tools used by test so that the bot will use the <build_dir>/bin/llvm-otool instead of the unqualified `llvm-otool` (which may not exist)
- update tests since the latest (TOT) llvm-otool prints a space between two bytes and the old one doesn't.
2021-11-09 11:52:46 -05:00
Vy Nguyen eb4a517816 Revert "[lld-macho] Fix assertion failure in registerCompactUnwind"
broke windows build - reverting to investigate
This reverts commit b2d9258474.
2021-11-09 10:31:47 -05:00
Vy Nguyen b2d9258474 [lld-macho] Fix assertion failure in registerCompactUnwind
PR/52372

  Differential Revision: https://reviews.llvm.org/D112977
2021-11-09 10:08:17 -05:00
Keith Smiley a7a2959901 [lld-macho] Replace LC_LINKER_OPTION parsing
This removes the tablegen based parsing of LC_LINKER_OPTION since it can
only actually contain a very small number of potential arguments. In our
project with tablegen this took 5 seconds before.

This replaces https://reviews.llvm.org/D113075

Differential Revision: https://reviews.llvm.org/D113235
2021-11-04 22:03:40 -07:00
Fangrui Song 005456e5fc [lld-macho] Fix an assertion failure when -u specifies an undefined section$start symbol
This matches ld64. Also improve the test for `-dead_strip`.

Reviewed By: #lld-macho, Jez Ng

Differential Revision: https://reviews.llvm.org/D113147
2021-11-04 21:28:33 -07:00
Keith Smiley 0bce3e3b84 [lld-macho] Clear resolvedReads cache
https://reviews.llvm.org/D113153#3108083

smeenai, int3

Differential Revision: https://reviews.llvm.org/D113198
2021-11-04 18:02:34 -07:00
Noah Shutty d788c44f5c [Support] Improve Caching conformance with Support library behavior
This diff makes several amendments to the local file caching mechanism
which was migrated from ThinLTO to Support in
rGe678c51177102845c93529d457b020f969125373 in response to follow-up
discussion on that commit.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D113080
2021-11-04 13:00:44 -07:00
Keith Smiley e7fdff403e [lld-macho] Silently ignore the -objc_abi_version
This undocumented ld64 flag, based on the most recent ld64 source dump
from Xcode 12, only applies to i386. It seems like on all newer
architectures this behavior is the default.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D113070
2021-11-03 22:16:09 -07:00
Keith Smiley d49e7244cc [lld-macho] Cache readFile results
In one of our links lld was reading 760k files, but the unique number of
files was only 1500. This takes that link from 30 seconds to 8.

This seems like a heavy hammer, especially since some things don't need
to be cached, like the filelist arguments and the passed static
archives (the latter is already cached as a one off), but it seems ld64
does something similar here to short circuit these duplicate reads:

82e429e186/src/ld/InputFiles.cpp (L644-L665)

Of the types of files being read for our iOS app, the biggest problem
was constantly re-reading small tbd files:

```
% wc -l /tmp/read.txt
761414 /tmp/read.txt
% cat /tmp/read.txt | sort -u | wc -l
1503

% cat /tmp/read.txt | grep "\.a$" | wc -l
43721
% cat /tmp/read.txt | grep "\.tbd$" | wc -l
717656
```

We could likely hoist this logic up to not cache at this level, but it
would be a more invasive change to make sure all callers that needed it
cached the results.

I could see this being an issue with OOMs, and I'm not a linker expert so
maybe there's another way we should solve this problem? Feedback welcome!

Reviewed By: int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D113153
2021-11-03 22:12:21 -07:00
Keith Smiley 6629ec3ecc [lld-macho] Implement -arch_errors_fatal
By default with ld64, architecture mismatches are just warnings, then
this flag can be passed to make these fail. This matches that behavior.

Reviewed By: int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D113082
2021-11-03 22:01:53 -07:00
Keith Smiley 63e65de3ff [lld-macho] Cache discovered framework paths
On our large iOS project this took a link from 1 minute 45 seconds to 45
seconds. For reference ld64 does the same link in ~20 seconds.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D113063
2021-11-03 11:11:54 -07:00
Keith Smiley f79e65e61f [lld-macho] Cache library paths from findLibrary
On top of https://reviews.llvm.org/D113063 this took another 10 seconds
off our overall link time.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D113073
2021-11-03 10:02:23 -07:00
Vy Nguyen 37f96cb478 Revert "[lld-macho] Change bitfield types to be identical."
This reverts commit ae31f9fbad.

Reason: bitfields can't be merged across parent/child classes anyway. So this change doesn't help.
2021-11-02 16:57:51 -04:00
Vy Nguyen ae31f9fbad [lld-macho] Change bitfield types to be identical.
Symbol's subclasses all have an additional bitfield of type uint8_t (RefState enum).
For the bitfields in the same block tomerge, they should be of the same type. (clang/gcc will work, but others like MSVC does not)

Differential Revision: https://reviews.llvm.org/D113040
2021-11-02 15:48:39 -04:00
Nico Weber 64c1734438 [lld/mac] Write -v output to stderr
This matches ld64, and it's conceivable that projects try to read
this information off stderr for that reason.

--version keeps writing to stdout.

Differential Revision: https://reviews.llvm.org/D113020
2021-11-02 13:59:14 -04:00
Vy Nguyen d7e5393af4 [lld-macho] Remove no_dtrace_dof from un-implemented group.
One fewer warning.
In practice, lld already "implements" it. (ie., it does not do dtrace-dof processing ever).

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D112934
2021-11-02 12:36:08 -04:00
Vy Nguyen 3f35dd06a5 [lld-macho][nfc][cleanup] Fix a few code style lints and clang-tidy findings
- Use .empty() instead of `size() == 0` when possible.
- Use const-ref to avoid copying

Differential Revision: https://reviews.llvm.org/D112978
2021-11-02 11:26:15 -04:00
Shoaib Meenai 264d3b6d4e [MachO] Use error instead of fatal for missing -arch
`fatal` should only be used for malformed inputs according to
ErrorHandler.h; `error` is more appropriate for missing arguments,
accompanied by a check to bail out early in case of the error. Some
tests need to be adjusted accordingly.

Makes `lld/test/MachO/arch.s` pass with `LLD_IN_TEST=2`.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D112879
2021-10-31 16:31:21 -07:00
Shoaib Meenai 0f6d720f1f [MachO] Properly reset global state
We need to reset global state between runs, similar to the other ports.
There's some file-static state which needs to be reset as well and we
need to add some new helpers for that.

With this change, most LLD Mach-O tests pass with `LLD_IN_TEST=2` (which
runs the linker twice on each test). Some tests will be fixed by the
remainder of this stack, and the rest are fundamentally incompatible
with that mode (e.g. they intentionally throw fatal errors).

Fixes PR52070.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D112878
2021-10-31 16:14:29 -07:00
Nico Weber 2d48b19136 [lld/mac] Fix mislink with ICF
When comparing relocations against two symbols, ICF's equalsConstant() did not
look at the value of the two symbols. With subsections_via_symbols, the value
is usually 0 but not always: In particular, it isn't 0 for constants in string
and literal sections. Since we ignored the value, comparing two constant string
symbols or two literal symbols always compared the 0th's element, so functions
in the same TU always compared as equal.

This can cause mislinks, and, with -dead_strip, crashes.

Fixes PR52349, see that bug for lots of details and examples of mislinks.

While here, make the existing assembly in icf-literals.s a bit more realistic
(use leaq instead of movq with strings, and use foo(%rip) instead of
foo@gotpcrel(%rip)). This has no interesting effect, it just maybe makes the
test look a bit less surprising.

Differential Revision: https://reviews.llvm.org/D112862
2021-10-30 18:58:59 -04:00
Jez Ng 6c2f26a159 [lld-macho] -all_load and -ObjC should not affect LC_LINKER_OPTION flags
In particular, they should not cause archives to be eagerly loaded. This
matches ld64's behavior.

Fixes PR52246.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D112756
2021-10-29 11:00:28 -04:00
Jez Ng a271f2410f [lld-macho][nfc] Canonicalize all pointers to InputSections early on
Having to remember to call `canonical()` all over the place is
error-prone; let's do it in a centralized location instead. It also
appears to improve performance slightly.

             base           diff           difference (95% CI)
  sys_time   0.984 ± 0.009  0.983 ± 0.014  [  -0.8% ..   +0.6%]
  user_time  6.508 ± 0.035  6.475 ± 0.036  [  -0.8% ..   -0.2%]
  wall_time  5.321 ± 0.034  5.300 ± 0.033  [  -0.7% ..   -0.1%]
  samples    36             23

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D112687
2021-10-29 11:00:28 -04:00
Fangrui Song 3a4b605bc1 [lld-macho] Internalize createFiles. NFC 2021-10-28 22:14:37 -07:00
Vincent Lee d54360cd32 [lld-macho] Implement -S
There are a couple internal builds that require the use of this flag.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D112594
2021-10-27 17:09:57 -07:00
Nico Weber 7f369304df [lld/mac] Don't crash on undefined symbols with --icf=all
ICF runs before relocation processing, but undefined symbol errors
are only emitted during relocation processing.

So just ignore Undefineds during ICF (instead of crashing) -- lld
will emit an error once ICF is done.

Fixes PR52330.

Differential Revision: https://reviews.llvm.org/D112643
2021-10-27 16:20:10 -04:00
Jez Ng b7e12ca7aa [lld-macho] If export_size is zero, export_off must be zero
Otherwise tools like codesign_allocate will choke. We were already
handling this correctly for the other DYLD_INFO sections.

Doing this correctly is a bit subtle: we don't know if export_size will
be zero until we have run `ExportSection::finalizeContents()`. However,
we must still add the ExportSection to the `__LINKEDIT` segment in order
that it gets sorted during `sortSectionsAndSegments()`.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D112589
2021-10-27 14:58:42 -04:00
Nico Weber 6503a68565 [lld/mac] Don't assert when ICFing arm64 code
WordLiteralSection dedupes literals by content.
WordLiteralInputSection::getOffset() used to read a literal at the passed-in
offset and look up this value in the deduping map to find the offset of the
deduped value.

But it's possible that (e.g.) a 16-byte literal's value is accessed 4 bytes in.
To get the offset at that address, we have to get the deduped value at offset 0
and then apply the offset 4 to the result.

(See also WordLiteralSection::finalizeContents() which fills in those maps.)

Only a problem on arm64 because in x86_64 the offset is part of the instruction
instead of a separate ARM64_RELOC_ADDEND relocation. (See bug for more details.)

Fixes PR51999.

Differential Revision: https://reviews.llvm.org/D112584
2021-10-27 14:02:07 -04:00
Jez Ng 1d2a4cd57d [lld-macho] Fix compact-unwind-bad-reloc.s test
Broken by a9353dbe51.

Now that the functions point to the compact unwind entries, instead of
the other way around, we need to perform the "invalid reference" check
in a different place.

This change was originally part of the stacked diff D109946, but should
have been included as part of D109945.
2021-10-26 18:59:12 -04:00
Nuri Amari a299b24712 Regenerate LC_CODE_SIGNATURE during llvm-objcopy operations
**Context:**

This is a second attempt at introducing signature regeneration to llvm-objcopy. In this diff: https://reviews.llvm.org/D109840, a script was introduced to test
the validity of a code signature. In this diff: https://reviews.llvm.org/D109803 (now reverted), an effort was made to extract the signature generation behavior out of LLD into a common location for use in llvm-objcopy. In this diff: https://reviews.llvm.org/D109972 it was decided that there was no appropriate common location and that a small amount of duplication to bring signature generation to llvm-objcopy would be better. This diff introduces this duplication.

**Summary**

Prior to this change, if a LC_CODE_SIGNATURE load command
was included in the binary passed to llvm-objcopy, the command and
associated section were simply copied and included verbatim in the
new binary. If rest of the binary was modified at all, this results
in an invalid Mach-O file. This change regenerates the signature
rather than copying it.

The code_signature_lc.test test was modified to include the yaml
representation of a small signed MachO executable in order to
effectively test the signature generation.

Reviewed By: alexander-shaposhnikov, #lld-macho

Differential Revision: https://reviews.llvm.org/D111164
2021-10-26 14:51:13 -07:00
Jez Ng a9353dbe51 [lld-macho] Simplify the handling of "no unwind info" functions
This diff does away with `addEntriesForFunctionsWithoutUnwindInfo()`,
because `addSymbol()` can now determine which functions need those
entries.

While overhauling UnwindInfoSection, I also parallelized the relocation
of the contents of the CUEs. This somewhat offsets the time regression
from creating one InputSection per CUE (which was done in D109944).

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D109945
2021-10-26 16:04:16 -04:00
Jez Ng 002eda7056 [lld-macho] Associate compact unwind entries with function symbols
Compact unwind entries (CUEs) contain pointers to their respective
function symbols. However, during the link process, it's far more useful
to have pointers from the function symbol to the CUE than vice versa.
This diff adds that pointer in the form of `Defined::compactUnwind`.

In particular, when doing dead-stripping, we want to mark CUEs live when
their function symbol is live; and when doing ICF, we want to dedup
sections iff the symbols in that section have identical CUEs. In both
cases, we want to be able to locate the symbols within a given section,
as well as locate the CUEs belonging to those symbols. So this diff also
adds `InputSection::symbols`.

The ultimate goal of this refactor is to have ICF support dedup'ing
functions with unwind info, but that will be handled in subsequent
diffs. This diff focuses on simplifying `-dead_strip` --
`findFunctionsWithUnwindInfo` is no longer necessary, and
`Defined::isLive()` is now a lot simpler. Moreover, UnwindInfoSection no
longer has to check for dead CUEs -- we simply avoid adding them in the
first place.

Additionally, we now support stripping of dead LSDAs, which follows
quite naturally since `markLive()` can now reach them via the CUEs.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D109944
2021-10-26 16:04:15 -04:00
Jez Ng 622150ad5f [lld-macho] Put GOT into `__DATA` segment where appropriate
We were previously always emitting the GOT into `__DATA_CONST`, even for
target platforms where it should end up in `__DATA`.

I stumbled onto this while trying to use the `class-dump` tool -- with
the wrong segment names, it fails to locate the ObjC runtime info and
therefore fails to dump any classes.

Reviewed By: #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D112500
2021-10-26 11:38:01 -04:00
Vy Nguyen 236197e2d0 [lld-macho] Implement -oso_prefix
https://bugs.llvm.org/show_bug.cgi?id=50229

Differential Revision: https://reviews.llvm.org/D112291
2021-10-22 16:32:42 -04:00
Vy Nguyen 6b715e9c4d [lld-macho][nfc] Added some notes on deliberate differences btw LD64 vs LLD-MACHO
For future references and to help with debugging crashes, this could be useful.

Differential Revision: https://reviews.llvm.org/D110464
2021-10-20 22:41:57 -04:00
Nico Weber 1412719066 [lld/mac] Remove else-after-return in ICF code
No behavior change.
2021-10-20 14:24:13 -04:00
Kaining Zhong aab0f2264a [lld-macho] Fix dangling string reference when adding frameworks
In Driver.cpp, addFramework used std::string instance to represent the path of a framework, which will be freed after the function returns. However, this string is stored in loadedArchive, which will be used later to compare with path of newly added frameworks. This caused https://bugs.llvm.org/show_bug.cgi?id=52133. A test is included in this commit to reproduce this bug.

Now resolveDylibPath returns a StringRef instance, and it uses StringSaver to save its data, then returns it to functions on the top. This ensures the resolved framework path is still valid after LC_LINKER_OPTION is parsed.

Reviewed By: int3, #lld-macho, oontvoo

Differential Revision: https://reviews.llvm.org/D111706
2021-10-20 11:21:40 -04:00
Noah Shutty e678c51177 [Support][ThinLTO] Move ThinLTO caching to LLVM Support library
We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371
2021-10-18 18:57:25 -07:00
Petr Hosek 8e46e34d24 Revert "[Support][ThinLTO] Move ThinLTO caching to LLVM Support library"
This reverts commit 92b8cc52bb since
it broke the gold plugin.
2021-10-18 12:24:05 -07:00
Noah Shutty 92b8cc52bb [Support][ThinLTO] Move ThinLTO caching to LLVM Support library
We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371
2021-10-18 12:08:49 -07:00
Nico Weber 4e572db0c2 [lld/mac] Mark private externs with GOT relocs as LOCAL in indirect symbtab
prepareSymbolRelocation() in Writer.cpp adds both symbols that need binding and
symbols relocated with a pointer relocation to the got.

Pointer relocations are emitted for non-movq GOTPCREL(%rip) loads.  (movqs
become GOT_LOADs so that the linker knows they can be relaxed to leaqs, while
others, such as addq, become just GOT -- a pointer relocation -- since they
can't be relaxed in that way).

For example, this C file produces a private_extern GOT relocation when
compiled with -O2 with clang:

    extern const char kString[];
    const char* g(int a) { return kString + a; }

Linkers need to put pointer-relocated symbols into the GOT, but ld64 marks them
as LOCAL in the indirect symbol table. This matters, since `strip -x` looks at
the indirect symbol table when deciding what to strip.

The indirect symtab emitting code was assuming that only symbols that need
binding are in the GOT, but pointer relocations where there too. Hence, the
code needs to explicitly check if a symbol is a private extern.

Fixes https://crbug.com/1242638, which has some more information in comments 14
and 15. With this patch, the output of `nm -U` on Chromium Framework after
stripping now contains just two symbols when using lld, just like with ld64.

Differential Revision: https://reviews.llvm.org/D111852
2021-10-15 13:24:47 -04:00
Nico Weber f3091831f4 [lld] Use checkError more
No behavior change.
2021-10-04 11:46:16 -04:00
Daniel Rodríguez Troitiño 657f02d458 Revert "Extract LC_CODE_SIGNATURE related implementation out of LLD"
This reverts commit cc8229603b.

As discussed in the review of https://reviews.llvm.org/D109972, this was not
right approach, so we are reverting to start with a different approach.

Differential Revision: https://reviews.llvm.org/D110974
2021-10-01 17:19:50 -07:00
Nico Weber c19315ef60 [lld/mac] Don't warn on both --icf=all and -no_deduplicate
Instead, just make the later flag win, like usual.
Implement this by making -no_deduplicate an actual alias for --icf=none
at the Options.td level.

Differential Revision: https://reviews.llvm.org/D110672
2021-09-29 08:25:21 -04:00
Mike Hommey 08ef24f6ab Wrap xar/xar.h include in extern "C" block
Without such wrapping, linking lld fails with missing symbols because of
C++ symbol mangling with older versions of the MacOSX SDK, in which
xar.h doesn't have an extern "C" block itself.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D110224
2021-09-23 09:37:30 +02:00
Nico Weber 1b2c36aa5f [lld/mac] Fix comment typo to cycle bots 2021-09-18 11:15:21 -04:00
Jez Ng 91ace9f062 [lld-macho] Construct CFString literals by copying the ConcatInputSection
... instead of constructing a new one each time. This allows us
to take advantage of {D105305}.

I didn't see a substantial difference when linking chromium_framework,
but this paves the way for reusing similar logic for splitting compact
unwind entries into sections. There are a lot more of those, so the
performance impact is significant.

Differential Revision: https://reviews.llvm.org/D109895
2021-09-17 19:46:20 -04:00
Vy Nguyen b428c3e8c1 [lld-macho] Ignore local personality symbols if non-local with the same name exisst, to avoid "too many personalities" error.
Sometimes people intentionally re-define a dylib personlity symbol as a local defined symbol as a workaround to a ld -r bug.
    As a result, we could see "too many personalities" to encode. This patch tries to handle this case by ignoring the local symbols entirely.

    Differential Revision: https://reviews.llvm.org/D107533
2021-09-17 12:59:42 -04:00
Nuri Amari cc8229603b Extract LC_CODE_SIGNATURE related implementation out of LLD
Move the functionality in lld that handles writing of the LC_CODE_SIGNATURE load command and associated data section to a central reusable location.

This change is in preparation for another change that modifies llvm-objcopy to reproduce the LC_CODE_SIGNATURE load command and corresponding
data section to maintain the validity of signed macho object files passed through llvm-objcopy.

Reviewed By: #lld-macho, int3, oontvoo

Differential Revision: https://reviews.llvm.org/D109803
2021-09-16 17:43:39 -07:00
Nico Weber ed2f0ad307 [lld/mac] Search .tbd before binary for framework files too
This matters for example for the iPhoneSimulator14.0.sdk, which has
a System/Library/Frameworks/UIKit.framework/UIKit that has
LC_BUILD_VERSION with minos of 14.0, so linking against that file
will produce warnings like:

   .../iPhoneSimulator14.0.sdk/System/Library/Frameworks/UIKit.framework/UIKit
   has version 14.0.0, which is newer than target minimum of 12.0.0

when targeting x86_64-apple-ios12.0-simulator. That doens't happen when
linking against UIKit.tbd instead, obviously.

Linking with RC_TRACE_DYLIB_SEARCHING=1 shows that ld64 also searches
the tbd file first, and we already get that right for non-framework
dylibs.

Fixes crbug.com/1249456.

Differential Revision: https://reviews.llvm.org/D109768
2021-09-14 15:26:45 -04:00
Jez Ng d9ab62ca3d [lld-macho] Initialize LTO backend with diagnostic handler
Failing to do so results in `std::bad_function_call` being
thrown when a pass tries to emit a diagnostic.

I've copied the relevant test over from LLD-ELF's test suite.

Reviewed By: #lld-macho, thevinster

Differential Revision: https://reviews.llvm.org/D109274
2021-09-04 17:40:07 -04:00
Nico Weber c15b588852 [lld/mac] Don't assert during thunk insertion if there are undefined symbols
We end up calling resolveBranchVA(), which asserts for Undefineds.

As fix, just return early in Writer::run() if there are any diagnostics
after processing relocations (which is where undefined symbol errors are
emitted). This matches what the ELF port does.

Differential Revision: https://reviews.llvm.org/D109079
2021-09-03 12:22:41 -04:00
Nico Weber 86c8f395ae [lld/mac] Leave more room for thunks in thunk placement code
Fixes PR51578 in practice.

Currently there's only enough room for a single thunk, which for real-life code
isn't enough. The error case only happens when there are many branch statements
very close to each other (0 or 1 instructions apart), with the function at the
finalization barrier small.

There's a FIXME on what to do if we hit this case, but that suggestion sounds
complicated to me (see end of PR51578 comment 5 for why).

Instead, just leave more room for thunks. Chromium's unit_tests links fine with
room for 3 thunks. Leave room for 100, which should fix this for most cases in
practice.

There's little cost for leaving lots of room: This slop value only determines
when we finalize sections, and we insert thunks for forward jumps into
unfinalized sections. So leaving room means we'll need a few more thunks, but
the thunk jump range is 128 MiB while a single thunk is just 12 bytes.

For Chromium's unit_tests:
With a slop of   3: thunk calls = 355418, thunks = 10903
With a slop of 100: thunk calls = 355426, thunks = 10904

Chances are 100 is enough for all use cases we'll hit in practice, but even
bumping it to 1000 would probably be fine.

Differential Revision: https://reviews.llvm.org/D108930
2021-08-30 22:09:05 -04:00
Nico Weber 83df94067d [lld/mac] Tweak estimateStubsInRangeVA a bit
- Move a few variables closer to their uses, remove some completely
  (no behavior change)
- Add some comments
- Make maxPotentialThunks include calls to stubs. It's possible that
  an earlier call to a stub late in the stub table will need a thunk,
  and that inserted thunk could push a stub earlier in the stub table
  out of range. This is unlikely to happen, but usually there are
  way fewer stub calls than non-stub calls, so if we're doing a
  conservative approximation here we might as well do it correctly.
  (For chromium's unit_tests target, 134421/242639 stub calls are
  direct calls without this change, compared to 134408/242639 with
  this change)

No real, meaningful behavior difference.

Differential Revision: https://reviews.llvm.org/D108924
2021-08-30 13:56:45 -04:00
Nico Weber 9721197520 [lld/mac] Set branchRange a bit more carefully
- Don't subtract thunkSize from branchRange. Most places care about
  the actual maximal branch range. Subtract thunkSize in the one place
  that wants to leave room for a thunk.
- Set it to 0x800_0000 instead of 0xFF_FFFF
- Subtract 4 for the positive branch direction since it's a
  two's complement 24bit number sign-extended mutiplied by 4,
  so its range is -0x800_0000..+0x7FF_FFFC
- Make boundary checks include the boundary values

This doesn't make a huge difference in practice. It's preparation
for a "real" fix for PR51578 -- but it also lets the repro in comment 0
in that bug place one more thunk before hitting the TODO.

Differential Revision: https://reviews.llvm.org/D108897
2021-08-30 12:36:06 -04:00
Nico Weber 28be02f334 [lld/mac] Don't assert on -dead_strip + arm64 range extension thunks
The assert is harmless and thinks worked fine in builds with asserts enabled,
but it's still nice to fix the assert.

Differential Revision: https://reviews.llvm.org/D108853
2021-08-27 23:27:45 -04:00
Jez Ng c74eb05f21 [lld-macho][nfc] Clean up InputSection constructors 2021-08-26 19:07:48 -04:00
Jez Ng 9b5148d426 [lld-macho] Have -ObjC load archive members before symbol resolution
This is what ld64 does. Deviating in behavior here can result
in some subtle duplicate symbol errors, as detailed in the objc.s test.

Differential Revision: https://reviews.llvm.org/D108781
2021-08-26 18:52:07 -04:00
Jez Ng 9065fe5591 [lld-macho] Refactor archive loading
The previous logic was duplicated between symbol-initiated
archive loads versus flag-initiated loads (i.e. `-force_load` and
`-ObjC`). This resulted in code duplication as well as redundant work --
we would create Archive instances twice whenever we had one of those
flags; once in `getArchiveMembers` and again when we constructed the
ArchiveFile.

This was motivated by an upcoming diff where we load archive members
containing ObjC-related symbols before loading those containing
ObjC-related sections, as well as before performing symbol resolution.
Without this refactor, it would be difficult to do that while avoiding
loading the same archive member twice.

Differential Revision: https://reviews.llvm.org/D108780
2021-08-26 18:52:07 -04:00
Jez Ng 2179930868 [lld-macho] Fix unwind info personality size
This was missed by {D107035}. This fix addresses the following warning:

  loop variable 'personality' has type 'const uint32_t &' (aka 'const unsigned int &') but is initialized with type 'const unsigned long long' resulting in a copy [-Wrange-loop-analysis]

In addition to fixing the size, I also removed the const reference,
since there's no performance benefit to avoiding copies of integer-sized
values.
2021-08-26 18:52:06 -04:00
Vincent Lee 08d55c5c01 [lld-macho] Refactor parseSections to avoid creating isec on LLVM segments
Address post follow up comment in D108016. Avoid creating isec for
LLVM segments since we are skipping over it.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D108167
2021-08-16 18:47:50 -07:00
Vincent Lee 15dc93e61c [lld-macho] Ignore LLVM segments to prevent duplicate syms
There was an instance of a third-party archive containing multiple
_llvm symbols from different files that clashed with each other
producing duplicate symbols. Symbols under the LLVM segment
don't seem to be producing any meaningful value, so just ignore them.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D108016
2021-08-16 12:41:03 -07:00
Fangrui Song 7a6482216f [CMake][gn] lldMachO=>lldMachOOld, lldMachO2=>lldMachO
Now that D95204 switched default to new Darwin backend, rename some CMake
targets to match.

Reviewed By: #lld-macho, smeenai, int3

Differential Revision: https://reviews.llvm.org/D107516
2021-08-04 18:52:41 -07:00
Vy Nguyen 0bd14711ac [lld-macho] Change personalities entry type to Ptr to avoid overflowing uint32
PR51262

Differential Revision: https://reviews.llvm.org/D107035
2021-07-29 14:26:07 -04:00
Jez Ng e49374f9e0 [lld-macho] Support common symbols in bitcode (but differently from ld64)
ld64 seems to handle common symbols in bitcode rather
bizarrely. They follow entirely different precedence rules from their
non-bitcode counterparts. I initially tried to emulate ld64 in D106597,
but I'm not sure the extra complexity is worth it, especially given that
common symbols are not, well, very common.

This diff accords common bitcode symbols the same precedence as regular
common symbols, just as we treat all other pairs of bitcode and
non-bitcode symbol types. The tests document ld64's behavior in detail,
just in case we want to revisit this.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D107027
2021-07-29 11:07:50 -04:00
Jez Ng dc9ee39251 [lld-macho] Downgrade "cannot export hidden symbol" to warning
This matches ld64's behavior, and makes it easier to fit LLD
into existing build systems.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D107011
2021-07-28 18:46:26 -04:00
Nico Weber dd57915b1e [lld/mac] Fix sub-library.s on Windows after 8e8701abca
The endswith() check for the framework name fails when joining
with the native path separator. Always use the posix separator as fix.
2021-07-27 15:25:52 -04:00
Nico Weber 8e8701abca [lld/mac] When loading reexports, look for basename in -F / -L first
Matches ld64 (cf Options::findIndirectDylib()), and fixes PR51218.

Differential Revision: https://reviews.llvm.org/D106842
2021-07-27 14:28:52 -04:00
Nico Weber 80caa1eb4a [lld/mac] Add support for segment$start$ and segment$end$ symbols
These symbols are somewhat interesting in that they create non-existing
segments, which as far as I know is the only way to create segments
that don't contain any sections.

Final part of part of PR50760. Like D106629, but for segments instead
of sections. I'm not aware of anything that needs this in practice.

Differential Revision: https://reviews.llvm.org/D106767
2021-07-25 18:25:13 -04:00
Nico Weber afdeb432f0 [lld/mac] Move output segment rename logic into OutputSegment
Fixes the output segment name if both -rename_section and
-rename_segment are used and the post-section-rename segment
name is the same as the pre-segment-rename segment name to
match ld64's behavior.

The motivation is that segment$start$ can create section-less segments,
and this makes a corner case in the interaction between segment$start and
-rename_segment in the upcoming segment$start patch.

Differential Revision: https://reviews.llvm.org/D106766
2021-07-25 18:20:09 -04:00
Nico Weber 04f5eb407c [lld/mac] Fix start-stop.s test with expensive checks enabled
See e.g. https://lab.llvm.org/buildbot/#/builders/16/builds/14317
Not 100% sure why this fails yet, but this fixes it. Let's get
the bots green again first :)

Differential Revision: https://reviews.llvm.org/D106711
2021-07-23 17:01:16 -04:00
Nico Weber 04e8d0b62d [lld/mac] Implement support for section$start and section$ end symbols
With this, libclang_rt.profile_osx.a can be linked, that is coverage
and PGO-instrumented builds should now work with lld.

section$start and section$end symbols can create non-existing sections.
They're also undefined symbols that are only magic if there isn't a
regular symbol with their name, which means the need to be handled
in treatUndefined() instead of just looping over all existing
sections and adding start and end symbols like the ELF port does.

To represent the actual symbols, this uses absolute symbols that
get their value updated once an output section is layed out.

segment$start and segment$end are still missing for now, but they produce a
nicer error message after this patch.

Main part of PR50760.

Differential Revision: https://reviews.llvm.org/D106629
2021-07-23 16:01:09 -04:00
Jez Ng 3313b84481 [lld-macho] ICF: Do more work in equalsConstant, less in equalsVariable
In particular, relocations to absolute symbols or literal sections can
be handled in equalsConstant(), since their output addresses will not
change across each iteration of ICF. Offsets and addends can also be
dealt with entirely in equalsConstant(), making the code somewhat easier
to reason about. Only ConcatInputSections need to be handled in
equalsVariable().

LLD-ELF's implementation takes a similar approach.

Although this should make ICF do less work, in practice it seems like
there is no stat sig difference in time taken when linking
chromium_framework.

This refactor is motivated by an upcoming diff which improves ICF's handling of
addends.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D106212
2021-07-23 11:49:00 -04:00
Nico Weber 9482aa98e5 [lld/mac] Let OutputSegment store its start address
segment$start$/segment$end$ symbols allow creating segments without
sections, so getting the segment address off the first section
won't work there. Storing the address on the segment is arguably a
bit simpler too.

No behavior change, part of PR50760.

Differential Revision: https://reviews.llvm.org/D106665
2021-07-23 11:43:25 -04:00
Nico Weber 2c508cf583 [lld/mac] Don't crash on absolute symbols in order files
Absolute symbols have a nullptr isec. buildInputSectionPriorities()
would defer isec, causing crashes. Ordering absolute symbols doesn't
make sense, so just ignore them. This seems to match ld64.

Differential Revision: https://reviews.llvm.org/D106628
2021-07-23 11:33:23 -04:00
Leonard Grey 5acc6d4572 [lld-macho] Disambiguate bitcode files with the same name by archive name/offset in archive
Ported from COFF/ELF; test is adapted from
test/COFF/thinlto-archivecollision.ll

LTO expects every bitcode file to have a unique name. If given multiple bitcode
files with the same name, it errors with "Expected at most one ThinLTO module
per bitcode file".

This change incorporates the archive name, to disambiguate members with the
same name in different archives and the offset in archive to disambiguate
members with the same name in the same archive.

Differential Revision: https://reviews.llvm.org/D106179
2021-07-22 22:50:25 -04:00
Nico Weber 393116faad [lld/mac] Remove "else" after return
No behavior change
2021-07-22 21:31:52 -04:00
Nico Weber 2d6fb62ef2 [lld/mac] Handle symbols from -U in treatUndefinedSymbol()
In ld64, `-U section$start$FOO$bar` handles `section$start$FOO$bar`
as a regular `section$start` symbol, that is section$start processing
happens before -U processing.

Likely, nobody uses that in practice so it doesn't seem very important
to be compatible with this, but it also moves the -U handling code next
to the `-undefined dynamic_lookup` handling code, which is nice because
they do the same thing. And, in fact, this did identify a bug in a corner
case in the intersection of `-undefined dynamic_lookup` and dead-stripping
(fix for that in D106565).

Vaguely related to PR50760.

No interesting behavior change.

Differential Revision: https://reviews.llvm.org/D106566
2021-07-22 19:43:57 -04:00
Nico Weber 5ae39d4f97 [lld/mac] Fix bug in interaction of -dead_strip and -undefined dynamic_lookup
We lost the `used` bit on the Undefined when we replaced it with a DylibSymbol
in treatUndefined().

Differential Revision: https://reviews.llvm.org/D106565
2021-07-22 19:30:46 -04:00
Nico Weber 9d43c000e1 [lld/mac] Move handling of special undefineds later
treatUndefinedSymbol() was previously called before gatherInputSections()
and markLive() for these special symbols, but after them for normal
undefineds.

For PR50760, treatUndefinedSymbol() will have to potentially create
sections, so it's good to move treatUndefinedSymbol() for special
undefineds later, so that it can assume that gatherInputSections()
and markLive() has already been called always.

No intended behavior change, but part of PR50760 (and covered in
tests in the patch for the full feature).

Differential Revision: https://reviews.llvm.org/D106552
2021-07-22 11:43:49 -04:00
Vincent Lee 33ab995617 Recommit "[lld-macho] Use DO_BIND_ADD_ADDR_IMM_SCALED for bind opcodes"
Implement pass 3 of bind opcodes from ld64 (which supports both 32-bit and 64-bit).
Pass 3 implementation condenses BIND_OPCODE_DO_BIND_ADD_ADDR_ULEB opcode
to BIND_OPCODE_DO_BIND_ADD_ADDR_IMM_SCALED.  This change is already behind an
O2 flag so it shouldn't impact current performance. I verified ld64's output with x86_64 LLD
and they were both emitting the same optimized bind opcodes (although in a slightly different
order). Tested with arm64_32 LLD and compared that with x86 LLD that the order of the bind
opcodes are the same (offset values are different which should be expected).

Reviewed By: int3, #lld-macho, MaskRay

Differential Revision: https://reviews.llvm.org/D106128
2021-07-20 13:45:24 -07:00
Fangrui Song 88e2268a34 Revert D106128 "[lld-macho] Use DO_BIND_ADD_ADDR_IMM_SCALED for bind opcodes"
This reverts commit 321b2bef09.

`for (BindIR *p = &opcodes[0]; p->opcode != BIND_OPCODE_DONE; ++p) {` has a heap-buffer-overflow with test/MachO/bind-opcodes.
2021-07-19 18:13:52 -07:00
Vincent Lee 321b2bef09 [lld-macho] Use DO_BIND_ADD_ADDR_IMM_SCALED for bind opcodes
Implement pass 3 of bind opcodes from ld64 (which supports both 32-bit and 64-bit).
Pass 3 implementation condenses BIND_OPCODE_DO_BIND_ADD_ADDR_ULEB opcode
to BIND_OPCODE_DO_BIND_ADD_ADDR_IMM_SCALED.  This change is already behind an
O2 flag so it shouldn't impact current performance. I verified ld64's output with x86_64 LLD
and they were both emitting the same optimized bind opcodes (although in a slightly different
order). Tested with arm64_32 LLD and compared that with x86 LLD that the order of the bind
opcodes are the same (offset values are different which should be expected).

Reviewed By: int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D106128
2021-07-19 16:18:33 -07:00
Nico Weber fbb45947b2 [lld/mac] Resolve defined symbols before undefined symbols
Ports https://reviews.llvm.org/D95985 to the MachO port.
Happens to fix PR51135; see that bug for details.
Also makes lld's behavior match ld64 for the included test case.

Differential Revision: https://reviews.llvm.org/D106293
2021-07-19 16:37:41 -04:00
Nico Weber bcbb3066ce [lld/mac] Change load command order to be more like ld64
No meaningful behavior change. Makes diffing `otool -l` output a bit easier.

Differential Revision: https://reviews.llvm.org/D106219
2021-07-19 15:04:32 -04:00
Jez Ng 428a7c1b38 [lld-macho] Have ICF operate on all sections at once
ICF previously operated only within a given OutputSection. We would
merge all CFStrings first, then merge all regular code sections in a
second phase. This worked fine since CFStrings would never reference
regular `__text` sections. However, I would like to expand ICF to merge
functions that reference unwind info. Unwind info references the LSDA
section, which can in turn reference the `__text` section, so we cannot
perform ICF in phases.

In order to have ICF operate on InputSections spanning multiple
OutputSections, we need a way to distinguish InputSections that are
destined for different OutputSections, so that we don't fold across
section boundaries. We achieve this by creating OutputSections early,
and setting `InputSection::parent` to point to them. This is what
LLD-ELF does. (This change should also make it easier to implement the
`section$start$` symbols.)

This diff also folds InputSections w/o checking their flags, which I
think is the right behavior -- if they are destined for the same
OutputSection, they will have the same flags in the output (even if
their input flags differ). I.e. the `parent` pointer check subsumes the
`flags` check. In practice this has nearly no effect (ICF did not become
any more effective on chromium_framework).

I've also updated ICF.cpp's block comment to better reflect its current
status.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D105641
2021-07-17 13:42:51 -04:00
Vincent Lee d695d0d6f6 [lld-macho] Optimize bind opcodes with multiple passes
In D105866, we used an intermediate container to store a list of opcodes. Here,
we use that data structure to help us perform optimization passes that would allow
a more efficient encoding of bind opcodes. Currently, the functionality mirrors the
optimization pass {1,2} done in ld64 for bind opcodes under optimization gate
to prevent slight regressions.

Reviewed By: int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D105867
2021-07-15 20:52:46 -07:00
Vincent Lee f2b1264141 [lld-macho] Use intermediate arrays to store opcodes
We want to incorporate some of the optimization passes in bind opcodes from ld64.
This revision makes no functional changes but to start storing opcodes in intermediate
containers in preparation for implementing the optimization passes in a follow-up revision.

Differential Revision: https://reviews.llvm.org/D105866
2021-07-15 16:57:45 -07:00
Leonard Grey c931ff72bd [lld-macho] Add LTO cache support
This adds support for the lld-only `--thinlto-cache-policy` option, as well as
implementations for ld64's `-cache_path_lto`, `-prune_interval_lto`,
`-prune_after_lto`, and `-max_relative_cache_size_lto`.

Test is adapted from lld/test/ELF/lto/cache.ll

Differential Revision: https://reviews.llvm.org/D105922
2021-07-15 12:56:13 -04:00
Alexander Shaposhnikov d21772fa21 [lld][MachO] Code cleanup
Make use of ArgList::getLastArgValue. NFC.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D105452
2021-07-14 04:33:09 -07:00
Nico Weber f21801dab2 [lld/mac] Implement -application_extension
Differential Revision: https://reviews.llvm.org/D105818
2021-07-12 13:42:16 -04:00
Jez Ng 11a0d23650 [lld-macho][nfc] clang-format 2021-07-11 18:36:59 -04:00
Jez Ng 28a2102ee3 [lld-macho][nfc] Remove unnecessary llvm:: namespace prefixes 2021-07-11 18:36:53 -04:00
Jez Ng f6e84a84f9 [lld-macho][nfc] Avoid using std::map for PlatformKinds
The mappings we were using had a small number of keys, so a vector is
probably better. This allows us to remove the last usage of std::map in
our codebase.

I also used `removeSimulator` to simplify the code a bit further.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105786
2021-07-11 18:24:53 -04:00
Nico Weber 6e05c1cd5f [lld/mac] Always reference dyld_stub_binder when linked with libSystem
lld currently only references dyld_stub_binder when it's needed.
ld64 always references it when libSystem is linked.
Match ld64.

The (somewhat lame) motivation is that `nm` on a binary without any
export writes a "no symbols" warning to stderr, and this change makes
it so that every binary in practice has at least a reference to
dyld_stub_binder, which suppresses that.

Every "real" output file will reference dyld_stub_binder, so most
of the time this shouldn't make much of a difference. And if you
really don't want to have this reference for whatever reason, you
can stop passing -lSystem, like you have to for ld64 anyways.

(After linking any dylib, we dump the exported list of symbols to
a txt file with `nm` and only relink downstream deps if that txt
file changes. A nicer fix is to make lld optionally write .tbd files
with the public interface of a linked dylib and use that instead,
but for now the txt files are what we do.)

Differential Revision: https://reviews.llvm.org/D105782
2021-07-11 13:37:48 -04:00
Nico Weber 10e28a7484 [lld/mac] Use normal Undefined machinery for dyld_stub_binder lookup
This is for aesthetic reasons, I'm not aware of anything that needs
this in practice. It does have a few effects:

- `-undefined dynamic_lookup` now has an effect for dyld_stub_binder.
  This matches ld64.

- `-U dyld_stub_binder` now works like you'd expect (it doesn't work in ld64).

- The error message for a missing dyld_stub_binder symbol now looks like
  other undefined reference symbols, it changes from

      symbol dyld_stub_binder not found (normally in libSystem.dylib). Needed to perform lazy binding.

  to

      error: undefined symbol: dyld_stub_binder
      >>> referenced by lazy binding (normally in libSystem.dylib)

Also add test coverage for that error message.

But in practice, this should have no interesting effects since everything links
in dyld_stub_binder via libSystem anyways.

Differential Revision: https://reviews.llvm.org/D105781
2021-07-11 12:48:59 -04:00
Vy Nguyen 3822e3d5b0 [lld-macho] Fix bug in handling unwind info from ld -r
Two changess:
- Drop assertions that all symbols are in GOT
- Set allEntriesAreOmitted correctly

Related bug: 50812

Differential Revision: https://reviews.llvm.org/D105364
2021-07-09 22:44:51 -04:00
Mikael Holmen 21fd875952 [lld/mac] Fix warning about unused variable [NFC]
Change "dyn_cast" to "isa" to get rid of the unused
variable "bitcodeFile".

gcc warned with

lld/MachO/Driver.cpp:531:17: warning: unused variable 'bitcodeFile' [-Wunused-variable]
531 |       if (auto *bitcodeFile = dyn_cast<BitcodeFile>(file)) {
    |                 ^~~~~~~~~~~
2021-07-08 09:46:30 +02:00
Nico Weber 8a7b5ebf4d [lld/mac] Don't crash when dead-stripping removes all unwind info
If the input has compact unwind info but all of it is removed
after dead stripping, we would crash. Now we don't write any
__unwind_info section at all, like ld64.

This is a bit awkward to implement because we only know the final
state of unwind info after UnwindInfoSectionImpl<Ptr>::finalize(),
which is called after sections are added. So add a small amount of
bookkeeping to relocateCompactUnwind() instead (which runs earlier)
so that we can predict what finalize() will do before it runs.

Fixes PR51010.

Differential Revision: https://reviews.llvm.org/D105557
2021-07-07 13:05:40 -04:00
Nico Weber d7e65757ed [lld/mac] Tweak reserve() argument in unwind code
addEntriesForFunctionsWithoutUnwindInfo() can add entries to cuVector, so
cuCount can be stale. Use cuVector.size() instead.

No behavior change.
2021-07-07 11:44:22 -04:00
Nico Weber 3eb2fc4b50 [lld/mac] Partially implement -export_dynamic
This implements the part of -export_dynamic that adds external
symbols as dead strip roots even for executables.

It does not yet implement the effect -export_dynamic has for LTO.
I tried just replacing `config->outputType != MH_EXECUTE` with
`(config->outputType != MH_EXECUTE || config->exportDynamic)` in
LTO.cpp, but then local symbols make it into the symbol table too,
which is too much (and also doesn't match ld64). So punt on this
for now until I understand it better.
(D91583 may or may not be related too).

Differential Revision: https://reviews.llvm.org/D105482
2021-07-06 11:22:18 -04:00
Nico Weber 64be5b7d87 [lld/mac] Implement -arch_multiple
This is the other flag clang passes when calling clang with two -arch
flags (which means with this, `clang -arch x86_64 -arch arm64 -fuse-ld=lld ...`
now no longer prints any warnings \o/). Since clang calls the linker several
times in that setup, it's not clear to the user from which invocation the
errors are. The flag's help text is

    Specifies that the linker should augment error and warning messages
    with the architecture name.

In ld64, the only effect of the flag is that undefined symbols are prefaced
with

    Undefined symbols for architecture x86_64:

instead of the usual "Undefined symbols:". So for now, let's add this
only to undefined symbol errors too. That's probably the most common
linker diagnostic.

Another idea would be to prefix errors and warnings with "ld64.lld(x86_64):"
instead of the usual "ld64.lld:", but I'm not sure if people would
misunderstand that as a comment about the arch of ld itself.
But open to suggestions on what effect this flag should have :) And we
don't have to get it perfect now, we can iterate on it.

Differential Revision: https://reviews.llvm.org/D105450
2021-07-06 00:25:18 -04:00
Nico Weber 2c25f39fcc [lld/mac] Implement -final_output
This is one of two flags clang passes to the linker when giving calling
clang with multiple -arch flags.

I think it'd make sense to also use finalOutput instead of outputFile
in CodeSignatureSection() and when replacing @executable_path, but
ld64 doesn't do that, so I'll at least put those in separate commits.

Differential Revision: https://reviews.llvm.org/D105449
2021-07-05 20:06:26 -04:00
Nico Weber db64306d99 [lld/mac] Implement -umbrella
I think this is an old way for doing what is done with
-reexport_library these days, but it's e.g. still used in libunwind's
build (the opensource.apple.com one, not the llvm one).

Differential Revision: https://reviews.llvm.org/D105448
2021-07-05 20:06:25 -04:00
Jez Ng 718c32175b [lld-macho] Only emit one BIND_OPCODE_SET_SYMBOL per symbol
Size-wise, BIND_OPCODE_SET_SYMBOL_TRAILING_FLAGS_IMM is the most
expensive opcode, since it comes with an associated symbol string. We
were previously emitting it once per binding, instead of once per
symbol. This diff groups all bindings for a given symbol together and
ensures we only emit one such opcode per symbol. This matches ld64's
behavior.

While this is a relatively small win on chromium_framework (-72KiB), for
programs that have more dynamic bindings, the difference can be quite
large.

This change is perf-neutral when linking chromium_framework.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105075
2021-07-05 20:00:19 -04:00
Jez Ng bcaf57cae8 [lld-macho] Parse relocations quickly by assuming sorted order
clang and gcc both seem to emit relocations in reverse order of
address. That means we can match relocations to their containing
subsections in `O(relocs + subsections)` rather than the `O(relocs *
log(subsections))` that our previous binary search implementation
required.

Unfortunately, `ld -r` can still emit unsorted relocations, so we have a
fallback code path for that (less common) case.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.04          4.11         4.075        4.0775   0.018027756
  +  20          3.95          4.02          3.98         3.985   0.020900768
  Difference at 95.0% confidence
          -0.0925 +/- 0.0124919
          -2.26855% +/- 0.306361%
          (Student's t, pooled s = 0.0195172)

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105410
2021-07-05 01:13:44 -04:00
Nico Weber 9e24979d73 [lld/mac] Fix function offset on 1st-level unwind table sentinel
Two bugs:
1. This tries to take the address of the last symbol plus the length
   of the last symbol. However, the sorted vector is cuPtrVector,
   not cuVector. Also, cuPtrVector has tombstone values removed
   and cuVector doesn't. If there was a stripped value at the end,
   the "last" element's value was UINT64_MAX, which meant the
   sentinel value was one less than the length of that "last"
   dead symbol.

2. We have to subtract in.header->addr. For 64-bit binaries that's
   (1 << 32) and functionAddress is 32-bit so this is a no-op, but
   for 32-bit binaries the sentinel's value was too large.

I believe this has no effect in practice since the first-level
binary search code in libunwind (in UnwindCursor.hpp) does:

    uint32_t low = 0;
    uint32_t high = sectionHeader.indexCount();
    uint32_t last = high - 1;
    while (low < high) {
      uint32_t mid = (low + high) / 2;
        if ((mid == last) ||
            (topIndex.functionOffset(mid + 1) > targetFunctionOffset)) {
          low = mid;
          break;
        } else {
        low = mid + 1;
      }

So the address of the last entry in the first-level table isn't really
checked -- except for the very end, but the check against `last` means
we just run the loop once more than necessary. But it makes `unwinddump` output
look less confusing, and it's what it looks was the intention here.

(No test since I can't think of a way to make FileCheck check that one
number is larger than another.)

Differential Revision: https://reviews.llvm.org/D105404
2021-07-04 18:06:20 -04:00
Nico Weber d2d6da3011 [lld/mac] Don't crash on 32-bit output binaries when dead-stripping
Fixes PR50974.

Differential Revision: https://reviews.llvm.org/D105399
2021-07-04 18:03:31 -04:00
Vy Nguyen c7c5a1c9ae [lld-macho] Ignore debug symbols while preparing relocations.
Details: see https://bugs.llvm.org/show_bug.cgi?id=50812

Differential Revision: https://reviews.llvm.org/D105210
2021-07-02 13:51:46 -04:00
Jez Ng f6b6e72143 [lld-macho] Factor out common InputSection members
We have been creating many ConcatInputSections with identical values due
to .subsections_via_symbols. This diff factors out the identical values
into a Shared struct, to reduce memory consumption and make copying
cheaper.

I also changed `callSiteCount` from a uint32_t to a 31-bit field to save an
extra word.

All in all, this takes InputSection from 120 to 72 bytes (and
ConcatInputSection from 160 to 112 bytes), i.e. 30% size reduction in
ConcatInputSection.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.14          4.24          4.18         4.183   0.027548999
  +  20          4.04          4.11         4.075        4.0775   0.018027756
  Difference at 95.0% confidence
          -0.1055 +/- 0.0149005
          -2.52211% +/- 0.356215%
          (Student's t, pooled s = 0.0232803)

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105305
2021-07-01 21:22:39 -04:00
Jez Ng 08715e6c47 [lld-macho][nfc] Remove unnecessary vertical spacing
This makes NonLazyPointerSectionBase's style more in line with the rest
of the classes in its file.
2021-07-01 21:22:38 -04:00
Jez Ng ac2dd06b91 [lld-macho] Deduplicate CFStrings
`__cfstring` is a special literal section, so instead of breaking it up
at symbol boundaries, we break it up at fixed-width boundaries (since
each literal is the same size). Symbols can only occur at one of those
boundaries, so this is strictly more powerful than
`.subsections_via_symbols`.

With that in place, we then run the section through ICF.

This change is about perf-neutral when linking chromium_framework.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D105045
2021-07-01 21:22:38 -04:00
Jez Ng 3a11528d97 [lld-macho] Move ICF earlier to avoid emitting redundant binds
This is a pretty big refactoring diff, so here are the motivations:

Previously, ICF ran after scanRelocations(), where we emitting
bind/rebase opcodes etc. So we had a bunch of redundant leftovers after
ICF. Having ICF run before Writer seems like a better design, and is
what LLD-ELF does, so this diff refactors it accordingly.

However, ICF had two dependencies on things occurring in Writer: 1) it
needs literals to be deduplicated beforehand and 2) it needs to know
which functions have unwind info, which was being handled by
`UnwindInfoSection::prepareRelocations()`.

In order to do literal deduplication earlier, we need to add literal
input sections to their corresponding output sections. So instead of
putting all input sections into the big `inputSections` vector, and then
filtering them by type later on, I've changed things so that literal
sections get added directly to their output sections during the 'gather'
phase. Likewise for compact unwind sections -- they get added directly
to the UnwindInfoSection now. This latter change is not strictly
necessary, but makes it easier for ICF to determine which functions have
unwind info.

Adding literal sections directly to their output sections means that we
can no longer determine `inputOrder` from iterating over
`inputSections`. Instead, we store that order explicitly on
InputSection. Bloating the size of InputSection for this purpose would
be unfortunate -- but LLD-ELF has already solved this problem: it reuses
`outSecOff` to store this order value.

One downside of this refactor is that we now make an additional pass
over the unwind info relocations to figure out which functions have
unwind info, since want to know that before `processRelocations()`. I've
made sure to run that extra loop only if ICF is enabled, so there should
be no overhead in non-optimizing runs of the linker.

The upside of all this is that the `inputSections` vector now contains
only ConcatInputSections that are destined for ConcatOutputSections, so
we can clean up a bunch of code that just existed to filter out other
elements from that vector.

I will test for the lack of redundant binds/rebases in the upcoming
cfstring deduplication diff. While binds/rebases can also happen in the
regular `.text` section, they're more common in `.data` sections, so it
seems more natural to test it that way.

This change is perf-neutral when linking chromium_framework.

Reviewed By: oontvoo

Differential Revision: https://reviews.llvm.org/D105044
2021-07-01 21:22:38 -04:00
Leonard Grey fe08e9c487 [lld-macho] Add support for LTO optimization level
Everything (including test) modified from ELF/COFF. Using the same syntax
(--lto-O3, etc) as ELF.

Differential Revision: https://reviews.llvm.org/D105223
2021-07-01 15:01:59 -04:00
Jez Ng b41b4148e7 [lld-macho] Only enable `__DATA_CONST` for newer platforms
Matches ld64.

Reviewed By: #lld-macho, alexander-shaposhnikov

Differential Revision: https://reviews.llvm.org/D105080
2021-06-30 18:55:48 -04:00
Jez Ng 0d6d35e63b [lld-macho] -section_rename should work on synthetic sections too
Previously, we only applied the renames to
ConcatOutputSections.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D105079
2021-06-30 18:55:48 -04:00
Nico Weber aed0a08c69 [lld/mac] Make symbol table order deterministic
SymtabSection::emitStabs() writes the symbol table in the order
of externalSymbols, which has the order of symtab->getSymbols(),
which is just the order symbols are added to the symbol table.

In practice, symbols in the symbol files of input .o files are
sorted, but since that's not guaranteed we sort them in
ObjFile::parseSymbols(). To make sure several symbols with the same
address keep the order they're in the input file, we have to use
stable_sort().

In practice, std::sort() on already-sorted inputs won't change the order
of just adjacent elements, and while in theory std::sort() could use a
random pivot, in practice the code should be deterministic as it was
previously too.

But now lld/test/MachO/stabs.s passes with LLVM_ENABLE_EXPENSIVE_CHECKS=ON
(the last test that was failing with that set).

Fixes a regression from D99972.

While here, remove an empty section in stabs.s and move
.subsections_via_symbols to the end where it usually is (this part no
behavior change).

Differential Revision: https://reviews.llvm.org/D105071
2021-06-29 09:29:49 -04:00
Leonard Grey a8a6e5b094 [lld-macho] Preserve alignment for non-deduplicated cstrings
Fixes PR50637.

Downstream bug: https://crbug.com/1218958

Currently, we split __cstring along symbol boundaries with .subsections_via_symbols
when not deduplicating, and along null bytes when deduplicating. This change splits
along null bytes unconditionally, and preserves original alignment in the non-
deduplicated case.

Removing subsections-section-relocs.s because with this change, __cstring
is never reordered based on the order file.

Differential Revision: https://reviews.llvm.org/D104919
2021-06-28 22:26:43 -04:00
Nico Weber f1969b74a7 [lld/mac] Fix nondeterminism in output section ordering
The two different thread_local_regular sections (__thread_data and
more_thread_data) had nondeterminstic ordering for two reasons:

1. https://reviews.llvm.org/D102972 changed concatOutputSections
   from MapVector to DenseMap, so when we iterate it to make
   output segments, we would add the two sections to the __DATA
   output segment in nondeterministic order.

2. The same change also moved the two stable_sort()s for segments
   and sections to sort(). Since sections with assigned priority
   (such as TLV data) have the same priority for all sections,
   this is incorrect -- we must use stable_sort() so that the
   initial (input-order-based) order remains.

As a side effect, we now (deterministically) put the __common
section in front of __bss (while previously we happened to
put it after it). (__common and __bss are both zerofill so
both have order INT_MAX, but common symbols are added to
inputSections before normal sections are collected.)

Makes lld/test/MachO/tlv.s and lld/test/MachO/tlv-dylib.s pass with
LLVM_ENABLE_EXPENSIVE_CHECKS=ON.

Differential Revision: https://reviews.llvm.org/D105054
2021-06-28 18:41:33 -04:00
Jez Ng bf457919f2 [lld-macho][nfc] Remove unnecessary dyn_cast and simplify code 2021-06-28 14:50:44 -04:00
Jez Ng 557e1fa02f [lld-macho] Extend ICF to literal sections
Literal sections can be deduplicated before running ICF. That makes it
easy to compare them during ICF: we can tell if two literals are
constant-equal by comparing their offsets in their OutputSection.

LLD-ELF takes a similar approach.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D104671
2021-06-28 14:49:39 -04:00
Nico Weber 0f24ffcdfa [lld/mac] Don't fold UNWIND_X86_64_MODE_STACK_IND unwind entries
libunwind uses unwind info to find the function address belonging
to the current instruction pointer. libunwind/src/CompactUnwinder.hpp's
step functions read functionStart for UNWIND_X86_64_MODE_STACK_IND
(and for nothing else), so these encodings need a dedicated entry
per function, so that the runtime can get the stacksize off the
`subq` instrunction in the function's prologue.

This matches ld64.

(CompactUnwinder.hpp from https://opensource.apple.com/source/libunwind/
also reads functionStart in a few more cases if `SUPPORT_OLD_BINARIES` is set,
but it defaults to 0, and ld64 seems to not worry about these additional
cases.)

Related upstream bug: https://crbug.com/1220175

Differential Revision: https://reviews.llvm.org/D104978
2021-06-27 06:49:32 -04:00
Jez Ng 8aa17d1eae [lld-macho] Move ICF members from InputSection to ConcatInputSection
`icfEqClass` only makes sense on ConcatInputSections since (in contrast
to literal sections) they are deduplicated as an atomic unit.

Similarly, `hasPersonality` and `replacement` don't make sense on
literal sections.

This mirrors LLD-ELF, which stores `icfEqClass` only on non-mergeable
sections.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D104670
2021-06-24 22:23:12 -04:00
Fangrui Song f1e2d5851b [OptTable] Rename PrintHelp to printHelp
To be consistent with other member functions and match the coding standard.
2021-06-24 14:47:03 -07:00
Jez Ng 4a8503c8e0 [lld-macho] Align all cstrings to 16 bytes when deduplicating
We previously did this only for x86_64, but it turns out that
arm64 needs this too -- see PR50791.

Ultimately this is a hack, and we should avoid over-aligning strings
that don't need it. I'm just having a hard time figuring out how ld64 is
determining the right alignment.

No new test for this since we were already testing this behavior for
x86_64, and extending it to arm64 seems too trivial.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104835
2021-06-24 16:53:29 -04:00
Greg McGary 8a8558ae27 [lld-macho] add tests for ICF, plus cleanups
Add tests for pending TODOs, plus some global cleanups:
* No fold: func has personality/LSDA
* Fold: reference to absolute symbol with different name but identical value
* No fold: reloc references to absolute symbols with different values
* No fold: N_ALT_ENTRY symbols

Differential Revision: https://reviews.llvm.org/D104721
2021-06-23 20:44:25 -07:00
Nico Weber ef75358080 [lld/mac] Delete incorrect FIXME
"""Bitcode symbols only exist before LTO runs, and only serve the purpose of
resolving visibility so LTO can better optimize. Running LTO creates ObjFiles
from BitcodeFiles, and those ObjFiles contain regular Defined symbols (with
isec set and all) that will replace the bitcode symbols. So things should
(hopefully) work as-is :)"""

  -- https://reviews.llvm.org/rGdbbc8d8333f29cf4ad6f4793da1adf71bbfdac69#inline-6081
2021-06-23 16:25:34 -04:00
Nico Weber dbbc8d8333 [lld/mac] Don't crash on absolute symbols in unwind info generation
Fixes a regression from d6565a2dbc and PR50820.
2021-06-23 14:25:34 -04:00
Nico Weber d6565a2dbc [lld/mac] Add explicit "no unwind info" entries for functions without unwind info
Fixes PR50529. With this, lld-linked Chromium base_unittests passes on arm macs.

Surprisingly, no measurable impact on link time.

Differential Revision: https://reviews.llvm.org/D104681
2021-06-22 06:12:42 -04:00
Nico Weber 3a6a60f6c9 [lld/mac] Make a variable more local; no behavior change
The variable used to need the wider scope, but doesn't after the
reland. See LC_LINKER_OPTIONS-related discussion on
https://reviews.llvm.org/D104353 for background.
2021-06-20 21:59:15 -04:00
Nico Weber e6cb55d5ce [lld/mac] Test zerofill sections after __thread_bss
Real zerofill sections go after __thread_bss, since zerofill sections
must all be at the end of their segment and __thread_bss must be right
after __thread_data.

Works fine already, but wasn't tested as far as I can tell.

Also tweak comment about zerofill sections a bit.

No behavior change.

Differential Revision: https://reviews.llvm.org/D104609
2021-06-20 20:44:29 -04:00
Jez Ng f79e7a5a48 [lld-macho] Have inputOrder default to less than INT_MAX
We make it less than INT_MAX in order not to conflict with the ordering
of zerofill sections, which must always be placed at the end of their
segment.

This is the more structural fix for the issue addressed in {D104596}.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104607
2021-06-20 19:49:27 -04:00
Nico Weber c931e12b1d [lld/mac] Make sure __thread_ptrs is in front of __thread_bss
The exact location doesn't matter, but it should be in front
of __thread_bss. We put it right in front of __thread_data
which is where ld64 seems to put it as well.

Fixes PR50769.

(As mentioned on the bug, there is probably a more structural
fix too, see comment 5. If we don't address this, it's likely
we'll run into this again with other synthetic sections. But
for now, let's fix the immediate breakage.)

Differential Revision: https://reviews.llvm.org/D104596
2021-06-19 12:56:43 -04:00
Nico Weber 17271ece0d [lld/mac] Give __DATA,__thread_ptrs type S_THREAD_LOCAL_VARIABLE_POINTERS
...instead of S_NON_LAZY_SYMBOL_POINTERS. This matches ld64.

Part of PR50769.

While here, also remove an old TODO that was done in D87178.

Differential Revision: https://reviews.llvm.org/D104594
2021-06-19 12:56:42 -04:00
Jez Ng 4507f64165 [re-land][lld-macho] Avoid force-loading the same archive twice
This reverts commit c9b241efd6, which was
a backout diff to fix the buildbots.

The real culprit of the crash is
1d31fb8d12,
which is being reverted.

Differential Revision: https://reviews.llvm.org/D104353
2021-06-18 22:43:50 -04:00
Jez Ng a79c018325 Revert "[lld-macho] Have path-related functions return std::string, not StringRef"
This reverts commit 1d31fb8d12.

Making `rerootPath` return a temporary std::string caused a
use-after-free:

https://ci.chromium.org/ui/p/chromium/builders/try/win_upload_clang/1608/overview
2021-06-18 22:43:49 -04:00
Nico Weber c9b241efd6 Revert "[lld-macho] Avoid force-loading the same archive twice"
This reverts commit 24706cd73c.
Test seems to fail flakily. See comments on https://reviews.llvm.org/D104353
for a hypothesis for why.
2021-06-18 20:25:27 -04:00
Jez Ng 1d31fb8d12 [lld-macho] Have path-related functions return std::string, not StringRef
findLibrary() returned a StringRef while findFramework & other helper
functions returned std::strings. Standardize on std::string.

(I initially tried making the helper functions all return StringRefs,
but I realized we shouldn't return input StringRefs since their
lifetimes would not be obvious from the calling code.)
2021-06-18 16:36:14 -04:00
Jez Ng 4c49f9ceaf [lld-macho] Handle non-extern symbols marked as private extern
Previously, we asserted that such a case was invalid, but in fact
`ld -r` can emit such symbols if the input contained a (true) private
extern, or if it contained a symbol started with "L".

Non-extern symbols marked as private extern are essentially equivalent
to regular TU-scoped symbols, so no new functionality is needed.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104502
2021-06-18 16:36:14 -04:00
Nico Weber f7366890c2 [lld/mac] Support -data_in_code_info, -function_starts flags
These are on by default, but there's also an explicit flag for them.

Differential Revision: https://reviews.llvm.org/D104543
2021-06-18 13:01:42 -04:00
Greg McGary 8120c9e379 Rename option -icf MODE to --icf=MODE
The `icf` command-line option is not present in ld64, so it should use the LLD option syntax, which begins with double dashes and separates primary option from any suboption with the equal sign.

Differential Revision: https://reviews.llvm.org/D104548
2021-06-18 09:52:15 -07:00
Muhammad Omair Javaid 9777f3fd06 Fix build failure on 32 bit Arm
This patch fixes build failure caused by commit
f27e4548fc on 32 bit arm.

Differential Revision: https://reviews.llvm.org/D103292
2021-06-18 15:27:09 +00:00
Vy Nguyen 366df11a35 [lld-macho] Rework mergeFlag to behave closer to what ld64 does.
Details:
I've been getting a few weird errors similar to the following from our internal tests:

```
ld64.lld.darwinnew: error: Cannot merge section __eh_frame (type=0x0) into __eh_frame (type=0xB): inconsistent types
ld64.lld.darwinnew: error: Cannot merge section __eh_frame (flags=0x0) into __eh_frame (flags=0x6800000B): strict flags differ
ld64.lld.darwinnew: error: Cannot merge section __eh_frame (type=0x0) into __eh_frame (type=0xB): inconsistent types
ld64.lld.darwinnew: error: Cannot merge section __eh_frame (flags=0x0) into __eh_frame (flags=0x6800000B): strict flags differ
```

Differential Revision: https://reviews.llvm.org/D103971
2021-06-17 14:22:58 -04:00
Greg McGary f27e4548fc [lld-macho] Implement ICF
ICF = Identical C(ode|OMDAT) Folding

This is the LLD ELF/COFF algorithm, adapted for MachO. So far, only `-icf all` is supported. In order to support `-icf safe`, we will need to port address-significance tables (`.addrsig` directives) to MachO, which will come in later diffs.

`check-{llvm,clang,lld}` have 0 regressions for `lld -icf all` vs. baseline ld64.

We only run ICF on `__TEXT,__text` for reasons explained in the block comment in `ConcatOutputSection.cpp`.

Here is the perf impact for linking `chromium_framekwork` on a Mac Pro (16-core Xeon W) for the non-ICF case vs. pre-ICF:
```
    N           Min           Max        Median           Avg        Stddev
x  20          4.27          4.44          4.34         4.349   0.043029977
+  20          4.37          4.46         4.405        4.4115   0.025188761
Difference at 95.0% confidence
        0.0625 +/- 0.0225658
        1.43711% +/- 0.518873%
        (Student's t, pooled s = 0.0352566)
```

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D103292
2021-06-17 10:07:44 -07:00
Jez Ng 24706cd73c [lld-macho] Avoid force-loading the same archive twice
We need to dedup archive loads (similar to what we do for dylib
loads).

I noticed this issue after building some Swift stuff that used
`-force_load_swift_libs`, as it caused some Swift archives to be loaded
many times.

Reviewed By: #lld-macho, thakis, MaskRay

Differential Revision: https://reviews.llvm.org/D104353
2021-06-17 11:13:54 -04:00
Xuanda Yang 01cb9c5fc5 [lld][MachO] Sort symbols in parallel in -map
source: https://bugs.llvm.org/show_bug.cgi?id=50689

When writing a map file, sort symbols in parallel using parallelSort.
Use address name to break ties if two symbols have the same address.

Reviewed By: thakis, int3

Differential Revision: https://reviews.llvm.org/D104346
2021-06-17 10:19:59 +08:00
Jez Ng 560636e549 [lld-macho] Put DATA_IN_CODE immediately after FUNCTION_STARTS
codesign checks for this.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104354
2021-06-16 15:23:07 -04:00
Jez Ng eeac6b2bec [lld-macho] Handle multiple LC_LINKER_OPTIONs
We previously only parsed the first one.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104352
2021-06-16 15:23:06 -04:00
Jez Ng b8bbb9723a [lld-macho][nfc] Put back shouldOmitFromOutput() asserts
I removed them in rG5de7467e982 but @thakis pointed out that
they were useful to keep, so here they are again. I've also converted
the `!isCoalescedWeak()` asserts into `!shouldOmitFromOutput()` asserts,
since the latter check subsumes the former.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104169
2021-06-16 15:23:04 -04:00
Jez Ng d52d1b93c3 [lld-macho] Downgrade version mismatch to warning
It's a warning in ld64. While having LLD be stricter would be nice, it
makes it harder for it to be a drop-in replacement into existing builds.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104333
2021-06-16 11:06:26 -04:00
Nico Weber b579938d40 [lld/mac] Add support for -no_data_in_code_info flag
Differential Revision: https://reviews.llvm.org/D104345
2021-06-16 06:40:42 -04:00
Nico Weber 46ac1b213a [lld/mac] Put lld-only flags in "LLD-SPECIFIC:" --help section
Differential Revision: https://reviews.llvm.org/D104347
2021-06-16 06:39:36 -04:00
Vitaly Buka b01bfdfda6 [lld][MachO] Fix UB after D103006
ubsan detected:
lld/MachO/SyntheticSections.cpp:636:15: runtime error: null pointer
passed as argument 2, which is declared to never be null
2021-06-14 21:15:54 -07:00
Alexander Shaposhnikov 928394d109 [lld][MachO] Add support for LC_DATA_IN_CODE
Add first bits for emitting LC_DATA_IN_CODE.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D103006
2021-06-14 19:21:59 -07:00
Jez Ng cc17bfe489 [lld-macho] Fix "shift exponent too large" UBSAN error
UBSAN seems to have added this check somewhere along the way...

This might also fix the PPC buildbot, which is failing on the same test
2021-06-14 13:47:25 -04:00
Jez Ng e06b9ba485 [lld-macho] Reword comment for clarity 2021-06-14 13:47:25 -04:00
Jez Ng 9c5d43fb55 [lld-macho] Try to fix MSAN "uninitialized memory" error
I *think* this is the fix, with the regression being introduced by
D104199. Not 100% sure since MSAN isn't supported on my Mac machine, and
it'll take some time to spin up a Linux box... will look at the
buildbots for answers
2021-06-13 23:47:09 -04:00
Jez Ng da24e6d43e [lld-macho][nfc] Add `final` to classes where possible
I wanted to see if we would get any perf wins out of this, but
it doesn't seem to be the case. But it still seems worth committing.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D104200
2021-06-13 19:52:03 -04:00
Jez Ng c5c05ffa45 [lld-macho][nfc] Represent the image loader cache with a ConcatInputSection
We don't need to define any special behavior for this section,
so creating a subclass for it is redundant.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104199
2021-06-13 19:51:31 -04:00
Jez Ng b2a0739012 [lld-macho][nfc] Remove InputSection::outSecFileOff
`outSecFileOff` and the associated `getFileOffset()` accessors were
unnecessary.

For all the cases we care about, `outSecFileOff` is the same as
`outSecOff`. The only time they deviate is if there are zerofill
sections within a given segment. But since zerofill sections are always
at the end of a segment, the only sections where the two values deviate
are zerofill sections themselves. And we never actually query the
outSecFileOff of zerofill sections.

As for `getFileOffset()`, the only place it was being used was to
calculate the offset of the entry symbol. However, we can compute that
value by just taking the difference between the address of the entry
symbol and the address of the Mach-O header. In fact, this appears to be
what ld64 itself does. This difference is the same as the file offset as
long as there are no intervening zerofill sections, but since `__text`
is the first section in `__TEXT`, this never happens, so our previous
use of `getFileOffset()` was not wrong -- just inefficient.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D104177
2021-06-13 19:51:30 -04:00
Nico Weber 7d4c8a2b8f [lld/mac] clarify comment
This is a "we should do X in the future" fixme, not an "X might go wrong"
fixme.
2021-06-13 13:30:07 -04:00
Nico Weber 5f9bc580d8 fix comment typos to cycle bots 2021-06-13 10:18:51 -04:00
Alexander Shaposhnikov b9095f5e1a [lld][MachO] Fix function starts section
Sort the addresses stored in FunctionStarts section.
Previously we were encoding potentially large numbers (due to unsigned overflow).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D103662
2021-06-11 17:47:28 -07:00
Jez Ng 5de7467e98 [lld-macho] Fix debug build
D103977 broke a bunch of stuff as I had only tested the release build
which eliminated asserts.

I've retained the asserts where possible, but I also removed a bunch
instead of adding a whole lot of verbose ConcatInputSection casts.
2021-06-11 20:21:27 -04:00
Jez Ng 464d3dc3d1 [lld-macho] Have dead-stripping work with literal sections
Literal sections are not atomically live or dead. Rather,
liveness is tracked for each individual literal they contain. CStrings
have their liveness tracked via a `live` bit in StringPiece, and
fixed-width literals have theirs tracked via a BitVector.

The live-marking code now needs to track the offset within each section
that is to be marked live, in order to identify the literal at that
particular offset.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W
with both `-dead_strip` and `--deduplicate-literals`, with and without this diff
applied:

```
    N           Min           Max        Median           Avg        Stddev
x  20          4.32          4.44         4.375         4.372    0.03105174
+  20           4.3          4.39          4.36        4.3595   0.023277502
No difference proven at 95.0% confidence
```
This gives us size savings of about 0.4%.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103979
2021-06-11 19:50:09 -04:00
Jez Ng 681cfeb591 [lld-macho][nfc] Have InputSection ctors take some parameters
This is motivated by an upcoming diff in which the
WordLiteralInputSection ctor sets itself up based on the value of its
section flags. As such, it needs to be passed the `flags` value as part
of its ctor parameters, instead of having them assigned after the fact
in `parseSection()`. While refactoring code to make that possible, I
figured it would make sense for the other InputSections to also take
their initial values as ctor parameters.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103978
2021-06-11 19:50:09 -04:00
Jez Ng 7f2ba39b16 [lld-macho][nfc] Move liveness-tracking fields into ConcatInputSection
These fields currently live in the parent InputSection class,
but they should be specific to ConcatInputSection, since the other
InputSection classes (that contain literals) aren't atomically live or
dead -- rather their component string/int literals should have
individual liveness states. (An upcoming diff will add liveness bits for
StringPieces and fixed-sized literals.)

I also factored out some asserts for isCoalescedWeak() in MarkLive.cpp.
We now avoid putting coalesced sections in the `inputSections` vector,
so we don't have to check/assert against it everywhere.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103977
2021-06-11 19:50:08 -04:00
Jez Ng 5d88f2dd94 [lld-macho] Deduplicate fixed-width literals
Conceptually, the implementation is pretty straightforward: we put each
literal value into a hashtable, and then write out the keys of that
hashtable at the end.

In contrast with ELF, the Mach-O format does not support variable-length
literals that aren't strings. Its literals are either 4, 8, or 16 bytes
in length. LLD-ELF dedups its literals via sorting + uniq'ing, but since
we don't need to worry about overly-long values, we should be able to do
a faster job by just hashing.

That said, the implementation right now is far from optimal, because we
add to those hashtables serially. To parallelize this, we'll need a
basic concurrent hashtable (only needs to support concurrent writes w/o
interleave reads), which shouldn't be to hard to implement, but I'd like
to punt on it for now.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          4.27          4.39         4.315        4.3225   0.033225703
  +  20          4.36          4.82          4.44        4.4845    0.13152846
  Difference at 95.0% confidence
          0.162 +/- 0.0613971
          3.74783% +/- 1.42041%
          (Student's t, pooled s = 0.0959262)

This corresponds to binary size savings of 2MB out of 335MB, or 0.6%.
It's not a great tradeoff as-is, but as mentioned our implementation can
be signficantly optimized, and literal dedup will unlock more
opportunities for ICF to identify identical structures that reference
the same literals.

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D103113
2021-06-11 19:50:08 -04:00
Nico Weber f2b1a1e10c [lld/mac] Use sectionType() more
Not sure sectionType() carries its weight, but while we have it
we should use it consistently.

No behavior change.

Differential Revision: https://reviews.llvm.org/D104027
2021-06-11 11:15:47 -04:00
Nico Weber 54418c5a35 [lld/mac] Make binaries written by lld strippable
Be less clever when writing the indirect symbols in LC_DYSYMTAB:
lld used to make point __stubs and __la_symbol_ptr point at the
same bytes in the indirect symbol table in the __LINKEDIT segment.
That confused strip, so write the same bytes twice and make
__stubs and __la_symbol_ptr point at one copy each, so that they
don't share data. This unconfuses strip, and seems to be what ld64
does too, so hopefully tools are generally more used to this.

This makes the output binaries a bit larger, but not much: 4 bytes
for roughly each called function from a dylib and each weak function.
Chromium Framewoork grows by 6536 bytes, clang-format by a few hundred.

With this, `strip -x Chromium\ Framework` works (244 MB before stripping
to 171 MB after stripping, compared to 236 MB=>164 MB with ld64). Running
strip without `-x` produces the same error message now for lld-linked
Chromium Framework as for when using ld64 as a linker.

`strip clang-format` also works now but didn't previously.

Fixes PR50657.

Differential Revision: https://reviews.llvm.org/D104081
2021-06-11 00:18:03 -04:00
Jez Ng 4b5c6c5c4b [lld-macho][nfc] Fix uninitialized members warning from Coverity
We were always assigning to this member before using it, but just to be
safe...

See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151029.html
2021-06-10 15:09:07 -04:00
Nico Weber e87c095af3 [lld/mac] Print dylib search details with --print-dylib-search or RC_TRACE_DYLIB_SEARCHING
For debugging dylib loading, it's useful to have some insight into what
the linker is doing.

ld64 has the undocumented RC_TRACE_DYLIB_SEARCHING env var
for this printing dylib search candidates.

This adds a flag --print-dylib-search to make lld print the seame information.
It's useful for users, but also for writing tests. The output is formatted
slightly differently than ld64, but we still support RC_TRACE_DYLIB_SEARCHING
to offer at least a compatible way to trigger this.

ld64 has both `-print_statistics` and `-trace_symbol_output` to enable
diagnostics output. I went with "print" since that seems like a more
straightforward name.

Differential Revision: https://reviews.llvm.org/D103985
2021-06-09 22:08:20 -04:00
Nico Weber 0e399eb527 [lld/mac] When handling @loader_path, use realpath() of symlinks
This is important for Frameworks, which are usually symlinks.

ld64 gets this right for @rpath that's replaced with @loader_path, but not for
bare @loader_path -- ld64's code calls realpath() in that case too, but ignores
the result.

ld64 somehow manages to find libbar1.dylib in the test without the
explicit `-rpath` in Foo1. I don't understand why or how. But this
change is a step forward and fixes an immediate problem I'm having,
so let's start with this :)

Differential Revision: https://reviews.llvm.org/D103990
2021-06-09 20:36:07 -04:00
Jez Ng 447dfbe005 [lld-macho] Implement -force_load_swift_libs
It causes libraries whose names start with "swift" to be force-loaded.
Note that unlike the more general `-force_load`, this flag only applies
to libraries specified via LC_LINKER_OPTIONS, and not those passed on
the command-line. This is what ld64 does.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103709
2021-06-07 23:48:35 -04:00
Jez Ng 04259cde15 [lld-macho] Implement cstring deduplication
Our implementation draws heavily from LLD-ELF's, which in turn delegates
its string deduplication to llvm-mc's StringTableBuilder. The messiness of
this diff is largely due to the fact that we've previously assumed that
all InputSections get concatenated together to form the output. This is
no longer true with CStringInputSections, which split their contents into
StringPieces. StringPieces are much more lightweight than InputSections,
which is important as we create a lot of them. They may also overlap in
the output, which makes it possible for strings to be tail-merged. In
fact, the initial version of this diff implemented tail merging, but
I've dropped it for reasons I'll explain later.

**Alignment Issues**

Mergeable cstring literals are found under the `__TEXT,__cstring`
section. In contrast to ELF, which puts strings that need different
alignments into different sections, clang's Mach-O backend puts them all
in one section. Strings that need to be aligned have the `.p2align`
directive emitted before them, which simply translates into zero padding
in the object file.

I *think* ld64 extracts the desired per-string alignment from this data
by preserving each string's offset from the last section-aligned
address. I'm not entirely certain since it doesn't seem consistent about
doing this; but perhaps this can be chalked up to cases where ld64 has
to deduplicate strings with different offset/alignment combos -- it
seems to pick one of their alignments to preserve. This doesn't seem
correct in general; we can in fact can induce ld64 to produce a crashing
binary just by linking in an additional object file that only contains
cstrings and no code. See PR50563 for details.

Moreover, this scheme seems rather inefficient: since unaligned and
aligned strings are all put in the same section, which has a single
alignment value, it doesn't seem possible to tell whether a given string
doesn't have any alignment requirements. Preserving offset+alignments
for strings that don't need it is wasteful.

In practice, the crashes seen so far seem to stem from x86_64 SIMD
operations on cstrings. X86_64 requires SIMD accesses to be
16-byte-aligned. So for now, I'm thinking of just aligning all strings
to 16 bytes on x86_64. This is indeed wasteful, but implementation-wise
it's simpler than preserving per-string alignment+offsets. It also
avoids the aforementioned crash after deduplication of
differently-aligned strings. Finally, the overhead is not huge: using
16-byte alignment (vs no alignment) is only a 0.5% size overhead when
linking chromium_framework.

With these alignment requirements, it doesn't make sense to attempt tail
merging -- most strings will not be eligible since their overlaps aren't
likely to start at a 16-byte boundary. Tail-merging (with alignment) for
chromium_framework only improves size by 0.3%.

It's worth noting that LLD-ELF only does tail merging at `-O2`. By
default (at `-O1`), it just deduplicates w/o tail merging. @thakis has
also mentioned that they saw it regress compressed size in some cases
and therefore turned it off. `ld64` does not seem to do tail merging at
all.

**Performance Numbers**

CString deduplication reduces chromium_framework from 250MB to 242MB, or
about a 3.2% reduction.

Numbers for linking chromium_framework on my 3.2 GHz 16-Core Intel Xeon W:

      N           Min           Max        Median           Avg        Stddev
  x  20          3.91          4.03         3.935          3.95   0.034641016
  +  20          3.99          4.14         4.015        4.0365     0.0492336
  Difference at 95.0% confidence
          0.0865 +/- 0.027245
          2.18987% +/- 0.689746%
          (Student's t, pooled s = 0.0425673)

As expected, cstring merging incurs some non-trivial overhead.

When passing `--no-literal-merge`, it seems that performance is the
same, i.e. the refactoring in this diff didn't cost us.

      N           Min           Max        Median           Avg        Stddev
  x  20          3.91          4.03         3.935          3.95   0.034641016
  +  20          3.89          4.02         3.935        3.9435   0.043197831
  No difference proven at 95.0% confidence

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D102964
2021-06-07 23:48:35 -04:00
Nico Weber 17c43c4045 [lld/mac] Add reexports after reexporter to inputFiles
When a library "host"'s reexports change their installName with
`$ld$os10.11$install_name$host`, we used to write a load command for "host" but
write the version numbers of the reexport instead of "host". This fixes that.

I first thought that the rule is to take the version numbers from the library
that originally had that install name (implemented in D103819), but that's not
what ld64 seems to be doing: It takes the version number from the first dylib
with that install name it loads, and it loads the reexporting library before
the reexports. We already did most of that, we just added reexports before the
reexporter. After this change, we add the reexporter before the reexports.

Addresses https://bugs.llvm.org/show_bug.cgi?id=49800#c11 part 1.

(ld64 seems to add reexports after processing _all_ files on the command line,
while we add them right after the reexporter. For the common case of reexport +
$ld$ symbol changing back to the exporter name, this doesn't make a difference,
but you can construct a case where it does. I expect this to not make a
difference in practice though.)

Differential Revision: https://reviews.llvm.org/D103821
2021-06-07 17:04:03 -04:00
Nico Weber c5ffe97988 [lld/mac] Implement support for searching dylibs with @rpath/ in install name
Also adjust a few comments, and move the DylibFile comment talking about
umbrella next to the parameter again.

Differential Revision: https://reviews.llvm.org/D103783
2021-06-07 06:22:52 -04:00
Nico Weber 52489021cf [lld/mac] Implement support for searching dylibs with @loader_path/ in install name
Differential Revision: https://reviews.llvm.org/D103779
2021-06-06 20:19:50 -04:00
Nico Weber a48bd587f7 [lld/mac] Implement support for searching dylibs with @executable_path/ in install name
Differential Revision: https://reviews.llvm.org/D103775
2021-06-06 20:01:50 -04:00
Nico Weber 7def700667 [lld/mac] Rename DylibFile::dylibName to DylibFile::installName
The flag to set it is called `-install_name`, and it's called `installName` in tbd files.

No behavior change.

Differential Revision: https://reviews.llvm.org/D103776
2021-06-06 20:00:35 -04:00
Nico Weber e910437443 [lld/mac] Use fewer magic numbers in magic $ld$ handling code
Also simply a conditional and de-alias a variable.
Minor cleanups, no behavior change.

Differential Revision: https://reviews.llvm.org/D103774
2021-06-06 18:13:16 -04:00
Alexander Shaposhnikov 5e49ee8794 [lld][MachO] Add support for $ld$install_name symbols
This diff adds support for $ld$install_name symbols.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D103746
2021-06-05 12:58:59 -07:00
Alexander Shaposhnikov 1309c181a8 [lld][MachO] Add first bits to support special symbols
This diff adds first bits to support special symbols $ld$previous* in LLD.
$ld$* symbols modify properties/behavior of the library
(e.g. its install name, compatibility version or hide/add symbols)
for specific target versions.

Test plan: make check-lld-macho

Differential revision: https://reviews.llvm.org/D103505
2021-06-04 23:32:26 -07:00
Alex Richardson 90344499ae [lld-macho] Fix BUILD_SHARED_LIBS build
ca6751043d added a dependency on XAR (at
least for the shared libs build), so without this change we get the
following linker error:

Undefined symbols for architecture x86_64:
  "_xar_close", referenced from:
      lld::macho::BitcodeBundleSection::finalize() in SyntheticSections.cpp.o

Reviewed By: #lld-macho, int3, thakis

Differential Revision: https://reviews.llvm.org/D100999
2021-06-03 19:58:43 +01:00
Nikita Popov d93b678abb [lld] Add missing includes (NFC)
Fix lld build after 983565a6fe.
2021-06-03 18:55:18 +02:00
Jez Ng 6881f29a36 [lld-macho] Parse re-exports of nested TAPI documents
D103423 neglected to call `parseReexports()` for nested TBD
documents, leading to symbol resolution failures when trying to look up
a symbol nested more than one level deep in a TBD file. This fixes the
regression and adds a test.

It also appears that `umbrella` wasn't being set properly when calling
`parseLoadCommands` -- it's supposed to resolve to `this` if `nullptr`
is passed. I didn't write a failing test case for this but I've made
`umbrella` a member so the previous behavior should be preserved.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D103586
2021-06-03 12:02:30 -04:00
Nico Weber a5645513db [lld/mac] Implement -dead_strip
Also adds support for live_support sections, no_dead_strip sections,
.no_dead_strip symbols.

Chromium Framework 345MB unstripped -> 250MB stripped
(vs 290MB unstripped -> 236M stripped with ld64).

Doing dead stripping is a bit faster than not, because so much less
data needs to be processed:

    % ministat lld_*
    x lld_nostrip.txt
    + lld_strip.txt
        N           Min           Max        Median           Avg        Stddev
    x  10      3.929414       4.07692     4.0269079     4.0089678   0.044214794
    +  10     3.8129408     3.9025559     3.8670411     3.8642573   0.024779651
    Difference at 95.0% confidence
            -0.144711 +/- 0.0336749
            -3.60967% +/- 0.839989%
            (Student's t, pooled s = 0.0358398)

This interacts with many parts of the linker. I tried to add test coverage
for all added `isLive()` checks, so that some test will fail if any of them
is removed. I checked that the test expectations for the most part match
ld64's behavior (except for live-support-iterations.s, see the comment
in the test). Interacts with:
- debug info
- export tries
- import opcodes
- flags like -exported_symbol(s_list)
- -U / dynamic_lookup
- mod_init_funcs, mod_term_funcs
- weak symbol handling
- unwind info
- stubs
- map files
- -sectcreate
- undefined, dylib, common, defined (both absolute and normal) symbols

It's possible it interacts with more features I didn't think of,
of course.

I also did some manual testing:
- check-llvm check-clang check-lld work with lld with this patch
  as host linker and -dead_strip enabled
- Chromium still starts
- Chromium's base_unittests still pass, including unwind tests

Implemenation-wise, this is InputSection-based, so it'll work for
object files with .subsections_via_symbols (which includes all
object files generated by clang). I first based this on the COFF
implementation, but later realized that things are more similar to ELF.
I think it'd be good to refactor MarkLive.cpp to look more like the ELF
part at some point, but I'd like to get a working state checked in first.

Mechanical parts:
- Rename canOmitFromOutput to wasCoalesced (no behavior change)
  since it really is for weak coalesced symbols
- Add noDeadStrip to Defined, corresponding to N_NO_DEAD_STRIP
  (`.no_dead_strip` in asm)

Fixes PR49276.

Differential Revision: https://reviews.llvm.org/D103324
2021-06-02 11:09:26 -04:00
Nico Weber 66a1ecd2cf [lld/mac] Implement -needed_framework, -needed_library, -needed-l
These allow overriding dead_strip_dylibs.

Differential Revision: https://reviews.llvm.org/D103499
2021-06-02 11:06:42 -04:00
Nico Weber e14fd7d879 [lld/mac] Don't strip explicit dylib also mentioned in LC_LINKER_OPTION
Noticed by Jez in D103499.

Differential Revision: https://reviews.llvm.org/D103521
2021-06-02 10:59:56 -04:00
Nico Weber 476e4d65d4 [lld/mac] Address review feedback and improve a comment
I forgot to move the message() call around as requested in D103428
before committing that change. Move it now.

Also, improve the ordinal uniq'ing comment. I hadn't realized that the
distinct-but-identical files happen with --reproduce and not in general.

No behavior change.

Differential Revision: https://reviews.llvm.org/D103522
2021-06-02 10:54:53 -04:00
Nico Weber 78ce89bb1e [lld/mac] Implement -reexport_framework, -reexport_library, -reexport-l
These are slightly easier-to-use versions of -sub_library and -sub_umbrella.

Differential Revision: https://reviews.llvm.org/D103497
2021-06-02 06:37:34 -04:00
Nico Weber 222a88a243 [lld/mac] Make -t work correctly with -flat_namespace
We used to not print dylibs referenced by other dylibs in `-t` mode. This
affected reexports, and with `-flat_namespace` also just dylibs loaded by
dylibs. Now we print them.

Fixes PR49514.

Differential Revision: https://reviews.llvm.org/D103428
2021-06-01 19:23:39 -04:00