Builds that error out on duplicate symbols can still succeed if the symbols
will be dead stripped. Currently, this is the current behavior in ld64.
https://github.com/apple-oss-distributions/ld64/blob/main/src/ld/Resolver.cpp#L2018.
In order to provide an easier to path for adoption, introduce a new flag that will
retain compatibility with ld64's behavior (similar to `--deduplicate-literals`). This is
turned off by default since we do not encourage this behavior in the linker.
Reviewed By: #lld-macho, thakis, int3
Differential Revision: https://reviews.llvm.org/D134794
If a value for a given key is not present, `DenseMap::operator[]`
default-constructs one, which is wasteful when we don't do anything with
it afterwards. Fix it by calling `lookup()` instead which only returns
the default value, but does not modify the map.
This speeds up linking a fair bit when only a small portion of all
sections are specified in the order file, like in the case of Chromium
Framework:
N Min Max Median Avg Stddev
x 25 3.727684 3.8808699 3.753552 3.7702461 0.0397282
+ 25 3.6469049 3.7523289 3.6764321 3.6841622 0.025525047
Difference at 95.0% confidence
-0.0860839 +/- 0.0189924
-2.28324% +/- 0.503745%
(Student's t, pooled s = 0.0333906)
Differential Revision: https://reviews.llvm.org/D134811
This is still undocumented and unsupported, but if someone passed it before you
would end up with a missing file error since this takes an argument that
wouldn't be handled.
Differential Revision: https://reviews.llvm.org/D133854
`__thread_vars` contains pointers to `__tlv_bootstrap`, which are fixed
up by dyld; however the section's alignment is not specified. This means
that the relocations might end up on odd addresses, which is not
representable by the soon to be added chained fixups.
This is arguably a bug in MC, but this behavior has been there since TLV
support was originally added.
This patch forces the `__thread_vars` sections to be aligned to the
target's pointer size. This is done by ld64 as well.
Differential Revision: https://reviews.llvm.org/D134594
This arg is undocumented but from looking at the code + experiment, it's used to add additional DYLD_ENVIRONMENT load commands to the output.
Differential Revision: https://reviews.llvm.org/D134058
This commit moves the parsing of linker optimization hints into
`ARM64::applyOptimizationHints`. This lets us avoid allocating memory
for holding the parsed information, and moves work out of
`ObjFile::parse`, which is not parallelized at the moment.
This change reduces the overhead of processing LOHs to 25-30 ms when
linking Chromium Framework on my M1 machine; previously it took close to
100 ms.
There's no statistically significant change in runtime for a --threads=1
link.
Performance figures with all 8 cores utilized:
N Min Max Median Avg Stddev
x 20 3.8027232 3.8760762 3.8505335 3.8454145 0.026352574
+ 20 3.7019017 3.8660538 3.7546209 3.7620371 0.032680043
Difference at 95.0% confidence
-0.0833775 +/- 0.019
-2.16823% +/- 0.494094%
(Student's t, pooled s = 0.0296854)
Differential Revision: https://reviews.llvm.org/D133439
This is similar to the `-alias` CLI option, but it gives finer-grained
control in that it allows the aliased symbols to be treated as private
externs.
While working on this, I realized that our `-alias` handling did not
cover the cases where the aliased symbol is a common or dylib symbol,
nor the case where we have an undefined that gets treated specially and
converted to a defined later on. My N_INDR handling neglects this too
for now; I've added checks and TODO messages for these.
`N_INDR` symbols cropped up as part of our attempt to link swift-stdlib.
Reviewed By: #lld-macho, thakis, thevinster
Differential Revision: https://reviews.llvm.org/D133825
This is similar to the `-alias` CLI option, but it gives finer-grained
control in that it allows the aliased symbols to be treated as private
externs.
While working on this, I realized that our `-alias` handling did not
cover the cases where the aliased symbol is a common or dylib symbol,
nor the case where we have an undefined that gets treated specially and
converted to a defined later on. My N_INDR handling neglects this too
for now; I've added checks and TODO messages for these.
`N_INDR` symbols cropped up as part of our attempt to link swift-stdlib.
Reviewed By: #lld-macho, thakis, thevinster
Differential Revision: https://reviews.llvm.org/D133825
This is what ld64 does (though it doesn't use ICF to do this; instead it
always dedups selrefs by default).
We'll want to dedup implicitly-defined selrefs as well, but I will leave
that for future work.
Additionally, I'm not *super* happy with the current LLD implementation
because I think it is rather janky and inefficient. But at least it
moves us toward the goal of closing the size gap with ld64. I've
described ideas for cleaning up our implementation here:
https://github.com/llvm/llvm-project/issues/57714
Differential Revision: https://reviews.llvm.org/D133780
So @keith observed
[here](https://reviews.llvm.org/D128108#inline-1263900) that the
StringRefs we were returning from `CStringInputSection::getStringRef()`
included the null terminator in their total length, but regular
StringRefs do not. Let's fix that so these StringRefs are less confusing
to use.
Reviewed By: #lld-macho, keith, Roger
Differential Revision: https://reviews.llvm.org/D133728
Previously, we would add entries to DataInCodeSection in the order they
appeared in input files. Because of this, entries would not be sorted if
sections were reordered due to e.g. `-order_file` or call graph profile
sorting. ld64 always keeps data-in-code information sorted.
This commit also fixes an incorrect assertion. The original assertion
from D103006 used to check that data-in-code entries are sorted in the
input objects -- likely because we use binary search on that data. In
D115556, the assertion was moved into `collectDataInCodeEntries`, but
the checked variable's name was not changed, so it ended up checking the
final contents of the DataInCodeSection.
We no longer crash when building LLVM with PGO using an asserts build of
LLD as the linker.
Fixes https://bugs.chromium.org/p/chromium/issues/detail?id=1265937
Numbers for linking the Chromium Framework reproducer from #48001, which
has 6829 data-in-code entries:
x before
+ after
N Min Max Median Avg Stddev
x 20 2.1076453 2.3059683 2.1132485 2.1350302 0.049905767
+ 20 2.1069031 2.3915262 2.14465 2.1728429 0.084065898
No difference proven at 95.0% confidence
Differential Revision: https://reviews.llvm.org/D133581
I wasn't able to find any docs for Mach-O in `lld/docs`, so here's an attempt at adding basic docs. One of my goals here is to make it easy for users who are unfamiliar with linkers to successfully use lld.
Reviewed By: #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D132893
This commit removes the `relocTargets` vector, and instead makes the
code reconstruct the referent addresses from the relocated instructions.
This will allow us to move `applyOptimizationHints` from
`ConcatInputSection::writeTo` to a separate pass that parses and applies
LOHs in one step, on a per-file basis. This will improve performance, as
parsing is currently done serially in `ObjFile::parse`.
I opted to remove the sanity check that ensures that all relocations
within a LOH point to the same symbol. This completely eliminates the
need to search through relocations. It is my understanding that
mismatched relocation targets should not be present in valid object
files, so it's unlikely that the removal will lead to mislinks.
Differential Revision: https://reviews.llvm.org/D133274
The LDR and STR instructions store their immediate offsets as a multiple
of the load/store's size. Therefore, if the target address is not
aligned, the relocation is not representable. We now emit an error if
that happens, similarly to ld64.
This commit removes a test case from loh-adrp-ldr.s that contained an
unaligned LDR.
Differential Revision: https://reviews.llvm.org/D133269
This flag instructs dyld to make the segment read-only after fixups have
been performed.
I'm not sure why this flag is needed, as on macOS 13 beta at least,
__DATA_CONST is read-only even without this flag; but ld64 sets it as
well.
Differential Revision: https://reviews.llvm.org/D133010
This section stores 32-bit `__TEXT` segment offsets of initializer
functions, and is used instead of `__mod_init_func` when chained fixups
are enabled.
Storing the offsets lets us avoid emitting fixups for the initializers.
Differential Revision: https://reviews.llvm.org/D132947
Folding them will cause the unwinder to compute the incorrect function
start address for the folded entries, which in turn will cause the
personality function to interpret the LSDA incorrectly and break
exception handling.
You can verify the end-to-end flow by creating a simple C++ file:
```
void h();
int main() { h(); }
```
and then linking this file against the liblsda.dylib produced by the
test case added here. Before this change, running the resulting program
would result in a program termination with an uncaught exception.
Afterwards, it works correctly.
Reviewed By: #lld-macho, thevinster
Differential Revision: https://reviews.llvm.org/D132845
This section is marked S_ATTR_LIVE_SUPPORT in input files, which meant
that on arm64, we were unnecessarily preserving FDEs if we e.g. had
multiple weak definitions for a function. Worse, we would actually
produce an invalid `__eh_frame` section in that case, because the CIE
associated with the unnecessary FDE would still get dead-stripped and
we'd end up with a dangling FDE. We set up associations from functions
to their FDEs, so dead-stripping will just work naturally, and we can
clear S_ATTR_LIVE_SUPPORT from our input `__eh_frame` sections to fix
dead-stripping.
Reviewed By: #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D132489
We now re-use the existing needsBinding() helper to determine if a
branch has to go through a stub. The logic for determining which type of
binding is needed is moved inside StubsSection::addEntry().
This is an NFC refactor that simplifies my diff that adds support for
chained fixups.
Differential Revision: https://reviews.llvm.org/D132476
The flag accidentally used Joined<> instead of Flag<>.
Previously, `--warn-dylib-install-namefoobarbaz` would be accepted and
had the same effect as `-warn-dylib-install-name`. Now the flag only
works if no suffix is attached to it, as originally intended.
Also fix a typo in the flag's help text.
Differential Revision: https://reviews.llvm.org/D131781
This refactors LTO compile to look more like COFF, where cache hits and misses are all funneled through the same code path.
Previously, cache hits were *not* being saved to -object_path_lto, which led to them sometimes falling out of the cache before dsymutil could process them. As a side effect of the refactor, cached objects are now saved with -save-temps as well, which seems desirable.
(Deleted lld/test/MachO/lto-cache-dsymutil.ll and rolled it into lld/test/MachO/lto-object-path.ll, since the cache-only, non object path approach is unreliable anyway).
Differential Revision: https://reviews.llvm.org/D131624
Apple Clang in Xcode 14 introduced a new feature for reducing the
overhead of objc_msgSend calls by deduplicating the setup calls for each
individual selector. This works by clang adding undefined symbols for
each selector called in a translation unit, such as `_objc_msgSend$foo`
for calling the `foo` method on any `NSObject`. There are 2
different modes for this behavior, the default directly does the setup
for `_objc_msgSend` and calls it, and the smaller option does the
selector setup, and then calls the standard `_objc_msgSend` stub
function.
The general overview of how this works is:
- Undefined symbols with the given prefix are collected
- The suffix of each matching undefined symbol is added as a string to
`__objc_methname`
- A pointer is added for every method name in the `__objc_selrefs`
section
- A `got` entry is emitted for `_objc_msgSend`
- Stubs are emitting pointing to the synthesized locations
Notes:
- Both `__objc_methname` and `__objc_selrefs` can also exist from object
files, so their contents are merged with our synthesized contents
- The compiler emits method names for defined methods, but not for
undefined symbols you call, but stubs are used for both
- This only implements the default "fast" mode currently just to reduce
the diff, I also doubt many folks will care to swap modes
- This only implements this for arm64 and x86_64, we don't need to
implement this for 32 bit iOS archs, but we should implement it for
watchOS archs in a later diff
Differential Revision: https://reviews.llvm.org/D128108
Normally we'd use LLVM_FALLTHROUGH, or now, [[fallthrough]].
But for case labels followed directly by other case labels, we
use neither.
No behavior change.
Some header files used
namespace lld {
namespace macho {
// ...
} // namespace macho
std::string toString(const Type &t);
} // namespace lld
In those files, I didn't use a nested namespace since it's not a big win there.
No behavior change.
Differential Revision: https://reviews.llvm.org/D131354
This fixes the following warnings produced by GCC 9:
../tools/lld/MachO/Arch/ARM64.cpp: In member function ‘void {anonymous}::OptimizationHintContext::applyAdrpLdr(const lld::macho::OptimizationHint&)’:
../tools/lld/MachO/Arch/ARM64.cpp:448:18: warning: comparison of integer expressions of different signedness: ‘int64_t’ {aka ‘long int’} and ‘uint64_t’ {aka ‘long unsigned int’} [-Wsign-compare]
448 | if (ldr.offset != (rel1->referentVA & 0xfff))
| ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../tools/lld/MachO/UnwindInfoSection.cpp: In function ‘bool canFoldEncoding(compact_unwind_encoding_t)’:
../tools/lld/MachO/UnwindInfoSection.cpp:404:44: warning: comparison between ‘enum<unnamed>’ and ‘enum<unnamed>’ [-Wenum-compare]
404 | static_assert(UNWIND_X86_64_MODE_MASK == UNWIND_X86_MODE_MASK, "");
| ^~~~~~~~~~~~~~~~~~~~
../tools/lld/MachO/UnwindInfoSection.cpp:405:49: warning: comparison between ‘enum<unnamed>’ and ‘enum<unnamed>’ [-Wenum-compare]
405 | static_assert(UNWIND_X86_64_MODE_STACK_IND == UNWIND_X86_MODE_STACK_IND, "");
| ^~~~~~~~~~~~~~~~~~~~~~~~~
Differential Revision: https://reviews.llvm.org/D130970
I went over the output of the following mess of a command:
`(ulimit -m 2000000; ulimit -v 2000000; git ls-files -z | parallel --xargs -0 cat | aspell list --mode=none --ignore-case | grep -E '^[A-Za-z][a-z]*$' | sort | uniq -c | sort -n | grep -vE '.{25}' | aspell pipe -W3 | grep : | cut -d' ' -f2 | less)`
and proceeded to spend a few days looking at it to find probable typos
and fixed a few hundred of them in all of the llvm project (note, the
ones I found are not anywhere near all of them, but it seems like a
good start).
Differential Revision: https://reviews.llvm.org/D130982
Previously we only supporting using the system pointer size (aka the
`absptr` encoding) because `llvm-mc`'s CFI directives always generate EH
frames with that encoding. But libffi uses 4-byte-encoded, hand-rolled
EH frames, so this patch adds support for it.
Fixes#56576.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D130804
A symbol `$ld$previous$/Another$1.2.3$1$3.0$14.0$_xxx$` means
"pretend symbol `_xxx` is in dylib `/Another` with version `1.2.3`
if the deployment target is between `3.0` and `14.0` and we're
targeting platform `1` (ie macOS)".
This means dylibs can now inject synthetic dylibs into the link, so
DylibFile needs to grow a 3rd constructor.
The only other interesting thing is that such an injected dylib
counts as a use of the original dylib. This patch gets this mostly
right (if _only_ `$ld$previous` symbols are used from a dylib,
we don't add a dep on the dylib itself, matching ld64), but one case
where we don't match ld64 yet is that ld64 even omits the original
dylib when linking it with `-needed-l`. Lld currently still adds a load
command for the original dylib in that case. (That's for a future
patch.)
Fixes#56074.
Differential Revision: https://reviews.llvm.org/D130725
Linking fails when targeting `x86_64-apple-darwin` for runtimes. The issue
is that LLD strictly assumes the target architecture be present in the tbd
files (which isn't always true). For example, when targeting `x86_64h`, it should
work with `x86_64` because they are ABI compatible. This is also inline with what
ld64 does.
An environment variable (which ld64 also supports) is also added to preserve the
existing behavior of strict architecture matching.
Reviewed By: #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D130683
Vincent Lee