Symbols for which `canBeOmittedFromSymbolTable()` is true should be
treated as private externs. This diff tries to do that by unsetting the
ExportDynamic bit. It seems to mostly work with the FullLTO backend, but
with the ThinLTO backend, the `local_unnamed_addr` symbols still fail to
be properly hidden. Nonetheless, this is a step in the right direction.
I've documented all the remaining differences between our behavior and
LD64's in the lto-internalized-unnamed-addr.ll test.
See also https://discourse.llvm.org/t/mach-o-lto-handling-of-linkonce-odr-unnamed-addr/60015
Reviewed By: #lld-macho, thevinster
Differential Revision: https://reviews.llvm.org/D119767
If both an order file and a call graph profile are present, the edges of the
call graph which use symbols present in the order file are not used. All of
the symbols in the order file will appear at the beginning of the section just
as they do currently. In other words, the highest priority derived from the
call graph will be below the lowest priority derived from the order file.
Practically, this change renames CallGraphSort.{h,cpp} to SectionPriorities.{h,cpp},
and most order file and call graph profile related code is moved into the new
file to reduce duplication.
Differential Revision: https://reviews.llvm.org/D117354
By unsetting this property, we are now able to internalize more symbols
during LTO. I compared the output of `-save-temps` for both LLD and
ld64, and we now match ld64's behavior as far as `lto-internalize.ll` is
concerned.
(Thanks @smeenai for working on an initial version of this diff!)
Fixes https://github.com/llvm/llvm-project/issues/50574.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D119372
... to use hyphens instead of underscores, making it consistent with
our other substitutions like %no-arg-lld and %lld-watchos.
Reviewed By: keith
Differential Revision: https://reviews.llvm.org/D119513
We have a mix of substituted lld (`%lld`) and hard-coded lld (`ld64.lld`) commands.
When testing with different versions of LLD, this would require going into every place
where lld is hard-coded and changing that. If we centralize it, this'll only require us
to modify it in only one place and will make it easy to run the same test suite. Plus,
this will make it be consistent with how we write other tests.
Reviewed By: #lld-macho, int3, oontvoo
Differential Revision: https://reviews.llvm.org/D119394
This makes it easier to pinpoint the source of the problem.
TODO: Have more relocation error messages make use of this
functionality.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D118798
Adds `-pagezero_size`. `-pagezero_size` commonly used for kernel development.
`-pagezero_size` changes the `__PAGEZERO` size, removing that segment if it is set to zero.
One of the four flags from {D118570}
Now with error messages and tests.
Differential Revision: https://reviews.llvm.org/D118724
In the case your framework bundles contain relocatable objects, and your
objects include LC_LINKER_OPTIONs for the framework, previously they
would not be deduplicated like they would have if they were static
archives. This was also the case if you passed `-framework` for the
framework as well.
Reviewed By: #lld-macho, thakis, oontvoo
Differential Revision: https://reviews.llvm.org/D114841
This is a ld64 option equivalent to `-sectcreate seg sect /dev/null`
that's useful for creating sections like the RESTRICT section.
Differential Revision: https://reviews.llvm.org/D117749
Previously functions that aren't included in the symtab were also
excluded from the function starts. Symbols missing from function starts
degrades the debugger experience in the case you don't have debug info
for them.
Differential Revision: https://reviews.llvm.org/D114275
Close#52781: for LTO, the inline asm diagnostic uses `<inline asm>` as the file
name (lib/CodeGen/AsmPrinter/AsmPrinterInlineAsm.cpp) and it is unclear which
module has the issue.
With this patch, we will see the module name (say `asm.o`) before `<inline asm>` with ThinLTO.
```
% clang -flto=thin -c asm.c && myld.lld asm.o -e f
ld.lld: error: asm.o <inline asm>:1:2: invalid instruction mnemonic 'invalid'
invalid
^~~~~~~
```
For regular LTO, unfortunately the original module name is lost and we only get
ld-temp.o.
Reviewed By: #lld-macho, ychen, Jez Ng
Differential Revision: https://reviews.llvm.org/D118434
In start-end.s there is a lit check line `# SEG: _main` to begin the
check at the start of the function main where `_main` is the Darwin name
mangling for C main. Because the text file that FileCheck is getting as
input has the path of the compiler build in it from llvm-mc and
llvm-objdump, and because of the lack of a trailing colon in this check
line we end up inadvertently matching against the line of text with the
compiler path in it in the case where said path contains "_main" some
place. This can be very likely if the compiler branch has "main" or
"_main" in it.
To fix this I include the training : since that will match on the
function label and not the path line.
This flag is the default, so in ld64 it is not implemented, but it can
be useful to negate previous -all_load arguments. Specifically if your
build system has some global linker flags, that you may want to negate
for specific links. We use something like this today to make sure some
C++ symbols are automatically discovered for all links, which passing
-all_load hides.
Differential Revision: https://reviews.llvm.org/D117629
In ld.lld, when an ObjFile/BitcodeFile is read in --start-lib state, the file is
given archive semantics. --end-lib closes the previous --start-lib. A build
system can use this feature as an alternative to archives. This patch ports
the feature to lld-macho.
--start-lib and --end-lib are positional, unlike usual ld64 options.
I think the slight drawback does not matter as (a) reusing option names
make build systems convenient (b) `--start-lib a.o b.o --end-lib` conveys more
information than an alternative design: `-objlib a.o -objlib b.o` because
--start-lib makes it clear which objects are in the same conceptual archive.
This provides flexibility (c) `-objlib`/`-filelist` interaction may be weird.
Close https://github.com/llvm/llvm-project/issues/52931
Reviewed By: #lld-macho, Jez Ng, oontvoo
Differential Revision: https://reviews.llvm.org/D116913
It's still uncertain but whether we want to have `deduplicate-literals` be the
default flag for LLD out of the box or not. If `deduplicate-literals` is the default
behavior, then we will need a way override it and not deduplicate. Luckily, we
have `no_deduplicate` to fill this gap. For now, I've set the default to be false
which aligns with the existing behavior. That can only always be changed after
discussions on D117250.
Reviewed By: #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D117387
Depends on D112160
This adds the new options `--call-graph-profile-sort` (default),
`--no-call-graph-profile-sort` and `--print-symbol-order=`. If call graph
profile sorting is enabled, reads `__LLVM,__cg_profile` sections from object
files and uses the resulting graph to put callees and callers close to each
other in the final binary via the C3 clustering heuristic.
Differential Revision: https://reviews.llvm.org/D112164
After {D117069}, map-file.s seems flaky. It seems that the "Total Write
map file" section always exists, but the "Write map file" sub-section
may or may not be emitted. So we check for the former.
After {D115416}, the "Write map file" event no longer shows up
in the time trace. Each time trace profiler instance is thread-local,
but we had neglected to initialize a separate instance for the mapfile
worker thread.
Reviewed By: keith
Differential Revision: https://reviews.llvm.org/D117069
When `file->fetch(sym)` is replaced with a no-op, no test fails.
The new test catches the case.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D116916
References from thread-local variable sections are treated as offsets
relative to the start of the thread-local data memory area, which is
initialized via copying all the TLV data sections (which are all
contiguous). If later data sections require a greater alignment than
earlier ones, the offsets of data within those sections won't be
guaranteed to aligned unless we normalize alignments. We therefore use
the largest alignment for all TLV data sections.
Reviewed By: #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D116263
Fixes#52778.
Probably fixes Chromium crashing on startup on macOS 10.15 (and older) systems
when building with LTO, but I haven't verified that yet.
Differential Revision: https://reviews.llvm.org/D115949
We were fetching archive symbols too eagerly, bloating binary size as well as
just screwing up binaries that expected to look up certain symbols only at
runtime.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D115092
Mach-O LLD uses the buffer identifier of the memory buffer backing an object
file to generate stabs which are used by `dsymutil` to find the object file for
dSYM generation.
When using thinLTO, these buffers are provided by the cache which initially
saves them to disk as temporary files beginning with "Thin-" but renames them
to persistent files beginning with "llvmcache-" before the buffer is provided
to the cache user.
However, the buffer is created before the file is renamed and is given the temp
file's name as an identifier. This causes the generated stabs to point to
nonexistent files.
This change names the buffer with the eventual persistent filename. I think
this is safe because failing to rename the temp file is a fatal error.
Differential Revision: https://reviews.llvm.org/D115055
Follup-up to D107533, where we replaced local syms with non-local.
It doesn't make sense to replace local symbol with lazy.
Differential Revision: https://reviews.llvm.org/D110040
Follow-up to https://reviews.llvm.org/D112643. Even after that change, we were
still asserting if two separate functions that are eligible for ICF (same size,
same data, same number of relocs, same reloc types, ...) referred to
Undefineds. This fixes that oversight.
Differential Revision: https://reviews.llvm.org/D114195
ld64 doesn't warn on builds using `-install_name` if it's a bundle. But, the
current warning is nice to have because `install_name` only works with dylib.
To prevent an overflow of warnings in build logs and have parity with ld64,
create a `--warn-dylib-install-name` and `--warn-no-dylib-install-name` flag
that enables this LLD specific warning.
Reviewed By: #lld-macho, thakis
Differential Revision: https://reviews.llvm.org/D113534
autohide symbols behaves similarly to private_extern symbols.
However, LD64 allows exporting autohide symbols. LLD currently does not.
This patch allows LLD to export them.
Differential Revision: https://reviews.llvm.org/D113167
Similar to D113702, but for the LSDAs. Clang seems to emit all LSDA
relocs as section relocs, but ld -r can turn those relocs into symbol
ones.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D113721
Dedup'ing unwind info is tricky because each CUE contains a different
function address, if ICF operated naively and compared the entire
contents of each CUE, entries with identical unwind info but belonging
to different functions would never be considered identical. To work
around this problem, we slice away the function address before
performing ICF. We rely on `relocateCompactUnwind()` to correctly handle
these truncated input sections.
Here are the numbers before and after D109944, D109945, and this diff
were applied, as tested on my 3.2 GHz 16-Core Intel Xeon W:
Without any optimizations:
base diff difference (95% CI)
sys_time 0.849 ± 0.015 0.896 ± 0.012 [ +4.8% .. +6.2%]
user_time 3.357 ± 0.030 3.512 ± 0.023 [ +4.3% .. +5.0%]
wall_time 3.944 ± 0.039 4.032 ± 0.031 [ +1.8% .. +2.6%]
samples 40 38
With `-dead_strip`:
base diff difference (95% CI)
sys_time 0.847 ± 0.010 0.896 ± 0.012 [ +5.2% .. +6.5%]
user_time 3.377 ± 0.014 3.532 ± 0.015 [ +4.4% .. +4.8%]
wall_time 3.962 ± 0.024 4.060 ± 0.030 [ +2.1% .. +2.8%]
samples 47 30
With `-dead_strip` and `--icf=all`:
base diff difference (95% CI)
sys_time 0.935 ± 0.013 0.957 ± 0.018 [ +1.5% .. +3.2%]
user_time 3.472 ± 0.022 6.531 ± 0.046 [ +87.6% .. +88.7%]
wall_time 4.080 ± 0.040 5.329 ± 0.060 [ +30.0% .. +31.2%]
samples 37 30
Unsurprisingly, ICF is now a lot slower, likely due to the much larger
number of input sections it needs to process. But the rest of the
linker only suffers a mild slowdown.
Note that the compact-unwind-bad-reloc.s test was expanded because we
now handle the relocation for CUE's function address in a separate code
path from the rest of the CUE relocations. The extended test covers both
code paths.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D109946
Clang seems to emit all functionAddress relocs as section relocs, but
`ld -r` can turn those relocs into symbol ones. It turns out that we
weren't handling that case correctly when the symbol was a weak def
whose definition did not prevail.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D113702
Previously if you passed `-oso_prefix path/to/foo/` with a trailing
slash at the end, using `real_path` would remove that slash, but that
slash is necessary to make sure OSO prefix paths end up as valid
relative paths instead of starting with `/`.
Differential Revision: https://reviews.llvm.org/D113541
Clang seems to emit all functionAddress relocs as section relocs, but
`ld -r` can turn those relocs into symbol ones. It turns out that we
weren't handling that case correctly when the symbol was a weak def
whose definition did not prevail.
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D113702
Previously, our unwind info finalization logic assumed that the LSDA
section referenced by `__compact_unwind` was already finalized before
`__TEXT,__unwind_info` itself. However, that assumption could be broken
by the use of `-rename_section` -- it could be (and is) used to move
`__gcc_except_tab` it into a different segment later in the file.
(__TEXT is always the first non-zerofill segment, so any rename
basically guarantees that the section will be ordered after
`__unwind_info`.)
To handle this case, we compare LSDA relocations instead of their final
values in `UnwindInfoSection::finalize()`, and we actually relocate
those LSDAs in `UnwindInfoSection::writeTo()`. In order to do this, we
need an easy way to track which Symbol a given CUE corresponds to. My
solution was to change our `cuPtrVector` into a vector of indices, with
each index used for both the symbols vector (`symbolsVec`) as well as
the CUE vector (`cuVector`).
This change seems perf neutral. Numbers for linking chromium_framework
on my 16 core Mac Pro:
base diff difference (95% CI)
sys_time 1.248 ± 0.025 1.245 ± 0.026 [ -1.3% .. +0.8%]
user_time 3.588 ± 0.045 3.587 ± 0.037 [ -0.6% .. +0.5%]
wall_time 4.605 ± 0.069 4.595 ± 0.069 [ -1.0% .. +0.5%]
samples 42 26
Reviewed By: #lld-macho, oontvoo
Differential Revision: https://reviews.llvm.org/D113582
PR/52372
Differential Revision: https://reviews.llvm.org/D112977
New changes:
- use llvm-otool instead of `otool` which doesn't in exist on non-OSX platforms
- add llvm-otool to the set of tools used by test so that the bot will use the <build_dir>/bin/llvm-otool instead of the unqualified `llvm-otool` (which may not exist)
- update tests since the latest (TOT) llvm-otool prints a space between two bytes and the old one doesn't.
This undocumented ld64 flag, based on the most recent ld64 source dump
from Xcode 12, only applies to i386. It seems like on all newer
architectures this behavior is the default.
Reviewed By: #lld-macho, int3
Differential Revision: https://reviews.llvm.org/D113070
Jez Ng