Commit Graph

709 Commits

Author SHA1 Message Date
Fangrui Song 02eab52866 [ELF][AArch64] Fix unneeded thunk for branches to hidden undefined weak
Similar to D119787 for PPC64.

A hidden undefined weak may change its binding to local before some
`isUndefinedWeak` code, so some `isUndefinedWeak` code needs to be changed to
`isUndefined`. The undefined non-weak case has been errored, so just using
`isUndefined` is fine.

The Linux kernel recently has a usage that a branch from 0xffff800008491ee0
references a hidden undefined weak symbol `vfio_group_set_kvm`.
It relies on the behavior that a branch to undefined weak resolving to the next
instruction, otherwise it'd see spurious relocation out of range errors.

Fixes https://github.com/ClangBuiltLinux/linux/issues/1624

Differential Revision: https://reviews.llvm.org/D123750
2022-04-14 11:32:30 -07:00
Fangrui Song 4645311933 [ELF] --emit-relocs: adjust offsets of .rel[a].eh_frame relocations
Two code paths may reach the EHFrame case in SectionBase::getOffset:

* .eh_frame reference
* relocation copy for --emit-relocs

The first may be used by clang_rt.crtbegin.o and GCC crtbeginT.o to get the
start address of the output .eh_frame. The relocation has an offset of 0 or
(x86-64 PC-relative leaq for clang_rt.crtbegin.o) -4. The current code just
returns `offset`, which handles this case well.

The second is related to InputSection::copyRelocations on .eh_frame (used by
--emit-relocs). .eh_frame pieces may be dropped due to GC/ICF, so we should
convert the input offset to the output offset. Use the same way as
MergeInputSection with a special case handling outSecOff==-1 for an invalid
piece (see eh-frame-marker.s).

This exposes an issue in mips64-eh-abs-reloc.s that we don't reliably
handle anyway. Just add --no-check-dynamic-relocations to paper over it.

Differential Revision: https://reviews.llvm.org/D122459
2022-03-29 09:51:41 -07:00
Fangrui Song 48e251b1d6 Revert D122459 "[ELF] --emit-relocs: adjust offsets of .rel[a].eh_frame relocations"
This reverts commit 6faba31e0d.

It may cause "offset is outside the section".
2022-03-28 20:26:21 -07:00
Fangrui Song 6faba31e0d [ELF] --emit-relocs: adjust offsets of .rel[a].eh_frame relocations
.eh_frame pieces may be dropped due to GC/ICF. When --emit-relocs adds
relocations against .eh_frame, the offsets need to be adjusted. Use the same
way as MergeInputSection with a special case handling outSecOff==-1 for an
invalid piece (see eh-frame-marker.s).

This exposes an issue in mips64-eh-abs-reloc.s that we don't reliably
handle anyway. Just add --no-check-dynamic-relocations to paper over it.

Original patch by Ayrton Muñoz

Differential Revision: https://reviews.llvm.org/D122459
2022-03-28 16:23:13 -07:00
Fangrui Song 8565a87fd4 [ELF] Simplify MergeInputSection::getParentOffset. NFC
and remove overly verbose comments.
2022-03-28 10:02:35 -07:00
Fangrui Song 72bedf46c7 [ELF] Inline InputSection::getParent. NFC
Combined with the previous change, lld executable is ~2K smaller and some code
paths using InputSection::getParent are more efficient.

The fragmented headers lead to a design limitation that OutputSection has to be
incomplete, so we cannot use static_cast.
2022-03-08 11:26:12 -08:00
Fangrui Song a815424cc5 Reland D119909 [ELF] Parallelize initializeLocalSymbols
ObjFile::parse combines symbol initialization and resolution. Many tasks
unrelated to symbol resolution can be postponed and parallelized. This patch
extracts local symbol initialization and parallelizes it.

Technically the new function initializeLocalSymbols can be merged into
ObjFile::postParse, but functions like getSrcMsg may access the
uninitialized (all nullptr) local part of InputFile::symbols.

Linking chrome: 1.02x as fast with glibc malloc, 1.04x as fast with mimalloc

Depends on f456c3ae3f and D119908

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D119909
2022-03-04 19:00:10 -08:00
Jorge Gorbe Moya 449b649fec Revert "[ELF] Parallelize initializeLocalSymbols"
This reverts commit 09602d3b47.
2022-03-04 15:01:17 -08:00
Fangrui Song bb3eeac773 [ELF] Make InputSection::classof inline. NFC 2022-02-28 00:16:45 -08:00
Fangrui Song 4976d1fe58 [ELF] Move SyntheticSection check from InputSection::writeTo to OutputSection::writeTo. NFC
Simplify code and make the heavyweight operation to the call site so that it is
clearer how to improve the inefficient scheduling in the future.
2022-02-27 23:28:52 -08:00
Fangrui Song 09602d3b47 [ELF] Parallelize initializeLocalSymbols
ObjFile::parse combines symbol initialization and resolution. Many tasks
unrelated to symbol resolution can be postponed and parallelized. This patch
extracts local symbol initialization and parallelizes it.

Technically the new function initializeLocalSymbols can be merged into
ObjFile::postParse, but functions like getSrcMsg may access the
uninitialized (all nullptr) local part of InputFile::symbols.

Linking chrome: 1.02x as fast with glibc malloc, 1.04x as fast with mimalloc

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D119909
2022-02-24 20:05:59 -08:00
Fangrui Song ae1ba6194f [ELF] Replace uncompressed InputSectionBase::data() with rawData. NFC
In many call sites we know uncompression cannot happen (non-SHF_ALLOC, or the
data (even if compressed) must have been uncompressed by a previous pass).
Prefer rawData in these cases. data() increases code size and prevents
optimization on rawData.
2022-02-21 00:39:26 -08:00
Fangrui Song 27bb799095 [ELF] Clean up headers. NFC 2022-02-07 21:53:34 -08:00
Alexander Shaposhnikov 4450a2a23d [lld][ELF] Add support for ADRP+ADD optimization for AArch64
This diff adds support for ADRP+ADD optimization for AArch64 described in
d2ca58c54b
i.e. under appropriate constraints

ADRP  x0, symbol
ADD   x0, x0, :lo12: symbol

can be turned into

NOP
ADR   x0, symbol

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D117614
2022-02-02 06:09:55 +00:00
Fangrui Song 17a39aecd1 [ELF] Simplify code with invokeELFT. NFC 2022-02-01 09:53:29 -08:00
Fangrui Song d97749fabc [ELF] Switch split-stack to use SmallVector. NFC
My x86-64 lld executable is 1.1KiB smaller.
2022-02-01 00:09:30 -08:00
Fangrui Song 457273fda5 [ELF] splitStrings: replace entSize==1 special case with manual loop unswitch. NFC
My x86-64 lld executable is actually smaller.
2022-01-30 17:15:45 -08:00
Fangrui Song 5a2020d069 [ELF] copyShtGroup: replace unordered_set<uint32_t> with DenseSet<uint32_t>. NFC
We don't need to support the empty/tombstone key section index.
2022-01-30 01:18:41 -08:00
Fangrui Song bc1369fae3 [ELF] Optimize MergeInputSection::splitNonStrings with resize_for_overwrite. NFC 2022-01-30 00:10:52 -08:00
Fangrui Song 14b7785c09 [ELF] Simplify InputSection::writeTo. NFC 2022-01-26 22:03:26 -08:00
Alexandre Ganea 83d59e05b2 Re-land [LLD] Remove global state in lldCommon
Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe3622 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac.

Differential Revision: https://reviews.llvm.org/D108850
2022-01-20 14:53:26 -05:00
Fangrui Song a7a4115bf3 [ELF] Replace .zdebug string comparison with SHF_COMPRESSED check. NFC 2022-01-19 22:33:32 -08:00
Fangrui Song 5bd38a2826 [ELF] Fix split-stack caller with hidden non-split-stack callee
Fix a regression after aabe901d57 (`[ELF] Remove
one redundant computeBinding`): isLocal() does not indicate that the symbol is
originally local. For simplicity, just drop this optimization.
2022-01-19 12:25:01 -08:00
Fangrui Song 5f404a749a [ELF] De-template InputSectionBase::getLocation. NFC 2022-01-18 17:33:58 -08:00
Fangrui Song eafd34581f [ELF] Simplify/optimize EhInputSection::split
and change some `fatal` to `errorOrWarn`.

EhFrame.cpp is a helper file. We don't place all .eh_frame implementation there,
so the code move is fine.
2022-01-18 17:03:23 -08:00
Fangrui Song 83c7f5d3fb [ELF] EhInputSection::split: remove unneeded check 2022-01-17 13:59:52 -08:00
Alexandre Ganea e6b153947d Revert [LLD] Remove global state in lldCommon
It seems to be causing issues on https://lab.llvm.org/buildbot/#/builders/123/builds/8383
2022-01-16 11:03:06 -05:00
Alexandre Ganea f860fe3622 [LLD] Remove global state in lldCommon
Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

Differential Revision: https://reviews.llvm.org/D108850
2022-01-16 08:57:57 -05:00
Fangrui Song 8b2f33231c [ELF] Make some diagnostics follow the convention 2022-01-15 10:46:25 -08:00
Igor Kudrin e00ac48df3 [ELF] Use tombstone values for discarded symbols in relocatable output
This extends D81784. Sections can be discarded when linking a
relocatable output. Before the patch, LLD did not update the content
of debug sections and only replaced the corresponding relocations with
R_*_NONE, which could break the debug information.

Differential Revision: https://reviews.llvm.org/D116946
2022-01-13 11:38:26 +07:00
Fangrui Song 7f1955dc96 [ELF] Support mixed TLSDESC and TLS GD
We only support both TLSDESC and TLS GD for x86 so this is an x86-specific
problem. If both are used, only one R_X86_64_TLSDESC is produced and TLS GD
accesses will incorrectly reference R_X86_64_TLSDESC. Fix this by introducing
SymbolAux::tlsDescIdx.

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D116900
2022-01-10 10:03:21 -08:00
Alexander Shaposhnikov 8acc3b4ab0 [lld][ELF] Support adrp+ldr GOT optimization for AArch64
This diff adds first bits to support relocation relaxations for AArch64
discussed on https://github.com/ARM-software/abi-aa/pull/106.
In particular, the case of

adrp x0, :got: symbol
ldr x0, [x0, :got_lo12: symbol]

is handled.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D112063
2022-01-10 05:20:37 +00:00
Fangrui Song e90c8c0422 [ELF] Optimize basic block section bytesDropped/jumpInstrMods
and make them more space efficient. This decreases sizeof(InputSection) from 176
to 160, and decreases peak memory usage by 0.3% when linking Chrome.
2021-12-26 22:17:30 -08:00
Fangrui Song 469144ffa3 [ELF] De-template InputSectionBase::getEnclosingFunction 2021-12-26 15:21:22 -08:00
Fangrui Song 213896bc5a [ELF] Remove unused InputSection::getOffsetInFile 2021-12-26 15:18:56 -08:00
Fangrui Song bf7f3dd74e [ELF] Move outSecOff addition from InputSection::writeTo to the caller
Simplify the code a bit and improve consistency with SyntheticSection::writeTo.
2021-12-26 12:11:41 -08:00
Fangrui Song 745420d3f4 [ELF] Cache global variable `target` in relocate*
This avoid repeated load of the unique_ptr in hot paths.
2021-12-24 17:54:12 -08:00
Fangrui Song b5a0f0f397 [ELF] Add ELFFileBase::{elfShdrs,numELFShdrs} to avoid duplicate llvm::object::ELFFile::sections()
This mainly avoid `relsOrRelas` cost in `InputSectionBase::relocate`.
`llvm::object::ELFFile::sections()` has redundant and expensive checks.
2021-12-24 17:10:38 -08:00
Fangrui Song e1b6b5be46 [ELF] Avoid referencing SectionBase::repl after ICF
It is fairly easy to forget SectionBase::repl after ICF.
Let ICF rewrite a Defined symbol's `section` field to avoid references to
SectionBase::repl in subsequent passes. This slightly improves the --icf=none
performance due to less indirection (maybe for --icf={safe,all} as well if most
symbols are Defined).

With this change, there is only one reference to `repl` (--gdb-index D89751).
We can undo f4fb5fd752 (`Move Repl to SectionBase.`)
but move `repl` to `InputSection` instead.

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D116093
2021-12-24 12:09:48 -08:00
Fangrui Song bf45624ba0 [ELF][PPC32] Support .got2 in an output section description
I added `PPC32Got2Section` D62464 to support .got2 but did not implement .got2
in another output section.

PR52799 has a linker script placing .got2 in .rodata, which causes a null
pointer dereference because a MergeSyntheticSection's file is nullptr.
Add the support.
2021-12-23 11:32:44 -08:00
Fangrui Song ba948c5a9c [ELF] Use SmallVector for some global variables (*Files and *Sections). NFC
My lld executable is 26+KiB smaller.
2021-12-22 22:30:08 -08:00
Fangrui Song aa27bab5a1 [ELF] InputSection::writeTo: reorder type checks and add LLVM_UNLIKELY 2021-12-16 23:42:50 -08:00
Fangrui Song 054cdb34a2 [ELF] Optimize MergeInputSection::splitNonStrings. NFC 2021-12-16 21:23:00 -08:00
Fangrui Song 4c98d08841 [ELF] Speed up MergeInputSection::split*. NFC 2021-12-16 21:17:02 -08:00
Fangrui Song 1ff1d50d9f [ELF] Make InputFile smaller
sizeof(ObjFile<ELF64LE>) is decreased from 344 to 272 on an ELF64 system.
In a large link with 30000 ObjFiles, this may be 2+MiB saving.

Change std::vector members to SmallVector, and std::string members to
SmallString<0> (these members typically don't benefit from small string optimization).
On Linux x86-64 the lld executable is ~6k smaller.
2021-12-14 20:55:32 -08:00
Fangrui Song d29766bb48 [ELF] relocateAlloc: remove variables type and expr. NFC 2021-12-12 19:31:30 -08:00
Fangrui Song 4cfff19b88 [ELF] Move adjustSplitStackFunctionPrologues's splitStack check to the caller. NFC
Avoid a function call in the majority of cases and make the output smaller.
2021-12-12 19:26:03 -08:00
Fangrui Song 85e50c1080 [ELF] Inline InputSection::getOffset into callers and remove it. NFC
This is an unneeded abstraction which may cause confusion:
SectionBase::getOffset has the same name but hard codes -1 as the size of OutputSection.
2021-11-28 16:09:04 -08:00
Petr Hosek d56b171ee9 [lld][ELF] Support for R_ARM_THM_JUMP8
This change implements support for R_ARM_THM_JUMP8 relocation in
addition to R_ARM_THM_JUMP11 which is already supported by LLD.

Differential Revision: https://reviews.llvm.org/D21225
2021-11-11 09:06:52 -08:00
Fangrui Song 6fcc19afb9 [ELF] Simplify R_TPREL formula after D111365 2021-10-28 21:03:53 -07:00