llvm-project/lld/ELF/InputSection.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

440 lines
15 KiB
C
Raw Normal View History

//===- InputSection.h -------------------------------------------*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#ifndef LLD_ELF_INPUT_SECTION_H
#define LLD_ELF_INPUT_SECTION_H
#include "Relocations.h"
[ELF] Relax R_RISCV_ALIGN Alternative to D125036. Implement R_RISCV_ALIGN relaxation so that we can handle -mrelax object files (i.e. -mno-relax is no longer needed) and creates a framework for future relaxation. `relaxAux` is placed in a union with InputSectionBase::jumpInstrMod, storing auxiliary information for relaxation. In the first pass, `relaxAux` is allocated. The main data structure is `relocDeltas`: when referencing `relocations[i]`, the actual offset is `r_offset - (i ? relocDeltas[i-1] : 0)`. `relaxOnce` performs one relaxation pass. It computes `relocDeltas` for all text section. Then, adjust st_value/st_size for symbols relative to this section based on `SymbolAnchor`. `bytesDropped` is set so that `assignAddresses` knows that the size has changed. Run `relaxOnce` in the `finalizeAddressDependentContent` loop to wait for convergence of text sections and other address dependent sections (e.g. SHT_RELR). Note: extrating `relaxOnce` into a separate loop works for many cases but has issues in some linker script edge cases. After convergence, compute section contents: shrink the NOP sequence of each R_RISCV_ALIGN as appropriate. Instead of deleting bytes, we run a sequence of memcpy on the content delimitered by relocation locations. For R_RISCV_ALIGN let the next memcpy skip the desired number of bytes. Section content computation is parallelizable, but let's ensure the implementation is mature before optimizations. Technically we can save a copy if we interleave some code with `OutputSection::writeTo`, but let's not pollute the generic code (we don't have templated relocation resolving, so using conditions can impose overhead to non-RISCV.) Tested: `make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LLVM=1 defconfig all` built Linux kernel using -mrelax is bootable. FreeBSD RISCV64 system using -mrelax is bootable. bash/curl/firefox/libevent/vim/tmux using -mrelax works. Differential Revision: https://reviews.llvm.org/D127581
2022-07-08 01:16:09 +08:00
#include "lld/Common/CommonLinkerContext.h"
#include "lld/Common/LLVM.h"
[ELF] Relax R_RISCV_ALIGN Alternative to D125036. Implement R_RISCV_ALIGN relaxation so that we can handle -mrelax object files (i.e. -mno-relax is no longer needed) and creates a framework for future relaxation. `relaxAux` is placed in a union with InputSectionBase::jumpInstrMod, storing auxiliary information for relaxation. In the first pass, `relaxAux` is allocated. The main data structure is `relocDeltas`: when referencing `relocations[i]`, the actual offset is `r_offset - (i ? relocDeltas[i-1] : 0)`. `relaxOnce` performs one relaxation pass. It computes `relocDeltas` for all text section. Then, adjust st_value/st_size for symbols relative to this section based on `SymbolAnchor`. `bytesDropped` is set so that `assignAddresses` knows that the size has changed. Run `relaxOnce` in the `finalizeAddressDependentContent` loop to wait for convergence of text sections and other address dependent sections (e.g. SHT_RELR). Note: extrating `relaxOnce` into a separate loop works for many cases but has issues in some linker script edge cases. After convergence, compute section contents: shrink the NOP sequence of each R_RISCV_ALIGN as appropriate. Instead of deleting bytes, we run a sequence of memcpy on the content delimitered by relocation locations. For R_RISCV_ALIGN let the next memcpy skip the desired number of bytes. Section content computation is parallelizable, but let's ensure the implementation is mature before optimizations. Technically we can save a copy if we interleave some code with `OutputSection::writeTo`, but let's not pollute the generic code (we don't have templated relocation resolving, so using conditions can impose overhead to non-RISCV.) Tested: `make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LLVM=1 defconfig all` built Linux kernel using -mrelax is bootable. FreeBSD RISCV64 system using -mrelax is bootable. bash/curl/firefox/libevent/vim/tmux using -mrelax works. Differential Revision: https://reviews.llvm.org/D127581
2022-07-08 01:16:09 +08:00
#include "lld/Common/Memory.h"
#include "llvm/ADT/CachedHashString.h"
#include "llvm/ADT/DenseSet.h"
#include "llvm/ADT/TinyPtrVector.h"
#include "llvm/Object/ELF.h"
namespace lld {
2016-02-28 08:25:54 +08:00
namespace elf {
2022-02-08 13:53:34 +08:00
class InputFile;
class Symbol;
class Defined;
struct Partition;
class SyntheticSection;
template <class ELFT> class ObjFile;
class OutputSection;
extern std::vector<Partition> partitions;
// Returned by InputSectionBase::relsOrRelas. At least one member is empty.
template <class ELFT> struct RelsOrRelas {
ArrayRef<typename ELFT::Rel> rels;
ArrayRef<typename ELFT::Rela> relas;
bool areRelocsRel() const { return rels.size(); }
};
// This is the base class of all sections that lld handles. Some are sections in
// input files, some are sections in the produced output file and some exist
// just as a convenience for implementing special ways of combining some
// sections.
class SectionBase {
public:
enum Kind { Regular, Synthetic, EHFrame, Merge, Output };
Kind kind() const { return (Kind)sectionKind; }
StringRef name;
uint8_t sectionKind : 3;
// The next two bit fields are only used by InputSectionBase, but we
// put them here so the struct packs better.
uint8_t bss : 1;
// Set for sections that should not be folded by ICF.
uint8_t keepUnique : 1;
// The 1-indexed partition that this section is assigned to by the garbage
// collector, or 0 if this section is dead. Normally there is only one
// partition, so this will either be 0 or 1.
uint8_t partition = 1;
elf::Partition &getPartition() const;
// These corresponds to the fields in Elf_Shdr.
2017-10-08 09:55:29 +08:00
uint32_t alignment;
uint64_t flags;
uint32_t entsize;
uint32_t type;
uint32_t link;
uint32_t info;
OutputSection *getOutputSection();
const OutputSection *getOutputSection() const {
return const_cast<SectionBase *>(this)->getOutputSection();
}
// Translate an offset in the input section to an offset in the output
// section.
uint64_t getOffset(uint64_t offset) const;
uint64_t getVA(uint64_t offset = 0) const;
bool isLive() const { return partition != 0; }
void markLive() { partition = 1; }
void markDead() { partition = 0; }
protected:
constexpr SectionBase(Kind sectionKind, StringRef name, uint64_t flags,
uint32_t entsize, uint32_t alignment, uint32_t type,
uint32_t info, uint32_t link)
: name(name), sectionKind(sectionKind), bss(false), keepUnique(false),
alignment(alignment), flags(flags), entsize(entsize), type(type),
link(link), info(info) {}
};
[ELF] Relax R_RISCV_ALIGN Alternative to D125036. Implement R_RISCV_ALIGN relaxation so that we can handle -mrelax object files (i.e. -mno-relax is no longer needed) and creates a framework for future relaxation. `relaxAux` is placed in a union with InputSectionBase::jumpInstrMod, storing auxiliary information for relaxation. In the first pass, `relaxAux` is allocated. The main data structure is `relocDeltas`: when referencing `relocations[i]`, the actual offset is `r_offset - (i ? relocDeltas[i-1] : 0)`. `relaxOnce` performs one relaxation pass. It computes `relocDeltas` for all text section. Then, adjust st_value/st_size for symbols relative to this section based on `SymbolAnchor`. `bytesDropped` is set so that `assignAddresses` knows that the size has changed. Run `relaxOnce` in the `finalizeAddressDependentContent` loop to wait for convergence of text sections and other address dependent sections (e.g. SHT_RELR). Note: extrating `relaxOnce` into a separate loop works for many cases but has issues in some linker script edge cases. After convergence, compute section contents: shrink the NOP sequence of each R_RISCV_ALIGN as appropriate. Instead of deleting bytes, we run a sequence of memcpy on the content delimitered by relocation locations. For R_RISCV_ALIGN let the next memcpy skip the desired number of bytes. Section content computation is parallelizable, but let's ensure the implementation is mature before optimizations. Technically we can save a copy if we interleave some code with `OutputSection::writeTo`, but let's not pollute the generic code (we don't have templated relocation resolving, so using conditions can impose overhead to non-RISCV.) Tested: `make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LLVM=1 defconfig all` built Linux kernel using -mrelax is bootable. FreeBSD RISCV64 system using -mrelax is bootable. bash/curl/firefox/libevent/vim/tmux using -mrelax works. Differential Revision: https://reviews.llvm.org/D127581
2022-07-08 01:16:09 +08:00
struct RISCVRelaxAux;
// This corresponds to a section of an input file.
class InputSectionBase : public SectionBase {
public:
template <class ELFT>
InputSectionBase(ObjFile<ELFT> &file, const typename ELFT::Shdr &header,
StringRef name, Kind sectionKind);
InputSectionBase(InputFile *file, uint64_t flags, uint32_t type,
uint64_t entsize, uint32_t link, uint32_t info,
uint32_t alignment, ArrayRef<uint8_t> data, StringRef name,
Kind sectionKind);
static bool classof(const SectionBase *s) { return s->kind() != Output; }
// The file which contains this section. Its dynamic type is always
// ObjFile<ELFT>, but in order to avoid ELFT, we use InputFile as
// its static type.
InputFile *file;
// Input sections are part of an output section. Special sections
// like .eh_frame and merge sections are first combined into a
// synthetic section that is then added to an output section. In all
// cases this points one level up.
SectionBase *parent = nullptr;
// Section index of the relocation section if exists.
uint32_t relSecIdx = 0;
template <class ELFT> ObjFile<ELFT> *getFile() const {
return cast_or_null<ObjFile<ELFT>>(file);
}
[ELF] Relax R_RISCV_ALIGN Alternative to D125036. Implement R_RISCV_ALIGN relaxation so that we can handle -mrelax object files (i.e. -mno-relax is no longer needed) and creates a framework for future relaxation. `relaxAux` is placed in a union with InputSectionBase::jumpInstrMod, storing auxiliary information for relaxation. In the first pass, `relaxAux` is allocated. The main data structure is `relocDeltas`: when referencing `relocations[i]`, the actual offset is `r_offset - (i ? relocDeltas[i-1] : 0)`. `relaxOnce` performs one relaxation pass. It computes `relocDeltas` for all text section. Then, adjust st_value/st_size for symbols relative to this section based on `SymbolAnchor`. `bytesDropped` is set so that `assignAddresses` knows that the size has changed. Run `relaxOnce` in the `finalizeAddressDependentContent` loop to wait for convergence of text sections and other address dependent sections (e.g. SHT_RELR). Note: extrating `relaxOnce` into a separate loop works for many cases but has issues in some linker script edge cases. After convergence, compute section contents: shrink the NOP sequence of each R_RISCV_ALIGN as appropriate. Instead of deleting bytes, we run a sequence of memcpy on the content delimitered by relocation locations. For R_RISCV_ALIGN let the next memcpy skip the desired number of bytes. Section content computation is parallelizable, but let's ensure the implementation is mature before optimizations. Technically we can save a copy if we interleave some code with `OutputSection::writeTo`, but let's not pollute the generic code (we don't have templated relocation resolving, so using conditions can impose overhead to non-RISCV.) Tested: `make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LLVM=1 defconfig all` built Linux kernel using -mrelax is bootable. FreeBSD RISCV64 system using -mrelax is bootable. bash/curl/firefox/libevent/vim/tmux using -mrelax works. Differential Revision: https://reviews.llvm.org/D127581
2022-07-08 01:16:09 +08:00
// Used by --optimize-bb-jumps and RISC-V linker relaxation temporarily to
// indicate the number of bytes which is not counted in the size. This should
// be reset to zero after uses.
uint16_t bytesDropped = 0;
LLD Support for Basic Block Sections This is part of the Propeller framework to do post link code layout optimizations. Please see the RFC here: https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the detailed RFC doc here: https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf This patch adds lld support for basic block sections and performs relaxations after the basic blocks have been reordered. After the linker has reordered the basic block sections according to the desired sequence, it runs a relaxation pass to optimize jump instructions. Currently, the compiler emits the long form of all jump instructions. AMD64 ISA supports variants of jump instructions with one byte offset or a four byte offset. The compiler generates jump instructions with R_X86_64 32-bit PC relative relocations. We would like to use a new relocation type for these jump instructions as it makes it easy and accurate while relaxing these instructions. The relaxation pass does two things: First, it deletes all explicit fall-through direct jump instructions between adjacent basic blocks. This is done by discarding the tail of the basic block section. Second, If there are consecutive jump instructions, it checks if the first conditional jump can be inverted to convert the second into a fall through and delete the second. The jump instructions are relaxed by using jump instruction mods, something like relocations. These are used to modify the opcode of the jump instruction. Jump instruction mods contain three values, instruction offset, jump type and size. While writing this jump instruction out to the final binary, the linker uses the jump instruction mod to determine the opcode and the size of the modified jump instruction. These mods are required because the input object files are memory-mapped without write permissions and directly modifying the object files requires copying these sections. Copying a large number of basic block sections significantly bloats memory. Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 21:48:18 +08:00
// Whether the section needs to be padded with a NOP filler due to
// deleteFallThruJmpInsn.
bool nopFiller = false;
void drop_back(unsigned num) {
assert(bytesDropped + num < 256);
bytesDropped += num;
}
LLD Support for Basic Block Sections This is part of the Propeller framework to do post link code layout optimizations. Please see the RFC here: https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the detailed RFC doc here: https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf This patch adds lld support for basic block sections and performs relaxations after the basic blocks have been reordered. After the linker has reordered the basic block sections according to the desired sequence, it runs a relaxation pass to optimize jump instructions. Currently, the compiler emits the long form of all jump instructions. AMD64 ISA supports variants of jump instructions with one byte offset or a four byte offset. The compiler generates jump instructions with R_X86_64 32-bit PC relative relocations. We would like to use a new relocation type for these jump instructions as it makes it easy and accurate while relaxing these instructions. The relaxation pass does two things: First, it deletes all explicit fall-through direct jump instructions between adjacent basic blocks. This is done by discarding the tail of the basic block section. Second, If there are consecutive jump instructions, it checks if the first conditional jump can be inverted to convert the second into a fall through and delete the second. The jump instructions are relaxed by using jump instruction mods, something like relocations. These are used to modify the opcode of the jump instruction. Jump instruction mods contain three values, instruction offset, jump type and size. While writing this jump instruction out to the final binary, the linker uses the jump instruction mod to determine the opcode and the size of the modified jump instruction. These mods are required because the input object files are memory-mapped without write permissions and directly modifying the object files requires copying these sections. Copying a large number of basic block sections significantly bloats memory. Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 21:48:18 +08:00
void push_back(uint64_t num) {
assert(bytesDropped >= num);
bytesDropped -= num;
}
mutable ArrayRef<uint8_t> rawData;
LLD Support for Basic Block Sections This is part of the Propeller framework to do post link code layout optimizations. Please see the RFC here: https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the detailed RFC doc here: https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf This patch adds lld support for basic block sections and performs relaxations after the basic blocks have been reordered. After the linker has reordered the basic block sections according to the desired sequence, it runs a relaxation pass to optimize jump instructions. Currently, the compiler emits the long form of all jump instructions. AMD64 ISA supports variants of jump instructions with one byte offset or a four byte offset. The compiler generates jump instructions with R_X86_64 32-bit PC relative relocations. We would like to use a new relocation type for these jump instructions as it makes it easy and accurate while relaxing these instructions. The relaxation pass does two things: First, it deletes all explicit fall-through direct jump instructions between adjacent basic blocks. This is done by discarding the tail of the basic block section. Second, If there are consecutive jump instructions, it checks if the first conditional jump can be inverted to convert the second into a fall through and delete the second. The jump instructions are relaxed by using jump instruction mods, something like relocations. These are used to modify the opcode of the jump instruction. Jump instruction mods contain three values, instruction offset, jump type and size. While writing this jump instruction out to the final binary, the linker uses the jump instruction mod to determine the opcode and the size of the modified jump instruction. These mods are required because the input object files are memory-mapped without write permissions and directly modifying the object files requires copying these sections. Copying a large number of basic block sections significantly bloats memory. Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 21:48:18 +08:00
void trim() {
if (bytesDropped) {
rawData = rawData.drop_back(bytesDropped);
bytesDropped = 0;
}
}
ArrayRef<uint8_t> data() const {
if (uncompressedSize >= 0)
uncompress();
return rawData;
}
[ELF] Improve --gc-sections compatibility with GNU ld regarding section groups Based on D70020 by serge-sans-paille. The ELF spec says: > Furthermore, there may be internal references among these sections that would not make sense if one of the sections were removed or replaced by a duplicate from another object. Therefore, such groups must be included or omitted from the linked object as a unit. A section cannot be a member of more than one group. GNU ld has 2 behaviors that we don't have: - Group members (nextInSectionGroup != nullptr) are subject to garbage collection. This includes non-SHF_ALLOC SHT_NOTE sections. In particular, discarding non-SHF_ALLOC SHT_NOTE sections is an expected behavior by the Annobin project. See https://developers.redhat.com/blog/2018/02/20/annobin-storing-information-binaries/ for more information. - Groups members are retained or discarded as a unit. Members may have internal references that are not expressed as SHF_LINK_ORDER, relocations, etc. It seems that we should be more conservative here: if a section is marked live, mark all the other member within the group. Both behaviors are reasonable. This patch implements them. A new field InputSectionBase::nextInSectionGroup tracks the next member within a group. on ELF64, this increases sizeof(InputSectionBase) froms 144 to 152. InputSectionBase::dependentSections tracks section dependencies, which is used by both --gc-sections and /DISCARD/. We can't overload it for the "next member" semantic, because we should allow /DISCARD/ to discard sections independent of --gc-sections (GNU ld behavior). This behavior may be reasonably used by `/DISCARD/ : { *(.ARM.exidx*) }` or `/DISCARD/ : { *(.note*) }` (new test `linkerscript/discard-group.s`). Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D70146
2019-11-19 13:56:58 +08:00
// The next member in the section group if this section is in a group. This is
// used by --gc-sections.
InputSectionBase *nextInSectionGroup = nullptr;
template <class ELFT> RelsOrRelas<ELFT> relsOrRelas() const;
// InputSections that are dependent on us (reverse dependency for GC)
llvm::TinyPtrVector<InputSection *> dependentSections;
// Returns the size of this section (even if this is a common or BSS.)
size_t getSize() const;
InputSection *getLinkOrderDep() const;
// Get the function symbol that encloses this offset from within the
// section.
Defined *getEnclosingFunction(uint64_t offset);
// Returns a source location string. Used to construct an error message.
std::string getLocation(uint64_t offset);
std::string getSrcMsg(const Symbol &sym, uint64_t offset);
std::string getObjMsg(uint64_t offset);
2017-10-10 11:40:57 +08:00
// Each section knows how to relocate itself. These functions apply
// relocations, assuming that Buf points to this section's copy in
// the mmap'ed output buffer.
template <class ELFT> void relocate(uint8_t *buf, uint8_t *bufEnd);
void relocateAlloc(uint8_t *buf, uint8_t *bufEnd);
LLD Support for Basic Block Sections This is part of the Propeller framework to do post link code layout optimizations. Please see the RFC here: https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the detailed RFC doc here: https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf This patch adds lld support for basic block sections and performs relaxations after the basic blocks have been reordered. After the linker has reordered the basic block sections according to the desired sequence, it runs a relaxation pass to optimize jump instructions. Currently, the compiler emits the long form of all jump instructions. AMD64 ISA supports variants of jump instructions with one byte offset or a four byte offset. The compiler generates jump instructions with R_X86_64 32-bit PC relative relocations. We would like to use a new relocation type for these jump instructions as it makes it easy and accurate while relaxing these instructions. The relaxation pass does two things: First, it deletes all explicit fall-through direct jump instructions between adjacent basic blocks. This is done by discarding the tail of the basic block section. Second, If there are consecutive jump instructions, it checks if the first conditional jump can be inverted to convert the second into a fall through and delete the second. The jump instructions are relaxed by using jump instruction mods, something like relocations. These are used to modify the opcode of the jump instruction. Jump instruction mods contain three values, instruction offset, jump type and size. While writing this jump instruction out to the final binary, the linker uses the jump instruction mod to determine the opcode and the size of the modified jump instruction. These mods are required because the input object files are memory-mapped without write permissions and directly modifying the object files requires copying these sections. Copying a large number of basic block sections significantly bloats memory. Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 21:48:18 +08:00
static uint64_t getRelocTargetVA(const InputFile *File, RelType Type,
int64_t A, uint64_t P, const Symbol &Sym,
RelExpr Expr);
2017-10-10 11:40:57 +08:00
// The native ELF reloc data type is not very convenient to handle.
// So we convert ELF reloc records to our own records in Relocations.cpp.
// This vector contains such "cooked" relocations.
SmallVector<Relocation, 0> relocations;
LLD Support for Basic Block Sections This is part of the Propeller framework to do post link code layout optimizations. Please see the RFC here: https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the detailed RFC doc here: https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf This patch adds lld support for basic block sections and performs relaxations after the basic blocks have been reordered. After the linker has reordered the basic block sections according to the desired sequence, it runs a relaxation pass to optimize jump instructions. Currently, the compiler emits the long form of all jump instructions. AMD64 ISA supports variants of jump instructions with one byte offset or a four byte offset. The compiler generates jump instructions with R_X86_64 32-bit PC relative relocations. We would like to use a new relocation type for these jump instructions as it makes it easy and accurate while relaxing these instructions. The relaxation pass does two things: First, it deletes all explicit fall-through direct jump instructions between adjacent basic blocks. This is done by discarding the tail of the basic block section. Second, If there are consecutive jump instructions, it checks if the first conditional jump can be inverted to convert the second into a fall through and delete the second. The jump instructions are relaxed by using jump instruction mods, something like relocations. These are used to modify the opcode of the jump instruction. Jump instruction mods contain three values, instruction offset, jump type and size. While writing this jump instruction out to the final binary, the linker uses the jump instruction mod to determine the opcode and the size of the modified jump instruction. These mods are required because the input object files are memory-mapped without write permissions and directly modifying the object files requires copying these sections. Copying a large number of basic block sections significantly bloats memory. Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 21:48:18 +08:00
[ELF] Relax R_RISCV_ALIGN Alternative to D125036. Implement R_RISCV_ALIGN relaxation so that we can handle -mrelax object files (i.e. -mno-relax is no longer needed) and creates a framework for future relaxation. `relaxAux` is placed in a union with InputSectionBase::jumpInstrMod, storing auxiliary information for relaxation. In the first pass, `relaxAux` is allocated. The main data structure is `relocDeltas`: when referencing `relocations[i]`, the actual offset is `r_offset - (i ? relocDeltas[i-1] : 0)`. `relaxOnce` performs one relaxation pass. It computes `relocDeltas` for all text section. Then, adjust st_value/st_size for symbols relative to this section based on `SymbolAnchor`. `bytesDropped` is set so that `assignAddresses` knows that the size has changed. Run `relaxOnce` in the `finalizeAddressDependentContent` loop to wait for convergence of text sections and other address dependent sections (e.g. SHT_RELR). Note: extrating `relaxOnce` into a separate loop works for many cases but has issues in some linker script edge cases. After convergence, compute section contents: shrink the NOP sequence of each R_RISCV_ALIGN as appropriate. Instead of deleting bytes, we run a sequence of memcpy on the content delimitered by relocation locations. For R_RISCV_ALIGN let the next memcpy skip the desired number of bytes. Section content computation is parallelizable, but let's ensure the implementation is mature before optimizations. Technically we can save a copy if we interleave some code with `OutputSection::writeTo`, but let's not pollute the generic code (we don't have templated relocation resolving, so using conditions can impose overhead to non-RISCV.) Tested: `make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- LLVM=1 defconfig all` built Linux kernel using -mrelax is bootable. FreeBSD RISCV64 system using -mrelax is bootable. bash/curl/firefox/libevent/vim/tmux using -mrelax works. Differential Revision: https://reviews.llvm.org/D127581
2022-07-08 01:16:09 +08:00
union {
// These are modifiers to jump instructions that are necessary when basic
// block sections are enabled. Basic block sections creates opportunities
// to relax jump instructions at basic block boundaries after reordering the
// basic blocks.
JumpInstrMod *jumpInstrMod = nullptr;
// Auxiliary information for RISC-V linker relaxation. RISC-V does not use
// jumpInstrMod.
RISCVRelaxAux *relaxAux;
};
LLD Support for Basic Block Sections This is part of the Propeller framework to do post link code layout optimizations. Please see the RFC here: https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the detailed RFC doc here: https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf This patch adds lld support for basic block sections and performs relaxations after the basic blocks have been reordered. After the linker has reordered the basic block sections according to the desired sequence, it runs a relaxation pass to optimize jump instructions. Currently, the compiler emits the long form of all jump instructions. AMD64 ISA supports variants of jump instructions with one byte offset or a four byte offset. The compiler generates jump instructions with R_X86_64 32-bit PC relative relocations. We would like to use a new relocation type for these jump instructions as it makes it easy and accurate while relaxing these instructions. The relaxation pass does two things: First, it deletes all explicit fall-through direct jump instructions between adjacent basic blocks. This is done by discarding the tail of the basic block section. Second, If there are consecutive jump instructions, it checks if the first conditional jump can be inverted to convert the second into a fall through and delete the second. The jump instructions are relaxed by using jump instruction mods, something like relocations. These are used to modify the opcode of the jump instruction. Jump instruction mods contain three values, instruction offset, jump type and size. While writing this jump instruction out to the final binary, the linker uses the jump instruction mod to determine the opcode and the size of the modified jump instruction. These mods are required because the input object files are memory-mapped without write permissions and directly modifying the object files requires copying these sections. Copying a large number of basic block sections significantly bloats memory. Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 21:48:18 +08:00
// A function compiled with -fsplit-stack calling a function
// compiled without -fsplit-stack needs its prologue adjusted. Find
// such functions and adjust their prologues. This is very similar
// to relocation. See https://gcc.gnu.org/wiki/SplitStacks for more
// information.
template <typename ELFT>
void adjustSplitStackFunctionPrologues(uint8_t *buf, uint8_t *end);
template <typename T> llvm::ArrayRef<T> getDataAs() const {
size_t s = rawData.size();
assert(s % sizeof(T) == 0);
return llvm::makeArrayRef<T>((const T *)rawData.data(), s / sizeof(T));
}
protected:
template <typename ELFT>
void parseCompressedHeader();
void uncompress() const;
// This field stores the uncompressed size of the compressed data in rawData,
// or -1 if rawData is not compressed (either because the section wasn't
// compressed in the first place, or because we ended up uncompressing it).
// Since the feature is not used often, this is usually -1.
mutable int64_t uncompressedSize = -1;
};
// SectionPiece represents a piece of splittable section contents.
// We allocate a lot of these and binary search on them. This means that they
// have to be as compact as possible, which is why we don't store the size (can
// be found by looking at the next one).
struct SectionPiece {
SectionPiece() = default;
SectionPiece(size_t off, uint32_t hash, bool live)
: inputOff(off), live(live), hash(hash >> 1) {}
uint32_t inputOff;
uint32_t live : 1;
uint32_t hash : 31;
uint64_t outputOff = 0;
};
static_assert(sizeof(SectionPiece) == 16, "SectionPiece is too big");
// This corresponds to a SHF_MERGE section of an input file.
class MergeInputSection : public InputSectionBase {
public:
template <class ELFT>
MergeInputSection(ObjFile<ELFT> &f, const typename ELFT::Shdr &header,
StringRef name);
MergeInputSection(uint64_t flags, uint32_t type, uint64_t entsize,
ArrayRef<uint8_t> data, StringRef name);
2017-10-02 07:46:31 +08:00
static bool classof(const SectionBase *s) { return s->kind() == Merge; }
void splitIntoPieces();
// Translate an offset in the input section to an offset in the parent
// MergeSyntheticSection.
uint64_t getParentOffset(uint64_t offset) const;
// Splittable sections are handled as a sequence of data
// rather than a single large blob of data.
SmallVector<SectionPiece, 0> pieces;
// Returns I'th piece's data. This function is very hot when
// string merging is enabled, so we want to inline.
LLVM_ATTRIBUTE_ALWAYS_INLINE
llvm::CachedHashStringRef getData(size_t i) const {
size_t begin = pieces[i].inputOff;
size_t end =
(pieces.size() - 1 == i) ? rawData.size() : pieces[i + 1].inputOff;
return {toStringRef(rawData.slice(begin, end - begin)), pieces[i].hash};
}
// Returns the SectionPiece at a given input section offset.
SectionPiece &getSectionPiece(uint64_t offset);
const SectionPiece &getSectionPiece(uint64_t offset) const {
return const_cast<MergeInputSection *>(this)->getSectionPiece(offset);
}
SyntheticSection *getParent() const {
return cast_or_null<SyntheticSection>(parent);
}
private:
void splitStrings(StringRef s, size_t size);
void splitNonStrings(ArrayRef<uint8_t> a, size_t size);
};
struct EhSectionPiece {
EhSectionPiece(size_t off, InputSectionBase *sec, uint32_t size,
unsigned firstRelocation)
: inputOff(off), sec(sec), size(size), firstRelocation(firstRelocation) {}
ArrayRef<uint8_t> data() const {
return {sec->rawData.data() + this->inputOff, size};
}
size_t inputOff;
ssize_t outputOff = -1;
InputSectionBase *sec;
uint32_t size;
unsigned firstRelocation;
};
// This corresponds to a .eh_frame section of an input file.
class EhInputSection : public InputSectionBase {
public:
template <class ELFT>
EhInputSection(ObjFile<ELFT> &f, const typename ELFT::Shdr &header,
StringRef name);
2017-10-02 07:46:31 +08:00
static bool classof(const SectionBase *s) { return s->kind() == EHFrame; }
template <class ELFT> void split();
template <class ELFT, class RelTy> void split(ArrayRef<RelTy> rels);
// Splittable sections are handled as a sequence of data
// rather than a single large blob of data.
SmallVector<EhSectionPiece, 0> cies, fdes;
SyntheticSection *getParent() const;
uint64_t getParentOffset(uint64_t offset) const;
};
// This is a section that is added directly to an output section
// instead of needing special combination via a synthetic section. This
// includes all input sections with the exceptions of SHF_MERGE and
// .eh_frame. It also includes the synthetic sections themselves.
class InputSection : public InputSectionBase {
public:
InputSection(InputFile *f, uint64_t flags, uint32_t type, uint32_t alignment,
ArrayRef<uint8_t> data, StringRef name, Kind k = Regular);
template <class ELFT>
InputSection(ObjFile<ELFT> &f, const typename ELFT::Shdr &header,
StringRef name);
static bool classof(const SectionBase *s) {
return s->kind() == SectionBase::Regular ||
s->kind() == SectionBase::Synthetic;
}
// Write this section to a mmap'ed file, assuming Buf is pointing to
// beginning of the output section.
template <class ELFT> void writeTo(uint8_t *buf);
OutputSection *getParent() const {
return reinterpret_cast<OutputSection *>(parent);
}
// This variable has two usages. Initially, it represents an index in the
// OutputSection's InputSection list, and is used when ordering SHF_LINK_ORDER
// sections. After assignAddresses is called, it represents the offset from
// the beginning of the output section this section was assigned to.
uint64_t outSecOff = 0;
InputSectionBase *getRelocatedSection() const;
template <class ELFT, class RelTy>
void relocateNonAlloc(uint8_t *buf, llvm::ArrayRef<RelTy> rels);
// Points to the canonical section. If ICF folds two sections, repl pointer of
// one section points to the other.
InputSection *repl = this;
// Used by ICF.
uint32_t eqClass[2] = {0, 0};
// Called by ICF to merge two input sections.
void replace(InputSection *other);
static InputSection discarded;
private:
template <class ELFT, class RelTy>
void copyRelocations(uint8_t *buf, llvm::ArrayRef<RelTy> rels);
2017-06-09 11:19:08 +08:00
template <class ELFT> void copyShtGroup(uint8_t *buf);
};
static_assert(sizeof(InputSection) <= 160, "InputSection is too big");
class SyntheticSection : public InputSection {
public:
SyntheticSection(uint64_t flags, uint32_t type, uint32_t alignment,
StringRef name)
: InputSection(nullptr, flags, type, alignment, {}, name,
InputSectionBase::Synthetic) {}
virtual ~SyntheticSection() = default;
virtual size_t getSize() const = 0;
virtual bool updateAllocSize() { return false; }
// If the section has the SHF_ALLOC flag and the size may be changed if
// thunks are added, update the section size.
virtual bool isNeeded() const { return true; }
virtual void finalizeContents() {}
virtual void writeTo(uint8_t *buf) = 0;
static bool classof(const SectionBase *sec) {
return sec->kind() == InputSectionBase::Synthetic;
}
};
inline bool isDebugSection(const InputSectionBase &sec) {
return (sec.flags & llvm::ELF::SHF_ALLOC) == 0 &&
sec.name.startswith(".debug");
}
// The list of all input sections.
extern SmallVector<InputSectionBase *, 0> inputSections;
extern SmallVector<EhInputSection *, 0> ehInputSections;
[ELF][PPC64] Suppress toc-indirect to toc-relative relaxation if R_PPC64_TOC16_LO is seen The current implementation assumes that R_PPC64_TOC16_HA is always followed by R_PPC64_TOC16_LO_DS. This can break with R_PPC64_TOC16_LO: // Load the address of the TOC entry, instead of the value stored at that address addis 3, 2, .LC0@tloc@ha # R_PPC64_TOC16_HA addi 3, 3, .LC0@tloc@l # R_PPC64_TOC16_LO blr which is used by boringssl's util/fipstools/delocate/delocate.go https://github.com/google/boringssl/blob/master/crypto/fipsmodule/FIPS.md has some documentation. In short, this tool converts an assembly file to avoid any potential relocations. The distance to an input .toc is not a constant after linking, so it cannot use an `addis;ld` pair. Instead, it jumps to a stub which loads the TOC entry address with `addis;addi`. This patch checks the presence of R_PPC64_TOC16_LO and suppresses toc-indirect to toc-relative relaxation if R_PPC64_TOC16_LO is seen. This approach is conservative and loses some relaxation opportunities but is easy to implement. addis 3, 2, .LC0@toc@ha # no relaxation addi 3, 3, .LC0@toc@l # no relaxation li 9, 0 addis 4, 2, .LC0@toc@ha # can relax but suppressed ld 4, .LC0@toc@l(4) # can relax but suppressed Also note that interleaved R_PPC64_TOC16_HA and R_PPC64_TOC16_LO_DS is possible and this patch accounts for that. addis 3, 2, .LC1@toc@ha # can relax addis 4, 2, .LC2@toc@ha # can relax ld 3, .LC1@toc@l(3) # can relax ld 4, .LC2@toc@l(4) # can relax Reviewed By: #powerpc, sfertile Differential Revision: https://reviews.llvm.org/D78431
2020-04-18 14:08:11 +08:00
// The set of TOC entries (.toc + addend) for which we should not apply
// toc-indirect to toc-relative relaxation. const Symbol * refers to the
// STT_SECTION symbol associated to the .toc input section.
extern llvm::DenseSet<std::pair<const Symbol *, uint64_t>> ppc64noTocRelax;
2016-02-28 08:25:54 +08:00
} // namespace elf
std::string toString(const elf::InputSectionBase *);
} // namespace lld
#endif