llvm-project/lld/ELF/Symbols.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

632 lines
22 KiB
C
Raw Normal View History

//===- Symbols.h ------------------------------------------------*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
2015-10-14 03:51:57 +08:00
//
// This file defines various types of Symbols.
2015-10-14 03:51:57 +08:00
//
//===----------------------------------------------------------------------===//
#ifndef LLD_ELF_SYMBOLS_H
#define LLD_ELF_SYMBOLS_H
#include "InputFiles.h"
#include "InputSection.h"
#include "lld/Common/LLVM.h"
#include "lld/Common/Memory.h"
#include "lld/Common/Strings.h"
[ELF] --warn-backrefs: don't warn for linking sandwich problems This is an alternative design to D77512. D45195 added --warn-backrefs to detect * A. certain input orders which GNU ld either errors ("undefined reference") or has different resolution semantics * B. (byproduct) some latent multiple definition problems (-ldef1 -lref -ldef2) which I call "linking sandwich problems". def2 may or may not be the same as def1. When an archive appears more than once (-ldef -lref -ldef), lld and GNU ld may have the same resolution but --warn-backrefs may warn. This is not uncommon. For example, currently lld itself has such a problem: ``` liblldCommon.a liblldCOFF.a ... liblldCommon.a _ZN3lld10DWARFCache13getDILineInfoEmm in liblldCOFF.a refers to liblldCommon.a(DWARF.cpp.o) libLLVMSupport.a also appears twice and has a similar warning ``` glibc has such problems. It is somewhat destined because of its separate libc/libpthread/... and arbitrary grouping. The situation is getting improved over time but I have seen: ``` -lc __isnanl references -lm -lc _IO_funlockfile references -lpthread ``` There are also various issues in interaction with other runtime libraries such as libgcc_eh and libunwind: ``` -lc __gcc_personality_v0 references -lgcc_eh -lpthread __gcc_personality_v0 references -lgcc_eh -lpthread _Unwind_GetCFA references -lunwind ``` These problems are actually benign. We want --warn-backrefs to focus on its main task A and defer task B (which is also useful) to a more specific future feature (see gold --detect-odr-violations and https://bugs.llvm.org/show_bug.cgi?id=43110). Instead of warning immediately, we store the message and only report it if no subsequent lazy definition exists. The use of the static variable `backrefDiags` is similar to `undefs` in Relocations.cpp Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D77522
2020-04-06 13:27:46 +08:00
#include "llvm/ADT/DenseMap.h"
#include "llvm/Object/Archive.h"
#include "llvm/Object/ELF.h"
[ELF] Add --why-extract= to query why archive members/lazy object files are extracted Similar to D69607 but for archive member extraction unrelated to GC. This patch adds --why-extract=. Prior art: GNU ld -M prints ``` Archive member included to satisfy reference by file (symbol) a.a(a.o) main.o (a) b.a(b.o) (b()) ``` -M is mainly for input section/symbol assignment <-> output section mapping (often huge output) and the information may appear ad-hoc. Apple ld64 ``` __Z1bv forced load of b.a(b.o) _a forced load of a.a(a.o) ``` It doesn't say the reference file. Arm's proprietary linker ``` Selecting member vsnprintf.o(c_wfu.l) to define vsnprintf. ... Loading member vsnprintf.o from c_wfu.l. definition: vsnprintf reference : _printf_a ``` --- --why-extract= gives the user the full data (which is much shorter than GNU ld -Map). It is easy to track a chain of references to one archive member with a one-liner, e.g. ``` % ld.lld main.o a_b.a b_c.a c.a -o /dev/null --why-extract=- | tee stdout reference extracted symbol main.o a_b.a(a_b.o) a a_b.a(a_b.o) b_c.a(b_c.o) b() b_c.a(b_c.o) c.a(c.o) c() % ruby -ane 'BEGIN{p={}}; p[$F[1]]=[$F[0],$F[2]] if $.>1; END{x="c.a(c.o)"; while y=p[x]; puts "#{y[0]} extracts #{x} to resolve #{y[1]}"; x=y[0] end}' stdout b_c.a(b_c.o) extracts c.a(c.o) to resolve c() a_b.a(a_b.o) extracts b_c.a(b_c.o) to resolve b() main.o extracts a_b.a(a_b.o) to resolve a ``` Archive member extraction happens before --gc-sections, so this may not be a live path under --gc-sections, but I think it is a good approximation in practice. * Specifying a file avoids output interleaving with --verbose. * Required `=` prevents accidental overwrite of an input if the user forgets `=`. (Most of compiler drivers' long options accept `=` but not ` `) Differential Revision: https://reviews.llvm.org/D109572
2021-09-21 00:52:30 +08:00
#include <tuple>
namespace lld {
// Returns a string representation for a symbol for diagnostics.
std::string toString(const elf::Symbol &);
// There are two different ways to convert an Archive::Symbol to a string:
// One for Microsoft name mangling and one for Itanium name mangling.
// Call the functions toCOFFString and toELFString, not just toString.
std::string toELFString(const llvm::object::Archive::Symbol &);
namespace elf {
class CommonSymbol;
class Defined;
class InputFile;
class LazyArchive;
class LazyObject;
class SharedSymbol;
class Symbol;
class Undefined;
// Some index properties of a symbol are stored separately in this auxiliary
// struct to decrease sizeof(SymbolUnion) in the majority of cases.
struct SymbolAux {
uint32_t gotIdx = -1;
uint32_t pltIdx = -1;
uint32_t tlsDescIdx = -1;
uint32_t tlsGdIdx = -1;
};
extern SmallVector<SymbolAux, 0> symAux;
// The base class for real symbol classes.
class Symbol {
public:
enum Kind {
PlaceholderKind,
DefinedKind,
CommonKind,
SharedKind,
UndefinedKind,
LazyArchiveKind,
LazyObjectKind,
};
Kind kind() const { return static_cast<Kind>(symbolKind); }
// The file from which this symbol was created.
InputFile *file;
protected:
const char *nameData;
// 32-bit size saves space.
uint32_t nameSize;
public:
// A symAux index used to access GOT/PLT entry indexes. This is allocated in
// postScanRelocations().
uint32_t auxIdx = -1;
uint32_t dynsymIndex = 0;
// This field is a index to the symbol's version definition.
uint16_t verdefIndex = -1;
// Version definition index.
uint16_t versionId;
// Symbol binding. This is not overwritten by replace() to track
// changes during resolution. In particular:
// - An undefined weak is still weak when it resolves to a shared library.
// - An undefined weak will not extract archive members, but we have to
// remember it is weak.
uint8_t binding;
ELF: New symbol table design. This patch implements a new design for the symbol table that stores SymbolBodies within a memory region of the Symbol object. Symbols are mutated by constructing SymbolBodies in place over existing SymbolBodies, rather than by mutating pointers. As mentioned in the initial proposal [1], this memory layout helps reduce the cache miss rate by improving memory locality. Performance numbers: old(s) new(s) Without debug info: chrome 7.178 6.432 (-11.5%) LLVMgold.so 0.505 0.502 (-0.5%) clang 0.954 0.827 (-15.4%) llvm-as 0.052 0.045 (-15.5%) With debug info: scylla 5.695 5.613 (-1.5%) clang 14.396 14.143 (-1.8%) Performance counter results show that the fewer required indirections is indeed the cause of the improved performance. For example, when linking chrome, stalled cycles decreases from 14,556,444,002 to 12,959,238,310, and instructions per cycle increases from 0.78 to 0.83. We are also executing many fewer instructions (15,516,401,933 down to 15,002,434,310), probably because we spend less time allocating SymbolBodies. The new mechanism by which symbols are added to the symbol table is by calling add* functions on the SymbolTable. In this patch, I handle local symbols by storing them inside "unparented" SymbolBodies. This is suboptimal, but if we do want to try to avoid allocating these SymbolBodies, we can probably do that separately. I also removed a few members from the SymbolBody class that were only being used to pass information from the input file to the symbol table. This patch implements the new design for the ELF linker only. I intend to prepare a similar patch for the COFF linker. [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098832.html Differential Revision: http://reviews.llvm.org/D19752 llvm-svn: 268178
2016-05-01 12:55:03 +08:00
// The following fields have the same meaning as the ELF symbol attributes.
uint8_t type; // symbol type
uint8_t stOther; // st_other field value
uint8_t symbolKind;
// Symbol visibility. This is the computed minimum visibility of all
// observed non-DSO symbols.
uint8_t visibility : 2;
// True if the symbol was used for linking and thus need to be added to the
// output file's symbol table. This is true for all symbols except for
// unreferenced DSO symbols, lazy (archive) symbols, and bitcode symbols that
// are unreferenced except by other bitcode objects.
uint8_t isUsedInRegularObj : 1;
2019-08-13 17:12:52 +08:00
// Used by a Defined symbol with protected or default visibility, to record
// whether it is required to be exported into .dynsym. This is set when any of
// the following conditions hold:
//
// - If there is an interposable symbol from a DSO.
// - If -shared or --export-dynamic is specified, any symbol in an object
// file/bitcode sets this property, unless suppressed by LTO
// canBeOmittedFromSymbolTable().
uint8_t exportDynamic : 1;
2019-08-13 17:12:52 +08:00
// True if the symbol is in the --dynamic-list file. A Defined symbol with
// protected or default visibility with this property is required to be
// exported into .dynsym.
uint8_t inDynamicList : 1;
2019-08-13 17:12:52 +08:00
// False if LTO shouldn't inline whatever this symbol points to. If a symbol
// is overwritten after LTO, LTO shouldn't inline the symbol because it
// doesn't know the final contents of the symbol.
uint8_t canInline : 1;
// Used to track if there has been at least one undefined reference to the
// symbol. For Undefined and SharedSymbol, the binding may change to STB_WEAK
// if the first undefined reference from a non-shared object is weak.
//
// This is also used to retain __wrap_foo when foo is referenced.
uint8_t referenced : 1;
[ELF] Make binding (weak or non-weak) logic consistent for Undefined and SharedSymbol This is a case missed by D64136. If %t1.o has a weak reference on foo, and %t2.so has a non-weak reference on foo: ``` 0. ld.lld %t1.o %t2.so # ok; STB_WEAK; accepted since D64136 1. ld.lld %t2.so %t1.o # undefined symbol: foo; STB_GLOBAL 2. gold %t1.o %t2.so # ok; STB_WEAK 3. gold %t2.so %t1.o # undefined reference to 'foo'; STB_GLOBAL 4. ld.bfd %t1.o %t2.so # undefined reference to `foo'; STB_WEAK 5. ld.bfd %t2.so %t1.o # undefined reference to `foo'; STB_WEAK ``` It can be argued that in both cases, the binding of the undefined foo should be set to STB_WEAK, because the binding should not be affected by referenced from shared objects. --allow-shlib-undefined doesn't suppress errors (3,4,5), but -shared or --noinhibit-exec allows ld.bfd/gold to produce a binary: ``` 3. gold -shared %t2.so %t1.o # ok; STB_GLOBAL 4. ld.bfd -shared %t2.so %t1.o # ok; STB_WEAK 5. ld.bfd -shared %t1.o %t1.o # ok; STB_WEAK ``` If %t2.so has DT_NEEDED entries, ld.bfd will load them (lld/gold don't have the behavior). If one of the DSO defines foo and it is in the link-time search path (e.g. DT_NEEDED entry is an absolute path, via -rpath=, via -rpath-link=, etc), `ld.bfd %t1.o %t2.so` and `ld.bfd %t1.o %t2.so` will not error. In this patch, we make Undefined and SharedSymbol share the same binding computing logic. Case 1 will be allowed: ``` 0. ld.lld %t1.o %t2.so # ok; STB_WEAK; accepted since D64136 1. ld.lld %t2.so %t1.o # ok; STB_WEAK; changed by this patch ``` In the future, we can explore the option that turns both (0,1) into errors if --no-allow-shlib-undefined (default when linking an executable) is in action. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D65584 llvm-svn: 368038
2019-08-06 22:03:45 +08:00
// True if this symbol is specified by --trace-symbol option.
uint8_t traced : 1;
// True if the name contains '@'.
uint8_t hasVersionSuffix : 1;
inline void replace(const Symbol &newSym);
bool includeInDynsym() const;
uint8_t computeBinding() const;
bool isWeak() const { return binding == llvm::ELF::STB_WEAK; }
bool isUndefined() const { return symbolKind == UndefinedKind; }
bool isCommon() const { return symbolKind == CommonKind; }
bool isDefined() const { return symbolKind == DefinedKind; }
bool isShared() const { return symbolKind == SharedKind; }
bool isPlaceholder() const { return symbolKind == PlaceholderKind; }
bool isLocal() const { return binding == llvm::ELF::STB_LOCAL; }
bool isLazy() const {
return symbolKind == LazyArchiveKind || symbolKind == LazyObjectKind;
}
// True if this is an undefined weak symbol. This only works once
// all input files have been added.
bool isUndefWeak() const { return isWeak() && isUndefined(); }
StringRef getName() const { return {nameData, nameSize}; }
void setName(StringRef s) {
nameData = s.data();
nameSize = s.size();
}
void parseSymbolVersion();
// Get the NUL-terminated version suffix ("", "@...", or "@@...").
//
// For @@, the name has been truncated by insert(). For @, the name has been
// truncated by Symbol::parseSymbolVersion().
const char *getVersionSuffix() const { return nameData + nameSize; }
uint32_t getGotIdx() const {
return auxIdx == uint32_t(-1) ? uint32_t(-1) : symAux[auxIdx].gotIdx;
}
uint32_t getPltIdx() const {
return auxIdx == uint32_t(-1) ? uint32_t(-1) : symAux[auxIdx].pltIdx;
}
uint32_t getTlsDescIdx() const {
return auxIdx == uint32_t(-1) ? uint32_t(-1) : symAux[auxIdx].tlsDescIdx;
}
uint32_t getTlsGdIdx() const {
return auxIdx == uint32_t(-1) ? uint32_t(-1) : symAux[auxIdx].tlsGdIdx;
}
bool isInGot() const { return getGotIdx() != uint32_t(-1); }
bool isInPlt() const { return getPltIdx() != uint32_t(-1); }
uint64_t getVA(int64_t addend = 0) const;
uint64_t getGotOffset() const;
uint64_t getGotVA() const;
uint64_t getGotPltOffset() const;
uint64_t getGotPltVA() const;
uint64_t getPltVA() const;
uint64_t getSize() const;
OutputSection *getOutputSection() const;
// The following two functions are used for symbol resolution.
//
// You are expected to call mergeProperties for all symbols in input
// files so that attributes that are attached to names rather than
// indivisual symbol (such as visibility) are merged together.
//
// Every time you read a new symbol from an input, you are supposed
// to call resolve() with the new symbol. That function replaces
// "this" object as a result of name resolution if the new symbol is
// more appropriate to be included in the output.
//
// For example, if "this" is an undefined symbol and a new symbol is
// a defined symbol, "this" is replaced with the new symbol.
void mergeProperties(const Symbol &other);
void resolve(const Symbol &other);
// If this is a lazy symbol, extract an input file and add the symbol
// in the file to the symbol table. Calling this function on
// non-lazy object causes a runtime error.
void extract() const;
static bool isExportDynamic(Kind k, uint8_t visibility) {
if (k == SharedKind)
return visibility == llvm::ELF::STV_DEFAULT;
return config->shared || config->exportDynamic;
}
private:
void resolveUndefined(const Undefined &other);
void resolveCommon(const CommonSymbol &other);
void resolveDefined(const Defined &other);
template <class LazyT> void resolveLazy(const LazyT &other);
void resolveShared(const SharedSymbol &other);
int compare(const Symbol *other) const;
inline size_t getSymbolSize() const;
protected:
Symbol(Kind k, InputFile *file, StringRef name, uint8_t binding,
uint8_t stOther, uint8_t type)
: file(file), nameData(name.data()), nameSize(name.size()),
binding(binding), type(type), stOther(stOther), symbolKind(k),
visibility(stOther & 3),
isUsedInRegularObj(!file || file->kind() == InputFile::ObjKind),
2019-08-13 17:12:52 +08:00
exportDynamic(isExportDynamic(k, visibility)), inDynamicList(false),
canInline(false), referenced(false), traced(false),
hasVersionSuffix(false), isInIplt(false), gotInIgot(false),
isPreemptible(false), used(!config->gcSections), folded(false),
needsTocRestore(false), scriptDefined(false), needsCopy(false),
needsGot(false), needsPlt(false), needsTlsDesc(false),
needsTlsGd(false), needsTlsGdToIe(false), needsTlsLd(false),
needsGotDtprel(false), needsTlsIe(false), hasDirectReloc(false) {}
2015-12-25 14:55:39 +08:00
public:
// True if this symbol is in the Iplt sub-section of the Plt and the Igot
// sub-section of the .got.plt or .got.
uint8_t isInIplt : 1;
// True if this symbol needs a GOT entry and its GOT entry is actually in
// Igot. This will be true only for certain non-preemptible ifuncs.
uint8_t gotInIgot : 1;
// True if this symbol is preemptible at load time.
uint8_t isPreemptible : 1;
// True if an undefined or shared symbol is used from a live section.
//
// NOTE: In Writer.cpp the field is used to mark local defined symbols
// which are referenced by relocations when -r or --emit-relocs is given.
uint8_t used : 1;
// True if defined relative to a section discarded by ICF.
uint8_t folded : 1;
// True if a call to this symbol needs to be followed by a restore of the
// PPC64 toc pointer.
uint8_t needsTocRestore : 1;
// True if this symbol is defined by a linker script.
uint8_t scriptDefined : 1;
// True if this symbol needs a canonical PLT entry, or (during
// postScanRelocations) a copy relocation.
uint8_t needsCopy : 1;
// Temporary flags used to communicate which symbol entries need PLT and GOT
// entries during postScanRelocations();
uint8_t needsGot : 1;
uint8_t needsPlt : 1;
uint8_t needsTlsDesc : 1;
uint8_t needsTlsGd : 1;
uint8_t needsTlsGdToIe : 1;
uint8_t needsTlsLd : 1;
uint8_t needsGotDtprel : 1;
uint8_t needsTlsIe : 1;
uint8_t hasDirectReloc : 1;
bool needsDynReloc() const {
return needsCopy || needsGot || needsPlt || needsTlsDesc || needsTlsGd ||
needsTlsGdToIe || needsTlsLd || needsGotDtprel || needsTlsIe;
}
void allocateAux() {
assert(auxIdx == uint32_t(-1));
auxIdx = symAux.size();
symAux.emplace_back();
}
// The partition whose dynamic symbol table contains this symbol's definition.
uint8_t partition = 1;
[Coding style change] Rename variables so that they start with a lowercase letter This patch is mechanically generated by clang-llvm-rename tool that I wrote using Clang Refactoring Engine just for creating this patch. You can see the source code of the tool at https://reviews.llvm.org/D64123. There's no manual post-processing; you can generate the same patch by re-running the tool against lld's code base. Here is the main discussion thread to change the LLVM coding style: https://lists.llvm.org/pipermail/llvm-dev/2019-February/130083.html In the discussion thread, I proposed we use lld as a testbed for variable naming scheme change, and this patch does that. I chose to rename variables so that they are in camelCase, just because that is a minimal change to make variables to start with a lowercase letter. Note to downstream patch maintainers: if you are maintaining a downstream lld repo, just rebasing ahead of this commit would cause massive merge conflicts because this patch essentially changes every line in the lld subdirectory. But there's a remedy. clang-llvm-rename tool is a batch tool, so you can rename variables in your downstream repo with the tool. Given that, here is how to rebase your repo to a commit after the mass renaming: 1. rebase to the commit just before the mass variable renaming, 2. apply the tool to your downstream repo to mass-rename variables locally, and 3. rebase again to the head. Most changes made by the tool should be identical for a downstream repo and for the head, so at the step 3, almost all changes should be merged and disappear. I'd expect that there would be some lines that you need to merge by hand, but that shouldn't be too many. Differential Revision: https://reviews.llvm.org/D64121 llvm-svn: 365595
2019-07-10 13:00:37 +08:00
bool isSection() const { return type == llvm::ELF::STT_SECTION; }
bool isTls() const { return type == llvm::ELF::STT_TLS; }
bool isFunc() const { return type == llvm::ELF::STT_FUNC; }
bool isGnuIFunc() const { return type == llvm::ELF::STT_GNU_IFUNC; }
bool isObject() const { return type == llvm::ELF::STT_OBJECT; }
bool isFile() const { return type == llvm::ELF::STT_FILE; }
};
// Represents a symbol that is defined in the current output file.
class Defined : public Symbol {
public:
Defined(InputFile *file, StringRef name, uint8_t binding, uint8_t stOther,
uint8_t type, uint64_t value, uint64_t size, SectionBase *section)
: Symbol(DefinedKind, file, name, binding, stOther, type), value(value),
size(size), section(section) {}
static bool classof(const Symbol *s) { return s->isDefined(); }
uint64_t value;
uint64_t size;
SectionBase *section;
};
// Represents a common symbol.
//
// On Unix, it is traditionally allowed to write variable definitions
// without initialization expressions (such as "int foo;") to header
// files. Such definition is called "tentative definition".
//
// Using tentative definition is usually considered a bad practice
// because you should write only declarations (such as "extern int
// foo;") to header files. Nevertheless, the linker and the compiler
// have to do something to support bad code by allowing duplicate
// definitions for this particular case.
//
// Common symbols represent variable definitions without initializations.
// The compiler creates common symbols when it sees variable definitions
// without initialization (you can suppress this behavior and let the
// compiler create a regular defined symbol by -fno-common).
//
// The linker allows common symbols to be replaced by regular defined
// symbols. If there are remaining common symbols after name resolution is
// complete, they are converted to regular defined symbols in a .bss
// section. (Therefore, the later passes don't see any CommonSymbols.)
class CommonSymbol : public Symbol {
public:
CommonSymbol(InputFile *file, StringRef name, uint8_t binding,
uint8_t stOther, uint8_t type, uint64_t alignment, uint64_t size)
: Symbol(CommonKind, file, name, binding, stOther, type),
alignment(alignment), size(size) {}
static bool classof(const Symbol *s) { return s->isCommon(); }
uint32_t alignment;
uint64_t size;
};
class Undefined : public Symbol {
public:
Undefined(InputFile *file, StringRef name, uint8_t binding, uint8_t stOther,
uint8_t type, uint32_t discardedSecIdx = 0)
: Symbol(UndefinedKind, file, name, binding, stOther, type),
discardedSecIdx(discardedSecIdx) {}
static bool classof(const Symbol *s) { return s->kind() == UndefinedKind; }
// The section index if in a discarded section, 0 otherwise.
uint32_t discardedSecIdx;
};
class SharedSymbol : public Symbol {
public:
static bool classof(const Symbol *s) { return s->kind() == SharedKind; }
[Coding style change] Rename variables so that they start with a lowercase letter This patch is mechanically generated by clang-llvm-rename tool that I wrote using Clang Refactoring Engine just for creating this patch. You can see the source code of the tool at https://reviews.llvm.org/D64123. There's no manual post-processing; you can generate the same patch by re-running the tool against lld's code base. Here is the main discussion thread to change the LLVM coding style: https://lists.llvm.org/pipermail/llvm-dev/2019-February/130083.html In the discussion thread, I proposed we use lld as a testbed for variable naming scheme change, and this patch does that. I chose to rename variables so that they are in camelCase, just because that is a minimal change to make variables to start with a lowercase letter. Note to downstream patch maintainers: if you are maintaining a downstream lld repo, just rebasing ahead of this commit would cause massive merge conflicts because this patch essentially changes every line in the lld subdirectory. But there's a remedy. clang-llvm-rename tool is a batch tool, so you can rename variables in your downstream repo with the tool. Given that, here is how to rebase your repo to a commit after the mass renaming: 1. rebase to the commit just before the mass variable renaming, 2. apply the tool to your downstream repo to mass-rename variables locally, and 3. rebase again to the head. Most changes made by the tool should be identical for a downstream repo and for the head, so at the step 3, almost all changes should be merged and disappear. I'd expect that there would be some lines that you need to merge by hand, but that shouldn't be too many. Differential Revision: https://reviews.llvm.org/D64121 llvm-svn: 365595
2019-07-10 13:00:37 +08:00
SharedSymbol(InputFile &file, StringRef name, uint8_t binding,
uint8_t stOther, uint8_t type, uint64_t value, uint64_t size,
uint32_t alignment, uint16_t verdefIndex)
[ELF] Only allow the binding of SharedSymbol to change for the first undef ref Fixes PR42442 t.o has a STB_GLOBAL undef ref to f t2.so has a STB_WEAK undef ref to f t1.so defines f ld.lld t.o t1.so t2.so currently sets the binding of `f` to STB_WEAK. This is not correct because there exists a STB_GLOBAL undef ref from a regular object. The problem is that resolveUndefined() doesn't check if the undef ref is seen for the first time: if (isShared() || isLazy() || (isUndefined() && Other.Binding != STB_WEAK)) Binding = Other.Binding; The isShared() condition should be `isShared() && !Referenced` where Referenced is set to true after an undef ref is seen. In practice, when linking a pthread program with glibc: // a.o #include <pthread.h> pthread_mutex_t mu = PTHREAD_MUTEX_INITIALIZER; int main() { pthread_mutex_unlock(&mu); } {clang,gcc} -fuse-ld=lld a.o -lpthread # libpthread.so is linked before libgcc_s.so.1 The weak undef pthread_mutex_unlock in libgcc_s.so.1 makes the result weak, which diverges from GNU linkers where STB_DEFAULT is used: 23: 0000000000000000 0 FUNC WEAK DEFAULT UND pthread_mutex_lock (Note, if -pthread is used instead, libpthread.so will be linked **after** libgcc_s.so.1 . lld sets the binding to the expected STB_GLOBAL) Similar linking sequences (ld.lld t.o t1.so t2.so) appear to be used by Go, which cause a build error https://github.com/golang/go/issues/31912. Reviewed By: grimar, ruiu Differential Revision: https://reviews.llvm.org/D63974 llvm-svn: 364913
2019-07-02 19:37:21 +08:00
: Symbol(SharedKind, &file, name, binding, stOther, type), value(value),
size(size), alignment(alignment) {
this->verdefIndex = verdefIndex;
2017-10-13 05:45:53 +08:00
// GNU ifunc is a mechanism to allow user-supplied functions to
// resolve PLT slot values at load-time. This is contrary to the
// regular symbol resolution scheme in which symbols are resolved just
2017-10-13 05:45:53 +08:00
// by name. Using this hook, you can program how symbols are solved
// for you program. For example, you can make "memcpy" to be resolved
// to a SSE-enabled version of memcpy only when a machine running the
// program supports the SSE instruction set.
//
// Naturally, such symbols should always be called through their PLT
// slots. What GNU ifunc symbols point to are resolver functions, and
// calling them directly doesn't make sense (unless you are writing a
// loader).
//
// For DSO symbols, we always call them through PLT slots anyway.
// So there's no difference between GNU ifunc and regular function
2017-10-13 05:52:14 +08:00
// symbols if they are in DSOs. So we can handle GNU_IFUNC as FUNC.
2017-10-13 05:45:53 +08:00
if (this->type == llvm::ELF::STT_GNU_IFUNC)
this->type = llvm::ELF::STT_FUNC;
}
SharedFile &getFile() const { return *cast<SharedFile>(file); }
uint64_t value; // st_value
uint64_t size; // st_size
[ELF] Only allow the binding of SharedSymbol to change for the first undef ref Fixes PR42442 t.o has a STB_GLOBAL undef ref to f t2.so has a STB_WEAK undef ref to f t1.so defines f ld.lld t.o t1.so t2.so currently sets the binding of `f` to STB_WEAK. This is not correct because there exists a STB_GLOBAL undef ref from a regular object. The problem is that resolveUndefined() doesn't check if the undef ref is seen for the first time: if (isShared() || isLazy() || (isUndefined() && Other.Binding != STB_WEAK)) Binding = Other.Binding; The isShared() condition should be `isShared() && !Referenced` where Referenced is set to true after an undef ref is seen. In practice, when linking a pthread program with glibc: // a.o #include <pthread.h> pthread_mutex_t mu = PTHREAD_MUTEX_INITIALIZER; int main() { pthread_mutex_unlock(&mu); } {clang,gcc} -fuse-ld=lld a.o -lpthread # libpthread.so is linked before libgcc_s.so.1 The weak undef pthread_mutex_unlock in libgcc_s.so.1 makes the result weak, which diverges from GNU linkers where STB_DEFAULT is used: 23: 0000000000000000 0 FUNC WEAK DEFAULT UND pthread_mutex_lock (Note, if -pthread is used instead, libpthread.so will be linked **after** libgcc_s.so.1 . lld sets the binding to the expected STB_GLOBAL) Similar linking sequences (ld.lld t.o t1.so t2.so) appear to be used by Go, which cause a build error https://github.com/golang/go/issues/31912. Reviewed By: grimar, ruiu Differential Revision: https://reviews.llvm.org/D63974 llvm-svn: 364913
2019-07-02 19:37:21 +08:00
uint32_t alignment;
};
// LazyArchive and LazyObject represent a symbols that is not yet in the link,
// but we know where to find it if needed. If the resolver finds both Undefined
// and Lazy for the same name, it will ask the Lazy to load a file.
//
// A special complication is the handling of weak undefined symbols. They should
// not load a file, but we have to remember we have seen both the weak undefined
// and the lazy. We represent that with a lazy symbol with a weak binding. This
// means that code looking for undefined symbols normally also has to take lazy
// symbols into consideration.
// This class represents a symbol defined in an archive file. It is
// created from an archive file header, and it knows how to load an
// object file from an archive to replace itself with a defined
// symbol.
class LazyArchive : public Symbol {
public:
LazyArchive(InputFile &file, const llvm::object::Archive::Symbol s)
: Symbol(LazyArchiveKind, &file, s.getName(), llvm::ELF::STB_GLOBAL,
llvm::ELF::STV_DEFAULT, llvm::ELF::STT_NOTYPE),
sym(s) {}
static bool classof(const Symbol *s) { return s->kind() == LazyArchiveKind; }
MemoryBufferRef getMemberBuffer();
const llvm::object::Archive::Symbol sym;
};
// LazyObject symbols represents symbols in object files between
// --start-lib and --end-lib options.
class LazyObject : public Symbol {
public:
LazyObject(InputFile &file, StringRef name)
: Symbol(LazyObjectKind, &file, name, llvm::ELF::STB_GLOBAL,
llvm::ELF::STV_DEFAULT, llvm::ELF::STT_NOTYPE) {
isUsedInRegularObj = false;
}
static bool classof(const Symbol *s) { return s->kind() == LazyObjectKind; }
};
// Some linker-generated symbols need to be created as
// Defined symbols.
struct ElfSym {
// __bss_start
static Defined *bss;
// etext and _etext
static Defined *etext1;
static Defined *etext2;
// edata and _edata
static Defined *edata1;
static Defined *edata2;
// end and _end
static Defined *end1;
static Defined *end2;
// The _GLOBAL_OFFSET_TABLE_ symbol is defined by target convention to
// be at some offset from the base of the .got section, usually 0 or
// the end of the .got.
static Defined *globalOffsetTable;
// _gp, _gp_disp and __gnu_local_gp symbols. Only for MIPS.
static Defined *mipsGp;
static Defined *mipsGpDisp;
static Defined *mipsLocalGp;
// __rel{,a}_iplt_{start,end} symbols.
static Defined *relaIpltStart;
static Defined *relaIpltEnd;
// __global_pointer$ for RISC-V.
static Defined *riscvGlobalPointer;
// _TLS_MODULE_BASE_ on targets that support TLSDESC.
static Defined *tlsModuleBase;
};
// A buffer class that is large enough to hold any Symbol-derived
// object. We allocate memory using this class and instantiate a symbol
// using the placement new.
union SymbolUnion {
alignas(Defined) char a[sizeof(Defined)];
alignas(CommonSymbol) char b[sizeof(CommonSymbol)];
alignas(Undefined) char c[sizeof(Undefined)];
alignas(SharedSymbol) char d[sizeof(SharedSymbol)];
alignas(LazyArchive) char e[sizeof(LazyArchive)];
alignas(LazyObject) char f[sizeof(LazyObject)];
ELF: New symbol table design. This patch implements a new design for the symbol table that stores SymbolBodies within a memory region of the Symbol object. Symbols are mutated by constructing SymbolBodies in place over existing SymbolBodies, rather than by mutating pointers. As mentioned in the initial proposal [1], this memory layout helps reduce the cache miss rate by improving memory locality. Performance numbers: old(s) new(s) Without debug info: chrome 7.178 6.432 (-11.5%) LLVMgold.so 0.505 0.502 (-0.5%) clang 0.954 0.827 (-15.4%) llvm-as 0.052 0.045 (-15.5%) With debug info: scylla 5.695 5.613 (-1.5%) clang 14.396 14.143 (-1.8%) Performance counter results show that the fewer required indirections is indeed the cause of the improved performance. For example, when linking chrome, stalled cycles decreases from 14,556,444,002 to 12,959,238,310, and instructions per cycle increases from 0.78 to 0.83. We are also executing many fewer instructions (15,516,401,933 down to 15,002,434,310), probably because we spend less time allocating SymbolBodies. The new mechanism by which symbols are added to the symbol table is by calling add* functions on the SymbolTable. In this patch, I handle local symbols by storing them inside "unparented" SymbolBodies. This is suboptimal, but if we do want to try to avoid allocating these SymbolBodies, we can probably do that separately. I also removed a few members from the SymbolBody class that were only being used to pass information from the input file to the symbol table. This patch implements the new design for the ELF linker only. I intend to prepare a similar patch for the COFF linker. [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098832.html Differential Revision: http://reviews.llvm.org/D19752 llvm-svn: 268178
2016-05-01 12:55:03 +08:00
};
// It is important to keep the size of SymbolUnion small for performance and
// memory usage reasons. 72 bytes is a soft limit based on the size of Defined
// on a 64-bit system.
static_assert(sizeof(SymbolUnion) <= 72, "SymbolUnion too large");
template <typename T> struct AssertSymbol {
static_assert(std::is_trivially_destructible<T>(),
"Symbol types must be trivially destructible");
static_assert(sizeof(T) <= sizeof(SymbolUnion), "SymbolUnion too small");
static_assert(alignof(T) <= alignof(SymbolUnion),
"SymbolUnion not aligned enough");
};
static inline void assertSymbols() {
AssertSymbol<Defined>();
AssertSymbol<CommonSymbol>();
AssertSymbol<Undefined>();
AssertSymbol<SharedSymbol>();
AssertSymbol<LazyArchive>();
AssertSymbol<LazyObject>();
}
void printTraceSymbol(const Symbol *sym);
size_t Symbol::getSymbolSize() const {
switch (kind()) {
case CommonKind:
return sizeof(CommonSymbol);
case DefinedKind:
return sizeof(Defined);
case LazyArchiveKind:
return sizeof(LazyArchive);
case LazyObjectKind:
return sizeof(LazyObject);
case SharedKind:
return sizeof(SharedSymbol);
case UndefinedKind:
return sizeof(Undefined);
case PlaceholderKind:
return sizeof(Symbol);
}
llvm_unreachable("unknown symbol kind");
}
// replace() replaces "this" object with a given symbol by memcpy'ing
// it over to "this". This function is called as a result of name
// resolution, e.g. to replace an undefind symbol with a defined symbol.
void Symbol::replace(const Symbol &newSym) {
using llvm::ELF::STT_TLS;
// st_value of STT_TLS represents the assigned offset, not the actual address
// which is used by STT_FUNC and STT_OBJECT. STT_TLS symbols can only be
// referenced by special TLS relocations. It is usually an error if a STT_TLS
// symbol is replaced by a non-STT_TLS symbol, vice versa. There are two
// exceptions: (a) a STT_NOTYPE lazy/undefined symbol can be replaced by a
// STT_TLS symbol, (b) a STT_TLS undefined symbol can be replaced by a
// STT_NOTYPE lazy symbol.
if (symbolKind != PlaceholderKind && !newSym.isLazy() &&
(type == STT_TLS) != (newSym.type == STT_TLS) &&
type != llvm::ELF::STT_NOTYPE)
error("TLS attribute mismatch: " + toString(*this) + "\n>>> defined in " +
toString(newSym.file) + "\n>>> defined in " + toString(file));
Symbol old = *this;
memcpy(this, &newSym, newSym.getSymbolSize());
// old may be a placeholder. The referenced fields must be initialized in
// SymbolTable::insert.
versionId = old.versionId;
visibility = old.visibility;
isUsedInRegularObj = old.isUsedInRegularObj;
exportDynamic = old.exportDynamic;
2019-08-13 17:12:52 +08:00
inDynamicList = old.inDynamicList;
canInline = old.canInline;
[ELF] Make binding (weak or non-weak) logic consistent for Undefined and SharedSymbol This is a case missed by D64136. If %t1.o has a weak reference on foo, and %t2.so has a non-weak reference on foo: ``` 0. ld.lld %t1.o %t2.so # ok; STB_WEAK; accepted since D64136 1. ld.lld %t2.so %t1.o # undefined symbol: foo; STB_GLOBAL 2. gold %t1.o %t2.so # ok; STB_WEAK 3. gold %t2.so %t1.o # undefined reference to 'foo'; STB_GLOBAL 4. ld.bfd %t1.o %t2.so # undefined reference to `foo'; STB_WEAK 5. ld.bfd %t2.so %t1.o # undefined reference to `foo'; STB_WEAK ``` It can be argued that in both cases, the binding of the undefined foo should be set to STB_WEAK, because the binding should not be affected by referenced from shared objects. --allow-shlib-undefined doesn't suppress errors (3,4,5), but -shared or --noinhibit-exec allows ld.bfd/gold to produce a binary: ``` 3. gold -shared %t2.so %t1.o # ok; STB_GLOBAL 4. ld.bfd -shared %t2.so %t1.o # ok; STB_WEAK 5. ld.bfd -shared %t1.o %t1.o # ok; STB_WEAK ``` If %t2.so has DT_NEEDED entries, ld.bfd will load them (lld/gold don't have the behavior). If one of the DSO defines foo and it is in the link-time search path (e.g. DT_NEEDED entry is an absolute path, via -rpath=, via -rpath-link=, etc), `ld.bfd %t1.o %t2.so` and `ld.bfd %t1.o %t2.so` will not error. In this patch, we make Undefined and SharedSymbol share the same binding computing logic. Case 1 will be allowed: ``` 0. ld.lld %t1.o %t2.so # ok; STB_WEAK; accepted since D64136 1. ld.lld %t2.so %t1.o # ok; STB_WEAK; changed by this patch ``` In the future, we can explore the option that turns both (0,1) into errors if --no-allow-shlib-undefined (default when linking an executable) is in action. Reviewed By: ruiu Differential Revision: https://reviews.llvm.org/D65584 llvm-svn: 368038
2019-08-06 22:03:45 +08:00
referenced = old.referenced;
traced = old.traced;
hasVersionSuffix = old.hasVersionSuffix;
isPreemptible = old.isPreemptible;
scriptDefined = old.scriptDefined;
partition = old.partition;
// Print out a log message if --trace-symbol was specified.
// This is for debugging.
if (traced)
printTraceSymbol(this);
ELF: New symbol table design. This patch implements a new design for the symbol table that stores SymbolBodies within a memory region of the Symbol object. Symbols are mutated by constructing SymbolBodies in place over existing SymbolBodies, rather than by mutating pointers. As mentioned in the initial proposal [1], this memory layout helps reduce the cache miss rate by improving memory locality. Performance numbers: old(s) new(s) Without debug info: chrome 7.178 6.432 (-11.5%) LLVMgold.so 0.505 0.502 (-0.5%) clang 0.954 0.827 (-15.4%) llvm-as 0.052 0.045 (-15.5%) With debug info: scylla 5.695 5.613 (-1.5%) clang 14.396 14.143 (-1.8%) Performance counter results show that the fewer required indirections is indeed the cause of the improved performance. For example, when linking chrome, stalled cycles decreases from 14,556,444,002 to 12,959,238,310, and instructions per cycle increases from 0.78 to 0.83. We are also executing many fewer instructions (15,516,401,933 down to 15,002,434,310), probably because we spend less time allocating SymbolBodies. The new mechanism by which symbols are added to the symbol table is by calling add* functions on the SymbolTable. In this patch, I handle local symbols by storing them inside "unparented" SymbolBodies. This is suboptimal, but if we do want to try to avoid allocating these SymbolBodies, we can probably do that separately. I also removed a few members from the SymbolBody class that were only being used to pass information from the input file to the symbol table. This patch implements the new design for the ELF linker only. I intend to prepare a similar patch for the COFF linker. [1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098832.html Differential Revision: http://reviews.llvm.org/D19752 llvm-svn: 268178
2016-05-01 12:55:03 +08:00
}
template <typename... T> Defined *makeDefined(T &&...args) {
return new (reinterpret_cast<Defined *>(
getSpecificAllocSingleton<SymbolUnion>().Allocate()))
Defined(std::forward<T>(args)...);
}
void maybeWarnUnorderableSymbol(const Symbol *sym);
bool computeIsPreemptible(const Symbol &sym);
[ELF] --warn-backrefs: don't warn for linking sandwich problems This is an alternative design to D77512. D45195 added --warn-backrefs to detect * A. certain input orders which GNU ld either errors ("undefined reference") or has different resolution semantics * B. (byproduct) some latent multiple definition problems (-ldef1 -lref -ldef2) which I call "linking sandwich problems". def2 may or may not be the same as def1. When an archive appears more than once (-ldef -lref -ldef), lld and GNU ld may have the same resolution but --warn-backrefs may warn. This is not uncommon. For example, currently lld itself has such a problem: ``` liblldCommon.a liblldCOFF.a ... liblldCommon.a _ZN3lld10DWARFCache13getDILineInfoEmm in liblldCOFF.a refers to liblldCommon.a(DWARF.cpp.o) libLLVMSupport.a also appears twice and has a similar warning ``` glibc has such problems. It is somewhat destined because of its separate libc/libpthread/... and arbitrary grouping. The situation is getting improved over time but I have seen: ``` -lc __isnanl references -lm -lc _IO_funlockfile references -lpthread ``` There are also various issues in interaction with other runtime libraries such as libgcc_eh and libunwind: ``` -lc __gcc_personality_v0 references -lgcc_eh -lpthread __gcc_personality_v0 references -lgcc_eh -lpthread _Unwind_GetCFA references -lunwind ``` These problems are actually benign. We want --warn-backrefs to focus on its main task A and defer task B (which is also useful) to a more specific future feature (see gold --detect-odr-violations and https://bugs.llvm.org/show_bug.cgi?id=43110). Instead of warning immediately, we store the message and only report it if no subsequent lazy definition exists. The use of the static variable `backrefDiags` is similar to `undefs` in Relocations.cpp Reviewed By: grimar Differential Revision: https://reviews.llvm.org/D77522
2020-04-06 13:27:46 +08:00
void reportBackrefs();
// A mapping from a symbol to an InputFile referencing it backward. Used by
// --warn-backrefs.
extern llvm::DenseMap<const Symbol *,
std::pair<const InputFile *, const InputFile *>>
backwardReferences;
[ELF] Add --why-extract= to query why archive members/lazy object files are extracted Similar to D69607 but for archive member extraction unrelated to GC. This patch adds --why-extract=. Prior art: GNU ld -M prints ``` Archive member included to satisfy reference by file (symbol) a.a(a.o) main.o (a) b.a(b.o) (b()) ``` -M is mainly for input section/symbol assignment <-> output section mapping (often huge output) and the information may appear ad-hoc. Apple ld64 ``` __Z1bv forced load of b.a(b.o) _a forced load of a.a(a.o) ``` It doesn't say the reference file. Arm's proprietary linker ``` Selecting member vsnprintf.o(c_wfu.l) to define vsnprintf. ... Loading member vsnprintf.o from c_wfu.l. definition: vsnprintf reference : _printf_a ``` --- --why-extract= gives the user the full data (which is much shorter than GNU ld -Map). It is easy to track a chain of references to one archive member with a one-liner, e.g. ``` % ld.lld main.o a_b.a b_c.a c.a -o /dev/null --why-extract=- | tee stdout reference extracted symbol main.o a_b.a(a_b.o) a a_b.a(a_b.o) b_c.a(b_c.o) b() b_c.a(b_c.o) c.a(c.o) c() % ruby -ane 'BEGIN{p={}}; p[$F[1]]=[$F[0],$F[2]] if $.>1; END{x="c.a(c.o)"; while y=p[x]; puts "#{y[0]} extracts #{x} to resolve #{y[1]}"; x=y[0] end}' stdout b_c.a(b_c.o) extracts c.a(c.o) to resolve c() a_b.a(a_b.o) extracts b_c.a(b_c.o) to resolve b() main.o extracts a_b.a(a_b.o) to resolve a ``` Archive member extraction happens before --gc-sections, so this may not be a live path under --gc-sections, but I think it is a good approximation in practice. * Specifying a file avoids output interleaving with --verbose. * Required `=` prevents accidental overwrite of an input if the user forgets `=`. (Most of compiler drivers' long options accept `=` but not ` `) Differential Revision: https://reviews.llvm.org/D109572
2021-09-21 00:52:30 +08:00
// A tuple of (reference, extractedFile, sym). Used by --why-extract=.
extern SmallVector<std::tuple<std::string, const InputFile *, const Symbol &>,
0>
whyExtract;
2016-02-28 08:25:54 +08:00
} // namespace elf
} // namespace lld
#endif