maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

[BOLT] Refactor global symbol handling code. 

Summary:
This is preparation work for static data reordering.

I've created a new class called BinaryData which represents a symbol
contained in a section.  It records almost all the information relevant
for dealing with data, e.g. names, address, size, alignment, profiling
data, etc.  BinaryContext still stores and manages BinaryData objects
similar to how it managed symbols and global addresses before.  The
interfaces are not changed too drastically from before either.  There is
a bit of overlap between BinaryData and BinaryFunction.  I would have
liked to do some more refactoring to make a BinaryFunctionFragment that
subclassed from BinaryData and then have BinaryFunction be composed or
associated with BinaryFunctionFragments.

I've also attempted to use (symbol + offset) for when addresses are
pointing into the middle of symbols with known sizes.  This changes the
simplify rodata loads optimization slightly since the expression on an
instruction can now also be a (symbol + offset) rather than just a symbol.

One of the overall goals for this refactoring is to make sure every
relocation is associated with a BinaryData object.  This requires adding
"hole" BinaryData's wherever there are gaps in a section's address space.
Most of the holes seem to be data that has no associated symbol info. In
this case we can't do any better than lumping all the adjacent hole
symbols into one big symbol (there may be more than one actual data
object that contributes to a hole). At least the combined holes should
be moveable.

Jump tables have similar issues. They appear to mostly be sub-objects
for top level local symbols. The main problem is that we can't recognize
jump tables at the time we scan the symbol table, we have to wait til
disassembly. When a jump table is discovered we add it as a sub-object
to the existing local symbol. If there are one or more existing
BinaryData's that appear in the address range of a newly created jump
table, those are added as sub-objects as well.

(cherry picked from FBD6362544)
This commit is contained in:
Bill Nell 2017-11-14 20:05:11 -08:00 committed by Maksim Panchenko
parent 32b332ad2d
commit 0e4d86bf19
29 changed files with 2275 additions and 1008 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
bolt/BinaryBasicBlock.cpp
View File

@ -78,13 +78,22 @@ bool BinaryBasicBlock::validateSuccessorInvariants() {
// Work on the assumption that jump table blocks don't
// have a conditional successor.
Valid = false;
errs() << "BOLT-WARNING: Jump table successor "
<< Succ->getName()
<< " not contained in the jump table.\n";
}
}
// If there are any leftover entries in the jump table, they
// must be one of the function end labels.
for (auto *Sym : UniqueSyms) {
Valid &= (Sym == Function->getFunctionEndLabel() ||
Sym == Function->getFunctionColdEndLabel());
if (Valid) {
for (auto *Sym : UniqueSyms) {
Valid &= (Sym == Function->getFunctionEndLabel() ||
Sym == Function->getFunctionColdEndLabel());
if (!Valid) {
errs() << "BOLT-WARNING: Jump table contains illegal entry: "
<< Sym->getName() << "\n";
}
}
}
} else {
const MCSymbol *TBB = nullptr;

392
bolt/BinaryContext.cpp
View File

@ -11,6 +11,7 @@
#include "BinaryContext.h"
#include "BinaryFunction.h"
#include "DataReader.h"
#include "llvm/ADT/Twine.h"
#include "llvm/DebugInfo/DWARF/DWARFFormValue.h"
#include "llvm/DebugInfo/DWARF/DWARFUnit.h"
@ -19,6 +20,7 @@
#include "llvm/MC/MCStreamer.h"
#include "llvm/MC/MCSymbol.h"
#include "llvm/Support/CommandLine.h"
#include <iterator>
using namespace llvm;
using namespace bolt;
@ -57,6 +59,7 @@ BinaryContext::~BinaryContext() {
for (auto *Section : Sections) {
delete Section;
}
clearBinaryData();
}
std::unique_ptr<MCObjectWriter>
@ -69,47 +72,224 @@ BinaryContext::createObjectWriter(raw_pwrite_stream &OS) {
return MAB->createObjectWriter(OS);
}
MCSymbol *BinaryContext::getOrCreateGlobalSymbol(uint64_t Address,
Twine Prefix) {
MCSymbol *Symbol{nullptr};
std::string Name;
auto NI = GlobalAddresses.find(Address);
if (NI != GlobalAddresses.end()) {
// Even though there could be multiple names registered at the address,
// we only use the first one.
Name = NI->second;
} else {
Name = (Prefix + "0x" + Twine::utohexstr(Address)).str();
assert(GlobalSymbols.find(Name) == GlobalSymbols.end() &&
"created name is not unique");
GlobalAddresses.emplace(std::make_pair(Address, Name));
bool BinaryContext::validateObjectNesting() const {
auto Itr = BinaryDataMap.begin();
auto End = BinaryDataMap.end();
bool Valid = true;
while (Itr != End) {
auto Next = std::next(Itr);
while (Next != End &&
Itr->second->getSection() == Next->second->getSection() &&
Itr->second->containsRange(Next->second->getAddress(),
Next->second->getSize())) {
if (Next->second->Parent != Itr->second) {
errs() << "BOLT-WARNING: object nesting incorrect for:\n"
<< "BOLT-WARNING: " << *Itr->second << "\n"
<< "BOLT-WARNING: " << *Next->second << "\n";
Valid = false;
}
++Next;
}
Itr = Next;
}
return Valid;
}
bool BinaryContext::validateHoles() const {
bool Valid = true;
for (auto &Section : sections()) {
for (const auto &Rel : Section.relocations()) {
auto RelAddr = Rel.Offset + Section.getAddress();
auto *BD = getBinaryDataContainingAddress(RelAddr);
if (!BD) {
errs() << "BOLT-WARNING: no BinaryData found for relocation at address"
<< " 0x" << Twine::utohexstr(RelAddr) << " in "
<< Section.getName() << "\n";
Valid = false;
} else if (!BD->getAtomicRoot()) {
errs() << "BOLT-WARNING: no atomic BinaryData found for relocation at "
<< "address 0x" << Twine::utohexstr(RelAddr) << " in "
<< Section.getName() << "\n";
Valid = false;
}
}
}
return Valid;
}
void BinaryContext::updateObjectNesting(BinaryDataMapType::iterator GAI) {
const auto Address = GAI->second->getAddress();
const auto Size = GAI->second->getSize();
auto fixParents =
[&](BinaryDataMapType::iterator Itr, BinaryData *NewParent) {
auto *OldParent = Itr->second->Parent;
Itr->second->Parent = NewParent;
++Itr;
while (Itr != BinaryDataMap.end() && OldParent &&
Itr->second->Parent == OldParent) {
Itr->second->Parent = NewParent;
++Itr;
}
};
// Check if the previous symbol contains the newly added symbol.
if (GAI != BinaryDataMap.begin()) {
auto *Prev = std::prev(GAI)->second;
while (Prev) {
if (Prev->getSection() == GAI->second->getSection() &&
Prev->containsRange(Address, Size)) {
fixParents(GAI, Prev);
} else {
fixParents(GAI, nullptr);
}
Prev = Prev->Parent;
}
}
Symbol = Ctx->lookupSymbol(Name);
if (Symbol)
return Symbol;
// Check if the newly added symbol contains any subsequent symbols.
if (Size != 0) {
auto *BD = GAI->second->Parent ? GAI->second->Parent : GAI->second;
auto Itr = std::next(GAI);
while (Itr != BinaryDataMap.end() &&
BD->containsRange(Itr->second->getAddress(),
Itr->second->getSize())) {
Itr->second->Parent = BD;
++Itr;
}
}
}
Symbol = Ctx->getOrCreateSymbol(Name);
GlobalSymbols[Name] = Address;
MCSymbol *BinaryContext::getOrCreateGlobalSymbol(uint64_t Address,
uint64_t Size,
uint16_t Alignment,
Twine Prefix) {
auto Itr = BinaryDataMap.find(Address);
if (Itr != BinaryDataMap.end()) {
assert(Itr->second->getSize() == Size || !Size);
return Itr->second->getSymbol();
}
std::string Name = (Prefix + "0x" + Twine::utohexstr(Address)).str();
assert(!GlobalSymbols.count(Name) && "created name is not unique");
return registerNameAtAddress(Name, Address, Size, Alignment);
}
MCSymbol *BinaryContext::registerNameAtAddress(StringRef Name,
uint64_t Address,
uint64_t Size,
uint16_t Alignment) {
auto SectionOrErr = getSectionForAddress(Address);
auto &Section = SectionOrErr ? SectionOrErr.get() : absoluteSection();
auto GAI = BinaryDataMap.find(Address);
BinaryData *BD;
if (GAI == BinaryDataMap.end()) {
BD = new BinaryData(Name,
Address,
Size,
Alignment ? Alignment : 1,
Section);
} else {
BD = GAI->second;
}
return registerNameAtAddress(Name, Address, BD);
}
MCSymbol *BinaryContext::registerNameAtAddress(StringRef Name,
uint64_t Address,
BinaryData *BD) {
auto GAI = BinaryDataMap.find(Address);
if (GAI != BinaryDataMap.end()) {
if (BD != GAI->second) {
// Note: this could be a source of bugs if client code holds
// on to BinaryData*'s in data structures for any length of time.
auto *OldBD = GAI->second;
BD->merge(GAI->second);
delete OldBD;
GAI->second = BD;
for (auto &Name : BD->names()) {
GlobalSymbols[Name] = BD;
}
updateObjectNesting(GAI);
} else if (!GAI->second->hasName(Name)) {
GAI->second->Names.push_back(Name);
GlobalSymbols[Name] = GAI->second;
}
BD = nullptr;
} else {
GAI = BinaryDataMap.emplace(Address, BD).first;
GlobalSymbols[Name] = BD;
updateObjectNesting(GAI);
}
// Register the name with MCContext.
auto *Symbol = Ctx->getOrCreateSymbol(Name);
if (BD) {
BD->Symbols.push_back(Symbol);
assert(BD->Symbols.size() == BD->Names.size());
}
return Symbol;
}
MCSymbol *BinaryContext::getGlobalSymbolAtAddress(uint64_t Address) const {
auto NI = GlobalAddresses.find(Address);
if (NI == GlobalAddresses.end())
return nullptr;
const BinaryData *
BinaryContext::getBinaryDataContainingAddressImpl(uint64_t Address,
bool IncludeEnd,
bool BestFit) const {
auto NI = BinaryDataMap.lower_bound(Address);
auto End = BinaryDataMap.end();
if ((NI != End && Address == NI->first) ||
(NI-- != BinaryDataMap.begin())) {
if (NI->second->containsAddress(Address) ||
(IncludeEnd && NI->second->getEndAddress() == Address)) {
while (BestFit &&
std::next(NI) != End &&
(std::next(NI)->second->containsAddress(Address) ||
(IncludeEnd && std::next(NI)->second->getEndAddress() == Address))) {
++NI;
}
return NI->second;
}
auto *Symbol = Ctx->lookupSymbol(NI->second);
assert(Symbol && "symbol cannot be NULL at this point");
return Symbol;
// If this is a sub-symbol, see if a parent data contains the address.
auto *BD = NI->second->getParent();
while (BD) {
if (BD->containsAddress(Address) ||
(IncludeEnd && NI->second->getEndAddress() == Address))
return BD;
BD = BD->getParent();
}
}
return nullptr;
}
MCSymbol *BinaryContext::getGlobalSymbolByName(const std::string &Name) const {
auto Itr = GlobalSymbols.find(Name);
return Itr == GlobalSymbols.end()
? nullptr : getGlobalSymbolAtAddress(Itr->second);
bool BinaryContext::setBinaryDataSize(uint64_t Address, uint64_t Size) {
auto NI = BinaryDataMap.find(Address);
assert(NI != BinaryDataMap.end());
if (NI == BinaryDataMap.end())
return false;
assert(!NI->second->Size || NI->second->Size == Size);
NI->second->Size = Size;
updateObjectNesting(NI);
return true;
}
void BinaryContext::postProcessSymbolTable() {
fixBinaryDataHoles();
bool Valid = true;
for (auto &Entry : BinaryDataMap) {
auto *BD = Entry.second;
if ((BD->getName().startswith("SYMBOLat") ||
BD->getName().startswith("DATAat")) &&
!BD->getParent() &&
!BD->getSize() &&
!BD->isAbsolute() &&
BD->getSection()) {
outs() << "BOLT-WARNING: zero sized top level symbol: " << *BD << "\n";
Valid = false;
}
}
assert(Valid);
assignMemData();
}
void BinaryContext::foldFunction(BinaryFunction &ChildBF,
@ -126,7 +306,7 @@ void BinaryContext::foldFunction(BinaryFunction &ChildBF,
assert(Symbol && "symbol cannot be NULL at this point");
SymbolToFunctionMap[Symbol] = &ParentBF;
// NB: there's no need to update GlobalAddresses and GlobalSymbols.
// NB: there's no need to update BinaryDataMap and GlobalSymbols.
}
// Merge execution counts of ChildBF into those of ParentBF.
@ -148,10 +328,138 @@ void BinaryContext::foldFunction(BinaryFunction &ChildBF,
}
}
void BinaryContext::fixBinaryDataHoles() {
assert(validateObjectNesting() && "object nesting inconsitency detected");
for (auto &Section : allocatableSections()) {
std::vector<std::pair<uint64_t, uint64_t>> Holes;
auto isNotHole = [&Section](const binary_data_iterator &Itr) {
auto *BD = Itr->second;
bool isHole = (!BD->getParent() &&
!BD->getSize() &&
BD->isObject() &&
(BD->getName().startswith("SYMBOLat0x") ||
BD->getName().startswith("DATAat0x") ||
BD->getName().startswith("ANONYMOUS")));
return !isHole && BD->getSection() == Section && !BD->getParent();
};
auto BDStart = BinaryDataMap.begin();
auto BDEnd = BinaryDataMap.end();
auto Itr = FilteredBinaryDataIterator(isNotHole, BDStart, BDEnd);
auto End = FilteredBinaryDataIterator(isNotHole, BDEnd, BDEnd);
uint64_t EndAddress = Section.getAddress();
while (Itr != End) {
auto Gap = Itr->second->getAddress() - EndAddress;
if (Gap > 0) {
assert(EndAddress < Itr->second->getAddress());
Holes.push_back(std::make_pair(EndAddress, Gap));
}
EndAddress = Itr->second->getEndAddress();
++Itr;
}
if (EndAddress < Section.getEndAddress()) {
Holes.push_back(std::make_pair(EndAddress,
Section.getEndAddress() - EndAddress));
}
// If there is already a symbol at the start of the hole, grow that symbol
// to cover the rest. Otherwise, create a new symbol to cover the hole.
for (auto &Hole : Holes) {
auto *BD = getBinaryDataAtAddress(Hole.first);
if (BD) {
// BD->getSection() can be != Section if there are sections that
// overlap. In this case it is probably safe to just skip the holes
// since the overlapping section will not(?) have any symbols in it.
if (BD->getSection() == Section)
setBinaryDataSize(Hole.first, Hole.second);
} else {
getOrCreateGlobalSymbol(Hole.first, Hole.second, 1, "HOLEat");
}
}
}
assert(validateObjectNesting() && "object nesting inconsitency detected");
assert(validateHoles() && "top level hole detected in object map");
}
void BinaryContext::printGlobalSymbols(raw_ostream& OS) const {
for (auto &Entry : GlobalSymbols) {
OS << "(" << Entry.first << " -> 0x"
<< Twine::utohexstr(Entry.second) << ")\n";
const BinarySection* CurrentSection = nullptr;
bool FirstSection = true;
for (auto &Entry : BinaryDataMap) {
const auto *BD = Entry.second;
const auto &Section = BD->getSection();
if (FirstSection || Section != *CurrentSection) {
uint64_t Address, Size;
StringRef Name = Section.getName();
if (Section) {
Address = Section.getAddress();
Size = Section.getSize();
} else {
Address = BD->getAddress();
Size = BD->getSize();
}
OS << "BOLT-INFO: Section " << Name << ", "
<< "0x" + Twine::utohexstr(Address) << ":"
<< "0x" + Twine::utohexstr(Address + Size) << "/"
<< Size << "\n";
CurrentSection = &Section;
FirstSection = false;
}
OS << "BOLT-INFO: ";
auto *P = BD->getParent();
while (P) {
OS << " ";
P = P->getParent();
}
OS << *BD << "\n";
}
}
void BinaryContext::assignMemData() {
auto getAddress = [&](const MemInfo &MI) {
if (!MI.Addr.IsSymbol)
return MI.Addr.Offset;
if (auto *BD = getBinaryDataByName(MI.Addr.Name))
return BD->getAddress() + MI.Addr.Offset;
return 0ul;
};
// Map of sections (or heap/stack) to count/size.
std::map<StringRef, uint64_t> Counts;
uint64_t TotalCount = 0;
for (auto &Entry : DR.getAllFuncsMemData()) {
for (auto &MI : Entry.second.Data) {
const auto Addr = getAddress(MI);
auto *BD = getBinaryDataContainingAddress(Addr);
if (BD) {
BD->getAtomicRoot()->addMemData(MI);
Counts[BD->getSectionName()] += MI.Count;
} else {
Counts["Heap/stack"] += MI.Count;
}
TotalCount += MI.Count;
}
}
if (!Counts.empty()) {
outs() << "BOLT-INFO: Memory stats breakdown:\n";
for (auto &Entry : Counts) {
const auto Section = Entry.first;
const auto Count = Entry.second;
outs() << "BOLT-INFO: " << Section << " = " << Count
<< format(" (%.1f%%)\n", 100.0*Count/TotalCount);
}
outs() << "BOLT-INFO: Total memory events: " << TotalCount << "\n";
}
}
@ -484,6 +792,14 @@ BinaryContext::getSectionForAddress(uint64_t Address) const {
return std::make_error_code(std::errc::bad_address);
}
ErrorOr<StringRef>
BinaryContext::getSectionNameForAddress(uint64_t Address) const {
if (auto Section = getSectionForAddress(Address)) {
return Section->getName();
}
return std::make_error_code(std::errc::bad_address);
}
BinarySection &BinaryContext::registerSection(BinarySection *Section) {
assert(!Section->getName().empty() &&
"can't register sections without a name");
@ -562,6 +878,12 @@ void BinaryContext::printSections(raw_ostream &OS) const {
}
}
BinarySection &BinaryContext::absoluteSection() {
if (auto Section = getUniqueSectionByName("<absolute>"))
return *Section;
return registerOrUpdateSection("<absolute>", ELF::SHT_NULL, 0u);
}
ErrorOr<uint64_t>
BinaryContext::extractPointerAtAddress(uint64_t Address) const {
auto Section = getSectionForAddress(Address);

316
bolt/BinaryContext.h
View File

@ -14,6 +14,7 @@
#ifndef LLVM_TOOLS_LLVM_BOLT_BINARY_CONTEXT_H
#define LLVM_TOOLS_LLVM_BOLT_BINARY_CONTEXT_H
#include "BinaryData.h"
#include "BinarySection.h"
#include "DebugData.h"
#include "llvm/ADT/iterator.h"
@ -55,6 +56,50 @@ namespace bolt {
class BinaryFunction;
class DataReader;
/// Filter iterator.
template <typename ItrType,
typename PredType = std::function<bool (const ItrType &)>>
class FilterIterator
: public std::iterator<std::bidirectional_iterator_tag,
typename std::iterator_traits<ItrType>::value_type> {
using Iterator = FilterIterator;
using T = typename std::iterator_traits<ItrType>::reference;
using PointerT = typename std::iterator_traits<ItrType>::pointer;
PredType Pred;
ItrType Itr, End;
void prev() {
while (!Pred(--Itr))
;
}
void next() {
++Itr;
nextMatching();
}
void nextMatching() {
while (Itr != End && !Pred(Itr))
++Itr;
}
public:
Iterator &operator++() { next(); return *this; }
Iterator &operator--() { prev(); return *this; }
Iterator operator++(int) { auto Tmp(Itr); next(); return Tmp; }
Iterator operator--(int) { auto Tmp(Itr); prev(); return Tmp; }
bool operator==(const Iterator& Other) const {
return Itr == Other.Itr;
}
bool operator!=(const Iterator& Other) const {
return !operator==(Other);
}
T operator*() { return *Itr; }
PointerT operator->() { return &operator*(); }
FilterIterator(PredType Pred, ItrType Itr, ItrType End)
: Pred(Pred), Itr(Itr), End(End) {
nextMatching();
}
};
class BinaryContext {
BinaryContext() = delete;
@ -70,6 +115,9 @@ class BinaryContext {
using SectionIterator = pointee_iterator<SectionSetType::iterator>;
using SectionConstIterator = pointee_iterator<SectionSetType::const_iterator>;
using FilteredSectionIterator = FilterIterator<SectionIterator>;
using FilteredSectionConstIterator = FilterIterator<SectionConstIterator>;
/// Map virtual address to a section. It is possible to have more than one
/// section mapped to the same address, e.g. non-allocatable sections.
using AddressToSectionMapType = std::multimap<uint64_t, BinarySection *>;
@ -84,13 +132,24 @@ class BinaryContext {
BinarySection &registerSection(BinarySection *Section);
public:
/// [name] -> [address] map used for global symbol resolution.
typedef std::map<std::string, uint64_t> SymbolMapType;
/// [name] -> [BinaryData*] map used for global symbol resolution.
using SymbolMapType = std::map<std::string, BinaryData *>;
SymbolMapType GlobalSymbols;
/// [address] -> [name1], [name2], ...
/// Global addresses never change.
std::multimap<uint64_t, std::string> GlobalAddresses;
/// [address] -> [BinaryData], ...
/// Addresses never change.
/// Note: it is important that clients do not hold on to instances of
/// BinaryData* while the map is still being modified during BinaryFunction
/// disassembly. This is because of the possibility that a regular
/// BinaryData is later discovered to be a JumpTable.
using BinaryDataMapType = std::map<uint64_t, BinaryData *>;
using binary_data_iterator = BinaryDataMapType::iterator;
using binary_data_const_iterator = BinaryDataMapType::const_iterator;
BinaryDataMapType BinaryDataMap;
using FilteredBinaryDataConstIterator =
FilterIterator<binary_data_const_iterator>;
using FilteredBinaryDataIterator = FilterIterator<binary_data_iterator>;
/// [MCSymbol] -> [BinaryFunction]
///
@ -99,6 +158,38 @@ public:
std::unordered_map<const MCSymbol *,
BinaryFunction *> SymbolToFunctionMap;
/// Look up the symbol entry that contains the given \p Address (based on
/// the start address and size for each symbol). Returns a pointer to
/// the BinaryData for that symbol. If no data is found, nullptr is returned.
const BinaryData *getBinaryDataContainingAddressImpl(uint64_t Address,
bool IncludeEnd,
bool BestFit) const;
/// Update the Parent fields in BinaryDatas after adding a new entry into
/// \p BinaryDataMap.
void updateObjectNesting(BinaryDataMapType::iterator GAI);
/// Validate that if object address ranges overlap that the object with
/// the larger range is a parent of the object with the smaller range.
bool validateObjectNesting() const;
/// Validate that there are no top level "holes" in each section
/// and that all relocations with a section are mapped to a valid
/// top level BinaryData.
bool validateHoles() const;
/// Get a bogus "absolute" section that will be associated with all
/// absolute BinaryDatas.
BinarySection &absoluteSection();
/// Process "holes" in between known BinaryData objects. For now,
/// symbols are padded with the space before the next BinaryData object.
void fixBinaryDataHoles();
/// Populate \p GlobalMemData. This should be done after all symbol discovery
/// is complete, e.g. after building CFGs for all functions.
void assignMemData();
public:
/// Map address to a constant island owner (constant data in code section)
std::map<uint64_t, BinaryFunction *> AddressToConstantIslandMap;
@ -204,28 +295,122 @@ public:
std::unique_ptr<MCObjectWriter> createObjectWriter(raw_pwrite_stream &OS);
/// Return a global symbol registered at a given \p Address. If no symbol
/// exists, create one with unique name using \p Prefix.
/// If there are multiple symbols registered at the \p Address, then
/// return the first one.
MCSymbol *getOrCreateGlobalSymbol(uint64_t Address, Twine Prefix);
/// Return MCSymbol registered at a given \p Address or nullptr if no
/// global symbol was registered at the location.
MCSymbol *getGlobalSymbolAtAddress(uint64_t Address) const;
/// Find the address of the global symbol with the given \p Name.
/// return an error if no such symbol exists.
ErrorOr<uint64_t> getAddressForGlobalSymbol(StringRef Name) const {
auto Itr = GlobalSymbols.find(Name);
if (Itr != GlobalSymbols.end())
return Itr->second;
return std::make_error_code(std::errc::bad_address);
/// Iterate over all BinaryData.
iterator_range<binary_data_const_iterator> getBinaryData() const {
return make_range(BinaryDataMap.begin(), BinaryDataMap.end());
}
/// Return MCSymbol for the given \p Name or nullptr if no
/// Iterate over all BinaryData.
iterator_range<binary_data_iterator> getBinaryData() {
return make_range(BinaryDataMap.begin(), BinaryDataMap.end());
}
/// Iterate over all BinaryData associated with the given \p Section.
iterator_range<FilteredBinaryDataConstIterator>
getBinaryDataForSection(StringRef SectionName) const {
auto Begin = BinaryDataMap.begin();
auto End = BinaryDataMap.end();
auto pred =
[&SectionName](const binary_data_const_iterator &Itr) -> bool {
return Itr->second->getSection().getName() == SectionName;
};
return make_range(FilteredBinaryDataConstIterator(pred, Begin, End),
FilteredBinaryDataConstIterator(pred, End, End));
}
/// Iterate over all BinaryData associated with the given \p Section.
iterator_range<FilteredBinaryDataIterator>
getBinaryDataForSection(StringRef SectionName) {
auto Begin = BinaryDataMap.begin();
auto End = BinaryDataMap.end();
auto pred = [&SectionName](const binary_data_iterator &Itr) -> bool {
return Itr->second->getSection().getName() == SectionName;
};
return make_range(FilteredBinaryDataIterator(pred, Begin, End),
FilteredBinaryDataIterator(pred, End, End));
}
/// Clear the global symbol address -> name(s) map.
void clearBinaryData() {
GlobalSymbols.clear();
for (auto &Entry : BinaryDataMap) {
delete Entry.second;
}
BinaryDataMap.clear();
}
/// Return a global symbol registered at a given \p Address and \p Size.
/// If no symbol exists, create one with unique name using \p Prefix.
/// If there are multiple symbols registered at the \p Address, then
/// return the first one.
MCSymbol *getOrCreateGlobalSymbol(uint64_t Address,
uint64_t Size,
uint16_t Alignment,
Twine Prefix);
/// Register a symbol with \p Name at a given \p Address and \p Size.
MCSymbol *registerNameAtAddress(StringRef Name,
uint64_t Address,
BinaryData* BD);
/// Register a symbol with \p Name at a given \p Address and \p Size.
MCSymbol *registerNameAtAddress(StringRef Name,
uint64_t Address,
uint64_t Size,
uint16_t Alignment);
/// Return BinaryData registered at a given \p Address or nullptr if no
/// global symbol was registered at the location.
const BinaryData *getBinaryDataAtAddress(uint64_t Address) const {
auto NI = BinaryDataMap.find(Address);
return NI != BinaryDataMap.end() ? NI->second : nullptr;
}
BinaryData *getBinaryDataAtAddress(uint64_t Address) {
auto NI = BinaryDataMap.find(Address);
return NI != BinaryDataMap.end() ? NI->second : nullptr;
}
/// Look up the symbol entry that contains the given \p Address (based on
/// the start address and size for each symbol). Returns a pointer to
/// the BinaryData for that symbol. If no data is found, nullptr is returned.
const BinaryData *getBinaryDataContainingAddress(uint64_t Address,
bool IncludeEnd = false,
bool BestFit = false) const {
return getBinaryDataContainingAddressImpl(Address, IncludeEnd, BestFit);
}
BinaryData *getBinaryDataContainingAddress(uint64_t Address,
bool IncludeEnd = false,
bool BestFit = false) {
return const_cast<BinaryData *>(getBinaryDataContainingAddressImpl(Address,
IncludeEnd,
BestFit));
}
/// Return BinaryData for the given \p Name or nullptr if no
/// global symbol with that name exists.
MCSymbol *getGlobalSymbolByName(const std::string &Name) const;
const BinaryData *getBinaryDataByName(StringRef Name) const {
auto Itr = GlobalSymbols.find(Name);
return Itr != GlobalSymbols.end() ? Itr->second : nullptr;
}
BinaryData *getBinaryDataByName(StringRef Name) {
auto Itr = GlobalSymbols.find(Name);
return Itr != GlobalSymbols.end() ? Itr->second : nullptr;
}
/// Perform any necessary post processing on the symbol table after
/// function disassembly is complete. This processing fixes top
/// level data holes and makes sure the symbol table is valid.
/// It also assigns all memory profiling info to the appropriate
/// BinaryData objects.
void postProcessSymbolTable();
/// Set the size of the global symbol located at \p Address. Return
/// false if no symbol exists, true otherwise.
bool setBinaryDataSize(uint64_t Address, uint64_t Size);
/// Print the global symbol table.
void printGlobalSymbols(raw_ostream& OS) const;
@ -269,15 +454,62 @@ public:
bool deregisterSection(BinarySection &Section);
/// Iterate over all registered sections.
iterator_range<SectionIterator> sections() {
return make_range(Sections.begin(), Sections.end());
iterator_range<FilteredSectionIterator> sections() {
auto notNull = [](const SectionIterator &Itr) {
return (bool)*Itr;
};
return make_range(FilteredSectionIterator(notNull,
Sections.begin(),
Sections.end()),
FilteredSectionIterator(notNull,
Sections.end(),
Sections.end()));
}
/// Iterate over all registered sections.
iterator_range<SectionConstIterator> sections() const {
return make_range(Sections.begin(), Sections.end());
iterator_range<FilteredSectionConstIterator> sections() const {
return const_cast<BinaryContext *>(this)->sections();
}
/// Iterate over all registered allocatable sections.
iterator_range<FilteredSectionIterator> allocatableSections() {
auto isAllocatable = [](const SectionIterator &Itr) {
return *Itr && Itr->isAllocatable();
};
return make_range(FilteredSectionIterator(isAllocatable,
Sections.begin(),
Sections.end()),
FilteredSectionIterator(isAllocatable,
Sections.end(),
Sections.end()));
}
/// Iterate over all registered allocatable sections.
iterator_range<FilteredSectionConstIterator> allocatableSections() const {
return const_cast<BinaryContext *>(this)->allocatableSections();
}
/// Iterate over all registered non-allocatable sections.
iterator_range<FilteredSectionIterator> nonAllocatableSections() {
auto notAllocated = [](const SectionIterator &Itr) {
return *Itr && !Itr->isAllocatable();
};
return make_range(FilteredSectionIterator(notAllocated,
Sections.begin(),
Sections.end()),
FilteredSectionIterator(notAllocated,
Sections.end(),
Sections.end()));
}
/// Iterate over all registered non-allocatable sections.
iterator_range<FilteredSectionConstIterator> nonAllocatableSections() const {
return const_cast<BinaryContext *>(this)->nonAllocatableSections();
}
/// Return section name containing the given \p Address.
ErrorOr<StringRef> getSectionNameForAddress(uint64_t Address) const;
/// Print all sections.
void printSections(raw_ostream& OS) const;
@ -321,28 +553,6 @@ public:
/// the binary.
ErrorOr<uint64_t> extractPointerAtAddress(uint64_t Address) const;
/// Register a symbol with \p Name at a given \p Address.
MCSymbol *registerNameAtAddress(const std::string &Name, uint64_t Address) {
// Check if the Name was already registered.
const auto GSI = GlobalSymbols.find(Name);
if (GSI != GlobalSymbols.end()) {
assert(GSI->second == Address && "addresses do not match");
auto *Symbol = Ctx->lookupSymbol(Name);
assert(Symbol && "symbol should be registered with MCContext");
return Symbol;
}
// Add the name to global symbols map.
GlobalSymbols[Name] = Address;
// Add to the reverse map. There could multiple names at the same address.
GlobalAddresses.emplace(std::make_pair(Address, Name));
// Register the name with MCContext.
return Ctx->getOrCreateSymbol(Name);
}
/// Replaces all references to \p ChildBF with \p ParentBF. \p ChildBF is then
/// removed from the list of functions \p BFs. The profile data of \p ChildBF
/// is merged into that of \p ParentBF.
@ -371,6 +581,12 @@ public:
return BFI == SymbolToFunctionMap.end() ? nullptr : BFI->second;
}
/// Associate the symbol \p Sym with the function \p BF for lookups with
/// getFunctionForSymbol().
void setSymbolToFunctionMap(const MCSymbol *Sym, BinaryFunction *BF) {
SymbolToFunctionMap[Sym] = BF;
}
/// Populate some internal data structures with debug info.
void preprocessDebugInfo(
std::map<uint64_t, BinaryFunction> &BinaryFunctions);

132
bolt/BinaryData.cpp Normal file
View File

@ -0,0 +1,132 @@
//===--- BinaryData.cpp - Representation of section data objects ----------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
//===----------------------------------------------------------------------===//
#include "BinaryData.h"
#include "BinarySection.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Regex.h"
using namespace llvm;
using namespace bolt;
#undef DEBUG_TYPE
#define DEBUG_TYPE "bolt"
namespace opts {
extern cl::OptionCategory BoltCategory;
extern cl::opt<unsigned> Verbosity;
cl::opt<bool>
PrintSymbolAliases("print-aliases",
cl::desc("print aliases when printing objects"),
cl::Hidden,
cl::ZeroOrMore,
cl::cat(BoltCategory));
}
bool BinaryData::isMoveable() const {
return (!isAbsolute() &&
(IsMoveable &&
(!Parent || isTopLevelJumpTable())));
}
void BinaryData::merge(const BinaryData *Other) {
assert(!Size || !Other->Size || Size == Other->Size);
assert(Address == Other->Address);
assert(*Section == *Other->Section);
assert(OutputOffset == Other->OutputOffset);
assert(OutputSection == Other->OutputSection);
Names.insert(Names.end(), Other->Names.begin(), Other->Names.end());
Symbols.insert(Symbols.end(), Other->Symbols.begin(), Other->Symbols.end());
MemData.insert(MemData.end(), Other->MemData.begin(), Other->MemData.end());
if (!Size)
Size = Other->Size;
}
bool BinaryData::hasNameRegex(StringRef NameRegex) const {
Regex MatchName(NameRegex);
for (auto &Name : Names)
if (MatchName.match(Name))
return true;
return false;
}
StringRef BinaryData::getSectionName() const {
return getSection().getName();
}
uint64_t BinaryData::computeOutputOffset() const {
return Address - getSection().getAddress();
}
void BinaryData::setSection(BinarySection &NewSection) {
Section = &NewSection;
if (OutputSection.empty())
OutputSection = getSection().getName();
}
bool BinaryData::isMoved() const {
return (computeOutputOffset() != OutputOffset ||
OutputSection != getSectionName());
}
void BinaryData::print(raw_ostream &OS) const {
printBrief(OS);
}
void BinaryData::printBrief(raw_ostream &OS) const {
OS << "(";
if (isJumpTable())
OS << "jump-table: ";
else
OS << "object: ";
OS << getName();
if ((opts::PrintSymbolAliases || opts::Verbosity > 1) && Names.size() > 1) {
OS << ", aliases:";
for (unsigned I = 1u; I < Names.size(); ++I) {
OS << (I == 1 ? " (" : ", ") << Names[I];
}
OS << ")";
}
if (opts::Verbosity > 1 && Parent) {
OS << " (" << Parent->getName() << "/" << Parent->getSize() << ")";
}
OS << ", 0x" << Twine::utohexstr(getAddress())
<< ":0x" << Twine::utohexstr(getEndAddress())
<< "/" << getSize();
if (opts::Verbosity > 1) {
for (auto &MI : memData()) {
OS << ", " << MI;
}
}
OS << ")";
}
BinaryData::BinaryData(StringRef Name,
uint64_t Address,
uint64_t Size,
uint16_t Alignment,
BinarySection &Section)
: Names({Name}),
Section(&Section),
Address(Address),
Size(Size),
Alignment(Alignment),
OutputSection(Section.getName()),
OutputOffset(computeOutputOffset())
{ }

207
bolt/BinaryData.h Normal file
View File

@ -0,0 +1,207 @@
//===--- BinaryData.h - Representation of section data objects -----------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_TOOLS_LLVM_BOLT_BINARY_DATA_H
#define LLVM_TOOLS_LLVM_BOLT_BINARY_DATA_H
#include "DataReader.h"
#include "llvm/MC/MCSymbol.h"
#include "llvm/Support/raw_ostream.h"
#include <algorithm>
#include <string>
#include <vector>
namespace llvm {
namespace bolt {
struct BinarySection;
/// \p BinaryData represents an indivisible part of a data section section.
/// BinaryData's may contain sub-components, e.g. jump tables but they are
/// considered to be part of the parent symbol in terms of divisibility and
/// reordering.
class BinaryData {
friend class BinaryContext;
/// Non-null if this BinaryData is contained in a larger BinaryData object,
/// i.e. the start and end addresses are contained within another object.
BinaryData *Parent{nullptr};
// non-copyable
BinaryData() = delete;
BinaryData(const BinaryData &) = delete;
BinaryData &operator=(const BinaryData &) = delete;
protected:
/// All names associated with this data. The first name is the primary one.
std::vector<std::string> Names;
/// All symbols associated with this data. This vector should have one entry
/// corresponding to every entry in \p Names.
std::vector<MCSymbol *> Symbols;
/// Section this data belongs to.
BinarySection *Section;
/// Start address of this symbol.
uint64_t Address{0};
/// Size of this data (can be 0).
uint64_t Size{0};
/// Alignment of this data.
uint16_t Alignment{1};
/// Output section for this data if it has been moved from the original
/// section.
std::string OutputSection;
/// The offset of this symbol in the output section. This is different
/// from \p Address - Section.getAddress() when the data has been reordered.
uint64_t OutputOffset{0};
/// Memory profiling data associated with this object.
std::vector<MemInfo> MemData;
bool IsMoveable{true};
void addMemData(const MemInfo &MI) {
MemData.push_back(MI);
}
BinaryData *getRootData() {
auto *BD = this;
while (BD->Parent)
BD = BD->Parent;
return BD;
}
BinaryData *getAtomicRoot() {
auto *BD = this;
while (!BD->isAtomic() && BD->Parent)
BD = BD->Parent;
return BD;
}
uint64_t computeOutputOffset() const;
public:
BinaryData(BinaryData &&) = default;
BinaryData(StringRef Name,
uint64_t Address,
uint64_t Size,
uint16_t Alignment,
BinarySection &Section);
virtual ~BinaryData() { }
virtual bool isJumpTable() const { return false; }
virtual bool isObject() const { return !isJumpTable(); }
virtual void merge(const BinaryData *Other);
bool isTopLevelJumpTable() const {
return (isJumpTable() &&
(!Parent || (!Parent->Parent && Parent->isObject())));
}
// BinaryData that is considered atomic and potentially moveable. All
// MemInfo data and relocations should be wrt. to atomic data.
bool isAtomic() const {
return isTopLevelJumpTable() || !Parent;
}
iterator_range<std::vector<std::string>::const_iterator> names() const {
return make_range(Names.begin(), Names.end());
}
iterator_range<std::vector<MCSymbol *>::const_iterator> symbols() const {
return make_range(Symbols.begin(), Symbols.end());
}
iterator_range<std::vector<MemInfo>::const_iterator> memData() const {
return make_range(MemData.begin(), MemData.end());
}
StringRef getName() const { return Names.front(); }
const std::vector<std::string> &getNames() const { return Names; }
MCSymbol *getSymbol() { return Symbols.front(); }
const MCSymbol *getSymbol() const { return Symbols.front(); }
bool hasName(StringRef Name) const {
return std::find(Names.begin(), Names.end(), Name) != Names.end();
}
bool hasNameRegex(StringRef Name) const;
bool nameStartsWith(StringRef Prefix) const {
for (const auto &Name : Names) {
if (StringRef(Name).startswith(Prefix))
return true;
}
return false;
}
bool hasSymbol(const MCSymbol *Symbol) const {
return std::find(Symbols.begin(), Symbols.end(), Symbol) != Symbols.end();
}
bool isAbsolute() const { return getSymbol()->isAbsolute(); }
bool isMoveable() const;
uint64_t getAddress() const { return Address; }
uint64_t getEndAddress() const { return Address + Size; }
uint64_t getSize() const { return Size; }
uint16_t getAlignment() const { return Alignment; }
uint64_t getOutputOffset() const { return OutputOffset; }
uint64_t getOutputSize() const { return Size; }
BinarySection &getSection() { return *Section; }
const BinarySection &getSection() const { return *Section; }
StringRef getSectionName() const;
StringRef getOutputSection() const { return OutputSection; }
bool isMoved() const;
bool containsAddress(uint64_t Address) const {
return ((getAddress() <= Address && Address < getEndAddress()) ||
(getAddress() == Address && !getSize()));
}
bool containsRange(uint64_t Address, uint64_t Size) const {
return (getAddress() <= Address && Address + Size <= getEndAddress());
}
const BinaryData *getParent() const {
return Parent;
}
const BinaryData *getRootData() const {
auto *BD = this;
while (BD->Parent)
BD = BD->Parent;
return BD;
}
const BinaryData *getAtomicRoot() const {
auto *BD = this;
while (!BD->isAtomic() && BD->Parent)
BD = BD->Parent;
return BD;
}
void setIsMoveable(bool Flag) { IsMoveable = Flag; }
void setOutputOffset(uint64_t Offset) { OutputOffset = Offset; }
void setOutputSection(StringRef Name) { OutputSection = Name; }
void setSection(BinarySection &NewSection);
virtual void printBrief(raw_ostream &OS) const;
virtual void print(raw_ostream &OS) const;
};
inline raw_ostream &operator<<(raw_ostream &OS, const BinaryData &BD) {
BD.printBrief(OS);
return OS;
}
} // namespace bolt
} // namespace llvm
#endif

374
bolt/BinaryFunction.cpp
View File

@ -129,7 +129,7 @@ PrintOnlyRegex("print-only-regex",
cl::Hidden,
cl::cat(BoltCategory));
cl::opt<bool>
static cl::opt<bool>
TimeBuild("time-build",
cl::desc("print time spent constructing binary functions"),
cl::ZeroOrMore,
@ -364,9 +364,9 @@ bool BinaryFunction::isForwardCall(const MCSymbol *CalleeSymbol) const {
}
} else {
// Absolute symbol.
auto const CalleeSI = BC.GlobalSymbols.find(CalleeSymbol->getName());
assert(CalleeSI != BC.GlobalSymbols.end() && "unregistered symbol found");
return CalleeSI->second > getAddress();
auto *CalleeSI = BC.getBinaryDataByName(CalleeSymbol->getName());
assert(CalleeSI && "unregistered symbol found");
return CalleeSI->getAddress() > getAddress();
}
}
@ -563,7 +563,7 @@ void BinaryFunction::print(raw_ostream &OS, std::string Annotation,
// Print all jump tables.
for (auto &JTI : JumpTables) {
JTI.second.print(OS);
JTI.second->print(OS);
}
OS << "DWARF CFI Instructions:\n";
@ -675,9 +675,8 @@ IndirectBranchType BinaryFunction::processIndirectBranch(MCInst &Instruction,
if (BC.TheTriple->getArch() == llvm::Triple::aarch64) {
const auto *Sym = BC.MIA->getTargetSymbol(*PCRelBaseInstr, 1);
assert (Sym && "Symbol extraction failed");
auto SI = BC.GlobalSymbols.find(Sym->getName());
if (SI != BC.GlobalSymbols.end()) {
PCRelAddr = SI->second;
if (auto *BD = BC.getBinaryDataByName(Sym->getName())) {
PCRelAddr = BD->getAddress();
} else {
for (auto &Elmt : Labels) {
if (Elmt.second == Sym) {
@ -708,10 +707,12 @@ IndirectBranchType BinaryFunction::processIndirectBranch(MCInst &Instruction,
// RIP-relative addressing should be converted to symbol form by now
// in processed instructions (but not in jump).
if (DispExpr) {
auto SI =
BC.GlobalSymbols.find(BC.MIA->getTargetSymbol(DispExpr)->getName());
assert(SI != BC.GlobalSymbols.end() && "global symbol needs a value");
ArrayStart = SI->second;
const MCSymbol *TargetSym;
uint64_t TargetOffset;
std::tie(TargetSym, TargetOffset) = BC.MIA->getTargetSymbolInfo(DispExpr);
auto *BD = BC.getBinaryDataByName(TargetSym->getName());
assert(BD && "global symbol needs a value");
ArrayStart = BD->getAddress() + TargetOffset;
BaseRegNum = 0;
if (BC.TheTriple->getArch() == llvm::Triple::aarch64) {
ArrayStart &= ~0xFFFULL;
@ -729,13 +730,13 @@ IndirectBranchType BinaryFunction::processIndirectBranch(MCInst &Instruction,
// Check if there's already a jump table registered at this address.
if (auto *JT = getJumpTableContainingAddress(ArrayStart)) {
auto JTOffset = ArrayStart - JT->Address;
auto JTOffset = ArrayStart - JT->getAddress();
if (Type == IndirectBranchType::POSSIBLE_PIC_JUMP_TABLE && JTOffset != 0) {
// Adjust the size of this jump table and create a new one if necessary.
// We cannot re-use the entries since the offsets are relative to the
// table start.
DEBUG(dbgs() << "BOLT-DEBUG: adjusting size of jump table at 0x"
<< Twine::utohexstr(JT->Address) << '\n');
<< Twine::utohexstr(JT->getAddress()) << '\n');
JT->OffsetEntries.resize(JTOffset / JT->EntrySize);
} else {
// Re-use an existing jump table. Perhaps parts of it.
@ -750,8 +751,10 @@ IndirectBranchType BinaryFunction::processIndirectBranch(MCInst &Instruction,
// Get or create a new label for the table.
auto LI = JT->Labels.find(JTOffset);
if (LI == JT->Labels.end()) {
auto *JTStartLabel = BC.getOrCreateGlobalSymbol(ArrayStart,
"JUMP_TABLEat");
auto *JTStartLabel = BC.registerNameAtAddress(generateJumpTableName(ArrayStart),
ArrayStart,
0,
JT->EntrySize);
auto Result = JT->Labels.emplace(JTOffset, JTStartLabel);
assert(Result.second && "error adding jump table label");
LI = Result.first;
@ -827,20 +830,33 @@ IndirectBranchType BinaryFunction::processIndirectBranch(MCInst &Instruction,
Type == IndirectBranchType::POSSIBLE_PIC_JUMP_TABLE) {
assert(JTOffsetCandidates.size() > 2 &&
"expected more than 2 jump table entries");
auto *JTStartLabel = BC.getOrCreateGlobalSymbol(ArrayStart, "JUMP_TABLEat");
DEBUG(dbgs() << "BOLT-DEBUG: creating jump table "
<< JTStartLabel->getName()
<< " in function " << *this << " with "
<< JTOffsetCandidates.size() << " entries.\n");
auto JumpTableName = generateJumpTableName(ArrayStart);
auto JumpTableType =
Type == IndirectBranchType::POSSIBLE_JUMP_TABLE
? JumpTable::JTT_NORMAL
: JumpTable::JTT_PIC;
JumpTables.emplace(ArrayStart, JumpTable{ArrayStart,
EntrySize,
JumpTableType,
std::move(JTOffsetCandidates),
{{0, JTStartLabel}}});
auto *JTStartLabel = BC.Ctx->getOrCreateSymbol(JumpTableName);
auto JT = llvm::make_unique<JumpTable>(JumpTableName,
ArrayStart,
EntrySize,
JumpTableType,
std::move(JTOffsetCandidates),
JumpTable::LabelMapType{{0, JTStartLabel}},
*BC.getSectionForAddress(ArrayStart));
auto *JTLabel = BC.registerNameAtAddress(JumpTableName,
ArrayStart,
JT.get());
assert(JTLabel == JTStartLabel);
DEBUG(dbgs() << "BOLT-DEBUG: creating jump table "
<< JTStartLabel->getName()
<< " in function " << *this << " with "
<< JTOffsetCandidates.size() << " entries.\n");
JumpTables.emplace(ArrayStart, JT.release());
BC.MIA->replaceMemOperandDisp(const_cast<MCInst &>(*MemLocInstr),
JTStartLabel, BC.Ctx.get());
BC.MIA->setJumpTable(BC.Ctx.get(), Instruction, ArrayStart, IndexRegNum);
@ -849,6 +865,7 @@ IndirectBranchType BinaryFunction::processIndirectBranch(MCInst &Instruction,
return Type;
}
assert(!Value || BC.getSectionForAddress(Value));
BC.InterproceduralReferences.insert(Value);
return IndirectBranchType::POSSIBLE_TAIL_CALL;
}
@ -865,9 +882,9 @@ MCSymbol *BinaryFunction::getOrCreateLocalLabel(uint64_t Address,
// Check if there's a global symbol registered at given address.
// If so - reuse it since we want to keep the symbol value updated.
if (Offset != 0) {
if (auto *Symbol = BC.getGlobalSymbolAtAddress(Address)) {
Labels[Offset] = Symbol;
return Symbol;
if (auto *BD = BC.getBinaryDataAtAddress(Address)) {
Labels[Offset] = BD->getSymbol();
return BD->getSymbol();
}
}
@ -903,6 +920,7 @@ void BinaryFunction::disassemble(ArrayRef<uint8_t> FunctionData) {
auto handlePCRelOperand =
[&](MCInst &Instruction, uint64_t Address, uint64_t Size) {
uint64_t TargetAddress{0};
uint64_t TargetOffset{0};
MCSymbol *TargetSymbol{nullptr};
if (!MIA->evaluateMemOperandTarget(Instruction, TargetAddress, Address,
Size)) {
@ -970,13 +988,31 @@ void BinaryFunction::disassemble(ArrayRef<uint8_t> FunctionData) {
BC.InterproceduralReferences.insert(TargetAddress);
}
}
if (!TargetSymbol)
TargetSymbol = BC.getOrCreateGlobalSymbol(TargetAddress, "DATAat");
if (!TargetSymbol) {
auto *BD = BC.getBinaryDataContainingAddress(TargetAddress);
if (BD) {
TargetSymbol = BD->getSymbol();
TargetOffset = TargetAddress - BD->getAddress();
} else {
// TODO: use DWARF info to get size/alignment here?
TargetSymbol = BC.getOrCreateGlobalSymbol(TargetAddress, 0, 0, "DATAat");
DEBUG(if (opts::Verbosity >= 2) {
dbgs() << "Created DATAat sym: " << TargetSymbol->getName()
<< " in section " << BD->getSectionName() << "\n";
});
}
}
const MCExpr *Expr = MCSymbolRefExpr::create(TargetSymbol,
MCSymbolRefExpr::VK_None,
*BC.Ctx);
if (TargetOffset) {
auto *Offset = MCConstantExpr::create(TargetOffset, *BC.Ctx);
Expr = MCBinaryExpr::createAdd(Expr, Offset, *BC.Ctx);
}
MIA->replaceMemOperandDisp(
Instruction, MCOperand::createExpr(BC.MIA->getTargetExprFor(
Instruction,
MCSymbolRefExpr::create(
TargetSymbol, MCSymbolRefExpr::VK_None, *BC.Ctx),
Expr,
*BC.Ctx, 0)));
return true;
};
@ -1050,33 +1086,39 @@ void BinaryFunction::disassemble(ArrayRef<uint8_t> FunctionData) {
// Check if there's a relocation associated with this instruction.
bool UsedReloc{false};
if (!Relocations.empty()) {
auto RI = Relocations.lower_bound(Offset);
if (RI != Relocations.end() && RI->first < Offset + Size) {
const auto &Relocation = RI->second;
DEBUG(dbgs() << "BOLT-DEBUG: replacing immediate with relocation"
" against " << Relocation.Symbol->getName()
<< " in function " << *this
<< " for instruction at offset 0x"
<< Twine::utohexstr(Offset) << '\n');
int64_t Value;
const auto Result = BC.MIA->replaceImmWithSymbol(
Instruction, Relocation.Symbol, Relocation.Addend, Ctx.get(), Value,
Relocation.Type);
(void)Result;
assert(Result && "cannot replace immediate with relocation");
// For aarch, if we replaced an immediate with a symbol from a
// relocation, we mark it so we do not try to further process a
// pc-relative operand. All we need is the symbol.
if (BC.TheTriple->getArch() == llvm::Triple::aarch64)
UsedReloc = true;
for (auto Itr = Relocations.lower_bound(Offset);
Itr != Relocations.upper_bound(Offset + Size);
++Itr) {
const auto &Relocation = Itr->second;
if (Relocation.Offset >= Offset + Size)
continue;
// Make sure we replaced the correct immediate (instruction
// can have multiple immediate operands).
assert((BC.TheTriple->getArch() == llvm::Triple::aarch64 ||
static_cast<uint64_t>(Value) == Relocation.Value) &&
"immediate value mismatch in function");
}
DEBUG(dbgs() << "BOLT-DEBUG: replacing immediate with relocation"
" against " << Relocation.Symbol->getName()
<< "+" << Relocation.Addend
<< " in function " << *this
<< " for instruction at offset 0x"
<< Twine::utohexstr(Offset) << '\n');
int64_t Value = Relocation.Value;
const auto Result = BC.MIA->replaceImmWithSymbol(Instruction,
Relocation.Symbol,
Relocation.Addend,
Ctx.get(),
Value,
Relocation.Type);
(void)Result;
assert(Result && "cannot replace immediate with relocation");
// For aarch, if we replaced an immediate with a symbol from a
// relocation, we mark it so we do not try to further process a
// pc-relative operand. All we need is the symbol.
if (BC.TheTriple->getArch() == llvm::Triple::aarch64)
UsedReloc = true;
// Make sure we replaced the correct immediate (instruction
// can have multiple immediate operands).
assert((BC.TheTriple->getArch() == llvm::Triple::aarch64 ||
static_cast<uint64_t>(Value) == Relocation.Value) &&
"immediate value mismatch in function");
}
// Convert instruction to a shorter version that could be relaxed if needed.
@ -1157,6 +1199,8 @@ void BinaryFunction::disassemble(ArrayRef<uint8_t> FunctionData) {
}
TargetSymbol = BC.getOrCreateGlobalSymbol(TargetAddress,
0,
0,
"FUNCat");
if (TargetAddress == 0) {
// We actually see calls to address 0 in presence of weak symbols
@ -1288,12 +1332,13 @@ add_instruction:
void BinaryFunction::postProcessJumpTables() {
// Create labels for all entries.
for (auto &JTI : JumpTables) {
auto &JT = JTI.second;
auto &JT = *JTI.second;
for (auto Offset : JT.OffsetEntries) {
auto *Label = getOrCreateLocalLabel(getAddress() + Offset,
/*CreatePastEnd*/ true);
JT.Entries.push_back(Label);
}
BC.setBinaryDataSize(JT.getAddress(), JT.getSize());
}
// Add TakenBranches from JumpTables.
@ -1305,7 +1350,7 @@ void BinaryFunction::postProcessJumpTables() {
const auto JTAddress = JTSite.second;
const auto *JT = getJumpTableContainingAddress(JTAddress);
assert(JT && "cannot find jump table for address");
auto EntryOffset = JTAddress - JT->Address;
auto EntryOffset = JTAddress - JT->getAddress();
while (EntryOffset < JT->getSize()) {
auto TargetOffset = JT->OffsetEntries[EntryOffset / JT->EntrySize];
if (TargetOffset < getSize())
@ -1313,7 +1358,7 @@ void BinaryFunction::postProcessJumpTables() {
// Take ownership of jump table relocations.
if (BC.HasRelocations) {
auto EntryAddress = JT->Address + EntryOffset;
auto EntryAddress = JT->getAddress() + EntryOffset;
auto Res = BC.removeRelocationAt(EntryAddress);
(void)Res;
DEBUG(
@ -1335,7 +1380,7 @@ void BinaryFunction::postProcessJumpTables() {
// Free memory used by jump table offsets.
for (auto &JTI : JumpTables) {
auto &JT = JTI.second;
auto &JT = *JTI.second;
clearList(JT.OffsetEntries);
}
@ -1755,7 +1800,8 @@ void BinaryFunction::addEntryPoint(uint64_t Address) {
<< " at offset 0x" << Twine::utohexstr(Address - getAddress())
<< '\n');
auto *EntrySymbol = BC.getGlobalSymbolAtAddress(Address);
auto *EntryBD = BC.getBinaryDataAtAddress(Address);
auto *EntrySymbol = EntryBD ? EntryBD->getSymbol() : nullptr;
// If we haven't disassembled the function yet we can add a new entry point
// even if it doesn't have an associated entry in the symbol table.
@ -2905,26 +2951,28 @@ bool BinaryFunction::isIdenticalWith(const BinaryFunction &OtherBF,
}
// Check if symbols are jump tables.
auto SIA = BC.GlobalSymbols.find(A->getName());
if (SIA == BC.GlobalSymbols.end())
auto *SIA = BC.getBinaryDataByName(A->getName());
if (!SIA)
return false;
auto SIB = BC.GlobalSymbols.find(B->getName());
if (SIB == BC.GlobalSymbols.end())
auto *SIB = BC.getBinaryDataByName(B->getName());
if (!SIB)
return false;
assert((SIA->second != SIB->second) &&
assert((SIA->getAddress() != SIB->getAddress()) &&
"different symbols should not have the same value");
const auto *JumpTableA = getJumpTableContainingAddress(SIA->second);
const auto *JumpTableA =
getJumpTableContainingAddress(SIA->getAddress());
if (!JumpTableA)
return false;
const auto *JumpTableB =
OtherBF.getJumpTableContainingAddress(SIB->second);
OtherBF.getJumpTableContainingAddress(SIB->getAddress());
if (!JumpTableB)
return false;
if ((SIA->second - JumpTableA->Address) !=
(SIB->second - JumpTableB->Address))
if ((SIA->getAddress() - JumpTableA->getAddress()) !=
(SIB->getAddress() - JumpTableB->getAddress()))
return false;
return equalJumpTables(JumpTableA, JumpTableB, OtherBF);
@ -2955,6 +3003,24 @@ bool BinaryFunction::isIdenticalWith(const BinaryFunction &OtherBF,
return true;
}
std::string BinaryFunction::generateJumpTableName(uint64_t Address) const {
auto *JT = getJumpTableContainingAddress(Address);
size_t Id;
uint64_t Offset = 0;
if (JT) {
Offset = Address - JT->getAddress();
auto Itr = JT->Labels.find(Offset);
if (Itr != JT->Labels.end()) {
return Itr->second->getName();
}
Id = JumpTableIds.at(JT->getAddress());
} else {
Id = JumpTableIds[Address] = JumpTables.size();
}
return ("JUMP_TABLE/" + Names[0] + "." + std::to_string(Id) +
(Offset ? ("." + std::to_string(Offset)) : ""));
}
bool BinaryFunction::equalJumpTables(const JumpTable *JumpTableA,
const JumpTable *JumpTableB,
const BinaryFunction &BFB) const {
@ -3282,17 +3348,18 @@ void BinaryFunction::emitJumpTables(MCStreamer *Streamer) {
outs() << "BOLT-INFO: jump tables for function " << *this << ":\n";
}
for (auto &JTI : JumpTables) {
auto &JT = JTI.second;
auto &JT = *JTI.second;
if (opts::PrintJumpTables)
JT.print(outs());
if (opts::JumpTables == JTS_BASIC && BC.HasRelocations) {
JT.updateOriginal(BC);
JT.updateOriginal();
} else {
MCSection *HotSection, *ColdSection;
if (opts::JumpTables == JTS_BASIC) {
JT.SectionName =
".local.JUMP_TABLEat0x" + Twine::utohexstr(JT.Address).str();
HotSection = BC.Ctx->getELFSection(JT.SectionName,
std::string Name = JT.Labels[0]->getName().str();
std::replace(Name.begin(), Name.end(), '/', '.');
JT.setOutputSection(".local." + Name);
HotSection = BC.Ctx->getELFSection(JT.getOutputSection(),
ELF::SHT_PROGBITS,
ELF::SHF_ALLOC);
ColdSection = HotSection;
@ -3311,157 +3378,6 @@ void BinaryFunction::emitJumpTables(MCStreamer *Streamer) {
}
}
std::pair<size_t, size_t>
BinaryFunction::JumpTable::getEntriesForAddress(const uint64_t Addr) const {
const uint64_t InstOffset = Addr - Address;
size_t StartIndex = 0, EndIndex = 0;
uint64_t Offset = 0;
for (size_t I = 0; I < Entries.size(); ++I) {
auto LI = Labels.find(Offset);
if (LI != Labels.end()) {
const auto NextLI = std::next(LI);
const auto NextOffset =
NextLI == Labels.end() ? getSize() : NextLI->first;
if (InstOffset >= LI->first && InstOffset < NextOffset) {
StartIndex = I;
EndIndex = I;
while (Offset < NextOffset) {
++EndIndex;
Offset += EntrySize;
}
break;
}
}
Offset += EntrySize;
}
return std::make_pair(StartIndex, EndIndex);
}
bool BinaryFunction::JumpTable::replaceDestination(uint64_t JTAddress,
const MCSymbol *OldDest,
MCSymbol *NewDest) {
bool Patched{false};
const auto Range = getEntriesForAddress(JTAddress);
for (auto I = &Entries[Range.first], E = &Entries[Range.second];
I != E; ++I) {
auto &Entry = *I;
if (Entry == OldDest) {
Patched = true;
Entry = NewDest;
}
}
return Patched;
}
void BinaryFunction::JumpTable::updateOriginal(BinaryContext &BC) {
// In non-relocation mode we have to emit jump tables in local sections.
// This way we only overwrite them when a corresponding function is
// overwritten.
assert(BC.HasRelocations && "relocation mode expected");
auto Section = BC.getSectionForAddress(Address);
assert(Section && "section not found for jump table");
uint64_t Offset = Address - Section->getAddress();
StringRef SectionName = Section->getName();
for (auto *Entry : Entries) {
const auto RelType = (Type == JTT_NORMAL) ? ELF::R_X86_64_64
: ELF::R_X86_64_PC32;
const uint64_t RelAddend = (Type == JTT_NORMAL)
? 0 : Offset - (Address - Section->getAddress());
DEBUG(dbgs() << "adding relocation to section " << SectionName
<< " at offset " << Twine::utohexstr(Offset) << " for symbol "
<< Entry->getName() << " with addend "
<< Twine::utohexstr(RelAddend) << '\n');
Section->addRelocation(Offset, Entry, RelType, RelAddend);
Offset += EntrySize;
}
}
uint64_t BinaryFunction::JumpTable::emit(MCStreamer *Streamer,
MCSection *HotSection,
MCSection *ColdSection) {
// Pre-process entries for aggressive splitting.
// Each label represents a separate switch table and gets its own count
// determining its destination.
std::map<MCSymbol *, uint64_t> LabelCounts;
if (opts::JumpTables > JTS_SPLIT && !Counts.empty()) {
MCSymbol *CurrentLabel = Labels[0];
uint64_t CurrentLabelCount = 0;
for (unsigned Index = 0; Index < Entries.size(); ++Index) {
auto LI = Labels.find(Index * EntrySize);
if (LI != Labels.end()) {
LabelCounts[CurrentLabel] = CurrentLabelCount;
CurrentLabel = LI->second;
CurrentLabelCount = 0;
}
CurrentLabelCount += Counts[Index].Count;
}
LabelCounts[CurrentLabel] = CurrentLabelCount;
} else {
Streamer->SwitchSection(Count > 0 ? HotSection : ColdSection);
Streamer->EmitValueToAlignment(EntrySize);
}
MCSymbol *LastLabel = nullptr;
uint64_t Offset = 0;
for (auto *Entry : Entries) {
auto LI = Labels.find(Offset);
if (LI != Labels.end()) {
DEBUG(dbgs() << "BOLT-DEBUG: emitting jump table "
<< LI->second->getName() << " (originally was at address 0x"
<< Twine::utohexstr(Address + Offset)
<< (Offset ? "as part of larger jump table\n" : "\n"));
if (!LabelCounts.empty()) {
DEBUG(dbgs() << "BOLT-DEBUG: jump table count: "
<< LabelCounts[LI->second] << '\n');
if (LabelCounts[LI->second] > 0) {
Streamer->SwitchSection(HotSection);
} else {
Streamer->SwitchSection(ColdSection);
}
Streamer->EmitValueToAlignment(EntrySize);
}
Streamer->EmitLabel(LI->second);
LastLabel = LI->second;
}
if (Type == JTT_NORMAL) {
Streamer->EmitSymbolValue(Entry, OutputEntrySize);
} else { // JTT_PIC
auto JT = MCSymbolRefExpr::create(LastLabel, Streamer->getContext());
auto E = MCSymbolRefExpr::create(Entry, Streamer->getContext());
auto Value = MCBinaryExpr::createSub(E, JT, Streamer->getContext());
Streamer->EmitValue(Value, EntrySize);
}
Offset += EntrySize;
}
return Offset;
}
void BinaryFunction::JumpTable::print(raw_ostream &OS) const {
uint64_t Offset = 0;
for (const auto *Entry : Entries) {
auto LI = Labels.find(Offset);
if (LI != Labels.end()) {
OS << "Jump Table " << LI->second->getName() << " at @0x"
<< Twine::utohexstr(Address+Offset);
if (Offset) {
OS << " (possibly part of larger jump table):\n";
} else {
OS << " with total count of " << Count << ":\n";
}
}
OS << format(" 0x%04" PRIx64 " : ", Offset) << Entry->getName();
if (!Counts.empty()) {
OS << " : " << Counts[Offset / EntrySize].Mispreds
<< "/" << Counts[Offset / EntrySize].Count;
}
OS << '\n';
Offset += EntrySize;
}
OS << "\n\n";
}
void BinaryFunction::calculateLoopInfo() {
// Discover loops.
BinaryDominatorTree DomTree;

136
bolt/BinaryFunction.h
View File

@ -22,6 +22,7 @@
#include "BinaryLoop.h"
#include "DataReader.h"
#include "DebugData.h"
#include "JumpTable.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/ADT/ilist.h"
#include "llvm/ADT/iterator.h"
@ -51,8 +52,6 @@ class DWARFDebugInfoEntryMinimal;
namespace bolt {
struct SectionInfo;
using DWARFUnitLineTable = std::pair<DWARFUnit *,
const DWARFDebugLine::LineTable *>;
@ -150,14 +149,6 @@ inline raw_ostream &operator<<(raw_ostream &OS, const DynoStats &Stats) {
DynoStats operator+(const DynoStats &A, const DynoStats &B);
enum JumpTableSupportLevel : char {
JTS_NONE = 0, /// Disable jump tables support.
JTS_BASIC = 1, /// Enable basic jump tables support (in-place).
JTS_MOVE = 2, /// Move jump tables to a separate section.
JTS_SPLIT = 3, /// Enable hot/cold splitting of jump tables.
JTS_AGGRESSIVE = 4, /// Aggressive splitting of jump tables.
};
enum IndirectCallPromotionType : char {
ICP_NONE, /// Don't perform ICP.
ICP_CALLS, /// Perform ICP on indirect calls.
@ -231,12 +222,6 @@ public:
ST_ALL, /// Split all functions
};
/// Branch statistics for jump table entries.
struct JumpInfo {
uint64_t Mispreds{0};
uint64_t Count{0};
};
static constexpr uint64_t COUNT_NO_PROFILE =
BinaryBasicBlock::COUNT_NO_PROFILE;
@ -567,90 +552,17 @@ private:
/// function and that apply before the entry basic block).
CFIInstrMapType CIEFrameInstructions;
public:
/// Representation of a jump table.
///
/// The jump table may include other jump tables that are referenced by
/// a different label at a different offset in this jump table.
struct JumpTable {
enum JumpTableType : char {
JTT_NORMAL,
JTT_PIC,
};
/// Original address.
uint64_t Address;
/// Size of the entry used for storage.
std::size_t EntrySize;
/// Size of the entry size we will write (we may use a more compact layout)
std::size_t OutputEntrySize;
/// The type of this jump table.
JumpTableType Type;
/// All the entries as labels.
std::vector<MCSymbol *> Entries;
/// All the entries as offsets into a function. Invalid after CFG is built.
std::vector<uint64_t> OffsetEntries;
/// Map <Offset> -> <Label> used for embedded jump tables. Label at 0 offset
/// is the main label for the jump table.
std::map<unsigned, MCSymbol *> Labels;
/// Corresponding section if any.
ErrorOr<BinarySection &> Section{std::errc::bad_address};
/// Corresponding section name if any.
std::string SectionName;
/// Return the size of the jump table.
uint64_t getSize() const {
return std::max(OffsetEntries.size(), Entries.size()) * EntrySize;
}
/// Get the indexes for symbol entries that correspond to the jump table
/// starting at (or containing) 'Addr'.
std::pair<size_t, size_t> getEntriesForAddress(const uint64_t Addr) const;
/// Constructor.
JumpTable(uint64_t Address, std::size_t EntrySize, JumpTableType Type,
decltype(OffsetEntries) &&OffsetEntries,
decltype(Labels) &&Labels)
: Address(Address), EntrySize(EntrySize), OutputEntrySize(EntrySize),
Type(Type), OffsetEntries(OffsetEntries), Labels(Labels) {}
/// Dynamic number of times each entry in the table was referenced.
/// Identical entries will have a shared count (identical for every
/// entry in the set).
std::vector<JumpInfo> Counts;
/// Total number of times this jump table was used.
uint64_t Count{0};
/// Change all entries of the jump table in \p JTAddress pointing to
/// \p OldDest to \p NewDest. Return false if unsuccessful.
bool replaceDestination(uint64_t JTAddress, const MCSymbol *OldDest,
MCSymbol *NewDest);
/// Update jump table at its original location.
void updateOriginal(BinaryContext &BC);
/// Emit jump table data. Callee supplies sections for the data.
/// Return the number of total bytes emitted.
uint64_t emit(MCStreamer *Streamer, MCSection *HotSection,
MCSection *ColdSection);
/// Print for debugging purposes.
void print(raw_ostream &OS) const;
};
private:
/// All compound jump tables for this function.
/// <OriginalAddress> -> <JumpTable>
std::map<uint64_t, JumpTable> JumpTables;
/// <OriginalAddress> -> <JumpTable *>
std::map<uint64_t, JumpTable *> JumpTables;
/// A map from jump table address to insertion order. Used for generating
/// jump table names.
mutable std::map<uint64_t, size_t> JumpTableIds;
/// Generate a unique name for this jump table at the given address that should
/// be repeatable no matter what the start address of the table is.
std::string generateJumpTableName(uint64_t Address) const;
/// Return jump table that covers a given \p Address in memory.
JumpTable *getJumpTableContainingAddress(uint64_t Address) {
@ -658,8 +570,8 @@ private:
if (JTI == JumpTables.begin())
return nullptr;
--JTI;
if (JTI->first + JTI->second.getSize() > Address) {
return &JTI->second;
if (JTI->first + JTI->second->getSize() > Address) {
return JTI->second;
}
return nullptr;
}
@ -669,12 +581,18 @@ private:
if (JTI == JumpTables.begin())
return nullptr;
--JTI;
if (JTI->first + JTI->second.getSize() > Address) {
return &JTI->second;
if (JTI->first + JTI->second->getSize() > Address) {
return JTI->second;
}
return nullptr;
}
/// Iterate over all jump tables associated with this function.
iterator_range<std::map<uint64_t, JumpTable *>::const_iterator>
jumpTables() const {
return make_range(JumpTables.begin(), JumpTables.end());
}
/// Compare two jump tables in 2 functions. The function relies on consistent
/// ordering of basic blocks in both binary functions (e.g. DFS).
bool equalJumpTables(const JumpTable *JumpTableA,
@ -1227,6 +1145,8 @@ public:
"address is outside of the function");
auto Offset = Address - getAddress();
switch (RelType) {
case ELF::R_X86_64_8:
case ELF::R_X86_64_16:
case ELF::R_X86_64_32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_64:
@ -1247,8 +1167,7 @@ public:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
Relocations.emplace(Offset,
Relocation{Offset, Symbol, RelType, Addend, Value});
Relocations[Offset] = Relocation{Offset, Symbol, RelType, Addend, Value};
break;
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_PC8:
@ -1332,7 +1251,7 @@ public:
/// Return true if the function uses jump tables.
bool hasJumpTables() const {
return JumpTables.size();
return !JumpTables.empty();
}
const JumpTable *getJumpTable(const MCInst &Inst) const {
@ -1661,7 +1580,8 @@ public:
}
BinaryFunction &setPersonalityFunction(uint64_t Addr) {
PersonalityFunction = BC.getOrCreateGlobalSymbol(Addr, "FUNCat");
assert(!PersonalityFunction && "can't set personality function twice");
PersonalityFunction = BC.getOrCreateGlobalSymbol(Addr, 0, 0, "FUNCat");
return *this;
}
@ -1815,7 +1735,7 @@ public:
return std::make_pair(nullptr, nullptr);
// Register our island at global namespace
Symbol = BC.getOrCreateGlobalSymbol(Address, "ISLANDat");
Symbol = BC.getOrCreateGlobalSymbol(Address, 0, 0, "ISLANDat");
// Internal bookkeeping
const auto Offset = Address - getAddress();
assert((!IslandSymbols.count(Offset) || IslandSymbols[Offset] == Symbol) &&

12
bolt/BinaryFunctionProfile.cpp
View File

@ -391,7 +391,7 @@ void BinaryFunction::postProcessProfile() {
if (JT->Counts.empty())
JT->Counts.resize(JT->Entries.size());
auto EI = JT->Entries.begin();
auto Delta = (JTAddress - JT->Address) / JT->EntrySize;
auto Delta = (JTAddress - JT->getAddress()) / JT->EntrySize;
EI += Delta;
while (EI != JT->Entries.end()) {
const auto *TargetBB = getBasicBlockForLabel(*EI);
@ -728,14 +728,8 @@ bool BinaryFunction::fetchProfileForOtherEntryPoints() {
if (BB->isEntryPoint()) {
uint64_t EntryAddress = BB->getOffset() + getAddress();
// Look for branch data associated with this entry point
std::vector<std::string> Names;
std::multimap<uint64_t, std::string>::iterator I, E;
for (std::tie(I, E) = BC.GlobalAddresses.equal_range(EntryAddress);
I != E; ++I) {
Names.push_back(I->second);
}
if (!Names.empty()) {
if (FuncBranchData *Data = BC.DR.getFuncBranchData(Names)) {
if (auto *BD = BC.getBinaryDataAtAddress(EntryAddress)) {
if (FuncBranchData *Data = BC.DR.getFuncBranchData(BD->getNames())) {
BranchData->appendFrom(*Data, BB->getOffset());
Data->Used = true;
Updated = true;

317
bolt/BinarySection.cpp
View File

@ -10,8 +10,6 @@
//===----------------------------------------------------------------------===//
#include "BinarySection.h"
#include "llvm/MC/MCContext.h"
#include "llvm/MC/MCStreamer.h"
#include "llvm/Support/CommandLine.h"
#undef DEBUG_TYPE
@ -24,315 +22,6 @@ namespace opts {
extern cl::opt<bool> PrintRelocations;
}
Triple::ArchType Relocation::Arch;
bool Relocation::isSupported(uint64_t Type) {
switch (Type) {
default:
return false;
case ELF::R_X86_64_8:
case ELF::R_X86_64_16:
case ELF::R_X86_64_32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_64:
case ELF::R_X86_64_PC8:
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_PC64:
case ELF::R_X86_64_PLT32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_TPOFF32:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_CALL26:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC:
case ELF::R_AARCH64_ADD_ABS_LO12_NC:
case ELF::R_AARCH64_LDST128_ABS_LO12_NC:
case ELF::R_AARCH64_LDST32_ABS_LO12_NC:
case ELF::R_AARCH64_LDST16_ABS_LO12_NC:
case ELF::R_AARCH64_LDST8_ABS_LO12_NC:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_PREL32:
case ELF::R_AARCH64_ABS64:
return true;
}
}
size_t Relocation::getSizeForType(uint64_t Type) {
switch (Type) {
default:
llvm_unreachable("unsupported relocation type");
case ELF::R_X86_64_8:
case ELF::R_X86_64_PC8:
return 1;
case ELF::R_X86_64_16:
return 2;
case ELF::R_X86_64_PLT32:
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_TPOFF32:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_CALL26:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC:
case ELF::R_AARCH64_ADD_ABS_LO12_NC:
case ELF::R_AARCH64_LDST128_ABS_LO12_NC:
case ELF::R_AARCH64_LDST32_ABS_LO12_NC:
case ELF::R_AARCH64_LDST16_ABS_LO12_NC:
case ELF::R_AARCH64_LDST8_ABS_LO12_NC:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_PREL32:
return 4;
case ELF::R_X86_64_PC64:
case ELF::R_X86_64_64:
case ELF::R_AARCH64_ABS64:
return 8;
}
}
uint64_t Relocation::extractValue(uint64_t Type, uint64_t Contents,
uint64_t PC) {
switch (Type) {
default:
llvm_unreachable("unsupported relocation type");
case ELF::R_AARCH64_ABS64:
return Contents;
case ELF::R_AARCH64_PREL32:
return static_cast<int64_t>(PC) + SignExtend64<32>(Contents & 0xffffffff);
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_CALL26:
// Immediate goes in bits 25:0 of B and BL.
Contents &= ~0xfffffffffc000000ULL;
return static_cast<int64_t>(PC) + SignExtend64<28>(Contents << 2);
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_ADR_PREL_PG_HI21: {
// Bits 32:12 of Symbol address goes in bits 30:29 + 23:5 of ADRP
// instruction
Contents &= ~0xffffffff9f00001fUll;
auto LowBits = (Contents >> 29) & 0x3;
auto HighBits = (Contents >> 5) & 0x7ffff;
Contents = LowBits | (HighBits << 2);
Contents = static_cast<int64_t>(PC) + SignExtend64<32>(Contents << 12);
Contents &= ~0xfffUll;
return Contents;
}
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of LD/ST instruction, taken
// from bits 11:3 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 3);
}
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_ADD_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 0);
}
case ELF::R_AARCH64_LDST128_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:4 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 4);
}
case ELF::R_AARCH64_LDST32_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:2 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 2);
}
case ELF::R_AARCH64_LDST16_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:1 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 1);
}
case ELF::R_AARCH64_LDST8_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:0 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 0);
}
}
}
bool Relocation::isGOT(uint64_t Type) {
switch (Type) {
default:
return false;
case ELF::R_X86_64_GOT32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_GOTOFF64:
case ELF::R_X86_64_GOTPC32:
case ELF::R_X86_64_GOT64:
case ELF::R_X86_64_GOTPCREL64:
case ELF::R_X86_64_GOTPC64:
case ELF::R_X86_64_GOTPLT64:
case ELF::R_X86_64_GOTPC32_TLSDESC:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
return true;
}
}
bool Relocation::isTLS(uint64_t Type) {
switch (Type) {
default:
return false;
case ELF::R_X86_64_TPOFF32:
case ELF::R_X86_64_TPOFF64:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
return true;
}
}
bool Relocation::isPCRelative(uint64_t Type) {
switch (Type) {
default:
llvm_unreachable("Unknown relocation type");
case ELF::R_X86_64_64:
case ELF::R_X86_64_32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_16:
case ELF::R_X86_64_8:
case ELF::R_X86_64_TPOFF32:
case ELF::R_AARCH64_ABS64:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC:
case ELF::R_AARCH64_ADD_ABS_LO12_NC:
case ELF::R_AARCH64_LDST128_ABS_LO12_NC:
case ELF::R_AARCH64_LDST32_ABS_LO12_NC:
case ELF::R_AARCH64_LDST16_ABS_LO12_NC:
case ELF::R_AARCH64_LDST8_ABS_LO12_NC:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
return false;
case ELF::R_X86_64_PC8:
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_PLT32:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_CALL26:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_PREL32:
return true;
}
}
size_t Relocation::emit(MCStreamer *Streamer) const {
const auto Size = getSizeForType(Type);
auto &Ctx = Streamer->getContext();
if (isPCRelative(Type)) {
auto *TempLabel = Ctx.createTempSymbol();
Streamer->EmitLabel(TempLabel);
auto Value =
MCBinaryExpr::createSub(MCSymbolRefExpr::create(Symbol, Ctx),
MCSymbolRefExpr::create(TempLabel, Ctx),
Ctx);
if (Addend) {
Value = MCBinaryExpr::createAdd(Value,
MCConstantExpr::create(Addend, Ctx),
Ctx);
}
Streamer->EmitValue(Value, Size);
} else if (Addend) {
auto Value = MCBinaryExpr::createAdd(MCSymbolRefExpr::create(Symbol, Ctx),
MCConstantExpr::create(Addend, Ctx),
Ctx);
Streamer->EmitValue(Value, Size);
} else {
Streamer->EmitSymbolValue(Symbol, Size);
}
return Size;
}
#define ELF_RELOC(name, value) #name,
void Relocation::print(raw_ostream &OS) const {
static const char *X86RelocNames[] = {
#include "llvm/BinaryFormat/ELFRelocs/x86_64.def"
};
static const char *AArch64RelocNames[] = {
#include "llvm/BinaryFormat/ELFRelocs/AArch64.def"
};
if (Arch == Triple::aarch64)
OS << AArch64RelocNames[Type];
else
OS << X86RelocNames[Type];
OS << ", 0x" << Twine::utohexstr(Offset);
if (Symbol) {
OS << ", " << Symbol->getName();
}
if (int64_t(Addend) < 0)
OS << ", -0x" << Twine::utohexstr(-int64_t(Addend));
else
OS << ", 0x" << Twine::utohexstr(Addend);
OS << ", 0x" << Twine::utohexstr(Value);
}
BinarySection::~BinarySection() {
if (!isAllocatable() &&
(!hasSectionRef() ||
@ -353,6 +42,12 @@ void BinarySection::print(raw_ostream &OS) const {
if (isAllocatable())
OS << " (allocatable)";
if (isVirtual())
OS << " (virtual)";
if (isTLS())
OS << " (tls)";
if (opts::PrintRelocations) {
for (auto &R : relocations())
OS << "\n " << R;

102
bolt/BinarySection.h
View File

@ -12,6 +12,7 @@
#ifndef LLVM_TOOLS_LLVM_BOLT_BINARY_SECTION_H
#define LLVM_TOOLS_LLVM_BOLT_BINARY_SECTION_H
#include "Relocation.h"
#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/Triple.h"
#include "llvm/BinaryFormat/ELF.h"
@ -28,87 +29,7 @@ using namespace object;
namespace bolt {
/// Relocation class.
struct Relocation {
static Triple::ArchType Arch; /// for printing, set by BinaryContext ctor.
/// The offset of this relocation in the object it is contained in.
uint64_t Offset;
/// The symbol this relocation is referring to.
MCSymbol *Symbol;
/// Relocation type.
uint64_t Type;
/// The offset from the \p Symbol base used to compute the final
/// value of this relocation.
uint64_t Addend;
/// The computed relocation value extracted from the binary file.
/// Used to validate relocation correctness.
uint64_t Value;
/// Return size of the given relocation \p Type.
static size_t getSizeForType(uint64_t Type);
/// Extract current relocated value from binary contents. This is used for
/// RISC architectures where values are encoded in specific bits depending
/// on the relocation value.
static uint64_t extractValue(uint64_t Type, uint64_t Contents, uint64_t PC);
/// Return true if relocation type is PC-relative. Return false otherwise.
static bool isPCRelative(uint64_t Type);
/// Check if \p Type is a supported relocation type.
static bool isSupported(uint64_t Type);
/// Return true if relocation type implies the creation of a GOT entry
static bool isGOT(uint64_t Type);
/// Return true if relocation type is for thread local storage.
static bool isTLS(uint64_t Type);
/// Return true if this relocation is PC-relative. Return false otherwise.
bool isPCRelative() const {
return isPCRelative(Type);
}
/// Emit relocation at a current \p Streamer' position. The caller is
/// responsible for setting the position correctly.
size_t emit(MCStreamer *Streamer) const;
/// Print a relocation to \p OS.
void print(raw_ostream &OS) const;
};
/// Relocation ordering by offset.
inline bool operator<(const Relocation &A, const Relocation &B) {
return A.Offset < B.Offset;
}
inline raw_ostream &operator<<(raw_ostream &OS, const Relocation &Rel) {
Rel.print(OS);
return OS;
}
inline uint8_t *copyByteArray(const uint8_t *Data, uint64_t Size) {
auto Array = new uint8_t[Size];
memcpy(Array, Data, Size);
return Array;
}
inline uint8_t *copyByteArray(StringRef Buffer) {
return copyByteArray(reinterpret_cast<const uint8_t*>(Buffer.data()),
Buffer.size());
}
inline uint8_t *copyByteArray(ArrayRef<char> Buffer) {
return copyByteArray(reinterpret_cast<const uint8_t*>(Buffer.data()),
Buffer.size());
}
/// A class to manage binary sections that also manages related relocations
/// A class to manage binary sections that also manages related relocations.
class BinarySection {
friend class BinaryContext;
@ -280,6 +201,9 @@ public:
return (ELFType == ELF::SHT_NOBITS &&
(ELFFlags & (ELF::SHF_ALLOC | ELF::SHF_WRITE)));
}
bool isTLS() const {
return (ELFFlags & ELF::SHF_TLS);
}
bool isNote() const { return ELFType == ELF::SHT_NOTE; }
bool isStrTab() const { return ELFType == ELF::SHT_STRTAB; }
bool isSymTab() const { return ELFType == ELF::SHT_SYMTAB; }
@ -419,6 +343,22 @@ public:
void print(raw_ostream &OS) const;
};
inline uint8_t *copyByteArray(const uint8_t *Data, uint64_t Size) {
auto Array = new uint8_t[Size];
memcpy(Array, Data, Size);
return Array;
}
inline uint8_t *copyByteArray(StringRef Buffer) {
return copyByteArray(reinterpret_cast<const uint8_t*>(Buffer.data()),
Buffer.size());
}
inline uint8_t *copyByteArray(ArrayRef<char> Buffer) {
return copyByteArray(reinterpret_cast<const uint8_t*>(Buffer.data()),
Buffer.size());
}
inline raw_ostream &operator<<(raw_ostream &OS, const BinarySection &Section) {
Section.print(OS);
return OS;

3
bolt/CMakeLists.txt
View File

@ -62,6 +62,7 @@ add_llvm_tool(llvm-bolt
llvm-bolt.cpp
BinaryBasicBlock.cpp
BinaryContext.cpp
BinaryData.cpp
BinaryFunction.cpp
BinaryFunctionProfile.cpp
BinaryPassManager.cpp
@ -72,8 +73,10 @@ add_llvm_tool(llvm-bolt
DebugData.cpp
DWARFRewriter.cpp
Exceptions.cpp
JumpTable.cpp
ProfileReader.cpp
ProfileWriter.cpp
Relocation.cpp
RewriteInstance.cpp
DEPENDS

2
bolt/DWARFRewriter.cpp
View File

@ -446,7 +446,7 @@ void RewriteInstance::updateLineTableOffsets() {
auto DbgInfoSection = BC->getUniqueSectionByName(".debug_info");
assert(DbgInfoSection && ".debug_info section must exist");
auto *Zero = BC->registerNameAtAddress("Zero", 0);
auto *Zero = BC->registerNameAtAddress("Zero", 0, 0, 0);
DbgInfoSection->addRelocation(LTOffset,
Zero,
ELF::R_X86_64_32,

10
bolt/DataAggregator.cpp
View File

@ -809,12 +809,10 @@ std::error_code DataAggregator::parseMemEvents() {
if (MemFunc) {
MemName = MemFunc->getNames()[0];
Addr -= MemFunc->getAddress();
} else {
// TODO: global symbol size?
auto Sym = BC->getGlobalSymbolAtAddress(Addr);
if (Sym) {
MemName = Sym->getName();
Addr = 0;
} else if (Addr) { // TODO: filter heap/stack/nulls here?
if (auto *BD = BC->getBinaryDataContainingAddress(Addr)) {
MemName = BD->getName();
Addr -= BD->getAddress();
}
}

14
bolt/Exceptions.cpp
View File

@ -269,9 +269,8 @@ void BinaryFunction::parseLSDA(ArrayRef<uint8_t> LSDASectionData,
assert(PointerOrErr && "failed to decode indirect address");
TypeAddress = *PointerOrErr;
}
auto NI = BC.GlobalAddresses.find(TypeAddress);
if (NI != BC.GlobalAddresses.end()) {
OS << NI->second;
if (auto *TypeSymBD = BC.getBinaryDataAtAddress(TypeAddress)) {
OS << TypeSymBD->getName();
} else {
OS << "0x" << Twine::utohexstr(TypeAddress);
}
@ -507,9 +506,10 @@ void BinaryFunction::emitLSDA(MCStreamer *Streamer, bool EmitColdPart) {
const auto TTypeEncoding = BC.MOFI->getTTypeEncoding();
const auto TTypeEncodingSize = getEncodingSize(TTypeEncoding, BC);
const auto TTypeAlignment = 4;
// Type tables have to be aligned at 4 bytes.
Streamer->EmitValueToAlignment(4);
Streamer->EmitValueToAlignment(TTypeAlignment);
// Emit the LSDA label.
auto LSDASymbol = EmitColdPart ? getColdLSDASymbol() : getLSDASymbol();
@ -635,7 +635,11 @@ void BinaryFunction::emitLSDA(MCStreamer *Streamer, bool EmitColdPart) {
break;
case dwarf::DW_EH_PE_pcrel: {
if (TypeAddress) {
const auto *TypeSymbol = BC.getOrCreateGlobalSymbol(TypeAddress, "TI");
const auto *TypeSymbol =
BC.getOrCreateGlobalSymbol(TypeAddress,
TTypeEncodingSize,
TTypeAlignment,
"TI");
auto *DotSymbol = BC.Ctx->createTempSymbol();
Streamer->EmitLabel(DotSymbol);
const auto *SubDotExpr = MCBinaryExpr::createSub(

191
bolt/JumpTable.cpp Normal file
View File

@ -0,0 +1,191 @@
//===--- JumpTable.h - Representation of a jump table ---------------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
//===----------------------------------------------------------------------===//
#include "JumpTable.h"
#include "BinarySection.h"
#include "Relocation.h"
#include "llvm/MC/MCStreamer.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h"
#undef DEBUG_TYPE
#define DEBUG_TYPE "bolt"
using namespace llvm;
using namespace bolt;
namespace opts {
extern cl::opt<JumpTableSupportLevel> JumpTables;
extern cl::opt<unsigned> Verbosity;
}
std::pair<size_t, size_t>
JumpTable::getEntriesForAddress(const uint64_t Addr) const {
const uint64_t InstOffset = Addr - getAddress();
size_t StartIndex = 0, EndIndex = 0;
uint64_t Offset = 0;
for (size_t I = 0; I < Entries.size(); ++I) {
auto LI = Labels.find(Offset);
if (LI != Labels.end()) {
const auto NextLI = std::next(LI);
const auto NextOffset =
NextLI == Labels.end() ? getSize() : NextLI->first;
if (InstOffset >= LI->first && InstOffset < NextOffset) {
StartIndex = I;
EndIndex = I;
while (Offset < NextOffset) {
++EndIndex;
Offset += EntrySize;
}
break;
}
}
Offset += EntrySize;
}
return std::make_pair(StartIndex, EndIndex);
}
bool JumpTable::replaceDestination(uint64_t JTAddress,
const MCSymbol *OldDest,
MCSymbol *NewDest) {
bool Patched{false};
const auto Range = getEntriesForAddress(JTAddress);
for (auto I = &Entries[Range.first], E = &Entries[Range.second];
I != E; ++I) {
auto &Entry = *I;
if (Entry == OldDest) {
Patched = true;
Entry = NewDest;
}
}
return Patched;
}
void JumpTable::updateOriginal() {
// In non-relocation mode we have to emit jump tables in local sections.
// This way we only overwrite them when a corresponding function is
// overwritten.
const uint64_t BaseOffset = getAddress() - getSection().getAddress();
uint64_t Offset = BaseOffset;
for (auto *Entry : Entries) {
const auto RelType =
Type == JTT_NORMAL ? ELF::R_X86_64_64 : ELF::R_X86_64_PC32;
const uint64_t RelAddend = (Type == JTT_NORMAL ? 0 : Offset - BaseOffset);
DEBUG(dbgs() << "BOLT-DEBUG: adding relocation to section "
<< getSectionName() << " at offset 0x"
<< Twine::utohexstr(Offset) << " for symbol "
<< Entry->getName() << " with addend "
<< Twine::utohexstr(RelAddend) << '\n');
getSection().addRelocation(Offset, Entry, RelType, RelAddend);
Offset += EntrySize;
}
}
uint64_t JumpTable::emit(MCStreamer *Streamer,
MCSection *HotSection,
MCSection *ColdSection) {
// Pre-process entries for aggressive splitting.
// Each label represents a separate switch table and gets its own count
// determining its destination.
std::map<MCSymbol *, uint64_t> LabelCounts;
if (opts::JumpTables > JTS_SPLIT && !Counts.empty()) {
MCSymbol *CurrentLabel = Labels[0];
uint64_t CurrentLabelCount = 0;
for (unsigned Index = 0; Index < Entries.size(); ++Index) {
auto LI = Labels.find(Index * EntrySize);
if (LI != Labels.end()) {
LabelCounts[CurrentLabel] = CurrentLabelCount;
CurrentLabel = LI->second;
CurrentLabelCount = 0;
}
CurrentLabelCount += Counts[Index].Count;
}
LabelCounts[CurrentLabel] = CurrentLabelCount;
} else {
Streamer->SwitchSection(Count > 0 ? HotSection : ColdSection);
Streamer->EmitValueToAlignment(EntrySize);
}
MCSymbol *LastLabel = nullptr;
uint64_t Offset = 0;
for (auto *Entry : Entries) {
auto LI = Labels.find(Offset);
if (LI != Labels.end()) {
DEBUG(dbgs() << "BOLT-DEBUG: emitting jump table "
<< LI->second->getName() << " (originally was at address 0x"
<< Twine::utohexstr(getAddress() + Offset)
<< (Offset ? "as part of larger jump table\n" : "\n"));
if (!LabelCounts.empty()) {
DEBUG(dbgs() << "BOLT-DEBUG: jump table count: "
<< LabelCounts[LI->second] << '\n');
if (LabelCounts[LI->second] > 0) {
Streamer->SwitchSection(HotSection);
} else {
Streamer->SwitchSection(ColdSection);
}
Streamer->EmitValueToAlignment(EntrySize);
}
Streamer->EmitLabel(LI->second);
LastLabel = LI->second;
}
if (Type == JTT_NORMAL) {
Streamer->EmitSymbolValue(Entry, OutputEntrySize);
} else { // JTT_PIC
auto JT = MCSymbolRefExpr::create(LastLabel, Streamer->getContext());
auto E = MCSymbolRefExpr::create(Entry, Streamer->getContext());
auto Value = MCBinaryExpr::createSub(E, JT, Streamer->getContext());
Streamer->EmitValue(Value, EntrySize);
}
Offset += EntrySize;
}
return Offset;
}
void JumpTable::print(raw_ostream &OS) const {
uint64_t Offset = 0;
for (const auto *Entry : Entries) {
auto LI = Labels.find(Offset);
if (LI != Labels.end()) {
OS << "Jump Table " << LI->second->getName() << " at @0x"
<< Twine::utohexstr(getAddress()+Offset);
if (Offset) {
OS << " (possibly part of larger jump table):\n";
} else {
OS << " with total count of " << Count << ":\n";
}
}
OS << format(" 0x%04" PRIx64 " : ", Offset) << Entry->getName();
if (!Counts.empty()) {
OS << " : " << Counts[Offset / EntrySize].Mispreds
<< "/" << Counts[Offset / EntrySize].Count;
}
OS << '\n';
Offset += EntrySize;
}
OS << "\n\n";
}
JumpTable::JumpTable(StringRef Name,
uint64_t Address,
std::size_t EntrySize,
JumpTableType Type,
decltype(OffsetEntries) &&OffsetEntries,
decltype(Labels) &&Labels,
BinarySection &Section)
: BinaryData(Name, Address, 0, EntrySize, Section),
EntrySize(EntrySize),
OutputEntrySize(EntrySize),
Type(Type),
OffsetEntries(OffsetEntries),
Labels(Labels)
{ }

123
bolt/JumpTable.h Normal file
View File

@ -0,0 +1,123 @@
//===--- JumpTable.h - Representation of a jump table ---------------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_TOOLS_LLVM_BOLT_JUMP_TABLE_H
#define LLVM_TOOLS_LLVM_BOLT_JUMP_TABLE_H
#include "BinaryData.h"
#include "llvm/MC/MCSection.h"
#include "llvm/MC/MCStreamer.h"
#include "llvm/MC/MCSymbol.h"
#include <map>
#include <vector>
namespace llvm {
namespace bolt {
enum JumpTableSupportLevel : char {
JTS_NONE = 0, /// Disable jump tables support.
JTS_BASIC = 1, /// Enable basic jump tables support (in-place).
JTS_MOVE = 2, /// Move jump tables to a separate section.
JTS_SPLIT = 3, /// Enable hot/cold splitting of jump tables.
JTS_AGGRESSIVE = 4, /// Aggressive splitting of jump tables.
};
/// Representation of a jump table.
///
/// The jump table may include other jump tables that are referenced by
/// a different label at a different offset in this jump table.
class JumpTable : public BinaryData {
public:
enum JumpTableType : char {
JTT_NORMAL,
JTT_PIC,
};
/// Branch statistics for jump table entries.
struct JumpInfo {
uint64_t Mispreds{0};
uint64_t Count{0};
};
/// Size of the entry used for storage.
std::size_t EntrySize;
/// Size of the entry size we will write (we may use a more compact layout)
std::size_t OutputEntrySize;
/// The type of this jump table.
JumpTableType Type;
/// All the entries as labels.
std::vector<MCSymbol *> Entries;
/// All the entries as offsets into a function. Invalid after CFG is built.
std::vector<uint64_t> OffsetEntries;
/// Map <Offset> -> <Label> used for embedded jump tables. Label at 0 offset
/// is the main label for the jump table.
using LabelMapType = std::map<unsigned, MCSymbol *>;
LabelMapType Labels;
/// Dynamic number of times each entry in the table was referenced.
/// Identical entries will have a shared count (identical for every
/// entry in the set).
std::vector<JumpInfo> Counts;
/// Total number of times this jump table was used.
uint64_t Count{0};
/// Return the size of the jump table.
uint64_t getSize() const {
return std::max(OffsetEntries.size(), Entries.size()) * EntrySize;
}
const MCSymbol *getFirstLabel() const {
assert(Labels.count(0) != 0 && "labels must have an entry at 0");
return Labels.find(0)->second;
}
/// Get the indexes for symbol entries that correspond to the jump table
/// starting at (or containing) 'Addr'.
std::pair<size_t, size_t> getEntriesForAddress(const uint64_t Addr) const;
/// Constructor.
JumpTable(StringRef Name,
uint64_t Address,
std::size_t EntrySize,
JumpTableType Type,
decltype(OffsetEntries) &&OffsetEntries,
LabelMapType &&Labels,
BinarySection &Section);
virtual bool isJumpTable() const override { return true; }
/// Change all entries of the jump table in \p JTAddress pointing to
/// \p OldDest to \p NewDest. Return false if unsuccessful.
bool replaceDestination(uint64_t JTAddress, const MCSymbol *OldDest,
MCSymbol *NewDest);
/// Update jump table at its original location.
void updateOriginal();
/// Emit jump table data. Callee supplies sections for the data.
/// Return the number of total bytes emitted.
uint64_t emit(MCStreamer *Streamer, MCSection *HotSection,
MCSection *ColdSection);
/// Print for debugging purposes.
virtual void print(raw_ostream &OS) const override;
};
} // namespace bolt
} // namespace llvm
#endif

10
bolt/Passes/BinaryFunctionCallGraph.cpp
View File

@ -180,8 +180,8 @@ BinaryFunctionCallGraph buildCallGraph(BinaryContext &BC,
for (const auto &CSI : ICSP) {
if (!CSI.IsFunction)
continue;
if (auto DstSym = BC.getGlobalSymbolByName(CSI.Name)) {
Counts.push_back(std::make_pair(DstSym, CSI.Count));
if (auto *DstBD = BC.getBinaryDataByName(CSI.Name)) {
Counts.push_back(std::make_pair(DstBD->getSymbol(), CSI.Count));
}
}
} else {
@ -207,8 +207,8 @@ BinaryFunctionCallGraph buildCallGraph(BinaryContext &BC,
if (!CSI.IsFunction)
continue;
auto *DstSym = BC.getGlobalSymbolByName(CSI.Name);
if (!DstSym)
auto *DstBD = BC.getBinaryDataByName(CSI.Name);
if (!DstBD)
continue;
// The computed offset may exceed the hot part of the function; hence,
@ -217,7 +217,7 @@ BinaryFunctionCallGraph buildCallGraph(BinaryContext &BC,
if (Offset > Size)
Offset = Size;
if (!recordCall(DstSym, CSI.Count)) {
if (!recordCall(DstBD->getSymbol(), CSI.Count)) {
++NotProcessed;
}
}

16
bolt/Passes/BinaryPasses.cpp
View File

@ -1139,17 +1139,21 @@ bool SimplifyRODataLoads::simplifyRODataLoads(
"found PC-relative with non-symbolic displacement");
// Get displacement symbol.
const MCSymbolRefExpr *DisplExpr;
if (!(DisplExpr = dyn_cast<MCSymbolRefExpr>(DispOpI->getExpr())))
const MCSymbol *DisplSymbol;
uint64_t DisplOffset;
std::tie(DisplSymbol, DisplOffset) =
BC.MIA->getTargetSymbolInfo(DispOpI->getExpr());
if (!DisplSymbol)
continue;
const MCSymbol &DisplSymbol = DisplExpr->getSymbol();
// Look up the symbol address in the global symbols map of the binary
// context object.
auto GI = BC.GlobalSymbols.find(DisplSymbol.getName());
if (GI == BC.GlobalSymbols.end())
auto *BD = BC.getBinaryDataByName(DisplSymbol->getName());
if (!BD)
continue;
TargetAddress = GI->second;
TargetAddress = BD->getAddress() + DisplOffset;
} else if (!MIA->evaluateMemOperandTarget(Inst, TargetAddress)) {
continue;
}

49
bolt/Passes/IndirectCallPromotion.cpp
View File

@ -147,7 +147,10 @@ IndirectCallPromotion::Callsite::Callsite(BinaryFunction &BF,
Mispreds(ICP.Mispreds),
Branches(ICP.Count) {
if (ICP.IsFunction) {
To.Sym = BF.getBinaryContext().getGlobalSymbolByName(ICP.Name);
if (auto *BD = BF.getBinaryContext().getBinaryDataByName(ICP.Name)) {
To.Sym = BD->getSymbol();
To.Addr = 0;
}
}
}
@ -163,16 +166,15 @@ IndirectCallPromotion::getCallTargets(
if (const auto *JT = BF.getJumpTable(Inst)) {
// Don't support PIC jump tables for now
if (!opts::ICPJumpTablesByTarget &&
JT->Type == BinaryFunction::JumpTable::JTT_PIC)
if (!opts::ICPJumpTablesByTarget && JT->Type == JumpTable::JTT_PIC)
return Targets;
const Location From(BF.getSymbol());
const auto Range = JT->getEntriesForAddress(BC.MIA->getJumpTable(Inst));
assert(JT->Counts.empty() || JT->Counts.size() >= Range.second);
BinaryFunction::JumpInfo DefaultJI;
JumpTable::JumpInfo DefaultJI;
const auto *JI = JT->Counts.empty() ? &DefaultJI : &JT->Counts[Range.first];
const size_t JIAdj = JT->Counts.empty() ? 0 : 1;
assert(JT->Type == BinaryFunction::JumpTable::JTT_PIC ||
assert(JT->Type == JumpTable::JTT_PIC ||
JT->EntrySize == BC.AsmInfo->getCodePointerSize());
for (size_t I = Range.first; I < Range.second; ++I, JI += JIAdj) {
auto *Entry = JT->Entries[I];
@ -290,7 +292,7 @@ IndirectCallPromotion::maybeGetHotJumpTableTargets(
BinaryBasicBlock *BB,
MCInst &CallInst,
MCInst *&TargetFetchInst,
const BinaryFunction::JumpTable *JT
const JumpTable *JT
) const {
const auto *MemData = Function.getMemData();
JumpTableInfoType HotTargets;
@ -349,9 +351,9 @@ IndirectCallPromotion::maybeGetHotJumpTableTargets(
uint64_t ArrayStart;
if (DispExpr) {
auto SI = BC.GlobalSymbols.find(DispExpr->getSymbol().getName());
assert(SI != BC.GlobalSymbols.end() && "global symbol needs a value");
ArrayStart = SI->second;
auto *BD = BC.getBinaryDataByName(DispExpr->getSymbol().getName());
assert(BD && "global symbol needs a value");
ArrayStart = BD->getAddress();
} else {
ArrayStart = static_cast<uint64_t>(DispValue);
}
@ -388,12 +390,8 @@ IndirectCallPromotion::maybeGetHotJumpTableTargets(
if (MI.Addr.IsSymbol) {
// Deal with bad/stale data
if (MI.Addr.Name != (std::string("JUMP_TABLEat0x") +
Twine::utohexstr(JT->Address).str()) &&
MI.Addr.Name != (std::string("JUMP_TABLEat0x") +
Twine::utohexstr(ArrayStart).str())) {
if (!MI.Addr.Name.startswith("JUMP_TABLE/" + Function.getNames().front()))
return JumpTableInfoType();
}
Index = MI.Addr.Offset / JT->EntrySize;
} else {
Index = (MI.Addr.Offset - ArrayStart) / JT->EntrySize;
@ -602,26 +600,31 @@ IndirectCallPromotion::maybeGetVtableAddrs(
std::map<const MCSymbol *, uint64_t> MethodToVtable;
for (auto &MI : MemData->getMemInfoRange(DataOffset.get())) {
ErrorOr<uint64_t> Address = MI.Addr.IsSymbol
? BC.getAddressForGlobalSymbol(MI.Addr.Name)
: MI.Addr.Offset;
uint64_t Address;
if (MI.Addr.IsSymbol) {
auto *BD = BC.getBinaryDataByName(MI.Addr.Name);
Address = BD ? BD->getAddress() + MI.Addr.Offset : 0;
} else {
Address = MI.Addr.Offset;
}
// Ignore bogus data.
if (!Address)
continue;
if (MI.Addr.IsSymbol)
Address = Address.get() + MI.Addr.Offset;
const auto VtableBase = Address.get() - MethodOffset;
const auto VtableBase = Address - MethodOffset;
DEBUG_VERBOSE(1, dbgs() << "BOLT-INFO: ICP vtable = "
<< Twine::utohexstr(VtableBase)
<< "+" << MethodOffset << "/" << MI.Count
<< "\n");
if (auto MethodAddr = BC.extractPointerAtAddress(Address.get())) {
auto *MethodSym = BC.getGlobalSymbolAtAddress(MethodAddr.get());
if (auto MethodAddr = BC.extractPointerAtAddress(Address)) {
auto *MethodBD = BC.getBinaryDataAtAddress(MethodAddr.get());
if (!MethodBD) // skip unknown methods
continue;
auto *MethodSym = MethodBD->getSymbol();
MethodToVtable[MethodSym] = VtableBase;
DEBUG_VERBOSE(1,
const auto *Method = BC.getFunctionForSymbol(MethodSym);

2
bolt/Passes/IndirectCallPromotion.h
View File

@ -197,7 +197,7 @@ class IndirectCallPromotion : public BinaryFunctionPass {
BinaryBasicBlock *BB,
MCInst &Inst,
MCInst *&TargetFetchInst,
const BinaryFunction::JumpTable *JT) const;
const JumpTable *JT) const;
SymTargetsType findCallTargetSymbols(BinaryContext &BC,
std::vector<Callsite> &Targets,

8
bolt/Passes/JTFootprintReduction.cpp
View File

@ -42,7 +42,7 @@ namespace bolt {
void JTFootprintReduction::checkOpportunities(BinaryContext &BC,
BinaryFunction &Function,
DataflowInfoManager &Info) {
std::map<BinaryFunction::JumpTable *, uint64_t> AllJTs;
std::map<JumpTable *, uint64_t> AllJTs;
for (auto &BB : Function) {
for (auto &Inst : BB) {
@ -125,7 +125,7 @@ void JTFootprintReduction::checkOpportunities(BinaryContext &BC,
bool JTFootprintReduction::tryOptimizeNonPIC(
BinaryContext &BC, BinaryBasicBlock &BB, MCInst &Inst, uint64_t JTAddr,
BinaryFunction::JumpTable *JumpTable, DataflowInfoManager &Info) {
JumpTable *JumpTable, DataflowInfoManager &Info) {
if (opts::JTFootprintOnlyPIC)
return false;
@ -165,7 +165,7 @@ bool JTFootprintReduction::tryOptimizeNonPIC(
bool JTFootprintReduction::tryOptimizePIC(
BinaryContext &BC, BinaryBasicBlock &BB, MCInst &Inst, uint64_t JTAddr,
BinaryFunction::JumpTable *JumpTable, DataflowInfoManager &Info) {
JumpTable *JumpTable, DataflowInfoManager &Info) {
MCPhysReg BaseReg;
uint64_t Scale;
MCPhysReg Index;
@ -195,7 +195,7 @@ bool JTFootprintReduction::tryOptimizePIC(
JumpTable->OutputEntrySize = 4;
// DePICify
JumpTable->Type = BinaryFunction::JumpTable::JTT_NORMAL;
JumpTable->Type = JumpTable::JTT_NORMAL;
BB.replaceInstruction(&Inst, NewFrag.begin(), NewFrag.end());
return true;

6
bolt/Passes/JTFootprintReduction.h
View File

@ -35,7 +35,7 @@ class JTFootprintReduction : public BinaryFunctionPass {
uint64_t NumJTsBadMatch{0};
uint64_t NumJTsNoReg{0};
uint64_t BytesSaved{0};
DenseSet<BinaryFunction::JumpTable *> BlacklistedJTs;
DenseSet<JumpTable *> BlacklistedJTs;
DenseSet<const BinaryFunction *> Modified;
/// Check if \p Function presents jump tables where all jump locations can
@ -50,13 +50,13 @@ class JTFootprintReduction : public BinaryFunctionPass {
/// instructions that depend on the availability of an extra register.
/// This saves dcache/dTLB at the expense of icache.
bool tryOptimizeNonPIC(BinaryContext &BC, BinaryBasicBlock &BB, MCInst &Inst,
uint64_t JTAddr, BinaryFunction::JumpTable *JumpTable,
uint64_t JTAddr, JumpTable *JumpTable,
DataflowInfoManager &Info);
/// The PIC jump table optimization consists of "de-pic-ifying" it, since the
/// PIC jump sequence is larger than its non-PIC counterpart, saving icache.
bool tryOptimizePIC(BinaryContext &BC, BinaryBasicBlock &BB, MCInst &Inst,
uint64_t JTAddr, BinaryFunction::JumpTable *JumpTable,
uint64_t JTAddr, JumpTable *JumpTable,
DataflowInfoManager &Info);
/// Run a pass for \p Function

6
bolt/Passes/LongJmp.cpp
View File

@ -357,9 +357,9 @@ uint64_t LongJmpPass::getSymbolAddress(const BinaryContext &BC,
if (Iter == HotAddresses.end()) {
// Look at BinaryContext's resolution for this symbol - this is a symbol not
// mapped to a BinaryFunction
auto SymIter = BC.GlobalSymbols.find(Target->getName());
assert (SymIter != BC.GlobalSymbols.end() && "Unrecognized symbol");
return SymIter->second;
auto *BD = BC.getBinaryDataByName(Target->getName());
assert(BD && "Unrecognized symbol");
return BD ? BD->getAddress() : 0;
}
return Iter->second;
}

19
bolt/Passes/ReorderFunctions.cpp
View File

@ -358,20 +358,21 @@ void ReorderFunctions::runOnFunctions(BinaryContext &BC,
for (const auto &Function : readFunctionOrderFile()) {
std::vector<uint64_t> FuncAddrs;
auto Itr = BC.GlobalSymbols.find(Function);
if (Itr == BC.GlobalSymbols.end()) {
auto *BD = BC.getBinaryDataByName(Function);
if (!BD) {
uint32_t LocalID = 1;
while(1) {
// If we can't find the main symbol name, look for alternates.
Itr = BC.GlobalSymbols.find(Function + "/" + std::to_string(LocalID));
if (Itr != BC.GlobalSymbols.end())
FuncAddrs.push_back(Itr->second);
const auto FuncName = Function + "/" + std::to_string(LocalID);
BD = BC.getBinaryDataByName(FuncName);
if (BD)
FuncAddrs.push_back(BD->getAddress());
else
break;
LocalID++;
}
} else {
FuncAddrs.push_back(Itr->second);
FuncAddrs.push_back(BD->getAddress());
}
if (FuncAddrs.empty()) {
@ -381,10 +382,10 @@ void ReorderFunctions::runOnFunctions(BinaryContext &BC,
}
for (const auto FuncAddr : FuncAddrs) {
const auto *FuncSym = BC.getOrCreateGlobalSymbol(FuncAddr, "FUNCat");
assert(FuncSym);
const auto *FuncBD = BC.getBinaryDataAtAddress(FuncAddr);
assert(FuncBD);
auto *BF = BC.getFunctionForSymbol(FuncSym);
auto *BF = BC.getFunctionForSymbol(FuncBD->getSymbol());
if (!BF) {
errs() << "BOLT-WARNING: Reorder functions: can't find function for "
<< Function << ".\n";

7
bolt/ProfileWriter.cpp
View File

@ -78,9 +78,10 @@ convert(const BinaryFunction &BF, yaml::bolt::BinaryFunctionProfile &YamlBF) {
CSI.DestId = 0; // designated for unknown functions
CSI.EntryDiscriminator = 0;
if (CSP.IsFunction) {
const auto *CalleeSymbol = BC.getGlobalSymbolByName(CSP.Name);
if (CalleeSymbol) {
const auto *Callee = BC.getFunctionForSymbol(CalleeSymbol);
const auto *CalleeBD = BC.getBinaryDataByName(CSP.Name);
if (CalleeBD) {
const auto *Callee =
BC.getFunctionForSymbol(CalleeBD->getSymbol());
if (Callee) {
CSI.DestId = Callee->getFunctionNumber();
}

326
bolt/Relocation.cpp Normal file
View File

@ -0,0 +1,326 @@
//===--- Relocation.cpp - Interface for object file relocations ----------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
//===----------------------------------------------------------------------===//
#include "Relocation.h"
#include "llvm/MC/MCContext.h"
#include "llvm/MC/MCStreamer.h"
using namespace llvm;
using namespace bolt;
Triple::ArchType Relocation::Arch;
bool Relocation::isSupported(uint64_t Type) {
switch (Type) {
default:
return false;
case ELF::R_X86_64_8:
case ELF::R_X86_64_16:
case ELF::R_X86_64_32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_64:
case ELF::R_X86_64_PC8:
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_PC64:
case ELF::R_X86_64_PLT32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_TPOFF32:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_CALL26:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC:
case ELF::R_AARCH64_ADD_ABS_LO12_NC:
case ELF::R_AARCH64_LDST128_ABS_LO12_NC:
case ELF::R_AARCH64_LDST32_ABS_LO12_NC:
case ELF::R_AARCH64_LDST16_ABS_LO12_NC:
case ELF::R_AARCH64_LDST8_ABS_LO12_NC:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_PREL32:
case ELF::R_AARCH64_ABS64:
return true;
}
}
size_t Relocation::getSizeForType(uint64_t Type) {
switch (Type) {
default:
llvm_unreachable("unsupported relocation type");
case ELF::R_X86_64_8:
case ELF::R_X86_64_PC8:
return 1;
case ELF::R_X86_64_16:
return 2;
case ELF::R_X86_64_PLT32:
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_TPOFF32:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_CALL26:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC:
case ELF::R_AARCH64_ADD_ABS_LO12_NC:
case ELF::R_AARCH64_LDST128_ABS_LO12_NC:
case ELF::R_AARCH64_LDST32_ABS_LO12_NC:
case ELF::R_AARCH64_LDST16_ABS_LO12_NC:
case ELF::R_AARCH64_LDST8_ABS_LO12_NC:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_PREL32:
return 4;
case ELF::R_X86_64_PC64:
case ELF::R_X86_64_64:
case ELF::R_AARCH64_ABS64:
return 8;
}
}
uint64_t Relocation::extractValue(uint64_t Type, uint64_t Contents,
uint64_t PC) {
switch (Type) {
default:
llvm_unreachable("unsupported relocation type");
case ELF::R_AARCH64_ABS64:
return Contents;
case ELF::R_AARCH64_PREL32:
return static_cast<int64_t>(PC) + SignExtend64<32>(Contents & 0xffffffff);
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_CALL26:
// Immediate goes in bits 25:0 of B and BL.
Contents &= ~0xfffffffffc000000ULL;
return static_cast<int64_t>(PC) + SignExtend64<28>(Contents << 2);
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_ADR_PREL_PG_HI21: {
// Bits 32:12 of Symbol address goes in bits 30:29 + 23:5 of ADRP
// instruction
Contents &= ~0xffffffff9f00001fUll;
auto LowBits = (Contents >> 29) & 0x3;
auto HighBits = (Contents >> 5) & 0x7ffff;
Contents = LowBits | (HighBits << 2);
Contents = static_cast<int64_t>(PC) + SignExtend64<32>(Contents << 12);
Contents &= ~0xfffUll;
return Contents;
}
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of LD/ST instruction, taken
// from bits 11:3 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 3);
}
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_ADD_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 0);
}
case ELF::R_AARCH64_LDST128_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:4 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 4);
}
case ELF::R_AARCH64_LDST32_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:2 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 2);
}
case ELF::R_AARCH64_LDST16_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:1 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 1);
}
case ELF::R_AARCH64_LDST8_ABS_LO12_NC: {
// Immediate goes in bits 21:10 of ADD instruction, taken
// from bits 11:0 of Symbol address
Contents &= ~0xffffffffffc003ffU;
return Contents >> (10 - 0);
}
}
}
bool Relocation::isGOT(uint64_t Type) {
switch (Type) {
default:
return false;
case ELF::R_X86_64_GOT32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_GOTOFF64:
case ELF::R_X86_64_GOTPC32:
case ELF::R_X86_64_GOT64:
case ELF::R_X86_64_GOTPCREL64:
case ELF::R_X86_64_GOTPC64:
case ELF::R_X86_64_GOTPLT64:
case ELF::R_X86_64_GOTPC32_TLSDESC:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
return true;
}
}
bool Relocation::isTLS(uint64_t Type) {
switch (Type) {
default:
return false;
case ELF::R_X86_64_TPOFF32:
case ELF::R_X86_64_TPOFF64:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
return true;
}
}
bool Relocation::isPCRelative(uint64_t Type) {
switch (Type) {
default:
llvm_unreachable("Unknown relocation type");
case ELF::R_X86_64_64:
case ELF::R_X86_64_32:
case ELF::R_X86_64_32S:
case ELF::R_X86_64_16:
case ELF::R_X86_64_8:
case ELF::R_X86_64_TPOFF32:
case ELF::R_AARCH64_ABS64:
case ELF::R_AARCH64_LDST64_ABS_LO12_NC:
case ELF::R_AARCH64_ADD_ABS_LO12_NC:
case ELF::R_AARCH64_LDST128_ABS_LO12_NC:
case ELF::R_AARCH64_LDST32_ABS_LO12_NC:
case ELF::R_AARCH64_LDST16_ABS_LO12_NC:
case ELF::R_AARCH64_LDST8_ABS_LO12_NC:
case ELF::R_AARCH64_TLSIE_LD64_GOTTPREL_LO12_NC:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_HI12:
case ELF::R_AARCH64_TLSLE_ADD_TPREL_LO12_NC:
case ELF::R_AARCH64_LD64_GOT_LO12_NC:
case ELF::R_AARCH64_TLSDESC_LD64_LO12:
case ELF::R_AARCH64_TLSDESC_ADD_LO12:
return false;
case ELF::R_X86_64_PC8:
case ELF::R_X86_64_PC32:
case ELF::R_X86_64_GOTPCREL:
case ELF::R_X86_64_PLT32:
case ELF::R_X86_64_GOTTPOFF:
case ELF::R_X86_64_GOTPCRELX:
case ELF::R_X86_64_REX_GOTPCRELX:
case ELF::R_AARCH64_TLSDESC_CALL:
case ELF::R_AARCH64_CALL26:
case ELF::R_AARCH64_ADR_PREL_PG_HI21:
case ELF::R_AARCH64_ADR_GOT_PAGE:
case ELF::R_AARCH64_TLSIE_ADR_GOTTPREL_PAGE21:
case ELF::R_AARCH64_TLSDESC_ADR_PAGE21:
case ELF::R_AARCH64_JUMP26:
case ELF::R_AARCH64_PREL32:
return true;
}
}
size_t Relocation::emit(MCStreamer *Streamer) const {
const auto Size = getSizeForType(Type);
auto &Ctx = Streamer->getContext();
if (isPCRelative(Type)) {
auto *TempLabel = Ctx.createTempSymbol();
Streamer->EmitLabel(TempLabel);
auto Value =
MCBinaryExpr::createSub(MCSymbolRefExpr::create(Symbol, Ctx),
MCSymbolRefExpr::create(TempLabel, Ctx),
Ctx);
if (Addend) {
Value = MCBinaryExpr::createAdd(Value,
MCConstantExpr::create(Addend, Ctx),
Ctx);
}
Streamer->EmitValue(Value, Size);
} else if (Addend) {
auto Value = MCBinaryExpr::createAdd(MCSymbolRefExpr::create(Symbol, Ctx),
MCConstantExpr::create(Addend, Ctx),
Ctx);
Streamer->EmitValue(Value, Size);
} else {
Streamer->EmitSymbolValue(Symbol, Size);
}
return Size;
}
#define ELF_RELOC(name, value) #name,
void Relocation::print(raw_ostream &OS) const {
static const char *X86RelocNames[] = {
#include "llvm/BinaryFormat/ELFRelocs/x86_64.def"
};
static const char *AArch64RelocNames[] = {
#include "llvm/BinaryFormat/ELFRelocs/AArch64.def"
};
if (Arch == Triple::aarch64)
OS << AArch64RelocNames[Type];
else
OS << X86RelocNames[Type];
OS << ", 0x" << Twine::utohexstr(Offset);
if (Symbol) {
OS << ", " << Symbol->getName();
}
if (int64_t(Addend) < 0)
OS << ", -0x" << Twine::utohexstr(-int64_t(Addend));
else
OS << ", 0x" << Twine::utohexstr(Addend);
OS << ", 0x" << Twine::utohexstr(Value);
}

90
bolt/Relocation.h Normal file
View File

@ -0,0 +1,90 @@
//===--- Relocation.h - Interface for object file relocations ------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_TOOLS_LLVM_BOLT_RELOCATION_H
#define LLVM_TOOLS_LLVM_BOLT_RELOCATION_H
#include "llvm/ADT/Triple.h"
#include "llvm/MC/MCSymbol.h"
#include "llvm/BinaryFormat/ELF.h"
#include "llvm/Support/raw_ostream.h"
namespace llvm {
namespace bolt {
/// Relocation class.
struct Relocation {
static Triple::ArchType Arch; /// for printing, set by BinaryContext ctor.
/// The offset of this relocation in the object it is contained in.
uint64_t Offset;
/// The symbol this relocation is referring to.
MCSymbol *Symbol;
/// Relocation type.
uint64_t Type;
/// The offset from the \p Symbol base used to compute the final
/// value of this relocation.
uint64_t Addend;
/// The computed relocation value extracted from the binary file.
/// Used to validate relocation correctness.
uint64_t Value;
/// Return size of the given relocation \p Type.
static size_t getSizeForType(uint64_t Type);
/// Extract current relocated value from binary contents. This is used for
/// RISC architectures where values are encoded in specific bits depending
/// on the relocation value.
static uint64_t extractValue(uint64_t Type, uint64_t Contents, uint64_t PC);
/// Return true if relocation type is PC-relative. Return false otherwise.
static bool isPCRelative(uint64_t Type);
/// Check if \p Type is a supported relocation type.
static bool isSupported(uint64_t Type);
/// Return true if relocation type implies the creation of a GOT entry
static bool isGOT(uint64_t Type);
/// Return true if relocation type is for thread local storage.
static bool isTLS(uint64_t Type);
/// Return true if this relocation is PC-relative. Return false otherwise.
bool isPCRelative() const {
return isPCRelative(Type);
}
/// Emit relocation at a current \p Streamer' position. The caller is
/// responsible for setting the position correctly.
size_t emit(MCStreamer *Streamer) const;
/// Print a relocation to \p OS.
void print(raw_ostream &OS) const;
};
/// Relocation ordering by offset.
inline bool operator<(const Relocation &A, const Relocation &B) {
return A.Offset < B.Offset;
}
inline raw_ostream &operator<<(raw_ostream &OS, const Relocation &Rel) {
Rel.print(OS);
return OS;
}
} // namespace bolt
} // namespace llvm
#endif

386
bolt/RewriteInstance.cpp
View File

@ -199,6 +199,13 @@ MaxFunctions("max-funcs",
cl::Hidden,
cl::cat(BoltCategory));
static cl::opt<unsigned>
MaxDataRelocations("max-data-relocations",
cl::desc("maximum number of data relocations to process"),
cl::ZeroOrMore,
cl::Hidden,
cl::cat(BoltCategory));
cl::opt<bool>
PrintAll("print-all",
cl::desc("print functions after each stage"),
@ -220,6 +227,13 @@ PrintDisasm("print-disasm",
cl::Hidden,
cl::cat(BoltCategory));
static cl::opt<bool>
PrintGlobals("print-globals",
cl::desc("print global symbols after disassembly"),
cl::ZeroOrMore,
cl::Hidden,
cl::cat(BoltCategory));
static cl::opt<bool>
PrintSections("print-sections",
cl::desc("print all registered sections"),
@ -456,6 +470,15 @@ void check_error(std::error_code EC, StringRef Message) {
}
}
namespace {
std::string uniquifyName(BinaryContext &BC, std::string NamePrefix) {
unsigned LocalID = 1;
while (BC.getBinaryDataByName(NamePrefix + std::to_string(LocalID)))
++LocalID;
return NamePrefix + std::to_string(LocalID);
}
}
uint8_t *ExecutableFileMemoryManager::allocateSection(intptr_t Size,
unsigned Alignment,
unsigned SectionID,
@ -980,7 +1003,7 @@ void RewriteInstance::discoverFileObjects() {
FileSymRefs.clear();
BinaryFunctions.clear();
BC->GlobalAddresses.clear();
BC->clearBinaryData();
// For local symbols we want to keep track of associated FILE symbol name for
// disambiguation by combined name.
@ -1053,7 +1076,37 @@ void RewriteInstance::discoverFileObjects() {
});
}
auto getNextAddress = [&](std::vector<SymbolRef>::const_iterator Itr) {
auto Section = cantFail(Itr->getSection());
const auto SymbolEndAddress =
(cantFail(Itr->getAddress()) + ELFSymbolRef(*Itr).getSize());
// absolute sym
if (Section == InputFile->section_end())
return SymbolEndAddress;
while (Itr != MarkersBegin - 1 &&
cantFail(std::next(Itr)->getSection()) == Section &&
cantFail(std::next(Itr)->getAddress()) ==
cantFail(Itr->getAddress())) {
++Itr;
}
if (Itr != MarkersBegin - 1 &&
cantFail(std::next(Itr)->getSection()) == Section)
return cantFail(std::next(Itr)->getAddress());
const auto SectionEndAddress = Section->getAddress() + Section->getSize();
if ((ELFSectionRef(*Section).getFlags() & ELF::SHF_TLS) ||
SymbolEndAddress > SectionEndAddress)
return SymbolEndAddress;
return SectionEndAddress;
};
BinaryFunction *PreviousFunction = nullptr;
unsigned AnonymousId = 0;
for (auto ISym = SortedFileSymbols.begin(); ISym != MarkersBegin; ++ISym) {
const auto &Symbol = *ISym;
// Keep undefined symbols for pretty printing?
@ -1075,11 +1128,6 @@ void RewriteInstance::discoverFileObjects() {
FileSymRefs[Address] = Symbol;
// There's nothing horribly wrong with anonymous symbols, but let's
// ignore them for now.
if (SymName.empty())
continue;
/// It is possible we are seeing a globalized local. LLVM might treat it as
/// a local if it has a "private global" prefix, e.g. ".L". Thus we have to
/// change the prefix to enforce global scope of the symbol.
@ -1095,9 +1143,14 @@ void RewriteInstance::discoverFileObjects() {
// the one we use for profile data.
std::string UniqueName;
std::string AlternativeName;
if (Symbol.getFlags() & SymbolRef::SF_Global) {
assert(BC->GlobalSymbols.find(Name) == BC->GlobalSymbols.end() &&
"global name not unique");
if (Name.empty()) {
if (PLTSection && PLTSection->getAddress() == Address) {
// Don't register BOLT_PLT_PSEUDO twice.
continue;
}
UniqueName = "ANONYMOUS." + std::to_string(AnonymousId++);
} else if (Symbol.getFlags() & SymbolRef::SF_Global) {
assert(!BC->getBinaryDataByName(Name) && "global name not unique");
UniqueName = Name;
} else {
// If we have a local file name, we should create 2 variants for the
@ -1118,27 +1171,32 @@ void RewriteInstance::discoverFileObjects() {
AltPrefix = Prefix + std::string(SFI->second) + "/";
}
auto uniquifyName = [&] (std::string NamePrefix) {
unsigned LocalID = 1;
while (BC->GlobalSymbols.find(NamePrefix + std::to_string(LocalID))
!= BC->GlobalSymbols.end())
++LocalID;
return NamePrefix + std::to_string(LocalID);
};
UniqueName = uniquifyName(Prefix);
UniqueName = uniquifyName(*BC, Prefix);
if (!AltPrefix.empty())
AlternativeName = uniquifyName(AltPrefix);
AlternativeName = uniquifyName(*BC, AltPrefix);
}
// Register names even if it's not a function, e.g. for an entry point.
BC->registerNameAtAddress(UniqueName, Address);
if (!AlternativeName.empty())
BC->registerNameAtAddress(AlternativeName, Address);
uint64_t SymbolSize = ELFSymbolRef(Symbol).getSize();
uint64_t NextAddress = getNextAddress(ISym);
uint64_t TentativeSize = !SymbolSize ? NextAddress - Address : SymbolSize;
uint64_t SymbolAlignment = Symbol.getAlignment();
auto registerName = [&](uint64_t FinalSize) {
// Register names even if it's not a function, e.g. for an entry point.
BC->registerNameAtAddress(UniqueName, Address, FinalSize, SymbolAlignment);
if (!AlternativeName.empty())
BC->registerNameAtAddress(AlternativeName, Address, FinalSize,
SymbolAlignment);
};
section_iterator Section =
cantFail(Symbol.getSection(), "cannot get symbol section");
if (Section == InputFile->section_end()) {
// Could be an absolute symbol. Could record for pretty printing.
DEBUG(if (opts::Verbosity > 1) {
dbgs() << "BOLT-INFO: absolute sym " << UniqueName << "\n";
});
registerName(TentativeSize);
continue;
}
@ -1149,11 +1207,10 @@ void RewriteInstance::discoverFileObjects() {
assert(cantFail(Symbol.getType()) != SymbolRef::ST_Function &&
"unexpected function inside non-code section");
DEBUG(dbgs() << "BOLT-DEBUG: rejecting as symbol is not in code\n");
registerName(TentativeSize);
continue;
}
auto SymbolSize = ELFSymbolRef(Symbol).getSize();
// Assembly functions could be ST_NONE with 0 size. Check that the
// corresponding section is a code section and they are not inside any
// other known function to consider them.
@ -1166,15 +1223,18 @@ void RewriteInstance::discoverFileObjects() {
if (PreviousFunction->getSize() == 0) {
if (PreviousFunction->isSymbolValidInScope(Symbol, SymbolSize)) {
DEBUG(dbgs() << "BOLT-DEBUG: symbol is a function local symbol\n");
registerName(SymbolSize);
continue;
}
} else if (PreviousFunction->containsAddress(Address)) {
if (PreviousFunction->isSymbolValidInScope(Symbol, SymbolSize)) {
DEBUG(dbgs() << "BOLT-DEBUG: symbol is a function local symbol\n");
registerName(SymbolSize);
continue;
} else {
if (Address == PreviousFunction->getAddress() && SymbolSize == 0) {
DEBUG(dbgs() << "BOLT-DEBUG: ignoring symbol as a marker\n");
registerName(SymbolSize);
continue;
}
if (opts::Verbosity > 1) {
@ -1182,6 +1242,7 @@ void RewriteInstance::discoverFileObjects() {
<< " seen in the middle of function "
<< *PreviousFunction << ". Could be a new entry.\n";
}
registerName(SymbolSize);
continue;
}
}
@ -1213,6 +1274,7 @@ void RewriteInstance::discoverFileObjects() {
assert(SI->second == Symbol && "wrong symbol found");
FileSymRefs.erase(SI);
}
registerName(SymbolSize);
continue;
}
@ -1246,7 +1308,13 @@ void RewriteInstance::discoverFileObjects() {
<< "; symbol table : " << SymbolSize << ". Using max size.\n";
}
SymbolSize = std::max(SymbolSize, FDE.getAddressRange());
if (BC->getBinaryDataAtAddress(Address)) {
BC->setBinaryDataSize(Address, SymbolSize);
} else {
DEBUG(dbgs() << "BOLT-DEBUG: No BD @ 0x" << Twine::utohexstr(Address) << "\n");
}
}
TentativeSize = SymbolSize;
}
BinaryFunction *BF{nullptr};
@ -1265,6 +1333,7 @@ void RewriteInstance::discoverFileObjects() {
<< " old " << BF->getSize() << " new " << SymbolSize << "\n";
}
BF->setSize(std::max(SymbolSize, BF->getSize()));
BC->setBinaryDataSize(Address, BF->getSize());
}
BF->addAlternativeName(UniqueName);
} else {
@ -1276,6 +1345,7 @@ void RewriteInstance::discoverFileObjects() {
if (!AlternativeName.empty())
BF->addAlternativeName(AlternativeName);
registerName(SymbolSize);
PreviousFunction = BF;
}
@ -1368,8 +1438,9 @@ void RewriteInstance::disassemblePLT() {
// Pseudo function for the start of PLT. The table could have a matching
// FDE that we want to match to pseudo function.
createBinaryFunction("__BOLT_PLT_PSEUDO", *PLTSection, PLTAddress, 0, false);
for (uint64_t Offset = 0; Offset < PLTSection->getSize(); Offset += 0x10) {
createBinaryFunction("__BOLT_PLT_PSEUDO", *PLTSection, PLTAddress, 0, false,
PLTSize, PLTAlignment);
for (uint64_t Offset = 0; Offset < PLTSection->getSize(); Offset += PLTSize) {
uint64_t InstrSize;
MCInst Instruction;
const uint64_t InstrAddr = PLTAddress + Offset;
@ -1407,13 +1478,18 @@ void RewriteInstance::disassemblePLT() {
"non-null symbol expected");
const auto SymbolName = cantFail((*SymbolIter).getName());
std::string Name = SymbolName.str() + "@PLT";
const auto PtrSize = BC->AsmInfo->getCodePointerSize();
auto *BF = createBinaryFunction(Name,
*PLTSection,
InstrAddr,
0,
/*IsSimple=*/false);
/*IsSimple=*/false,
PLTSize,
PLTAlignment);
auto TargetSymbol = BC->registerNameAtAddress(SymbolName.str() + "@GOT",
TargetAddress);
TargetAddress,
PtrSize,
PLTAlignment);
BF->setPLTSymbol(TargetSymbol);
break;
}
@ -1428,7 +1504,8 @@ void RewriteInstance::disassemblePLT() {
*PLTGOTSection,
PLTGOTSection->getAddress(),
0,
false);
false,
PLTAlignment);
}
}
}
@ -1538,17 +1615,19 @@ void RewriteInstance::relocateEHFrameSection() {
break;
}
auto *Symbol = BC->getGlobalSymbolAtAddress(Value);
auto *BD = BC->getBinaryDataContainingAddress(Value);
auto *Symbol = BD ? BD->getSymbol() : nullptr;
auto Addend = BD ? Value - BD->getAddress() : 0;
if (!Symbol) {
DEBUG(dbgs() << "BOLT-DEBUG: creating symbol for DWARF reference at 0x"
<< Twine::utohexstr(Value) << '\n');
Symbol = BC->getOrCreateGlobalSymbol(Value, "FUNCat");
Symbol = BC->getOrCreateGlobalSymbol(Value, 0, 0, "FUNCat");
}
DEBUG(dbgs() << "BOLT-DEBUG: adding DWARF reference against symbol "
<< Symbol->getName() << '\n');
EHFrameSection->addRelocation(Offset, Symbol, RelType, 0);
EHFrameSection->addRelocation(Offset, Symbol, RelType, Addend);
};
EHFrame.parse(DE, createReloc);
@ -1556,13 +1635,16 @@ void RewriteInstance::relocateEHFrameSection() {
BinaryFunction *RewriteInstance::createBinaryFunction(
const std::string &Name, BinarySection &Section, uint64_t Address,
uint64_t Size, bool IsSimple) {
uint64_t Size, bool IsSimple, uint64_t SymbolSize, uint16_t Alignment) {
auto Result = BinaryFunctions.emplace(
Address, BinaryFunction(Name, Section, Address, Size, *BC, IsSimple));
assert(Result.second == true && "unexpected duplicate function");
auto *BF = &Result.first->second;
BC->registerNameAtAddress(Name, Address);
BC->SymbolToFunctionMap[BF->getSymbol()] = BF;
BC->registerNameAtAddress(Name,
Address,
SymbolSize ? SymbolSize : Size,
Alignment);
BC->setSymbolToFunctionMap(BF->getSymbol(), BF);
return BF;
}
@ -1610,6 +1692,11 @@ void RewriteInstance::readSpecialSections() {
BC->printSections(outs());
}
if (opts::PrintSections) {
outs() << "BOLT-INFO: Sections:\n";
BC->printSections(outs());
}
EHFrameSection = BC->getUniqueSectionByName(".eh_frame");
GdbIndexSection = BC->getUniqueSectionByName(".gdb_index");
PLTSection = BC->getUniqueSectionByName(".plt");
@ -1728,10 +1815,10 @@ bool RewriteInstance::analyzeRelocation(const RelocationRef &Rel,
// non-zero and the relocation is not pc-rel. Using the previous logic,
// the SymbolAddress would end up as a huge number. Seen in
// exceptions_pic.test.
DEBUG(dbgs() << "BOLT-DEBUG: relocation @ "
DEBUG(dbgs() << "BOLT-DEBUG: relocation @ 0x"
<< Twine::utohexstr(Rel.getOffset())
<< " value does not match addend for "
<< "relocation to undefined symbol.");
<< "relocation to undefined symbol.\n");
SymbolAddress += PCRelOffset;
return true;
}
@ -1849,6 +1936,35 @@ void RewriteInstance::readRelocations(const SectionRef &Section) {
const bool IsAArch64 = BC->TheTriple->getArch() == llvm::Triple::aarch64;
const bool IsFromCode = RelocatedSection.isText();
auto printRelocationInfo = [&](const RelocationRef &Rel,
StringRef SymbolName,
uint64_t SymbolAddress,
uint64_t Addend,
uint64_t ExtractedValue) {
SmallString<16> TypeName;
Rel.getTypeName(TypeName);
const auto Address = SymbolAddress + Addend;
auto Section = BC->getSectionForAddress(SymbolAddress);
dbgs() << "Relocation: offset = 0x"
<< Twine::utohexstr(Rel.getOffset())
<< "; type = " << Rel.getType()
<< "; type name = " << TypeName
<< "; value = 0x" << Twine::utohexstr(ExtractedValue)
<< "; symbol = " << SymbolName
<< " (" << (Section ? Section->getName() : "") << ")"
<< "; symbol address = 0x" << Twine::utohexstr(SymbolAddress)
<< "; addend = 0x" << Twine::utohexstr(Addend)
<< "; address = 0x" << Twine::utohexstr(Address)
<< "; in = ";
if (auto *Func = getBinaryFunctionContainingAddress(Rel.getOffset(),
false,
IsAArch64)) {
dbgs() << Func->getPrintName() << "\n";
} else {
dbgs() << BC->getSectionForAddress(Rel.getOffset())->getName() << "\n";
}
};
for (const auto &Rel : Section.relocations()) {
SmallString<16> TypeName;
Rel.getTypeName(TypeName);
@ -1877,16 +1993,13 @@ void RewriteInstance::readRelocations(const SectionRef &Section) {
SymbolName == "__hot_end"))
|| Rel.getType() == ELF::R_AARCH64_ADR_GOT_PAGE;
DEBUG(dbgs() << "BOLT-DEBUG: offset = 0x"
<< Twine::utohexstr(Rel.getOffset())
<< "; type = " << Rel.getType()
<< "; type name = " << TypeName
<< "; value = 0x" << Twine::utohexstr(ExtractedValue)
<< "; symbol = " << SymbolName
<< "; symbol address = 0x" << Twine::utohexstr(SymbolAddress)
<< "; addend = 0x" << Twine::utohexstr(Addend)
<< "; address = 0x" << Twine::utohexstr(Address)
<< '\n');
DEBUG(
dbgs() << "BOLT-DEBUG: ";
printRelocationInfo(Rel,
SymbolName,
SymbolAddress,
Addend,
ExtractedValue));
BinaryFunction *ContainingBF = nullptr;
if (IsFromCode) {
@ -1895,8 +2008,6 @@ void RewriteInstance::readRelocations(const SectionRef &Section) {
/*CheckPastEnd*/ false,
/*UseMaxSize*/ IsAArch64);
assert(ContainingBF && "cannot find function for address in code");
DEBUG(dbgs() << "BOLT-DEBUG: relocation belongs to " << *ContainingBF
<< '\n');
}
// PC-relative relocations from data to code are tricky since the original
@ -1935,10 +2046,8 @@ void RewriteInstance::readRelocations(const SectionRef &Section) {
uint64_t RefFunctionOffset = 0;
MCSymbol *ReferencedSymbol = nullptr;
if (ForceRelocation) {
if (Relocation::isGOT(Rel.getType()))
ReferencedSymbol = BC->getOrCreateGlobalSymbol(0, "Zero");
else
ReferencedSymbol = BC->registerNameAtAddress(SymbolName, 0);
auto Name = Relocation::isGOT(Rel.getType()) ? "Zero" : SymbolName;
ReferencedSymbol = BC->registerNameAtAddress(Name, 0, 0, 0);
SymbolAddress = 0;
Addend = Address;
DEBUG(dbgs() << "BOLT-DEBUG: creating relocations for huge pages against"
@ -1965,12 +2074,66 @@ void RewriteInstance::readRelocations(const SectionRef &Section) {
DEBUG(dbgs() << "BOLT-DEBUG: no corresponding function for "
"relocation against code\n");
}
ReferencedSymbol = BC->getOrCreateGlobalSymbol(SymbolAddress, "SYMBOLat");
if (auto *BD = BC->getBinaryDataContainingAddress(SymbolAddress)) {
assert(cantFail(Rel.getSymbol()->getType()) == SymbolRef::ST_Debug ||
BD->nameStartsWith(SymbolName) ||
BD->nameStartsWith("PG" + SymbolName) ||
(BD->nameStartsWith("ANONYMOUS") &&
(BD->getSectionName().startswith(".plt") ||
BD->getSectionName().endswith(".plt"))));
ReferencedSymbol = BD->getSymbol();
Addend += (SymbolAddress - BD->getAddress());
SymbolAddress = BD->getAddress();
assert(Address == SymbolAddress + Addend);
} else {
auto Symbol = *Rel.getSymbol();
// These are mostly local data symbols but undefined symbols
// in relocation sections can get through here too, from .plt.
assert(cantFail(Symbol.getType()) == SymbolRef::ST_Debug ||
BC->getSectionForAddress(SymbolAddress)->getName().startswith(".plt"));
const uint64_t SymbolSize = ELFSymbolRef(Symbol).getSize();
const uint64_t SymbolAlignment = Symbol.getAlignment();
if (cantFail(Symbol.getType()) != SymbolRef::ST_Debug) {
std::string Name;
if (Symbol.getFlags() & SymbolRef::SF_Global)
Name = SymbolName;
else // TODO: add PG prefix?
Name = uniquifyName(*BC, SymbolName + "/");
ReferencedSymbol = BC->registerNameAtAddress(Name,
SymbolAddress,
SymbolSize,
SymbolAlignment);
} else {
ReferencedSymbol = BC->getOrCreateGlobalSymbol(SymbolAddress,
SymbolSize,
SymbolAlignment,
"SYMBOLat");
}
}
}
auto checkMaxDataRelocations = [&]() {
++NumDataRelocations;
if (opts::MaxDataRelocations &&
NumDataRelocations + 1 == opts::MaxDataRelocations) {
dbgs() << "BOLT-DEBUG: processing ending on data relocation "
<< NumDataRelocations << ": ";
printRelocationInfo(Rel,
ReferencedSymbol->getName(),
SymbolAddress,
Addend,
ExtractedValue);
}
return (!opts::MaxDataRelocations ||
NumDataRelocations < opts::MaxDataRelocations);
};
if (IsFromCode) {
if (ReferencedBF || ForceRelocation || opts::ForceToDataRelocations ||
IsAArch64) {
if (ReferencedBF || ForceRelocation || IsAArch64 ||
(opts::ForceToDataRelocations && checkMaxDataRelocations())) {
ContainingBF->addRelocation(Rel.getOffset(),
ReferencedSymbol,
Rel.getType(),
@ -1982,7 +2145,7 @@ void RewriteInstance::readRelocations(const SectionRef &Section) {
}
} else if (IsToCode) {
BC->addRelocation(Rel.getOffset(), ReferencedSymbol, Rel.getType(), Addend);
} else if (opts::ForceToDataRelocations) {
} else if (opts::ForceToDataRelocations && checkMaxDataRelocations()) {
BC->addRelocation(Rel.getOffset(),
ReferencedSymbol,
Rel.getType(),
@ -2203,6 +2366,13 @@ void RewriteInstance::postProcessFunctions() {
BC->TotalScore += Function.getFunctionScore();
BC->SumExecutionCount += Function.getKnownExecutionCount();
}
BC->postProcessSymbolTable();
if (opts::PrintGlobals) {
outs() << "BOLT-INFO: Global symbols:\n";
BC->printGlobalSymbols(outs());
}
}
void RewriteInstance::runOptimizationPasses() {
@ -2486,10 +2656,9 @@ void RewriteInstance::emitFunctions() {
auto Resolver = orc::createLambdaResolver(
[&](const std::string &Name) -> JITSymbol {
DEBUG(dbgs() << "BOLT: looking for " << Name << "\n");
auto I = BC->GlobalSymbols.find(Name);
if (I == BC->GlobalSymbols.end())
return JITSymbol(nullptr);
return JITSymbol(I->second, JITSymbolFlags());
if (auto *I = BC->getBinaryDataByName(Name))
return JITSymbol(I->getAddress(), JITSymbolFlags());
return JITSymbol(nullptr);
},
[](const std::string &S) {
DEBUG(dbgs() << "BOLT: resolving " << S << "\n");
@ -2605,15 +2774,16 @@ void RewriteInstance::mapFileSections(
// Map jump tables if updating in-place.
if (opts::JumpTables == JTS_BASIC) {
for (auto &JTI : Function.JumpTables) {
auto &JT = JTI.second;
JT.Section = BC->getUniqueSectionByName(JT.SectionName);
assert(JT.Section && "cannot find section for jump table");
JT.Section->setFileAddress(JT.Address);
DEBUG(dbgs() << "BOLT-DEBUG: mapping " << JT.SectionName << " to 0x"
<< Twine::utohexstr(JT.Address) << '\n');
OLT->mapSectionAddress(ObjectsHandle,
JT.Section->getSectionID(),
JT.Address);
auto *JT = JTI.second;
auto Section = BC->getUniqueSectionByName(JT->getOutputSection());
assert(Section && "cannot find section for jump table");
JT->setSection(*Section);
Section->setFileAddress(JT->getAddress());
DEBUG(dbgs() << "BOLT-DEBUG: mapping " << Section->getName()
<< " to 0x" << Twine::utohexstr(JT->getAddress())
<< '\n');
OLT->mapSectionAddress(ObjectsHandle, Section->getSectionID(),
JT->getAddress());
}
}
@ -2701,9 +2871,7 @@ void RewriteInstance::mapFileSections(
// Handling for sections with relocations.
for (const auto &Section : BC->sections()) {
if (!Section ||
!Section.hasRelocations() ||
!Section.hasSectionRef())
if (!Section.hasRelocations() || !Section.hasSectionRef())
continue;
StringRef SectionName = Section.getName();
@ -2845,7 +3013,7 @@ void RewriteInstance::emitDataSection(MCStreamer *Streamer,
assert(Relocation.Offset < Section.getSize() && "overflow detected");
if (SectionOffset < Relocation.Offset) {
Streamer->EmitBytes(
SectionContents.substr(SectionOffset,
SectionContents.substr(SectionOffset,
Relocation.Offset - SectionOffset));
SectionOffset = Relocation.Offset;
}
@ -2865,13 +3033,11 @@ void RewriteInstance::emitDataSection(MCStreamer *Streamer,
void RewriteInstance::emitDataSections(MCStreamer *Streamer) {
for (const auto &Section : BC->sections()) {
if (!Section || !Section.hasRelocations() || !Section.hasSectionRef())
if (!Section.hasRelocations() || !Section.hasSectionRef())
continue;
StringRef SectionName = Section.getName();
assert(SectionName != ".eh_frame" && "should not emit .eh_frame as data");
auto EmitName = OrgSecPrefix + std::string(SectionName);
emitDataSection(Streamer, Section, EmitName);
}
@ -3100,11 +3266,8 @@ void RewriteInstance::rewriteNoteSections() {
}
// Write new note sections.
for (auto &Section : BC->sections()) {
if (!Section ||
Section.getFileOffset() ||
!Section.getAllocAddress() ||
Section.isAllocatable())
for (auto &Section : BC->nonAllocatableSections()) {
if (Section.getFileOffset() || !Section.getAllocAddress())
continue;
assert(!Section.hasPendingRelocations() && "cannot have pending relocs");
@ -3138,10 +3301,8 @@ void RewriteInstance::finalizeSectionStringTable(ELFObjectFile<ELFT> *File) {
SHStrTab.add(*AllSHStrTabStrings.back());
}
}
for (auto &Section : BC->sections()) {
if (Section) {
SHStrTab.add(Section.getName());
}
for (const auto &Section : BC->sections()) {
SHStrTab.add(Section.getName());
}
SHStrTab.finalize();
@ -3272,8 +3433,8 @@ RewriteInstance::getOutputSections(ELFObjectFile<ELFT> *File,
}
// Process entries for all new allocatable sections.
for (auto &Section : BC->sections()) {
if (!Section || !Section.isAllocatable() || !Section.isFinalized())
for (auto &Section : BC->allocatableSections()) {
if (!Section.isFinalized())
continue;
// Ignore function sections.
@ -3347,10 +3508,8 @@ RewriteInstance::getOutputSections(ELFObjectFile<ELFT> *File,
return NewSectionIndex;
// Create entries for new non-allocatable sections.
for (auto &Section : BC->sections()) {
if (!Section ||
Section.isAllocatable() ||
Section.getFileOffset() <= LastFileOffset)
for (auto &Section : BC->nonAllocatableSections()) {
if (Section.getFileOffset() <= LastFileOffset)
continue;
if (opts::Verbosity >= 1) {
@ -3397,7 +3556,7 @@ void RewriteInstance::patchELFSectionHeaderTable(ELFObjectFile<ELFT> *File) {
auto &OS = Out->os();
auto *Obj = File->getELFFile();
auto NewSectionIndex = getOutputSections(File, &OutputSections);
auto NewSectionIndex = getOutputSections(File, &OutputSections);
// Sort sections by their offset prior to writing. Only newly created sections
// were unsorted, hence this wouldn't ruin indices in NewSectionIndex.
@ -3468,6 +3627,14 @@ void RewriteInstance::patchELFSymTabs(ELFObjectFile<ELFT> *File) {
auto StringSection = cantFail(Obj->getStringTableForSymtab(*Section));
unsigned IsHotTextUpdated = 0;
std::map<const BinaryFunction *, uint64_t> IslandSizes;
auto getConstantIslandSize = [&IslandSizes](const BinaryFunction *BF) {
auto Itr = IslandSizes.find(BF);
if (Itr != IslandSizes.end())
return Itr->second;
return IslandSizes[BF] = BF->estimateConstantIslandSize();
};
for (const Elf_Sym &Symbol : cantFail(Obj->symbols(Section))) {
auto NewSymbol = Symbol;
const auto *Function = getBinaryFunctionAtAddress(Symbol.st_value);
@ -3496,7 +3663,7 @@ void RewriteInstance::patchELFSymTabs(ELFObjectFile<ELFT> *File) {
}
if (!PatchExisting && Function->hasConstantIsland()) {
auto DataMark = Function->getOutputDataAddress();
auto CISize = Function->estimateConstantIslandSize();
auto CISize = getConstantIslandSize(Function);
auto CodeMark = DataMark + CISize;
auto DataMarkSym = NewSymbol;
DataMarkSym.st_name = AddToStrTab("$d");
@ -3515,7 +3682,7 @@ void RewriteInstance::patchELFSymTabs(ELFObjectFile<ELFT> *File) {
if (!PatchExisting && Function->hasConstantIsland() &&
Function->isSplit()) {
auto DataMark = Function->getOutputColdDataAddress();
auto CISize = Function->estimateConstantIslandSize();
auto CISize = getConstantIslandSize(Function);
auto CodeMark = DataMark + CISize;
auto DataMarkSym = NewSymbol;
DataMarkSym.st_name = AddToStrTab("$d");
@ -3877,13 +4044,13 @@ void RewriteInstance::rewriteFile() {
// Write jump tables if updating in-place.
if (opts::JumpTables == JTS_BASIC) {
for (auto &JTI : Function.JumpTables) {
auto &JT = JTI.second;
assert(JT.Section && "section for jump table expected");
JT.Section->setFileOffset(getFileOffsetForAddress(JT.Address));
assert(JT.Section->getFileOffset() && "no matching offset in file");
OS.pwrite(reinterpret_cast<const char*>(JT.Section->getOutputData()),
JT.Section->getOutputSize(),
JT.Section->getFileOffset());
auto *JT = JTI.second;
auto &Section = JT->getSection();
Section.setFileOffset(getFileOffsetForAddress(JT->getAddress()));
assert(Section.getFileOffset() && "no matching offset in file");
OS.pwrite(reinterpret_cast<const char*>(Section.getOutputData()),
Section.getOutputSize(),
Section.getFileOffset());
}
}
@ -3944,11 +4111,8 @@ void RewriteInstance::rewriteFile() {
}
// Write all non-local sections, i.e. those not emitted with the function.
for (auto &Section : BC->sections()) {
if (!Section ||
!Section.isAllocatable() ||
!Section.isFinalized() ||
Section.isLocal())
for (auto &Section : BC->allocatableSections()) {
if (!Section.isFinalized() || Section.isLocal())
continue;
if (opts::Verbosity >= 1) {
outs() << "BOLT: writing new section " << Section.getName() << '\n';
@ -4127,11 +4291,9 @@ RewriteInstance::getBinaryFunctionContainingAddress(uint64_t Address,
const BinaryFunction *
RewriteInstance::getBinaryFunctionAtAddress(uint64_t Address) const {
const auto *Symbol = BC->getGlobalSymbolAtAddress(Address);
if (!Symbol)
return nullptr;
return BC->getFunctionForSymbol(Symbol);
if (const auto *BD = BC->getBinaryDataAtAddress(Address))
return BC->getFunctionForSymbol(BD->getSymbol());
return nullptr;
}
DWARFAddressRangesVector RewriteInstance::translateModuleAddressRanges(

12
bolt/RewriteInstance.h
View File

@ -367,7 +367,9 @@ private:
BinarySection &Section,
uint64_t Address,
uint64_t Size,
bool IsSimple);
bool IsSimple,
uint64_t SymbolSize = 0,
uint16_t Alignment = 0);
public:
/// When updating debug info, these are the sections we overwrite.
@ -394,6 +396,10 @@ private:
/// Alignment value used for .eh_frame_hdr.
static constexpr uint64_t EHFrameHdrAlign = 4;
// TODO: these are platform (x86, aarch64) specific.
static constexpr uint64_t PLTSize = 16;
static constexpr uint16_t PLTAlignment = 16;
/// An instance of the input binary we are processing, externally owned.
llvm::object::ELFObjectFileBase *InputFile;
@ -509,6 +515,10 @@ private:
static const std::string OrgSecPrefix;
static const std::string BOLTSecPrefix;
/// Number of processed to data relocations. Used to implement the
/// -max-relocations debugging option.
uint64_t NumDataRelocations{0};
};
} // namespace bolt