maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/bolt/lib/Passes/RegReAssign.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

443 lines
15 KiB
C++
Raw Normal View History

[BOLT][NFC] Fix file-description comments Summary: Fix comments at the start of source files. (cherry picked from FBD33274597)
2021-12-22 02:21:41 +08:00
//===- bolt/Passes/RegReAssign.cpp ----------------------------------------===//
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
//
[BOLT] Update license headers Summary: Update license and fix headers for some files. (cherry picked from FBD28112041)
2021-03-16 09:04:18 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
//
//===----------------------------------------------------------------------===//
//
[BOLT][NFC] Fix file-description comments Summary: Fix comments at the start of source files. (cherry picked from FBD33274597)
2021-12-22 02:21:41 +08:00
// This file implements the RegReAssign class.
//
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
//===----------------------------------------------------------------------===//
Rebase: [NFC] Refactor sources to be buildable in shared mode Summary: Moves source files into separate components, and make explicit component dependency on each other, so LLVM build system knows how to build BOLT in BUILD_SHARED_LIBS=ON. Please use the -c merge.renamelimit=230 git option when rebasing your work on top of this change. To achieve this, we create a new library to hold core IR files (most classes beginning with Binary in their names), a new library to hold Utils, some command line options shared across both RewriteInstance and core IR files, a new library called Rewrite to hold most classes concerned with running top-level functions coordinating the binary rewriting process, and a new library called Profile to hold classes dealing with profile reading and writing. To remove the dependency from BinaryContext into X86-specific classes, we do some refactoring on the BinaryContext constructor to receive a reference to the specific backend directly from RewriteInstance. Then, the dependency on X86 or AArch64-specific classes is transfered to the Rewrite library. We can't have the Core library depend on targets because targets depend on Core (which would create a cycle). Files implementing the entry point of a tool are transferred to the tools/ folder. All header files are transferred to the include/ folder. The src/ folder was renamed to lib/. (cherry picked from FBD32746834)
2021-10-09 02:47:10 +08:00
#include "bolt/Passes/RegReAssign.h"
#include "bolt/Core/MCPlus.h"
#include "bolt/Passes/BinaryFunctionCallGraph.h"
#include "bolt/Passes/DataflowAnalysis.h"
#include "bolt/Passes/DataflowInfoManager.h"
#include "bolt/Utils/Utils.h"
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
#include <numeric>
#define DEBUG_TYPE "regreassign"
using namespace llvm;
namespace opts {
extern cl::OptionCategory BoltOptCategory;
extern cl::opt<bool> UpdateDebugSections;
[bolt] Remove unneeded cl::ZeroOrMore for cl::opt options
2022-06-06 04:29:49 +08:00
static cl::opt<bool> AggressiveReAssign(
"use-aggr-reg-reassign",
cl::desc("use register liveness analysis to try to find more opportunities "
"for -reg-reassign optimization"),
cl::cat(BoltOptCategory));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
}
namespace llvm {
namespace bolt {
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
void RegReAssign::swap(BinaryFunction &Function, MCPhysReg A, MCPhysReg B) {
BinaryContext &BC = Function.getBinaryContext();
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
const BitVector &AliasA = BC.MIB->getAliases(A, false);
const BitVector &AliasB = BC.MIB->getAliases(B, false);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Regular instructions
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
for (BinaryBasicBlock &BB : Function) {
for (MCInst &Inst : BB) {
[BOLT][NFC] Add MCPlus::primeOperands iterator_range Reviewed By: yota9 Differential Revision: https://reviews.llvm.org/D125397
2022-05-12 00:34:10 +08:00
for (MCOperand &Operand : MCPlus::primeOperands(Inst)) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (!Operand.isReg())
continue;
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
unsigned Reg = Operand.getReg();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (AliasA.test(Reg)) {
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
Operand.setReg(BC.MIB->getAliasSized(B, BC.MIB->getRegSize(Reg)));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
--StaticBytesSaved;
DynBytesSaved -= BB.getKnownExecutionCount();
continue;
}
if (!AliasB.test(Reg))
continue;
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
Operand.setReg(BC.MIB->getAliasSized(A, BC.MIB->getRegSize(Reg)));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
++StaticBytesSaved;
DynBytesSaved += BB.getKnownExecutionCount();
}
}
}
// CFI
DenseSet<const MCCFIInstruction *> Changed;
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
for (BinaryBasicBlock &BB : Function) {
for (MCInst &Inst : BB) {
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
if (!BC.MIB->isCFI(Inst))
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
continue;
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
const MCCFIInstruction *CFI = Function.getCFIFor(Inst);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (Changed.count(CFI))
continue;
Changed.insert(CFI);
switch (CFI->getOperation()) {
case MCCFIInstruction::OpRegister: {
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
const unsigned CFIReg2 = CFI->getRegister2();
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
const MCPhysReg Reg2 = *BC.MRI->getLLVMRegNum(CFIReg2, /*isEH=*/false);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (AliasA.test(Reg2)) {
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
Function.setCFIFor(
Inst, MCCFIInstruction::createRegister(
nullptr, CFI->getRegister(),
BC.MRI->getDwarfRegNum(
BC.MIB->getAliasSized(B, BC.MIB->getRegSize(Reg2)),
false)));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
} else if (AliasB.test(Reg2)) {
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
Function.setCFIFor(
Inst, MCCFIInstruction::createRegister(
nullptr, CFI->getRegister(),
BC.MRI->getDwarfRegNum(
BC.MIB->getAliasSized(A, BC.MIB->getRegSize(Reg2)),
false)));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
}
}
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_FALLTHROUGH;
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
case MCCFIInstruction::OpUndefined:
case MCCFIInstruction::OpDefCfa:
case MCCFIInstruction::OpOffset:
case MCCFIInstruction::OpRestore:
case MCCFIInstruction::OpSameValue:
case MCCFIInstruction::OpDefCfaRegister:
case MCCFIInstruction::OpRelOffset:
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
case MCCFIInstruction::OpEscape: {
unsigned CFIReg;
if (CFI->getOperation() != MCCFIInstruction::OpEscape) {
CFIReg = CFI->getRegister();
} else {
Optional<uint8_t> Reg =
readDWARFExpressionTargetReg(CFI->getValues());
// Handle DW_CFA_def_cfa_expression
if (!Reg)
break;
CFIReg = *Reg;
}
const MCPhysReg Reg = *BC.MRI->getLLVMRegNum(CFIReg, /*isEH=*/false);
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
if (AliasA.test(Reg))
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
Function.mutateCFIRegisterFor(
Inst,
BC.MRI->getDwarfRegNum(
BC.MIB->getAliasSized(B, BC.MIB->getRegSize(Reg)), false));
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
else if (AliasB.test(Reg))
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
Function.mutateCFIRegisterFor(
Inst,
BC.MRI->getDwarfRegNum(
BC.MIB->getAliasSized(A, BC.MIB->getRegSize(Reg)), false));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
break;
}
default:
break;
}
}
}
}
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
void RegReAssign::rankRegisters(BinaryFunction &Function) {
BinaryContext &BC = Function.getBinaryContext();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
std::fill(RegScore.begin(), RegScore.end(), 0);
std::fill(RankedRegs.begin(), RankedRegs.end(), 0);
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
for (BinaryBasicBlock &BB : Function) {
for (MCInst &Inst : BB) {
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
const bool CannotUseREX = BC.MIB->cannotUseREX(Inst);
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
const MCInstrDesc &Desc = BC.MII->get(Inst.getOpcode());
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Disallow substituitions involving regs in implicit uses lists
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
const MCPhysReg *ImplicitUses = Desc.getImplicitUses();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
while (ImplicitUses && *ImplicitUses) {
const size_t RegEC =
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getAliases(*ImplicitUses, false).find_first();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
RegScore[RegEC] =
std::numeric_limits<decltype(RegScore)::value_type>::min();
++ImplicitUses;
}
// Disallow substituitions involving regs in implicit defs lists
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
const MCPhysReg *ImplicitDefs = Desc.getImplicitDefs();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
while (ImplicitDefs && *ImplicitDefs) {
const size_t RegEC =
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getAliases(*ImplicitDefs, false).find_first();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
RegScore[RegEC] =
std::numeric_limits<decltype(RegScore)::value_type>::min();
++ImplicitDefs;
}
[BOLT][Refactoring] Isolate changes to MC layer Summary: Changes that we made to MCInst, MCOperand, MCExpr, etc. are now all moved into tools/llvm-bolt. That required a change to the way we handle annotations and any extra operands for MCInst. Any MCPlus information is now attached via an extra operand of type MCInst with an opcode ANNOTATION_LABEL. Since this operand is MCInst, we attach extra info as operands to this instruction. For first-level annotations use functions to access the information, such as getConditionalTailCall() or getEHInfo(), etc. For the rest, optional or second-class annotations, use a general named-annotation interface such as getAnnotationAs<uint64_t>(Inst, "Count"). I did a test on HHVM binary, and a memory consumption went down a little bit while the runtime remained the same. (cherry picked from FBD7405412)
2018-03-20 09:32:12 +08:00
for (int I = 0, E = MCPlus::getNumPrimeOperands(Inst); I != E; ++I) {
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
const MCOperand &Operand = Inst.getOperand(I);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (!Operand.isReg())
continue;
if (Desc.getOperandConstraint(I, MCOI::TIED_TO) != -1)
continue;
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
unsigned Reg = Operand.getReg();
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
size_t RegEC = BC.MIB->getAliases(Reg, false).find_first();
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (RegEC == 0)
continue;
// Disallow substituitions involving regs in instrs that cannot use REX
if (CannotUseREX) {
RegScore[RegEC] =
std::numeric_limits<decltype(RegScore)::value_type>::min();
continue;
}
// Unsupported substitution, cannot swap BH with R* regs, bail
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
if (BC.MIB->isUpper8BitReg(Reg) && ClassicCSR.test(Reg)) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
RegScore[RegEC] =
std::numeric_limits<decltype(RegScore)::value_type>::min();
continue;
}
RegScore[RegEC] += BB.getKnownExecutionCount();
}
}
}
std::iota(RankedRegs.begin(), RankedRegs.end(), 0); // 0, 1, 2, 3...
std::sort(RankedRegs.begin(), RankedRegs.end(),
[&](size_t A, size_t B) { return RegScore[A] > RegScore[B]; });
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG({
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
for (size_t Reg : RankedRegs) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (RegScore[Reg] == 0)
continue;
dbgs() << Reg << " ";
if (RegScore[Reg] > 0)
dbgs() << BC.MRI->getName(Reg) << ": " << RegScore[Reg] << "\n";
else
dbgs() << BC.MRI->getName(Reg) << ": (blacklisted)\n";
}
});
}
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
void RegReAssign::aggressivePassOverFunction(BinaryFunction &Function) {
BinaryContext &BC = Function.getBinaryContext();
rankRegisters(Function);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Bail early if our registers are all black listed, before running expensive
// analysis passes
bool Bail = true;
int64_t LowScoreClassic = std::numeric_limits<int64_t>::max();
[BOLT][NFC] Use BitVector::set_bits Refactor and use `set_bits` BitVector interface. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D125374
2022-05-12 07:23:27 +08:00
for (int J : ClassicRegs.set_bits()) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (RegScore[J] <= 0)
continue;
Bail = false;
if (RegScore[J] < LowScoreClassic)
LowScoreClassic = RegScore[J];
}
if (Bail)
return;
BitVector Extended = ClassicRegs;
Extended.flip();
Extended &= GPRegs;
Bail = true;
int64_t HighScoreExtended = 0;
[BOLT][NFC] Use BitVector::set_bits Refactor and use `set_bits` BitVector interface. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D125374
2022-05-12 07:23:27 +08:00
for (int J : Extended.set_bits()) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (RegScore[J] <= 0)
continue;
Bail = false;
if (RegScore[J] > HighScoreExtended)
HighScoreExtended = RegScore[J];
}
// Also bail early if there is no profitable substitution even if we assume
// all registers can be exchanged
if (Bail || (LowScoreClassic << 1) >= HighScoreExtended)
return;
// -- expensive pass -- determine all regs alive during func start
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
DataflowInfoManager Info(Function, RA.get(), nullptr);
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
BitVector AliveAtStart = *Info.getLivenessAnalysis().getStateAt(
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
ProgramPoint::getFirstPointAt(*Function.begin()));
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
for (BinaryBasicBlock &BB : Function)
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (BB.pred_size() == 0)
AliveAtStart |= *Info.getLivenessAnalysis().getStateAt(
ProgramPoint::getFirstPointAt(BB));
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Mark frame pointer alive because of CFI
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
AliveAtStart |= BC.MIB->getAliases(BC.MIB->getFramePointer(), false);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Never touch return registers
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getDefaultLiveOut(AliveAtStart);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Try swapping more profitable options first
auto Begin = RankedRegs.begin();
auto End = std::prev(RankedRegs.end());
while (Begin != End) {
MCPhysReg ClassicReg = *End;
if (!ClassicRegs[ClassicReg] || RegScore[ClassicReg] <= 0) {
--End;
continue;
}
MCPhysReg ExtReg = *Begin;
if (!Extended[ExtReg] || RegScore[ExtReg] <= 0) {
++Begin;
continue;
}
if (RegScore[ClassicReg] << 1 >= RegScore[ExtReg]) {
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG(dbgs() << " Ending at " << BC.MRI->getName(ClassicReg)
<< " with " << BC.MRI->getName(ExtReg)
<< " because exchange is not profitable\n");
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
break;
}
BitVector AnyAliasAlive = AliveAtStart;
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
AnyAliasAlive &= BC.MIB->getAliases(ClassicReg);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (AnyAliasAlive.any()) {
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG(dbgs() << " Bailed on " << BC.MRI->getName(ClassicReg)
<< " with " << BC.MRI->getName(ExtReg)
<< " because classic reg is alive\n");
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
--End;
continue;
}
AnyAliasAlive = AliveAtStart;
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
AnyAliasAlive &= BC.MIB->getAliases(ExtReg);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (AnyAliasAlive.any()) {
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG(dbgs() << " Bailed on " << BC.MRI->getName(ClassicReg)
<< " with " << BC.MRI->getName(ExtReg)
<< " because extended reg is alive\n");
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
++Begin;
continue;
}
// Opportunity detected. Swap.
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG(dbgs() << "\n ** Swapping " << BC.MRI->getName(ClassicReg)
<< " with " << BC.MRI->getName(ExtReg) << "\n\n");
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
swap(Function, ClassicReg, ExtReg);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
FuncsChanged.insert(&Function);
++Begin;
if (Begin == End)
break;
--End;
}
}
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
bool RegReAssign::conservativePassOverFunction(BinaryFunction &Function) {
BinaryContext &BC = Function.getBinaryContext();
rankRegisters(Function);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
// Try swapping R12, R13, R14 or R15 with RBX (we work with all callee-saved
// regs except RBP)
MCPhysReg Candidate = 0;
[BOLT][NFC] Use BitVector::set_bits Refactor and use `set_bits` BitVector interface. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D125374
2022-05-12 07:23:27 +08:00
for (int J : ExtendedCSR.set_bits())
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (RegScore[J] > RegScore[Candidate])
Candidate = J;
if (!Candidate || RegScore[Candidate] < 0)
return false;
// Check if our classic callee-saved reg (RBX is the only one) has lower
// score / utilization rate
MCPhysReg RBX = 0;
[BOLT][NFC] Use BitVector::set_bits Refactor and use `set_bits` BitVector interface. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D125374
2022-05-12 07:23:27 +08:00
for (int I : ClassicCSR.set_bits()) {
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
int64_t ScoreRBX = RegScore[I];
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
if (ScoreRBX <= 0)
continue;
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
if (RegScore[Candidate] > (ScoreRBX + 10))
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
RBX = I;
}
if (!RBX)
return false;
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG(dbgs() << "\n ** Swapping " << BC.MRI->getName(RBX) << " with "
<< BC.MRI->getName(Candidate) << "\n\n");
[BOLT][NFC] Suppress unused variable warnings Address warnings in Release build without assertions. Tip @tschuett for reporting the issue #55404. Reviewed By: rafauler Differential Revision: https://reviews.llvm.org/D125475
2022-05-14 02:56:45 +08:00
(void)BC;
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
swap(Function, RBX, Candidate);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
FuncsChanged.insert(&Function);
return true;
}
void RegReAssign::setupAggressivePass(BinaryContext &BC,
[BOLT][NFC] Reformat with clang-format Summary: Selectively apply clang-format to BOLT code base. (cherry picked from FBD33119052)
2021-12-15 08:52:51 +08:00
std::map<uint64_t, BinaryFunction> &BFs) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
setupConservativePass(BC, BFs);
[BOLT] Move BinaryFunctions into a BinaryContext and more Summary: A long due refactoring that makes interfaces cleaner and less awkward. Mainly makes the future work way easier. (cherry picked from FBD14766284)
2019-04-04 06:52:01 +08:00
CG.reset(new BinaryFunctionCallGraph(buildCallGraph(BC)));
[X86] Support a subset of internal calls Summary: Add support for functions with internal calls, necessary for handling Intel MKL library and some code observed in google core dumper library. This is not optimizing these functions, but only identifying them, running analyses to assure we will not break those functions if we move them, and then "freezing" these functions (marking as not simple so Bolt will not try to reorder it or touch it in any way). (cherry picked from FBD8364381)
2018-06-12 04:18:44 +08:00
RA.reset(new RegAnalysis(BC, &BFs, &*CG));
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
GPRegs = BitVector(BC.MRI->getNumRegs(), false);
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getGPRegs(GPRegs);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
}
void RegReAssign::setupConservativePass(
BinaryContext &BC, std::map<uint64_t, BinaryFunction> &BFs) {
// Set up constant bitvectors used throughout this analysis
ClassicRegs = BitVector(BC.MRI->getNumRegs(), false);
CalleeSaved = BitVector(BC.MRI->getNumRegs(), false);
ClassicCSR = BitVector(BC.MRI->getNumRegs(), false);
ExtendedCSR = BitVector(BC.MRI->getNumRegs(), false);
// Never consider the frame pointer
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getClassicGPRegs(ClassicRegs);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
ClassicRegs.flip();
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
ClassicRegs |= BC.MIB->getAliases(BC.MIB->getFramePointer(), false);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
ClassicRegs.flip();
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getCalleeSavedRegs(CalleeSaved);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
ClassicCSR |= ClassicRegs;
ClassicCSR &= CalleeSaved;
[BOLT] Introduce MCPlus layer Summary: Refactor architecture-specific code out of llvm into llvm-bolt. Introduce MCPlusBuilder, a class that is taking over MCInstrAnalysis responsibilities, i.e. creating, analyzing, and modifying instructions. To access the builder use BC->MIB, i.e. substitute MIA with MIB. MIB is an acronym for MCInstBuilder, that's what MCPlusBuilder used to be. The name stuck, and I find it better than MPB. Instructions are still MCInst, and a bunch of BOLT-specific code still lives in LLVM, but the staff under Target/* is significantly reduced. (cherry picked from FBD7300101)
2018-03-10 01:45:13 +08:00
BC.MIB->getClassicGPRegs(ClassicRegs);
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
ExtendedCSR |= ClassicRegs;
ExtendedCSR.flip();
ExtendedCSR &= CalleeSaved;
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG({
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
RegStatePrinter P(BC);
dbgs() << "Starting register reassignment\nClassicRegs: ";
P.print(dbgs(), ClassicRegs);
dbgs() << "\nCalleeSaved: ";
P.print(dbgs(), CalleeSaved);
dbgs() << "\nClassicCSR: ";
P.print(dbgs(), ClassicCSR);
dbgs() << "\nExtendedCSR: ";
P.print(dbgs(), ExtendedCSR);
dbgs() << "\n";
});
}
[BOLT] Move BinaryFunctions into a BinaryContext and more Summary: A long due refactoring that makes interfaces cleaner and less awkward. Mainly makes the future work way easier. (cherry picked from FBD14766284)
2019-04-04 06:52:01 +08:00
void RegReAssign::runOnFunctions(BinaryContext &BC) {
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
RegScore = std::vector<int64_t>(BC.MRI->getNumRegs(), 0);
RankedRegs = std::vector<size_t>(BC.MRI->getNumRegs(), 0);
if (opts::AggressiveReAssign)
[BOLT] Move BinaryFunctions into a BinaryContext and more Summary: A long due refactoring that makes interfaces cleaner and less awkward. Mainly makes the future work way easier. (cherry picked from FBD14766284)
2019-04-04 06:52:01 +08:00
setupAggressivePass(BC, BC.getBinaryFunctions());
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
else
[BOLT] Move BinaryFunctions into a BinaryContext and more Summary: A long due refactoring that makes interfaces cleaner and less awkward. Mainly makes the future work way easier. (cherry picked from FBD14766284)
2019-04-04 06:52:01 +08:00
setupConservativePass(BC, BC.getBinaryFunctions());
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
[BOLT] Move BinaryFunctions into a BinaryContext and more Summary: A long due refactoring that makes interfaces cleaner and less awkward. Mainly makes the future work way easier. (cherry picked from FBD14766284)
2019-04-04 06:52:01 +08:00
for (auto &I : BC.getBinaryFunctions()) {
Rebase: [BOLT][NFC] Expand auto types Summary: Expanded auto types across BOLT semi-automatically with the aid of clangd LSP (cherry picked from FBD33289309)
2021-04-08 15:19:26 +08:00
BinaryFunction &Function = I.second;
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
[BOLT] Introduce isIgnored() function attribute Summary: Whenever a function is not meant for processing, e.g. when the user requests to optimize only a subset of functions, mark the function as ignored. Use this attribute instead of opts::shouldProcess(). (cherry picked from FBD21374806)
2020-05-04 04:54:45 +08:00
if (!Function.isSimple() || Function.isIgnored())
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
continue;
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG(dbgs() << "====================================\n");
LLVM_DEBUG(dbgs() << " - " << Function.getPrintName() << "\n");
[BOLT][NFC] Do not pass BinaryContext alongside BinaryFunction Summary: BinaryContext is available via BinaryFunction::getBinaryContext(), hence there's no reason to pass both as arguments to a function. In a similar fashion, BinaryBasicBlock has an access to BinaryFunction via getFunction(). Eliminate unneeded arguments. (cherry picked from FBD31921680)
2021-10-26 15:06:34 +08:00
if (!conservativePassOverFunction(Function) && opts::AggressiveReAssign) {
aggressivePassOverFunction(Function);
Rebase: Merge BOLT codebase in monorepo Summary: This commit is the first step in rebasing all of BOLT history in the LLVM monorepo. It also solves trivial build issues by updating BOLT codebase to use current LLVM. There is still work left in rebasing some BOLT features and in making sure everything is working as intended. History has been rewritten to put BOLT in the /bolt folder, as opposed to /tools/llvm-bolt. (cherry picked from FBD33289252)
2020-12-02 08:29:39 +08:00
LLVM_DEBUG({
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
if (FuncsChanged.count(&Function))
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
dbgs() << "Aggressive pass successful on " << Function.getPrintName()
<< "\n";
});
}
}
if (FuncsChanged.empty()) {
outs() << "BOLT-INFO: Reg Reassignment Pass: no changes were made.\n";
return;
}
[BOLT][NFC] Fix braces usage in Passes Summary: Refactor bolt/*/Passes to follow the braces rule for if/else/loop from [LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html). (cherry picked from FBD33344642)
2021-12-29 08:36:17 +08:00
if (opts::UpdateDebugSections)
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
outs() << "BOLT-WARNING: You used -reg-reassign and -update-debug-sections."
<< " Some registers were changed but associated AT_LOCATION for "
<< "impacted variables were NOT updated! This operation is "
Revert "[Bolt][NFC] Change capitalization s/BOLT/Bolt/g" Summary: (cherry picked from FBD8431879)
2018-06-15 05:27:20 +08:00
<< "currently unsupported by BOLT.\n";
[BOLT] Add REX prefix rebalancing pass Summary: Add a pass to rebalance the usage of REX prefixes, moving them from the hot code path to the cold path whenever possible. To do this, we rank the usage frequency of each register and exchange an X86 classic reg with an extended one (which requires a REX prefix) whenever the classic register is being used less times than the extended one. There are two versions of this pass: regular one will only consider RBX as classic and R12-R15 as extended registers because those are callee-saved, which means their scope is local to the function and therefore they can be easily interchanged within the function without further consequences. The aggressive version relies on liveness analysis to detect if the value of a register is being used as a caller-saved value (written to without being read first), which also is eligible for reallocation. However, it showed limited results and is not the default option because it is expensive. Currently, this pass does not update debug info. This means that if a substitution is made, the AT_LOCATION of a variable inside a function may be outdated and GDB will display the wrong value if you ask it to print the value of the affected variable. Updating DWARF involves a painful task of writing a new DWARF expression parser/writer similar to the one we already have for CFI expressions. I'll defer the task of writing this until we determine this optimization is enabled in production. So far, it is experimental to be combined with other optimizations to help us find a new set of optimizations that is beneficial. (cherry picked from FBD6476659)
2017-11-15 10:20:40 +08:00
outs() << "BOLT-INFO: Reg Reassignment Pass Stats:\n";
outs() << "\t " << FuncsChanged.size() << " functions affected.\n";
outs() << "\t " << StaticBytesSaved << " static bytes saved.\n";
outs() << "\t " << DynBytesSaved << " dynamic bytes saved.\n";
}
[BOLT][NFC] Reformat with clang-format Summary: Selectively apply clang-format to BOLT code base. (cherry picked from FBD33119052)
2021-12-15 08:52:51 +08:00
} // namespace bolt
} // namespace llvm