This adds a test to verify that when splitting all blocks, landing pad
trampolines are inserted in all blocks.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132426
For exception handling, LSDA call sites have to be emitted for each
fragment individually. With this patch, call sites and respective LSDA
symbols are generated and associated with each fragment of their
function, such that they can be used by the emitter.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132052
To enable split strategies that require view of the entire CFG (e.g. to
estimate cost of path from entry block), with this patch, all blocks of
a function are passed to `SplitStrategy::fragment`. Because this might
move non-outlineable blocks into a split fragment, these blocks are
moved back into the main fragment after fragmenting. This also gives
strategies the option to specify whether empty fragments should be
kept or removed.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132423
ICP has two modes: jump table promotion and indirect call promotion.
The selection is based on whether an instruction has a jump table or not.
An instruction with unknown control flow doesn't have a jump table and will
fall under indirect call promotion policy which might be incorrect/unsafe
(if an instruction is not a tail call, i.e. has local jump targets).
Prevent ICP for functions containing instructions with unknown control flow.
Follow-up to https://reviews.llvm.org/D128870.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132882
This introduces an abstract base class for splitting strategies to
document the interface a strategy needs to implement, and also to avoid
code bloat of the `splitFunction` method.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132054
Output different letters for different sections in the heatmap to
visually separate sections.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D133068
The command to detect whether the stash is needed is `git status --porcelain`
which includes untracked files by default. We want to stash untracked files
as well as they may affect compilation (LLVM CMake checks that all source files
should be included in CMakeLists).
Update the stash command to include untracked files as well.
Reviewed By: #bolt, rafauler
Differential Revision: https://reviews.llvm.org/D132610
We were trying to process .debug_addr for CU that doesn't have it. This resulted
in assert. Example came from GCC that also doesn't use DW_OP_addrx in
DW_FORM_exprloc.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132422
To generate all symbols correctly, it is necessary to record the address
of each fragment. This patch moves the address info for the main and
cold fragments from BinaryFunction to FunctionFragment, where this data
is recorded for all fragments.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132051
This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-specific information.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132050
A const-qualified reference to function layout allows accessing
non-const qualified basic blocks on a const-qualified function. This
patch adds or removes const-qualifiers where necessary to indicate where
basic blocks are used in a non-const manner.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132049
To generate all symbols correctly, it is necessary to record the address
of each fragment. This patch moves the address info for the main and
cold fragments from BinaryFunction to FunctionFragment, where this data
is recorded for all fragments.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132051
This changes `FunctionFragment` from being used as a temporary proxy
object to access basic block ranges to a heap-allocated object that can
store fragment-specific information.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132050
A const-qualified reference to function layout allows accessing
non-const qualified basic blocks on a const-qualified function. This
patch adds or removes const-qualifiers where necessary to indicate where
basic blocks are used in a non-const manner.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132049
This test contained some data tables that llvm-objdump was
disassembling as code, so the test was recovering the 32-bit values in
the table from the instruction encoding column of the disassembly.
D131589 changed how llvm-objdump decides what to disassemble as code
or as data. As a result, these data tables are now being disassembled
as data, which I think is actually more sensible -- but the test
wasn't expecting it, and got confused.
A simple sed doing these substitutions:
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?lib(${LLVM_LIBDIR_SUFFIX})?\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`
where `\>` means "word boundary".
The only manual modifications were reverting changes in
- `compiler-rt/cmake/Modules/CompilerRTUtils.cmake
- `runtimes/CMakeLists.txt`
because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.
This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.
Reviewed By: sebastian-ne
Differential Revision: https://reviews.llvm.org/D132316
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
255 to 233 LoC.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132104
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
295 to 255 LoC.
Differential Revision: https://reviews.llvm.org/D132101
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
338 to 295 LoC.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132100
Move the large lambda out of BinaryFunction::disassemble, reducing its size from
377 to 338 LoC.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132099
Working back towards D130586.
Bolt didn't use `LLVM_LIBDIR_SUFFIX` before, and has no in-tree reverse
dependencies, it seems easier to add.
The change in LLVM itself is to prevent some unexpected `lib64` from
cropping up due to the `CMAKE_INSTALL_LIBDIR` defaulting logic.
Differential Revision: https://reviews.llvm.org/D132297
Wasn't handling correctly creating DW_AT_rnglists_base in UnitDie when
converting access pattern for DW_AT_ranges from offset to index for DWARF5.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132087
This patch adds exception handling trampolines when a function is split
into more than two fragments. Trampolines are tracked per-fragment, such
that they can be removed if splitting is reversed.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132048
Use isSplit() instead of isCold() when building the call graph and
update parameter names to reflect this.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132047
This adds a strategy to split functions into a random number of
fragments at randomly chosen split points.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D130647
This adds a function splitting strategy that splits each outlineable
basic block into its own fragment. This is exposed through a new command
line option `--split-strategy`.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D129827
This patch adds support to generate any number of sections that are
assigned to fragments of functions that are split more than two-way.
With this, a function's *nth* split fragment goes into section
`.text.cold.n`.
This also changes `FunctionLayout::erase` to make sure, that there are
no empty fragments at the end of the function. This sometimes happens
when blocks are erased from the function. To avoid creating symbols
pointing to these fragments, they need to be removed.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D130521
This adds basic fragment awareness in the exception handling passes and
generates the necessary symbols for fragments.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D130520
This changes code emission such that it can emit specific function
fragments instead of scanning all basic blocks of a function and just
emitting those that are hot or cold.
To implement this, `FunctionLayout` explicitly distinguishes the "main"
fragment (i.e. the one that contains the entry block and is associated
with the original symbol) from "split" fragments. Additionally,
`BinaryFunction` receives support for multiple cold symbols - one for
each split fragment.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D130052
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.
`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.
I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.
Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi
Differential Revision: https://reviews.llvm.org/D130586
With reduced indentation, some strings can be reformatted to take less lines.
Also strategically apply `formatv` to shorten them.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132088
Split out the body of a for-loop in `RewriteInstance::readRelocations` into a
separate function (`handleRelocation`). It's still over 300 lines of code,
so it's worth splitting down further.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132078
Move logging into LLVM_DEBUG scope.
Remove redundant printing of jump table parents:
Old logging:
```
failed to analyze jump table in function _ZN12_GLOBAL__N_116InitHeaderSearch23Ad
dDefaultCIncludePathsERKN4llvm6TripleERKN5clang19HeaderSearchOptionsE/1(*2)
PIC Jump table JUMP_TABLE/_ZN12_GLOBAL__N_116InitHeaderSearch23AddDefaultCInclud
ePathsERKN4llvm6TripleERKN5clang19HeaderSearchOptionsE/1.1 for function _ZN12_GL
OBAL__N_116InitHeaderSearch23AddDefaultCIncludePathsERKN4llvm6TripleERKN5clang19
HeaderSearchOptionsE/1(*2) at 0x65996e0 with a total count of 0:
0x9dc
next jump table at 0x659a810 belongs to function _ZN5clang5Lexer40LexDependencyD
irectiveTokenWhileSkippingERNS_5TokenE
PIC Jump table JUMP_TABLE/_ZN5clang5Lexer40LexDependencyDirectiveTokenWhileSkipp
ingERNS_5TokenE.0 for function _ZN5clang5Lexer40LexDependencyDirectiveTokenWhile
SkippingERNS_5TokenE at 0x659a810 with a total count of 0:
jump table heuristic failure
```
New logging:
```
failed to analyze PIC Jump table JUMP_TABLE/_ZN12_GLOBAL__N_116InitHeaderSearch2
3AddDefaultCIncludePathsERKN4llvm6TripleERKN5clang19HeaderSearchOptionsE/1.1 for
function _ZN12_GLOBAL__N_116InitHeaderSearch23AddDefaultCIncludePathsERKN4llvm6T
ripleERKN5clang19HeaderSearchOptionsE/1(*2) at 0x65996e0 with a total count of 0:
absolute offset: 0x52ac58c
next PIC Jump table JUMP_TABLE/_ZN5clang5Lexer40LexDependencyDirectiveTokenWhile
SkippingERNS_5TokenE.0 for function _ZN5clang5Lexer40LexDependencyDirectiveToken
WhileSkippingERNS_5TokenE at 0x659a810 with a total count of 0:
jump table heuristic failure
```
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D131243
Fabian Parzefall