This adds a test to verify that when splitting all blocks, landing pad
trampolines are inserted in all blocks.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132426
For exception handling, LSDA call sites have to be emitted for each
fragment individually. With this patch, call sites and respective LSDA
symbols are generated and associated with each fragment of their
function, such that they can be used by the emitter.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132052
We were trying to process .debug_addr for CU that doesn't have it. This resulted
in assert. Example came from GCC that also doesn't use DW_OP_addrx in
DW_FORM_exprloc.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D132422
This test contained some data tables that llvm-objdump was
disassembling as code, so the test was recovering the 32-bit values in
the table from the instruction encoding column of the disassembly.
D131589 changed how llvm-objdump decides what to disassemble as code
or as data. As a result, these data tables are now being disassembled
as data, which I think is actually more sensible -- but the test
wasn't expecting it, and got confused.
Wasn't handling correctly creating DW_AT_rnglists_base in UnitDie when
converting access pattern for DW_AT_ranges from offset to index for DWARF5.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132087
This adds a strategy to split functions into a random number of
fragments at randomly chosen split points.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D130647
This adds a function splitting strategy that splits each outlineable
basic block into its own fragment. This is exposed through a new command
line option `--split-strategy`.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D129827
We were resetting DW_AT_low_pc to zero when DW_AT_high_pc was zero, or
DW_AT_low_pc == DW_AT_high_pc. This resulted in LLDB to print error "adding
range [0x0-0x0) which has a base that is less than the function's low PC".
Changed it so that when this case arises we set DW_AT_low_pc to the start
address.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132059
If the first block of a fragment is also a landing pad, the landing pad
is not used if an exception is thrown. This is because the landing pad
is at the same start address that the corresponding LSDA describes. In
that case, the offset in the call site records to refer to that landing
pad is zero, and a zero offset is interpreted by the personality
function as "no handler" and ignored.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D132053
DW_TAG_call_site/DW_AT_call_return_pc can contain address that is not in any
function. In this case getBinaryFunctionContainingAddress returns nullptr. For
this case preserving original address.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D132057
We were not handling correclty multiple DW_OP_addrx in the location expression.
This was exposed by clang-15 build in release mode with debug information.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D130812
This diff uncovers an ASAN leak in getOrCreateJumpTable:
```
Indirect leak of 264 byte(s) in 1 object(s) allocated from:
#1 0x4f6e48c in llvm::bolt::BinaryContext::getOrCreateJumpTable ...
```
The removal of an assertion needs to be accompanied by proper deallocation of
a `JumpTable` object for which `analyzeJumpTable` was unsuccessful.
This reverts commit 52cd00cabf.
This patch refactors BAT to be testable as a library, so we
can have open-source tests on it. This further fixes an issue with
basic blocks that lack a valid input offset, making BAT omit those
when writing translation tables.
Test Plan: new testcases added, new testing tool added (llvm-bat-dump)
Differential Revision: https://reviews.llvm.org/D129382
Disassembly and branch target analysis are not decoupled, so any
analysis that depends on disassembly may not operate properly.
In specific, analyzeJumpTable uses instruction bounds check property.
A jump table was analyzed twice: (a) during disassembly, and (b) after
disassembly, so there are potentially some mismatched results.
In this update, functions that access JTs which fail the second check
will be marked as ignored.
Test Plan:
```
ninja check-bolt
```
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D130431
Determine stripped status of a binary based on .symtab
Test Plan:
```
ninja check-bolt
```
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D130034
This one actually makes the test simpler, because lit doesn't have to
reconstitute a 32-bit little-endian value from individual bytes any
more: llvm-objdump is printing the desired 32-bit value in the first
place, so we can move straight on to doing the arithmetic on it.
The test exercises an implicit ptr-to-int conversion which is made an error in
D129881. We acknowledge the error but still want to test this case.
Add `-Wno-int-conversion` to silence the error.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D130546
Strip tools cause a few symbols in .dynsym to have bad section index.
This update safely keeps such broken symbols intact.
Test Plan:
```
ninja check-bolt
```
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D130073
There should not be a end of child mark before DW_AT_ranges, removed it and fixed unit offset.
Reviewed By: ayermolo
Differential Revision: https://reviews.llvm.org/D130335
Some unit tests has incorrect DW_AT_type offset since they are manual crafted, fix them to the correct offset.
Reviewed By: Amir, ayermolo
Differential Revision: https://reviews.llvm.org/D129828
Add a test for `-icp-inline` knob, which ensures that ICP is only performed for
functions that can be subsequently inlined.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D129803
We previously support split jump table, where some jump table entries
target different fragments of same function. In this fix, we provide
support for another type of intra-indirect transfer: landing pad.
When C++ exception handling is used, compiler emits .gcc_except_table
that describes the location of catch block (landing pad) for specific
range that potentially invokes a throw(). Normally landing pads reside
in the function, but with -fsplit-machine-functions, landing pads can
be moved to another fragment. The intuition is, landing pads are rarely
executed, so compiler can move them to .cold section.
This update will mark all fragments that have landing pad to another
fragment as non-simple, and later propagate non-simple to all related
fragments.
This update also includes one manual test case: split-landing-pad.s
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D128561
There are two assumptions regarding jump table:
(a) It is accessed by only one fragment, say, Parent
(b) All entries target instructions in Parent
For (a), BOLT stores jump table entries as relative offset to Parent.
For (b), BOLT treats jump table entries target somewhere out of Parent
as INVALID_OFFSET, including fragment of same split function.
In this update, we extend (a) and (b) to include fragment of same split
functinon. For (a), we store jump table entries in absolute offset
instead. In addition, jump table will store all fragments that access
it. A fragment uses this information to only create label for jump table
entries that target to that fragment.
For (b), using absolute offset allows jump table entries to target
fragments of same split function, i.e., extend support for split jump
table. This can be done using relocation (fragment start/size) and
fragment detection heuristics (e.g., using symbol name pattern for
non-stripped binaries).
For jump table targets that can only be reached by one fragment, we
mark them as local label; otherwise, they would be the secondary
function entry to the target fragment.
Test Plan
```
ninja check-bolt
```
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D128474
The gold linker veneers are written between functions without symbols,
so we to handle it specially in BOLT.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
Differential Revision: https://reviews.llvm.org/D129260
Add -experimental-shrink-wrapping flag to control when we
want to move callee-saved registers even when addresses of the stack
frame are captured and used in pointer arithmetic, making it more
challenging to do alias analysis to prove that we do not access
optimized stack positions. This alias analysis is not yet implemented,
hence, it is experimental. In practice, though, no compiler would emit
code to do pointer arithmetic to access a saved callee-saved register
unless there is a memory bug or we are failing to identify a
callee-saved reg, so I'm not sure how useful it would be to formally
prove that.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D126115
Add the option to run -equalize-bb-counts before shrink
wrapping to avoid unnecessarily optimizing some CFGs where profile is
inaccurate but we can prove two blocks have the same frequency.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D126113
Change how function score is calculated and provide more
detailed statistics when reporting back frame optimizer and shrink
wrapping results. In this new statistics, we provide dynamic coverage
numbers. The main metric for shrink wrapping is the number of executed
stores that were saved because of shrink wrapping (push instructions
that were either entirely moved away from the hot block or converted
to a stack adjustment instruction). There is still a number of reduced
load instructions (pop) that we are not counting at the moment. Also
update alloc combiner to report dynamic numbers, as well as frame
optimizer.
For debugging purposes, we also include a list of top 10 functions
optimized by shrink wrapping. These changes are aimed at better
understanding the impact of shrink wrapping in a given binary.
We also remove an assertion in dataflow analysis to do not choke on
empty functions (which makes no sense).
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D126111
If BOLT instrumentation runtime uses XMM registers, it can interfere
with the user program causing crashes and unexpected behavior. This
happens as the instrumentation code preserves general purpose registers
only.
Build BOLT instrumentation runtime with "-mno-sse".
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D128960
Added support for mixing monolithic DWARF5 with legacy DWARF, and monolithic legacy and DWARF5 split dwarf.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D128232
When SplitFunctions pass adds a trampoline code for exception landing
pads (limited to shared objects), it may increase the size of the hot
fragment making it larger than the whole function pre-split. When this
happens, the pass reverts the splitting action by restoring the original
block order and marking all blocks hot.
However, if createEHTrampolines() added new blocks to the CFG and
modified invoke instructions, simply restoring the original block layout
will not suffice as the new CFG has more blocks.
For proper backout of the split, modify the original layout by merging
in trampoline blocks immediately before their matching targets. As a
result, the number of blocks increases, but the number of instructions
and the function size remains the same as pre-split.
Add an assertion for the number of blocks when updating a function
layout.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D128696
For test purposes, we want to split functions at a random split point
to be able to test different layouts without relying on the profile.
This patch introduces an option, that randomly chooses a split point
to partition blocks of a function into hot and cold regions.
Reviewed By: Amir, yota9
Differential Revision: https://reviews.llvm.org/D128773
This reverts commit 425dda76e9.
This commit is currently causing BOLT to crash in one of our
binaries and needs a bit more checking to make sure it is safe
to land.
The gold linker veneers are written between functions without symbols,
so we to handle it specially in BOLT.
Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei
Differential Revision: https://reviews.llvm.org/D128082
The SplitFunctions pass does not distinguish between various splitting
modes anymore. This change updates the command line interface to
reflect this behavior by deprecating values passed to the
--split-function option.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D128558
DWARF 5 added two new attributes DW_AT_call_pc and DW_AT_call_return_pc.
Adding support for them.
Reviewed By: maksfb
Differential Revision: https://reviews.llvm.org/D128526
Add functionality to allow splitting code with C++ exceptions in shared
libraries and PIEs. To overcome a limitation in exception ranges format,
for functions with fragments spanning multiple sections, add trampoline
landing pads in the same section as the corresponding throwing range.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D127936
Allow cold fragment to get new address.
Our previous assumption is that a fragment (.cold) is only reached
through the main fragment of same function. In addition, .cold fragment
must be reached through either (a) direct transfer, or (b) split jump
table. For (a), we perform a simple fix-up. For (b), we currently mark
all relevant fragments as non-simple. Therefore, there is no need to
get new address for .cold fragment.
This is not always the case, as function entry can be rarely executed,
and is placed in .text.cold segment. Essentially we cannot tell which
the source-level function entry is based on hot and cold segments,
so we must treat each fragment a function on its own. Therfore, we
remove the assertion that a function entry cannot be cold fragment.
Test Plan:
```
ninja check-bolt
```
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D128111
Resolve a crash related to split functions
Due to split function optimization, a function can be divided to two
fragments, and both fragments can access same jump table. This
violates the assumption that a jump table can only have one parent
function, which causes a crash during instrumentation.
We want to support the case: different functions cannot access same
jump tables, but different fragments of same function can!
As all fragments are from same function, we point JT::Parent to one
specific fragment. Right now it is the first disassembled fragment, but
we can point it to the function's main fragment later.
Functions are disassembled sequentially. Previously, at the end of
processing a function, JT::OffsetEntries is cleared, so other fragment
can no longer reuse JT::OffsetEntries. To extend the support for split
function, we only clear JT::OffsetEntries after all functions are
disassembled.
Let say A.hot and A.cold access JT of three targets {X, Y, Z}, where
X and Y are in A.hot, and Z is in A.cold. Suppose that A.hot is
disassembled first, JT::OffsetEntries = {X',Y',INVALID_OFFSET}. When
A.cold is disassembled, it cannot reuse JT::OffsetEntries above due to
different fragment start. A simple solution:
A.hot = {X',Y',INVALID_OFFSET}
A.cold = {INVALID_OFFSET, INVALID_OFFSET, INVALID_OFFSET}
We update the assertion to allow different fragments of same function
to get the same JumpTable object.
Potential improvements:
A.hot = {X',Y',INVALID_OFFSET}
A.cold = {INVALID_OFFSET, INVALID_OFFSET, Z'}
The main issue is A.hot and A.cold have separate CFGs, thus jump table
targets are still constrained within fragment bounds.
Future improvements:
A.hot = {X, Y, Z}
A.cold = {X, Y, Z}
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D127924
Replace a single dash with a double dash for options that have more
than a single letter.
llvm-bolt-wrapper.py has special treatment for output options such as
"-o" and "-w" causing issues when a single dash is used, e.g. for
"-write-dwp". The wrapper can be fixed as well, but using a double dash
has other advantages as well.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D127538
Mark fragments related to split jump table as non-simple.
A function could be splitted into hot and cold fragments. A split jump table is
challenging for correctly reconstructing control flow graphs, so it was marked
as ignored. This update marks those fragments as non-simple, allowing them
to be printed and partial control flow graph construction.
Test Plan:
```
llvm-lit -a tools/bolt/test/X86/split-func-icf.s
```
This test has two functions (main, main2), each has a jump table target to the
same cold portion main2.cold.1(*2). We try to print out only this cold portion.
If it is ignored, it cannot be printed. If it is non-simple, it can be printed. We
verify that it can be printed.
Reviewed By: Amir
Differential Revision: https://reviews.llvm.org/D127464
This patch adds getFirstInstructionOffset method for BinaryFunction
which is used to properly handle cases where data is at zero offset in
a function. The main change is that we add basic block at first
instruction offset when disassembling, which prevents assertion
failures in buildCFG.
Reviewed By: yota9, rafauler
Differential Revision: https://reviews.llvm.org/D127111
The linker can convert instructions with GOTPCRELX relocations into a
form that uses an absolute addressing with an immediate. BOLT needs to
recognize such conversions and symbolize the immediates.
Reviewed By: rafauler
Differential Revision: https://reviews.llvm.org/D126747
Fabian Parzefall