Commit Graph

183 Commits

Author SHA1 Message Date
Vladislav Khmelevsky fd9604952d [BOLT] Set valid index for functions with profiles
Some of the passes that calculates tentative layout like LongJmp and
Golang are expecting that only functions with valid index will be
located in hot text section. But currently functions with valid profiles
and not set index are breaking this logic, to fix this we can move the
hasValidProfile() condition from AssignSections pass to ReorderFunctions.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D127223
2022-06-08 14:13:12 +03:00
Fangrui Song 15d82c62dc [MC] De-capitalize MCStreamer functions
Follow-up to c031378ce0 .
The class is mostly consistent now.
2022-06-07 00:31:02 -07:00
Fangrui Song b92436efcb [bolt] Remove unneeded cl::ZeroOrMore for cl::opt options 2022-06-05 13:29:49 -07:00
Fangrui Song 36c7d79dc4 Remove unneeded cl::ZeroOrMore for cl::opt options
Similar to 557efc9a8b.
This commit handles options where cl::ZeroOrMore is more than one line below
cl::opt.
2022-06-04 00:10:42 -07:00
Fangrui Song 72f9c69421 [Hexagon][bolt] Remove unneeded cl::ZeroOrMore for cl::opt options. NFC
Similar to 557efc9a8b
2022-06-03 22:04:57 -07:00
Huan Nguyen 5ac26156fe [BOLT][NFC] Warning for deprecated option '-reorder-blocks=cache+'
Emit warning when using deprecated option '-reorder-blocks=cache+'.
Auto switch to option '-reorder-blocks=ext-tsp'.

Test Plan:
```
ninja check-bolt
```
Added a new test cache+-deprecated.test.
Run and verify that the upstream tests are passed.

Reviewed By: rafauler, Amir, maksfb

Differential Revision: https://reviews.llvm.org/D126722
2022-06-03 14:16:55 -07:00
spupyrev 5904836b8a [BOLT] Cache-Aware Tail Duplication
A new "cache-aware" strategy for tail duplication.

Differential Revision: https://reviews.llvm.org/D123050
2022-06-03 09:08:45 -07:00
Amir Ayupov e2142ff47c [BOLT][NFC] Make ICP::verifyProfile static
Follow LLVM style guide suggestion to avoid function definitions in anonymous
namespaces: https://llvm.org/docs/CodingStandards.html#anonymous-namespaces

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124896
2022-06-02 19:09:29 -07:00
Maksim Panchenko 986e5dedf2 [BOLT][NFC] Fix braces in BinaryEmitter
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126844
2022-06-02 12:45:25 -07:00
Amir Ayupov 6333e5dde9 [BOLT][NFC] Use colors in CFG dumps
Use color coding to distinguish nodes:
- Entry nodes have bold border
- Scalar (non-loopy) code is milk white
- Outer loops are light yellow
- Innermost loops are light blue

`-print-loops` needs to be enabled to provide BinaryLoopInfo.
Examples:
{F23170673}
{F23170680}

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126248
2022-06-02 00:27:12 -07:00
Amir Ayupov cc23c64ff1 [BOLT][NFC] Print block instructions in dumpGraph as part of node label
Reuse the option `-dot-tooltip-code` to put block instructions into the label.
This way, the instructions are displayed by default when used with dot viewer.

When the .dot file is used with dot2html, instructions are hidden by default,
and are shown by clicking on a node.

{F23169510}

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126237
2022-06-01 23:41:41 -07:00
Alexander Yermolovich ab9a175990 [BOLT][DWARF] Fix TU Index handling for DWARF4/5
When we generate split dwarf with -fdebug-types-section we will have
.debug_types.dwo sections. These go into TU Index when we run llvm-dwp. BOLT was
not handling DWP input correctly with this section.

Added support for handling DWP with TU Index as an input and output for DWARF4.
Added support for handling DWP with TU Index as an input for DWARF5

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D126087
2022-06-01 18:16:12 -07:00
Maksim Panchenko 0426100ff4 [BOLT][NFC] Remove unused variable
Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126808
2022-06-01 13:43:10 -07:00
Maksim Panchenko e290133c76 [BOLT] Add new class for symbolizing X86 instructions
Summary:
While disassembling instructions, we need to replace certain immediate
operands with symbols. This symbolizing process relies on reading
relocations against instructions. However, some X86 instructions can
have multiple immediate operands and up to two relocations against
them. Thus, correctly matching a relocation to an operand is not
always possible without knowing the operand offset within the
instruction.

Luckily, LLVM provides an interface for passing the required info from
the disassembler via a virtual MCSymbolizer class. Creating a
target-specific version allows a precise matching of relocations to
operands.

This diff adds X86MCSymbolizer class that performs X86-specific
symbolizing (currently limited to non-branch instructions).

Reviewers: yota9, Amir, ayermolo, rafauler, zr33

Differential Revision: https://reviews.llvm.org/D120928
2022-05-31 17:48:19 -07:00
Denis Revunov 8579db96e8 [BOLT] [AArch64] Handle constant islands spanning multiple functions
Fix BOLT's constant island mapping when a constant island marked by $d
spans multiple functions. Currently, because BOLT only marks the
constant island in the first function where $d is located, if the next
function contains data at its start, BOLT will miss the data and try
to disassemble it. This patch adds code to explicitly go through all
symbols between $d and $x markers and mark their respective offsets as
data, which stops BOLT from trying to disassemble data. It also adds
MarkerType enum and refactors related functions.

Reviewed By: yota9, rafauler

Differential Revision: https://reviews.llvm.org/D126177
2022-05-31 13:51:35 -07:00
Balazs Benics a73b50ad06 Revert "[llvm][clang][bolt][NFC] Use llvm::less_first() when applicable"
This reverts commit 3988bd1398.

Did not build on this bot:
https://lab.llvm.org/buildbot#builders/215/builds/6372

/usr/include/c++/9/bits/predefined_ops.h:177:11: error: no match for call to
‘(llvm::less_first) (std::pair<long unsigned int, llvm::bolt::BinaryBasicBlock*>&, const std::pair<long unsigned int, std::nullptr_t>&)’
  177 |  { return bool(_M_comp(*__it, __val)); }
2022-05-27 11:19:18 +02:00
Balazs Benics 3988bd1398 [llvm][clang][bolt][NFC] Use llvm::less_first() when applicable
One could reuse this functor instead of rolling out your own version.
There were a couple other cases where the code was similar, but not
quite the same, such as it might have an assertion in the lambda or other
constructs. Thus, I've not touched any of those, as it might change the
behavior in some way.

As per https://discourse.llvm.org/t/submitting-simple-nfc-patches/62640/3?u=steakhal
Chris Lattner
> LLVM intentionally has a “yes, you can apply common sense judgement to
> things” policy when it comes to code review. If you are doing mechanical
> patches (e.g. adopting less_first) that apply to the entire monorepo,
> then you don’t need everyone in the monorepo to sign off on it. Having
> some +1 validation from someone is useful, but you don’t need everyone
> whose code you touch to weigh in.

Differential Revision: https://reviews.llvm.org/D126068
2022-05-27 11:15:23 +02:00
Rafael Auler c09cd64e5c [BOLT] Fix AND evaluation bug in shrink wrapping
Fix a bug where shrink-wrapping would use wrong stack offsets
because the stack was being aligned with an AND instruction, hence,
making its true offsets only available during runtime (we can't
statically determine where are the stack elements and we must give up
on this case).

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126110
2022-05-26 14:59:28 -07:00
Amir Ayupov f7581a3969 [BOLT][NFC] Use ListSeparator in BinaryFunction print methods
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126243
2022-05-24 18:29:24 -07:00
Amir Ayupov 69f87b6c29 [BOLT][NFC] Customize endline character for printInstruction(s)
This would be used in `BF::dumpGraph` to dump left-justified text.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126232
2022-05-24 18:26:12 -07:00
Amir Ayupov 5d8247d4c7 [BOLT][NFC] Use for_each to simplify printLoopInfo
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126242
2022-05-24 18:05:43 -07:00
Amir Ayupov c907d6e0e9 [BOLT][NFC] Suppress unused variable warnings
Addresses the warnings emitted by Apple Clang 13.1.6 (Xcode 13.3.1).
Tip @tschuett issue #55404.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125733
2022-05-17 14:30:23 -07:00
Amir Ayupov a7b69dbdd1 [BOLT][NFC] Move BinaryDominatorTree out of BinaryLoop header
Split up the BinaryLoop header and move BinaryDominatorTree into its own header,
preparing it for a standalone use.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125664
2022-05-17 14:20:11 -07:00
Amir Ayupov bdba3d091c [BOLT][CMAKE] Fix DYLIB build
Move BOLT libraries out of `LLVM_LINK_COMPONENTS` to `target_link_libraries`.
Addresses issue #55432.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125568
2022-05-13 13:27:21 -07:00
Amir Ayupov 253b8f0abd [BOLT][NFC] Use refs for loop variables to avoid copies
Addresses warnings when built with Apple Clang.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D125483
2022-05-13 20:18:29 +01:00
Amir Ayupov 139744ac53 [BOLT][NFC] Suppress unused variable warnings
Address warnings in Release build without assertions.
Tip @tschuett for reporting the issue #55404.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125475
2022-05-13 20:10:19 +01:00
Amir Ayupov d63c5a38fe [BOLT][NFC] Use BitVector::set_bits
Refactor and use `set_bits` BitVector interface.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125374
2022-05-11 16:23:44 -07:00
Amir Ayupov 8cb7a873ab [BOLT][NFC] Add MCPlus::primeOperands iterator_range
Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D125397
2022-05-11 09:34:51 -07:00
Amir Ayupov c2d40f1dfb [BOLT] Add icp-inline option
Add an option to only peel ICP targets that can be subsequently inlined.
Yet there's no guarantee that they will be inlined.

The mode is independent from the heuristic used to choose ICP targets: by exec
count, mispredictions, or memory profile.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124900
2022-05-11 03:21:24 -07:00
Alexander Yermolovich 3abb68a626 [BOLT][DWARF] Fix assert for split dwarf.
Fixing a small bug where it would assert if CU does not modify .debug_addr section.

Differential Revision: https://reviews.llvm.org/D125181
2022-05-08 19:18:17 -07:00
Alexander Yermolovich ba1ac98c62 [BOLT][DWARF] Add version 5 split dwarf support
Added support for DWARF5 Split Dwarf.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D122988
2022-05-05 14:59:05 -07:00
Rahman Lavaee 733dc3e50b [BOLT] Report per-section hotness in bolt-heatmap.
This patch adds a new feature to bolt heatmap to print the hotness of each section in terms of the percentage of samples within that section.

Sample output generated for the clang binary:

Section Name, Begin Address, End Address, Percentage Hotness
.text, 0x1a7b9b0, 0x20a2cc0, 1.4709
.init, 0x20a2cc0, 0x20a2ce1, 0.0001
.fini, 0x20a2ce4, 0x20a2cf2, 0.0000
.text.unlikely, 0x20a2d00, 0x431990c, 0.3061
.text.hot, 0x4319910, 0x4bc6927, 97.2197
.text.startup, 0x4bc6930, 0x4c10c89, 0.0058
.plt, 0x4c10c90, 0x4c12010, 0.9974

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124412
2022-05-05 11:37:46 -07:00
Amir Ayupov f8d2d8b587 [BOLT][NFC] Move getInliningInfo out of Inliner class
`getInliningInfo` is useful in other passes that need to check inlining
eligibility for some function. Move the declaration and InliningInfo definition
out of Inliner class. Prepare for subsequent use in ICP.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124899
2022-05-04 14:08:06 -07:00
Amir Ayupov 2ad1c7540e [BOLT][NFC] Minor cleanup in ICP getCallTargets and canPromoteCallsite
Minor refactoring. NFC.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124898
2022-05-04 14:06:53 -07:00
Amir Ayupov 68c7299f16 [BOLT][NFC] Fix MCPlusBuilder::getAliases caching behavior
Caching behavior of `getAliases` causes a failure in unit tests where two
MCPlusBuilder objects are created corresponding to AArch64 and X86:
the alias cache is created for AArch64 but then used for X86.

https://lab.llvm.org/staging/#/builders/211/builds/126

The issue only affects unit tests as we only construct one MCPlusBuilder
for ELF binary.

Resolve the issue by moving alias bitvectors to MCPlusBuilder object.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D124942
2022-05-04 12:53:26 -07:00
Amir Ayupov 60957a5a08 [BOLT] Fix ICPJumpTablesTopN option use
Fix non-sensical `opts::ICPJumpTablesTopN != 0 ? opts::ICPTopN : opts::ICPTopN`.
Refactor/simplify another similar assignment.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124880
2022-05-03 19:34:10 -07:00
Amir Ayupov c3d5372093 [BOLT][NFC] Make ICP options naming uniform
Rename `opts::IndirectCallPromotion*` to `opts::ICP*`, making option naming
uniform and easier to follow.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124879
2022-05-03 19:32:45 -07:00
Amir Ayupov d0b1c98c96 [BOLT][NFC] ICP: simplify findTargetsIndex
Unnest lambda and use `llvm::is_contained`.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124877
2022-05-03 19:31:20 -07:00
Amir Ayupov ec02227bf7 [BOLT][NFC] Refactor ICP::findCallTargetSymbols
Reduce nesting making it easier to read.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124876
2022-05-03 19:29:22 -07:00
Paul Kirth 625e0e611b [BOLT] [NFC] Remove unused variable
This patch fixes a warning from -Wunused-but-set-variable
MismatchedBranches are counted, but are never reported.
Since evaluateProfileData() should already identify and report
these cases, we can safely remove the unused variable.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124588
2022-05-03 15:15:56 +00:00
Amir Ayupov 64421e191b [BOLT][NFC] Reduce Target/{AArch64,X86} dependencies
We don't actually depend on entire X86/AArch64 components that pull in CodeGen,
SelectionDAG etc., just the Desc part with opcode and other definitions.

Note that it doesn't decouple BOLT from these components - we still pull in X86
and AArch64 from top-level llvm-bolt dependencies as we use assembler and
disassembler. It's difficult to reduce these as this requires non-trivial
changes to X86/AArch64 components themselves (e.g. moving out AsmPrinter).

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124206
2022-04-29 20:37:53 -07:00
Paul Kirth a0b8ab1ba3 [BOLT][NFC] Fix warning for unqualified call to std::move
Fixes warning from RetpolineInsertion.cpp:171:44:
warning: unqualified call to std::move [-Wunqualified-std-cast-call]

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D124482
2022-04-26 23:18:20 +00:00
Rahman Lavaee e59e580116 [BOLT] Refactor DataAggregator::printLBRHeatMap.
This also fixes some logs that were impacted by D123067.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D124281
2022-04-25 11:39:44 -07:00
Alexander Yermolovich 014cd37f51 [BOLT][DWARF] Implement monolithic DWARF5
Added implementation to support DWARF5 in monolithic mode.
Next step DWARF5 split dwarf support.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D121876
2022-04-21 16:02:23 -07:00
Alexey Moksyakov 48e894a536 [BOLT] Add R_AARCH64_PREL16/32/64 relocations support
Reviewed By: yota9, rafauler

Differential Revision: https://reviews.llvm.org/D122294
2022-04-21 13:52:47 +03:00
Vladislav Khmelevsky 63686af1e1 [BOLT] Fix build with GCC 7.3.0
The gcc 7.3.0 version raises "could not covert" error without std::move
used explicitly.

Differential Revision: https://reviews.llvm.org/D124009
2022-04-21 13:47:58 +03:00
Maksim Panchenko 76981fbcf6 [BOLT] Add fuzzy function name matching for LLVM LTO
LLVM with LTO can generate function names in the form
func.llvm.<number>, where <number> could vary based on the compilation
environment. As a result, if a profiled binary originated from a
different build than a corresponding binary used for BOLT optimization,
then profiles for such LTO functions will be ignored.

To fix the problem, use "fuzzy" matching with "func.llvm.*" form.

Reviewed By: yota9, Amir

Differential Revision: https://reviews.llvm.org/D124117
2022-04-20 17:00:21 -07:00
Alexander Yermolovich 7d6716786f [BOLT][DWARF] Handle Error returned by visitLocationList
Looks like implementation in llvm changed, and now we need to process error
being returned.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D124133
2022-04-20 16:40:46 -07:00
Amir Ayupov 4f277f28ab [BOLT] Check if LLVM_REVISION is defined
Handle the case where LLVM_REVISION is undefined (due to LLVM_APPEND_VC_REV=OFF
or otherwise) by setting "<unknown>" value as before D123549.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D123852
2022-04-15 06:33:14 -07:00
Amir Ayupov 2a9386726b [BOLT][NFC] Use LLVM_REVISION instead of BOLT_VERSION_STRING
Remove duplicate version string identification

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D123549
2022-04-14 19:16:35 -07:00