Commit Graph

49 Commits

Author SHA1 Message Date
Fangrui Song 0972a390b9 LLVM_FALLTHROUGH => [[fallthrough]]. NFC 2022-08-09 04:06:52 +00:00
Kazu Hirata f081ec20b5 [bolt] Remove redundaunt virtual specifiers (NFC)
Identified with modernize-use-override.
2022-07-30 10:35:51 -07:00
Vladislav Khmelevsky 35efe1d806 [BOLT][AArch64] Handle gold linker veneers
The gold linker veneers are written between functions without symbols,
so we to handle it specially in BOLT.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D129260
2022-07-13 14:47:22 +03:00
Rafael Auler a3cfdd746e [BOLT] Increase coverage of shrink wrapping [5/5]
Add -experimental-shrink-wrapping flag to control when we
want to move callee-saved registers even when addresses of the stack
frame are captured and used in pointer arithmetic, making it more
challenging to do alias analysis to prove that we do not access
optimized stack positions. This alias analysis is not yet implemented,
hence, it is experimental. In practice, though, no compiler would emit
code to do pointer arithmetic to access a saved callee-saved register
unless there is a memory bug or we are failing to identify a
callee-saved reg, so I'm not sure how useful it would be to formally
prove that.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126115
2022-07-11 17:30:13 -07:00
Rafael Auler 3508ced6ea [BOLT] Increase coverage of shrink wrapping [2/5]
Refactor isStackAccess() to reflect updates by D126116. Now we only
handle simple stack accesses and delegate the rest of the cases to
getMemDataSize.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126112
2022-07-11 17:29:54 -07:00
Amir Ayupov cb75faf40c [X86][BOLT] Use getOperandType to determine memory access size
Generate INSTRINFO_OPERAND_TYPE table in X86GenInstrInfo.inc.

This diff adds support for instructions that were previously reported as having
memory access size 0. It replaces the heuristic of looking at instruction
register width to determine memory access width by instead checking the memory
operand type using tablegen-provided tables.

Reviewed By: skan

Differential Revision: https://reviews.llvm.org/D126116
2022-06-30 00:25:32 -07:00
Rafael Auler fc2d96c334 Revert "[BOLT][AArch64] Handle gold linker veneers"
This reverts commit 425dda76e9.

This commit is currently causing BOLT to crash in one of our
binaries and needs a bit more checking to make sure it is safe
to land.
2022-06-28 19:23:28 -07:00
Vladislav Khmelevsky 425dda76e9 [BOLT][AArch64] Handle gold linker veneers
The gold linker veneers are written between functions without symbols,
so we to handle it specially in BOLT.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D128082
2022-06-28 16:14:05 +03:00
Amir Ayupov 445bc88501 [BOLT] Use 32-bit MOV to zero 64-bit register in instrumentation code
Instead of `movabsq $0x0, %rax` emit shorter equivalent `movl $0x0, %eax`.
Intel SDM, 3.4.1.1 General-Purpose Registers in 64-Bit Mode:
>32-bit operands generate a 32-bit result, zero-extended to a 64-bit result in
> the destination general-purpose register.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D127045
2022-06-19 11:34:32 -07:00
Maksim Panchenko 1817642684 [BOLT] Add support for GOTPCRELX relocations
The linker can convert instructions with GOTPCRELX relocations into a
form that uses an absolute addressing with an immediate. BOLT needs to
recognize such conversions and symbolize the immediates.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126747
2022-06-09 13:37:04 -07:00
Fangrui Song b92436efcb [bolt] Remove unneeded cl::ZeroOrMore for cl::opt options 2022-06-05 13:29:49 -07:00
Maksim Panchenko e290133c76 [BOLT] Add new class for symbolizing X86 instructions
Summary:
While disassembling instructions, we need to replace certain immediate
operands with symbols. This symbolizing process relies on reading
relocations against instructions. However, some X86 instructions can
have multiple immediate operands and up to two relocations against
them. Thus, correctly matching a relocation to an operand is not
always possible without knowing the operand offset within the
instruction.

Luckily, LLVM provides an interface for passing the required info from
the disassembler via a virtual MCSymbolizer class. Creating a
target-specific version allows a precise matching of relocations to
operands.

This diff adds X86MCSymbolizer class that performs X86-specific
symbolizing (currently limited to non-branch instructions).

Reviewers: yota9, Amir, ayermolo, rafauler, zr33

Differential Revision: https://reviews.llvm.org/D120928
2022-05-31 17:48:19 -07:00
Rafael Auler c09cd64e5c [BOLT] Fix AND evaluation bug in shrink wrapping
Fix a bug where shrink-wrapping would use wrong stack offsets
because the stack was being aligned with an AND instruction, hence,
making its true offsets only available during runtime (we can't
statically determine where are the stack elements and we must give up
on this case).

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126110
2022-05-26 14:59:28 -07:00
Amir Ayupov 139744ac53 [BOLT][NFC] Suppress unused variable warnings
Address warnings in Release build without assertions.
Tip @tschuett for reporting the issue #55404.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125475
2022-05-13 20:10:19 +01:00
Amir Ayupov d63c5a38fe [BOLT][NFC] Use BitVector::set_bits
Refactor and use `set_bits` BitVector interface.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125374
2022-05-11 16:23:44 -07:00
Amir Ayupov 8cb7a873ab [BOLT][NFC] Add MCPlus::primeOperands iterator_range
Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D125397
2022-05-11 09:34:51 -07:00
Amir Ayupov 64421e191b [BOLT][NFC] Reduce Target/{AArch64,X86} dependencies
We don't actually depend on entire X86/AArch64 components that pull in CodeGen,
SelectionDAG etc., just the Desc part with opcode and other definitions.

Note that it doesn't decouple BOLT from these components - we still pull in X86
and AArch64 from top-level llvm-bolt dependencies as we use assembler and
disassembler. It's difficult to reduce these as this requires non-trivial
changes to X86/AArch64 components themselves (e.g. moving out AsmPrinter).

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D124206
2022-04-29 20:37:53 -07:00
Amir Ayupov f99398fe0e [BOLT][NFC] Move isADD64rr and isADDri out of MCPlusBuilder class
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D123077
2022-04-05 14:32:07 -07:00
Vladislav Khmelevsky 2e51a32219 [BOLT] Check for !isTailCall in isUnconditionalBranch
Add !isTailCall in isUnconditionalBranch check in order to sync the x86
and aarch64 and fix the fixDoubleJumps pass on aarch64.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D122929
2022-04-05 23:39:34 +03:00
Amir Ayupov 686406a006 [BOLT][NFC] Use X86 mnemonic checks
Remove switches in X86MCPlusBuilder.cpp, use mnemonic checks instead

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D122853
2022-04-04 14:05:46 -07:00
Amir Ayupov 42e8e00189 [BOLT][NFC] Use X86 mnemonic tables
Remove tables from X86MCPlusBuilder, make use of llvm::X86 mnemonic tables.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121573
2022-03-18 01:52:11 -07:00
Amir Ayupov dc1cf838a5 [BOLT] Strip redundant AdSize override prefix
Since LLVM MC now preserves redundant AdSize override prefix (0x67), remove it
in BOLT explicitly (-x86-strip-redundant-adsize, on by default).

Test Plan:
`bin/llvm-lit -a bolt/test/X86/addr32.s`

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120975
2022-03-16 09:38:17 -07:00
Amir Ayupov 698127df51 [BOLT][NFC] Move isMOVSX64rm32 out of MCPlusBuilder
Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121669
2022-03-16 08:18:56 -07:00
Amir Ayupov 5790441c45 [BOLT][NFC] Use getShortOpcodeArith in X86MCPlusBuilder
Unify `llvm::X86::getRelaxedOpcodeArith` and `getShortArithOpcode` in
X86MCPlusBuilder.cpp.

Addresses https://lists.llvm.org/pipermail/llvm-dev/2022-January/154526.html

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D121404
2022-03-12 09:07:28 -08:00
Amir Ayupov 687e4af1c0 [BOLT] CMOVConversion pass
Convert simple hammocks into cmov based on misprediction rate.

Test Plan:
- Assembly test: `cmov-conversion.s`
- Testing on a binary:
  # Bootstrap clang with `-x86-cmov-converter-force-all` and `-Wl,--emit-relocs`
  (Release build)
  # Collect perf.data:

    - `clang++ <opts> bolt/lib/Core/BinaryFunction.cpp -E > bf.cpp`
    - `perf record -e cycles:u -j any,u -- clang-15 bf.cpp -O2 -std=c++14 -c -o bf.o`
  # Optimize clang-15 with and w/o -cmov-conversion:
    - `llvm-bolt clang-15 -p perf.data -o clang-15.bolt`
    - `llvm-bolt clang-15 -p perf.data -cmov-conversion -o clang-15.bolt.cmovconv`
  # Run perf experiment:
    - test: `clang-15.bolt.cmovconv`,
    - control: `clang-15.bolt`,
    - workload (clang options): `bf.cpp -O2 -std=c++14 -c -o bf.o`
Results:
```
  task-clock [delta: -360.21 ± 356.75, delta(%): -1.7760 ± 1.7589, p-value: 0.047951, balance: -6]
  instructions  [delta: 44061118 ± 13246382, delta(%): 0.0690 ± 0.0207, p-value: 0.000001, balance: 50]
  icache-misses [delta: -5534468 ± 2779620, delta(%): -0.4331 ± 0.2175, p-value: 0.028014, balance: -28]
  branch-misses [delta: -1624270 ± 1113244, delta(%): -0.3456 ± 0.2368, p-value: 0.030300, balance: -22]
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120177
2022-03-08 10:44:31 -08:00
Maksim Panchenko fada230920 [BOLT][NFC] Return MCRegister::NoRegister from MCPlusBuilder::getNoRegister()
Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D120863
2022-03-03 13:25:13 -08:00
Vladislav Khmelevsky 00b6efc830 [BOLT] Enable PLT analysis for aarch64
This patch enables PLT analysis for aarch64. It is used by the static
relocations in order to provide final symbol address of PLT entry for some
instructions like ADRP.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D118088
2022-03-02 22:14:48 +03:00
Amir Ayupov 08dcbed92f [BOLT] Fix X86MCPlusBuilder::replaceRegWithImm
Reassigning the operand didn't update the operand type which resulted in an
assertion (`Assertion `isReg() && "This is not a register operand!"' failed.`)
Reset the instruction instead.

Test Plan:
```
ninja check-bolt
...
PASS: BOLT-Unit :: Core/./CoreTests/X86/MCPlusBuilderTester.ReplaceRegWithImm/0 (90 of 136)
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120263
2022-02-28 19:24:46 -08:00
serge-sans-paille 57f7c7d90e Add missing MC includes in bolt/
Changes needed after ef736a1c39 that removes some implicit
dependencies from MrCV headers.
2022-02-09 08:28:34 -05:00
Vladislav Khmelevsky 19fb5a210d [BOLT] Add aarch64 support for peephole passes
Enable peephole optimizations for aarch64.
Also small code refactoring - add PeepholeOpts under Peepholes class.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D118732
2022-02-08 03:04:40 +03:00
Amir Ayupov 167b623a6a [BOLT][NFC] Use isInt<> instead of range checks
Summary: Reuse LLVM isInt check

Reviewed By: maksfb

FBD33945182
2022-02-02 20:32:05 -08:00
Amir Ayupov a9cd49d50e [BOLT][NFC] Move Offset annotation to Group 1
Summary:
Move the annotation to avoid dynamic memory allocations.
Improves the CPU time of instrumenting a large binary by 1% (+-0.8%, p-value 0.01)

Test Plan: NFC

Reviewers: maksfb

FBD30091656
2022-01-18 13:24:50 -08:00
Amir Ayupov c34adaa3ca [BOLT][CMAKE] Use IN_LIST check
Summary:
Address @smeenai feedback https://reviews.llvm.org/D117061#inline-1122106:
>CMake has if(IN_LIST) now, which you can use instead of the string(FIND)

IN_LIST is available since CMake 3.3 released in 2015.

Reviewed By: smeenai

FBD33590959
2022-01-14 15:47:14 -08:00
Maksim Panchenko 8aab58ba65 [BOLT][NFC] Refactor AArch64MCPlusBuilder
Summary: Selectively apply clang-format to the code in AArch64MCPlusBuilder.cpp.

(cherry picked from FBD33495653)
2022-01-08 18:17:31 -08:00
Maksim Panchenko 82278a8f29 [BOLT][NFC] Refactor X86MCPlusBuilder
Summary:
Selectively apply clang-format and other minor refactoring to the code
in X86MCPlusBuilder.cpp

(cherry picked from FBD33495550)
2022-01-08 17:48:33 -08:00
Amir Ayupov 799cbbb797 [BOLT][NFC] Reuse X86BaseInfo interfaces for macrofusion checks
Summary:
Remove X86MCPlusBuilder code that duplicates checks in X86BaseInfo.
Remove isINC and isDEC as redundant.

The new code of `X86MCPlusBuilder::isMacroOpFusionPair` is functionally
equivalent to `X86AsmBackend::isMacroFused`. However, as the method is
declared/defined in X86AsmBackend.cpp and not exported in a header file,
there's no way to use it in BOLT without changes in LLVM code.

(cherry picked from FBD33440373)
2022-01-05 15:58:01 -08:00
Amir Ayupov 1d3c150748 [BOLT] Remove ineligible macro-fusion patterns
Summary:
Remove patterns ineligible for macro-fusion:
- First instruction has a memory destination

This is a temporary commit to align BOLT with LLVM MC interfaces.
(cherry picked from FBD33479340)
2022-01-07 09:40:04 -08:00
Maksim Panchenko bc9032c7fa [BOLT][NFC] Use uniform DEBUG_TYPE for MCPlus builders
(cherry picked from FBD33435121)
2022-01-05 12:02:54 -08:00
Amir Ayupov 89ceb77997 [BOLT][NFC] Fix braces usage in Target
Summary:
Refactor bolt/lib/Target to follow the braces rule for if/else/loop from
[LLVM Coding Standards](https://llvm.org/docs/CodingStandards.html).

(cherry picked from FBD33345353)
2021-12-28 17:52:08 -08:00
Maksim Panchenko 2f09f445b2 [BOLT][NFC] Fix file-description comments
Summary: Fix comments at the start of source files.

(cherry picked from FBD33274597)
2021-12-21 10:21:41 -08:00
Rafael Auler 46e93fb427 Fix frameopt crash when processing POPF
Summary: POPF instruction was triggering an assertion in our analysis.

(cherry picked from FBD33141809)
2021-12-15 13:29:46 -08:00
Maksim Panchenko 40c2e0fafe [BOLT][NFC] Reformat with clang-format
Summary: Selectively apply clang-format to BOLT code base.

(cherry picked from FBD33119052)
2021-12-14 16:52:51 -08:00
Maksim Panchenko 69706eafab [BOLT] Refactor BinaryBasicBlock to use ADT
Summary:
Refactor members of BinaryBasicBlock. Replace some std containers with
ADT equivalents. The size of BinaryBasicBlock on x86-64 Linux is reduced
from 232 bytes to 192 bytes.

(cherry picked from FBD33081850)
2021-12-09 11:53:12 -08:00
Amir Ayupov b69d487a62 [BOLT][NFC] Remove unused MCPlusBuilder::isEnter
Summary: Remove unused code identified via coverage report.

(cherry picked from FBD32818608)
2021-12-02 17:23:58 -08:00
Amir Ayupov 8e632eae56 [BOLT][NFC] Remove unused MCPlusBuilder::createIndirectCall method
Summary: Remove unused code identified via coverage report.

(cherry picked from FBD32818329)
2021-12-02 17:19:33 -08:00
Maksim Panchenko 933df2a460 [BOLT][NFC] Remove references to internal tasks
(cherry picked from FBD32272387)
2021-11-08 19:54:05 -08:00
Alexey Moksyakov 85ffa8e4ba [PR][BOLT][Instrumentation] Optimize eflags load/store
Summary:
This commit uses reviews.llvm.org/D6629 as a reference to optimize
X86::EFLAGS load/store in the instrumentation snippet by using lahf/sahf
instructions instead of pushf/popf.

(cherry picked from FBD31662303)
2021-10-11 16:10:06 +00:00
Rafael Auler 0559dab546 [BOLT] Improve cmake configs for opensource
Summary:
Change cmake config in BOLT to only support Linux. In other
platforms, we print a warning that we won't build BOLT.  Change
configs to determine whether we will build BOLT runtime libs. This
only happens in x86 hosts. If true, we will build the runtime and
enable bolt-runtime tests. New tests that depend on the bolt_rt lib
needs to be marked REQUIRES:bolt-runtime. I updated the relevant
tests.  Fix cmake to do not crash when building llvm with a target
that BOLT does not support.

(cherry picked from FBD31935760)
2021-10-26 12:26:23 -07:00
Rafael Auler a34c753fe7 Rebase: [NFC] Refactor sources to be buildable in shared mode
Summary:
Moves source files into separate components, and make explicit
component dependency on each other, so LLVM build system knows how to
build BOLT in BUILD_SHARED_LIBS=ON.

Please use the -c merge.renamelimit=230 git option when rebasing your
work on top of this change.

To achieve this, we create a new library to hold core IR files (most
classes beginning with Binary in their names), a new library to hold
Utils, some command line options shared across both RewriteInstance
and core IR files, a new library called Rewrite to hold most classes
concerned with running top-level functions coordinating the binary
rewriting process, and a new library called Profile to hold classes
dealing with profile reading and writing.

To remove the dependency from BinaryContext into X86-specific classes,
we do some refactoring on the BinaryContext constructor to receive a
reference to the specific backend directly from RewriteInstance. Then,
the dependency on X86 or AArch64-specific classes is transfered to the
Rewrite library. We can't have the Core library depend on targets
because targets depend on Core (which would create a cycle).

Files implementing the entry point of a tool are transferred to the
tools/ folder. All header files are transferred to the include/
folder. The src/ folder was renamed to lib/.

(cherry picked from FBD32746834)
2021-10-08 11:47:10 -07:00