Commit Graph

185 Commits

Author SHA1 Message Date
Alexander Yermolovich 1c6dc43de9 [BOLT]DWARF] Eagerly write out loclists
Taking advantage of us being able to re-write .debug_info to reduce memory
footprint loclists. Writing out loc-list as they are added, similar to how
we handle ranges.

Collected on clang-14
trunk
4:41.20 real,   389.50 user,    59.50 sys,      0 amem, 38412532 mmem
4:30.08 real,   376.10 user,    63.75 sys,      0 amem, 38477844 mmem
4:25.58 real,   373.76 user,    54.71 sys,      0 amem, 38439660 mmem
diff
4:34.66 real,   392.83 user,    57.73 sys,      0 amem, 38382560 mmem
4:35.96 real,   377.70 user,    58.62 sys,      0 amem, 38255840 mmem
4:27.61 real,    390.18 user,    57.02 sys,      0 amem, 38223224 mmem

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D126999
2022-06-08 16:52:59 -07:00
Huan Nguyen 5ac26156fe [BOLT][NFC] Warning for deprecated option '-reorder-blocks=cache+'
Emit warning when using deprecated option '-reorder-blocks=cache+'.
Auto switch to option '-reorder-blocks=ext-tsp'.

Test Plan:
```
ninja check-bolt
```
Added a new test cache+-deprecated.test.
Run and verify that the upstream tests are passed.

Reviewed By: rafauler, Amir, maksfb

Differential Revision: https://reviews.llvm.org/D126722
2022-06-03 14:16:55 -07:00
spupyrev 5904836b8a [BOLT] Cache-Aware Tail Duplication
A new "cache-aware" strategy for tail duplication.

Differential Revision: https://reviews.llvm.org/D123050
2022-06-03 09:08:45 -07:00
Alexander Yermolovich ab9a175990 [BOLT][DWARF] Fix TU Index handling for DWARF4/5
When we generate split dwarf with -fdebug-types-section we will have
.debug_types.dwo sections. These go into TU Index when we run llvm-dwp. BOLT was
not handling DWP input correctly with this section.

Added support for handling DWP with TU Index as an input and output for DWARF4.
Added support for handling DWP with TU Index as an input for DWARF5

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D126087
2022-06-01 18:16:12 -07:00
Huan Nguyen 38fb7d56e5 [BOLT][TEST] Replace cache+ option with ext-tsp
Replace "cache+" with "ext-tsp" in all BOLT tests

Test Plan:
```
ninja check-bolt
grep -rnw . -e "cache+"
```
no more tests containing "cache+"
"cache+" and "ext-tsp" are aliases

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D126714
2022-06-01 14:00:16 -07:00
Yi Kong 716d428ab5 [BOLT] Add `-o` option to merge-fdata
Differential Revision: https://reviews.llvm.org/D126788
2022-06-02 01:29:04 +08:00
Alexander Yermolovich ec2711b354 [BOLT][DWARF] Fix dwarf5-debug-line test
After D126484, order in .debug-line-str and .debug-line is different. Changed
test accordingly.

Differential Revision: https://reviews.llvm.org/D126733
2022-05-31 18:17:16 -07:00
Maksim Panchenko e290133c76 [BOLT] Add new class for symbolizing X86 instructions
Summary:
While disassembling instructions, we need to replace certain immediate
operands with symbols. This symbolizing process relies on reading
relocations against instructions. However, some X86 instructions can
have multiple immediate operands and up to two relocations against
them. Thus, correctly matching a relocation to an operand is not
always possible without knowing the operand offset within the
instruction.

Luckily, LLVM provides an interface for passing the required info from
the disassembler via a virtual MCSymbolizer class. Creating a
target-specific version allows a precise matching of relocations to
operands.

This diff adds X86MCSymbolizer class that performs X86-specific
symbolizing (currently limited to non-branch instructions).

Reviewers: yota9, Amir, ayermolo, rafauler, zr33

Differential Revision: https://reviews.llvm.org/D120928
2022-05-31 17:48:19 -07:00
Denis Revunov 8579db96e8 [BOLT] [AArch64] Handle constant islands spanning multiple functions
Fix BOLT's constant island mapping when a constant island marked by $d
spans multiple functions. Currently, because BOLT only marks the
constant island in the first function where $d is located, if the next
function contains data at its start, BOLT will miss the data and try
to disassemble it. This patch adds code to explicitly go through all
symbols between $d and $x markers and mark their respective offsets as
data, which stops BOLT from trying to disassemble data. It also adds
MarkerType enum and refactors related functions.

Reviewed By: yota9, rafauler

Differential Revision: https://reviews.llvm.org/D126177
2022-05-31 13:51:35 -07:00
Rafael Auler b8a6345554 [BOLT] Fix LIT tests on Windows VS2019
Fix newline issue in link_fdata.py, as well as how to call the tool.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126437
2022-05-31 11:45:39 -07:00
Rafael Auler c09cd64e5c [BOLT] Fix AND evaluation bug in shrink wrapping
Fix a bug where shrink-wrapping would use wrong stack offsets
because the stack was being aligned with an AND instruction, hence,
making its true offsets only available during runtime (we can't
statically determine where are the stack elements and we must give up
on this case).

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D126110
2022-05-26 14:59:28 -07:00
zr33 e51a6b7374 [BOLT][DWARF] Convert dwarf5-df-* tests to assembly tests
Reviewed By: ayermolo

Differential Revision: https://reviews.llvm.org/D126086
2022-05-25 13:41:18 -07:00
Rafael Auler 6cc741bcbf [BOLT] Testcase to repro R_X86_64_REX_GOTPCRELX bug
Add a new testcase that reproduces a bug when BOLTing current
trunk LLD bootstrapped with trunk clang. This makes it official
that we do not support this transformation but are working on
it. When the support is ready, XFAIL should be removed.

Reviewed By: maksfb, Amir, yota9

Differential Revision: https://reviews.llvm.org/D125843
2022-05-18 16:07:14 -07:00
Rafael Auler 2fdc5d336e [BOLT] Fix merge-fdata handling of BAT profiles
When a profile is collected in a BOLTed binary, the generated
profile is tagged with a header string "boltedcollection" in the first
line of the fdata file. Fix merge-fdata to recognize this header
string and preserve it into the output.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D125591
2022-05-13 19:41:55 -07:00
Amir Ayupov da766cea56 [BOLT][TEST] Fix testing on macos
- Fix common (arch-independent) tests to explicitly target -linux triple.
- Override the triple inside arch-specific tests.
- Add cflags to common tests.
- Update individual tests.
- Expand pipe stderr `|&` shorthand.

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D125548
2022-05-13 13:03:47 -07:00
Amir Ayupov 4a58eb9e4e [BOLT][TEST] Remove -gdwarf-4 override from %cflags
As BOLT support for monolithic and split DWARF5 is added, remove DWARF version
override for BOLT tests.

Reviewed By: ayermolo

Differential Revision: https://reviews.llvm.org/D125366
2022-05-11 03:38:26 -07:00
Alexander Yermolovich 3abb68a626 [BOLT][DWARF] Fix assert for split dwarf.
Fixing a small bug where it would assert if CU does not modify .debug_addr section.

Differential Revision: https://reviews.llvm.org/D125181
2022-05-08 19:18:17 -07:00
Alexander Yermolovich ba1ac98c62 [BOLT][DWARF] Add version 5 split dwarf support
Added support for DWARF5 Split Dwarf.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D122988
2022-05-05 14:59:05 -07:00
Amir Ayupov 1d5263c554 [BOLT][TEST] Fix test failures on AArch64 builder
Address X86 tests failures on AArch64 builder:
https://lab.llvm.org/staging/#/builders/211/builds/82

Inputs fail to cross-compile due to a missing header:
```
/usr/include/stdio.h:27:10: fatal error: 'bits/libc-header-start.h' file not found
#include <bits/libc-header-start.h>
```

As inputs are linked with `-nostdlib` anyway, don't include stdio.h.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D124863
2022-05-03 15:39:43 -07:00
Amir Ayupov 39492ba5d6 Revert "[BOLT][TEST] Fix test failures on AArch64 builder"
This reverts commit 88b6d3211c.
2022-05-03 12:45:15 -07:00
Amir Ayupov 88b6d3211c [BOLT][TEST] Fix test failures on AArch64 builder
Address X86 tests failures on AArch64 builder:
https://lab.llvm.org/staging/#/builders/211/builds/82

Inputs fail to cross-compile due to a missing header:
```
/usr/include/stdio.h:27:10: fatal error: 'bits/libc-header-start.h' file not found
#include <bits/libc-header-start.h>
```

As inputs are linked with `-nostdlib` anyway, don't include stdio.h.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D124863
2022-05-03 12:42:30 -07:00
Alexey Moksyakov 61d54259ed [BOLT] Fix r_aarch64_prelxx test
The relocation value is calculated using the formula S + A - P,
the verification of the value is performed by inversely calculating the location address

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D124270
2022-04-28 23:52:24 +03:00
Alexander Yermolovich 014cd37f51 [BOLT][DWARF] Implement monolithic DWARF5
Added implementation to support DWARF5 in monolithic mode.
Next step DWARF5 split dwarf support.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D121876
2022-04-21 16:02:23 -07:00
Alexey Moksyakov 48e894a536 [BOLT] Add R_AARCH64_PREL16/32/64 relocations support
Reviewed By: yota9, rafauler

Differential Revision: https://reviews.llvm.org/D122294
2022-04-21 13:52:47 +03:00
Maksim Panchenko 76981fbcf6 [BOLT] Add fuzzy function name matching for LLVM LTO
LLVM with LTO can generate function names in the form
func.llvm.<number>, where <number> could vary based on the compilation
environment. As a result, if a profiled binary originated from a
different build than a corresponding binary used for BOLT optimization,
then profiles for such LTO functions will be ignored.

To fix the problem, use "fuzzy" matching with "func.llvm.*" form.

Reviewed By: yota9, Amir

Differential Revision: https://reviews.llvm.org/D124117
2022-04-20 17:00:21 -07:00
Alexander Yermolovich 7d6716786f [BOLT][DWARF] Handle Error returned by visitLocationList
Looks like implementation in llvm changed, and now we need to process error
being returned.

Reviewed By: maksfb

Differential Revision: https://reviews.llvm.org/D124133
2022-04-20 16:40:46 -07:00
Aaron Ballman e0ee080574 Speculatively fix build bots
This should address the issue found in:
https://lab.llvm.org/buildbot/#/builders/215/builds/4610
2022-04-20 12:05:33 -04:00
Aaron Ballman ef50d817b6 Speculatively fix build bots
This should address build failures found in:

https://lab.llvm.org/buildbot/#/builders/217/builds/3610
https://lab.llvm.org/buildbot/#/builders/215/builds/4609
https://lab.llvm.org/buildbot/#/builders/68/builds/31012
2022-04-20 11:48:06 -04:00
Vladislav Khmelevsky 2f98c5febc [BOLT] Update skipRelocation for aarch64
The ld might relax ADRP+ADD or ADRP+LDR sequences to the ADR+NOP, add
the new case to the skipRelocation for aarch64.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D123334
2022-04-13 22:54:06 +03:00
Amir Ayupov 04641b2121 [BOLT][TEST] Add -no-pie to two tests
Missed these two tests in D123329 in a rebase.
2022-04-13 11:48:13 -07:00
Amir Ayupov 487570fb86 [BOLT][TEST] Remove -no-pie from cflags/cxxflags
Align with an upstream change D120305 to make PIE the default on linux-gnu.

Add `-no-pie` to tests that require it.

Reviewed By: maksfb, yota9

Differential Revision: https://reviews.llvm.org/D123329
2022-04-13 11:38:40 -07:00
Maksim Panchenko 36cb736665 [BOLT] Ignore PC-relative relocations from data to data
BOLT expects PC-relative relocations in data sections to reference code
and the relocated data to form a jump table. However, there are cases
where PC-relative addressing is used for data-to-data references
(e.g. clang-15 can generate such code). BOLT should recognize and ignore
such relocations. Otherwise, they will be considered relocations not
claimed by any jump table and cause a failure in the strict mode.

Reviewed By: yota9, Amir

Differential Revision: https://reviews.llvm.org/D123650
2022-04-13 11:13:51 -07:00
Vladislav Khmelevsky caf9eb6e4d [BOLT] Fix two aarch64 tests
tls-lld test might be broken since compiler might optimize plt function
call and use address directly from got table. The test is removed since
plt-gnu-ld checks the same functionality + versioning symbol matching,
no need to keep both of the tests.
The toolchain might optimize relocations in runtime-relocs test, replace
the test compilation with yaml files.

Differential Revision: https://reviews.llvm.org/D123332
2022-04-13 13:38:32 +03:00
Vladislav Khmelevsky 87a57aada3 [BOLT][test] Fix X86 tests
Differential Revision: https://reviews.llvm.org/D123133
2022-04-06 21:16:42 +03:00
Vladislav Khmelevsky 2e51a32219 [BOLT] Check for !isTailCall in isUnconditionalBranch
Add !isTailCall in isUnconditionalBranch check in order to sync the x86
and aarch64 and fix the fixDoubleJumps pass on aarch64.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D122929
2022-04-05 23:39:34 +03:00
Vladislav Khmelevsky 4956e0e197 [BOLT] Fix plt relocations symbol match
The bfd linker adds the symbol versioning string to the symbol name in symtab.
Skip the versioning part in order to find the registered PLT function.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D122039
2022-04-05 15:57:26 +03:00
Maksim Panchenko 163e188e3e [BOLT][test] Fix AArch64 test
Remove header dependency from cross-platform test.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D123107
2022-04-04 23:28:47 -07:00
Maksim Panchenko f927106e10 [BOLT][test] Enable cross-target testing
Check for supported target architecture instead of the host arch when
deciding to execute non-runtime tests.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D122498
2022-04-04 23:00:54 -07:00
Maksim Panchenko f0f5d19a36 [BOLT][test] Fix X86 cross-platform tests
Use target-specific flags for building X86 non-runnable tests.

Reviewed By: Amir

Differential Revision: https://reviews.llvm.org/D123072
2022-04-04 22:25:42 -07:00
Vladislav Khmelevsky 3b1314f4de [BOLT] AArch64: Read all static relocations
Read static relocs on the same address, as dynamic in order to update
constant island data address properly.

Differential Revision: https://reviews.llvm.org/D122100
2022-04-03 19:03:35 +03:00
Maksim Panchenko 5679a3ce87 [BOLT][test] Fix AArch64 cross-platform tests
Use target-specific flags for building AArch64 non-runnable tests.

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D122520
2022-04-01 17:10:47 -07:00
Vladislav Khmelevsky fed958c6cc [BOLT] AArch64: Emit text objects
BOLT treats aarch64 objects located in text as empty functions with
contant islands. Emit them with at least 8-byte alignment to the new
text section.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D122097
2022-03-31 22:28:50 +03:00
Vladislav Khmelevsky af9bdcfc46 [BOLT] Align constant islands to 8 bytes
AArch64 requires CI to be aligned to 8 bytes due to access instructions
restrictions. E.g. the ldr with imm, where imm must be aligned to 8 bytes.

Differential Revision: https://reviews.llvm.org/D122065
2022-03-27 22:30:42 +03:00
spupyrev 4609f60ebc [BOLT] Avoid pointless loop rotation
It seems the earlier implementation does not follow the description
in LoopRotationPass.h: It rotates loops even if they are already laid out
correctly. The diff adjusts the behaviour.

Given that the impact of LoopInversionPass is minor, this change won't
yield significant perf differences. Tested on clang-10: there seems to be a
0.1%-0.3% cpu win and a small reduction of branch misses.

**Before:**
BOLT-INFO: 120 Functions were reordered by LoopInversionPass

**After:**
BOLT-INFO: 79 Functions were reordered by LoopInversionPass

Reviewed By: yota9

Differential Revision: https://reviews.llvm.org/D121921
2022-03-22 12:42:42 -07:00
Amir Ayupov dc1cf838a5 [BOLT] Strip redundant AdSize override prefix
Since LLVM MC now preserves redundant AdSize override prefix (0x67), remove it
in BOLT explicitly (-x86-strip-redundant-adsize, on by default).

Test Plan:
`bin/llvm-lit -a bolt/test/X86/addr32.s`

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120975
2022-03-16 09:38:17 -07:00
Sam McCall 75acad41bc Use lit_config.substitute instead of foo % lit_config.params everywhere
This mechanically applies the same changes from D121427 everywhere.

Differential Revision: https://reviews.llvm.org/D121746
2022-03-16 09:57:41 +01:00
Petr Hosek 0c0f6cfb7b [CMake] Rename TARGET_TRIPLE to LLVM_TARGET_TRIPLE
This clarifies that this is an LLVM specific variable and avoids
potential conflicts with other projects.

Differential Revision: https://reviews.llvm.org/D119918
2022-03-11 15:43:01 -08:00
Vladislav Khmelevsky 506a91c089 [BOLT] Move some of the tests to common directory
Some of the tests are not x86-specific, move them to common directory.

Differential Revision: https://reviews.llvm.org/D121261
2022-03-09 14:51:17 +03:00
Vladislav Khmelevsky 8bdbcfe7d8 [BOLT] Handle ifuncs trampolines for aarch64
The aarch64 uses the trampolines located in .iplt section, which
contains plt-like trampolines on the value stored in .got. In this case
we don't have JUMP_SLOT relocation, but we have a symbol that belongs to
ifunc trampoline, so use it and set set plt symbol for such functions.

Vladislav Khmelevsky,
Advanced Software Technology Lab, Huawei

Differential Revision: https://reviews.llvm.org/D120850
2022-03-09 01:38:19 +03:00
Amir Ayupov 687e4af1c0 [BOLT] CMOVConversion pass
Convert simple hammocks into cmov based on misprediction rate.

Test Plan:
- Assembly test: `cmov-conversion.s`
- Testing on a binary:
  # Bootstrap clang with `-x86-cmov-converter-force-all` and `-Wl,--emit-relocs`
  (Release build)
  # Collect perf.data:

    - `clang++ <opts> bolt/lib/Core/BinaryFunction.cpp -E > bf.cpp`
    - `perf record -e cycles:u -j any,u -- clang-15 bf.cpp -O2 -std=c++14 -c -o bf.o`
  # Optimize clang-15 with and w/o -cmov-conversion:
    - `llvm-bolt clang-15 -p perf.data -o clang-15.bolt`
    - `llvm-bolt clang-15 -p perf.data -cmov-conversion -o clang-15.bolt.cmovconv`
  # Run perf experiment:
    - test: `clang-15.bolt.cmovconv`,
    - control: `clang-15.bolt`,
    - workload (clang options): `bf.cpp -O2 -std=c++14 -c -o bf.o`
Results:
```
  task-clock [delta: -360.21 ± 356.75, delta(%): -1.7760 ± 1.7589, p-value: 0.047951, balance: -6]
  instructions  [delta: 44061118 ± 13246382, delta(%): 0.0690 ± 0.0207, p-value: 0.000001, balance: 50]
  icache-misses [delta: -5534468 ± 2779620, delta(%): -0.4331 ± 0.2175, p-value: 0.028014, balance: -28]
  branch-misses [delta: -1624270 ± 1113244, delta(%): -0.3456 ± 0.2368, p-value: 0.030300, balance: -22]
```

Reviewed By: rafauler

Differential Revision: https://reviews.llvm.org/D120177
2022-03-08 10:44:31 -08:00