Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
* We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
* We use a single byte per function rather than 8 bytes per block
The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.
When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).
[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116180
https://reviews.llvm.org/D116179 introduced some changes to
`InstrProfData.inc` which broke some downstream builds. This commit
reverts those changes since they only changes two field names.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D117631
Existing code tended to assume that counters had type `uint64_t` and
computed size from the number of counters. Fix this code to directly
compute the counters size in number of bytes where possible. When the
number of counters is needed, use `__llvm_profile_counter_entry_size()`
or `getCounterTypeSize()`. In a later diff these functions will depend
on the profile mode.
Change the meaning of `DataSize` and `CountersSize` to make them more clear.
* `DataSize` (`CountersSize`) - the size of the data (counter) section in bytes.
* `NumData` (`NumCounters`) - the number of data (counter) entries.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D116179
This patch adds support to read all the PT_NOTE segments in the
executable to find the binary ids. Previously, it was only reading
the first PT_NOTE segment, and this was missing the cases where
binary id is in the following segments. As a result, binary-id.c
and binary-id-padding.c test were failing in the following cases:
1) sanitizer-x86_64-linux bot
https://lab.llvm.org/staging/#/builders/97
2) OpenSuse Tumbleweed
https://github.com/llvm/llvm-project/issues/52695
Differential Revision: https://reviews.llvm.org/D115830
Profile merging is not supported when using debug info profile
correlation because the data section won't be in the binary at runtime.
Change the default profile name in this mode to `default_%p.proflite` so
we don't use profile merging.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115979
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.
Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
The original diff https://reviews.llvm.org/D114565 was reverted because of the `Instrumentation/InstrProfiling/debug-info-correlate.ll` test, which is fixed in this commit.
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D115693
This reverts commit 800bf8ed29.
The `Instrumentation/InstrProfiling/debug-info-correlate.ll` test was
failing because I forgot the `llc` commands are architecture specific.
I'll follow up with a fix.
Differential Revision: https://reviews.llvm.org/D115689
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.
Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4
Reviewed By: kyulee
Differential Revision: https://reviews.llvm.org/D114565
The INSTR_PROF_RAW_MAGIC_* number in profraw files should match during
profile merging. This causes an error with 32-bit and 64-bit variants
of the same code. The module signatures for the two binaries are
identical but they use different INSTR_PROF_RAW_MAGIC_* causing a
failure when profile-merging is used. Including it when computing the
module signature yields different signatures for the 32-bit and 64-bit
profiles.
Differential Revision: https://reviews.llvm.org/D114054
Since libclang_rt.profile is added later in the command line, a
definition of __llvm_profile_raw_version is not included if it is
provided from an earlier object, e.g. from a shared dependency.
This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version. This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols). Such a scenario can occur in Android's
mainline modules.
E.g.:
ld -o libB.so libclang_rt.profile-x86_64.a
ld -o libA.so -l B libclang_rt.profile-x86_64.a
libB.so has a global definition of __llvm_profile_raw_version. libA.so
uses libB.so's definition of __llvm_profile_raw_version. At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.
Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile. The visiblity
is unchanged for Apple platforms where its presence is checked by the
TAPI tool.
Reviewed By: MaskRay, phosek, davidxl
Differential Revision: https://reviews.llvm.org/D111759
Some compiler-rt tests are inherently incompatible with VE because..
* No consistent denormal support on VE. We skip denormal fp inputs in builtin tests.
* `madvise` unsupported on VE.
* Instruction alignment requirements.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D113093
Hide __llvm_profile_raw_version so as not to resolve reference from a
dependent shared object. Since libclang_rt.profile is added later in
the command line, a definition of __llvm_profile_raw_version is not
included if it is provided from an earlier object, e.g. from a shared
dependency.
This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version. This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols). Such a scenario can occur in Android's
mainline modules.
E.g.:
ld -o libB.so libclang_rt.profile-x86_64.a
ld -o libA.so -l B libclang_rt.profile-x86_64.a
libB.so has a global definition of __llvm_profile_raw_version. libA.so
uses libB.so's definition of __llvm_profile_raw_version. At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.
Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile.
Differential Revision: https://reviews.llvm.org/D111759
Fuchsia Clang code coverage pipeline started to use binary ids that are
embedded in profiles. This patch removes emitting symbolizer markup,
which is not necessary in the coverage pipeline anymore.
Differential Revision: https://reviews.llvm.org/D111674
Some tests with binary IDs would fail with error: no profile can be merged.
This is because raw profiles could have unaligned headers when emitting binary
IDs. This means padding should be emitted after binary IDs are emitted to
ensure everything else is aligned. This patch adds padding after each binary ID
to ensure the next binary ID size is 8-byte aligned. This also adds extra
checks to ensure we aren't reading corrupted data when printing binary IDs.
Differential Revision: https://reviews.llvm.org/D110365
Some tests with binary IDs would fail with error: no profile can be merged.
This is because raw profiles could have unaligned headers when emitting binary
IDs. This means padding should be emitted after binary IDs are emitted to
ensure everything else is aligned. This patch accounts for that padding in
__llvm_write_binary_ids.
Differential Revision: https://reviews.llvm.org/D110188
Replace D107203, because __llvm_profile_set_file_object is usually used when the
process doesn't have permission to open/create file. That patch trying to copy
from old profile to new profile contradicts with the usage.
Differential Revision: https://reviews.llvm.org/D108242
In the raw profile, binary ids immediately follow the header so when
computing counters offset we need to account for the new section.
Differential Revision: https://reviews.llvm.org/D108608
With unquoted ${CMAKE_CXX_FLAGS}, the REGEX fails when it's empty:
```CMake Error at lib/scudo/standalone/CMakeLists.txt:14 (string):
string sub-command REGEX, mode REPLACE needs at least 6 arguments total to
command.```
D104556 change the CountersPtr to be relative, however, it did not
update the pointer initialization in __llvm_profile_register_function,
so the platform (eg:AIX) that use __llvm_profile_register_function is now totaly
broken, any PGO code will SEGV.
This patch update the code to reflect that the Data->CountersPtr is now
relative.
Reviewed By: MaskRay, davidxl
Differential Revision: https://reviews.llvm.org/D108304
This patch adds static keyword to internal functions that write
binary id to restrict visibility to the file that they are declared.
Differential Revision: https://reviews.llvm.org/D108154
At least when compiling with gcc, this is not supported and will
result in errors when linking against the profiler runtime. Only
use the pragma comment linker based code with MSVC, but not with
a mingw toolchain. This also undoes D107620, which shouldn't be
relevant anymore.
Differential Revision: https://reviews.llvm.org/D108095
54902e00d1 added a use of
/alternatename via a #pragma comment(linker); in MinGW mode, this
requires building with -fms-extensions. (This flag is added to
SANITIZER_COMMON_CFLAGS in the toplevel CMakeLists.txt.)
This avoids a warning when building in MinGW mode (about an unknown
pragma being ignored), and presumably also makes the code work as
intended.
Differential Revision: https://reviews.llvm.org/D107620
The Headers.CountersDelta field is an uint64_t, not a pointer,
so just cast to uint32_t to truncate it.
Differential Revision: https://reviews.llvm.org/D107619
The _zx_vmar_root_self function is not a system call but
a libc function declared in a separate header.
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D107616
The Solaris buildbots have been broken for some time by the unconditional
use of `NT_GNU_BUILD_ID`, e.g. Solaris/sparcv9
<https://lab.llvm.org/staging/#/builders/50/builds/4910> and Solaris/amd64
<https://lab.llvm.org/staging/#/builders/101/builds/3751>. Being a GNU
extension, it is not defined in `<sys/elf.h>`. However, providing a
fallback definition doesn't help because the code also relies on
`__ehdr_start`, another unportable GNU extension that most likely never
will be implemented in Solaris `ld`. Besides, there's reallly no point in
supporting build ids since they aren't used on Solaris at all.
This patch fixes this by making the relevant code conditional on the
definition of `NT_GNU_BUILD_ID`.
Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11`.
Differential Revision: https://reviews.llvm.org/D107556
Fixes PR51331. On FreeBSD, the elf headers don't (yet) provide the
ElfW(type) macro. However, there is a similar set of macros in the
<sys/elf-generic.h> header, of which `__ElfN(type)` exactly matches the
indended purpose.
Reviewed By: gulfem
Differential Revision: https://reviews.llvm.org/D107388
... and rename it to 'warnIfNonZero' to better-reflect what it actually
does.
The goal is to minimize the amount of logic that's conditionally
compiled under '#if __APPLE__'.
This fixes support for merging profiles which broke as a consequence
of e50a38840d. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.
In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.
Differential Revision: https://reviews.llvm.org/D107143
This fixes support for merging profiles which broke as a consequence
of e50a38840d. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.
In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.
Differential Revision: https://reviews.llvm.org/D107143
This fixes support for merging profiles which broke as a consequence
of e50a38840d. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.
In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.
Differential Revision: https://reviews.llvm.org/D107143
Change `CountersPtr` in `__profd_` to a label difference, which is a link-time
constant. On ELF, when linking a shared object, this requires that `__profc_` is
either private or linkonce/linkonce_odr hidden. On COFF, we need D104564 so that
`.quad a-b` (64-bit label difference) can lower to a 32-bit PC-relative relocation.
```
# ELF: R_X86_64_PC64 (PC-relative)
.quad .L__profc_foo-.L__profd_foo
# Mach-O: a pair of 8-byte X86_64_RELOC_UNSIGNED and X86_64_RELOC_SUBTRACTOR
.quad l___profc_foo-l___profd_foo
# COFF: we actually use IMAGE_REL_AMD64_REL32/IMAGE_REL_ARM64_REL32 so
# the high 32-bit value is zero even if .L__profc_foo < .L__profd_foo
# As compensation, we truncate CountersDelta in the header so that
# __llvm_profile_merge_from_buffer and llvm-profdata reader keep working.
.quad .L__profc_foo-.L__profd_foo
```
(Note: link.exe sorts `.lprfc` before `.lprfd` even if the object writer
has `.lprfd` before `.lprfc`, so we cannot work around by reordering
`.lprfc` and `.lprfd`.)
With this change, a stage 2 (`-DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_BUILD_INSTRUMENTED=IR`)
`ld -pie` linked clang is 1.74% smaller due to fewer R_X86_64_RELATIVE relocations.
```
% readelf -r pie | awk '$3~/R.*/{s[$3]++} END {for (k in s) print k, s[k]}'
R_X86_64_JUMP_SLO 331
R_X86_64_TPOFF64 2
R_X86_64_RELATIVE 476059 # was: 607712
R_X86_64_64 2616
R_X86_64_GLOB_DAT 31
```
The absolute function address (used by llvm-profdata to collect indirect call
targets) can be converted to relative as well, but is not done in this patch.
Differential Revision: https://reviews.llvm.org/D104556
This patch changes the linkage type of a compiler-rt func
(__llvm_write_binary_ids) to fix the sanitizer-windows bot
build issue introduced in change f984ac271.
The issue is as the following:
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingInternal.h(201):
error C2496: '__llvm_write_binary_ids': 'selectany' can only be applied
to data items with external linkage
Ellis Hoag