Commit Graph

384 Commits

Author SHA1 Message Date
Leonard Chan c579c658cd [compiler-rt][profile] Make corrupted-profile.c more robust
This test specifically checks that profiles are not mergeable if there's a
change in the CounterPtr in the profile header. The test manually changes
CounterPtr by explicitly calling memset on some offset into the profile file.
This test would fail if binary IDs were emitted because the offset calculation
does not take into account the binary ID sizes.

This patch updates the test to use types provided in profile/InstrProfData.inc
to make it more resistant to profile layout changes.

Differential Revision: https://reviews.llvm.org/D110277
2021-09-23 17:16:47 -07:00
David Spickett 915a8bb5ee Revert "[compiler-rt][Profile] Disable test on Arm/AArch64 Linux"
This reverts commit 8b86f8a325.

The inconsistent behaviour has been fixed with
5e50d3073a.
2021-09-06 08:45:06 +00:00
David Spickett 5e50d3073a [compiler-rt][Profile] Wait for child threads in set-file-object test
We've been seeing this test return 31 instead of 32 for the "functions"
line in this test on our AArch64 bots.

One possible cause is some of the children not finishing in time
before the llvm-profdata commands are run, if the machine is heavily loaded.

Wait for all the children to finish before exiting the parent.

Reviewed By: zequanwu

Differential Revision: https://reviews.llvm.org/D109222
2021-09-03 11:48:50 -07:00
David Spickett 8b86f8a325 [compiler-rt][Profile] Disable test on Arm/AArch64 Linux
While a fix for flaky results is being reviewed.
2021-09-03 12:58:09 +00:00
Fangrui Song 68745a557e [InstrProfiling] Use llvm.compiler.used if applicable for Mach-O
Similar to D97585.

D25456 used `S_ATTR_LIVE_SUPPORT` to ensure the data variable will be retained
or discarded as a unit with the counter variable, so llvm.compiler.used is
sufficient. It allows ld to dead strip unneeded profc and profd variables.

Reviewed By: vsk

Differential Revision: https://reviews.llvm.org/D105445
2021-09-01 14:46:51 -07:00
Vedant Kumar 6c439a3817 [profile] Specify "-V" to otool to get expected test output
Newer Xcode toolchains ship a new otool implementation that prints out
section contents in a slightly different way than otool-classic. Specify
"-V" to otool to get the expected test output.

Differential Revision: https://reviews.llvm.org/D108929
2021-08-31 10:49:51 -07:00
Zequan Wu 1b05245119 [Profile] Support __llvm_profile_set_file_object in continuous mode.
Replace D107203, because __llvm_profile_set_file_object is usually used when the
process doesn't have permission to open/create file. That patch trying to copy
from old profile to new profile contradicts with the usage.

Differential Revision: https://reviews.llvm.org/D108242
2021-08-27 13:06:46 -07:00
Rong Xu 9b8425e42c Reapply commit b7425e956
The commit b7425e956: [NFC] fix typos
is harmless but was reverted by accident. Reapply.
2021-08-16 12:18:40 -07:00
Kostya Kortchinsky 80ed75e7fb Revert "[NFC] Fix typos"
This reverts commit b7425e956b.
2021-08-16 11:13:05 -07:00
Rong Xu b7425e956b [NFC] Fix typos
s/senstive/senstive/g
2021-08-16 10:15:30 -07:00
Fangrui Song d54c62cc32 [profile][test] Add -no-pie to make value profile merge work on Linux with default PIE
Alpine enables PIE by default.
2021-08-11 22:41:52 -07:00
Fangrui Song 18ec93d9e6 [profile][test] Delete --path-equivalence=/tmp,%S
This causes the test to fail if %S is under /tmp
2021-07-31 00:36:17 -07:00
Petr Hosek 83302c8489 [profile] Fix profile merging with binary IDs
This fixes support for merging profiles which broke as a consequence
of e50a38840d. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.

In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.

Differential Revision: https://reviews.llvm.org/D107143
2021-07-30 18:54:27 -07:00
Petr Hosek d3dd07e3d0 Revert "[profile] Fix profile merging with binary IDs"
This reverts commit dcadd64986.
2021-07-30 18:53:48 -07:00
Petr Hosek dcadd64986 [profile] Fix profile merging with binary IDs
This fixes support for merging profiles which broke as a consequence
of e50a38840d. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.

In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.

Differential Revision: https://reviews.llvm.org/D107143
2021-07-30 17:38:53 -07:00
Petr Hosek 6ea2f31f3d Revert "[profile] Fix profile merging with binary IDs"
This reverts commit 89d6eb6f8c, this
seemed to have break a few builders.
2021-07-30 14:32:52 -07:00
Petr Hosek 89d6eb6f8c [profile] Fix profile merging with binary IDs
This fixes support for merging profiles which broke as a consequence
of e50a38840d. The issue was missing
adjustment in merge logic to account for the binary IDs which are
now included in the raw profile just after header.

In addition, this change also:
* Includes the version in module signature that's used for merging
to avoid accidental attempts to merge incompatible profiles.
* Moves the binary IDs size field after version field in the header
as was suggested in the review.

Differential Revision: https://reviews.llvm.org/D107143
2021-07-30 13:30:30 -07:00
Gulfem Savrun Yeniceri e50a38840d [profile] Add binary id into profiles
This patch adds binary id into profiles to easily associate binaries
with the corresponding profiles. There is an RFC that discusses
the motivation, design and implementation in more detail:
https://lists.llvm.org/pipermail/llvm-dev/2021-June/151154.html

Differential Revision: https://reviews.llvm.org/D102039
2021-07-23 00:19:12 +00:00
Gulfem Savrun Yeniceri fd895bc81b Revert "[profile] Add binary id into profiles"
Revert "[profile] Change linkage type of a compiler-rt func"
This reverts commits f984ac2715 and
467c719124 because it broke some builds.
2021-07-21 19:15:18 +00:00
Gulfem Savrun Yeniceri f984ac2715 [profile] Add binary id into profiles
This patch adds binary id into profiles to easily associate binaries
with the corresponding profiles. There is an RFC that discusses
the motivation, design and implementation in more detail:
https://lists.llvm.org/pipermail/llvm-dev/2021-June/151154.html

Differential Revision: https://reviews.llvm.org/D102039
2021-07-21 17:55:43 +00:00
Fangrui Song 7b6b15e010 [profile][test] Improve coverage-linkage.cpp with ld.lld --gc-sections
The __llvm_prf_names section uses SHF_GNU_RETAIN.  However, GNU ld before 2015-10
(https://sourceware.org/bugzilla/show_bug.cgi?id=19161) neither supports it nor
retains __llvm_prf_names according to __start___llvm_prf_names. So --gc-sections
does not work on such old GNU ld.

This is not a problem for gold and sufficiently new lld.
2021-07-06 11:08:47 -07:00
Nico Weber f814cd7406 Revert "[profile][test] Improve coverage-linkage.cpp"
This reverts commit 36ba86fe8a.
Fails on some bots, see comments on
https://reviews.llvm.org/rG36ba86fe8a29cdf3251b786db7f342efde666cb2
2021-07-06 08:49:43 -04:00
Fangrui Song 36ba86fe8a [profile][test] Improve coverage-linkage.cpp 2021-07-05 12:46:06 -07:00
Fangrui Song 8ea2a58a2e [llvm-profdata] Make diagnostics consistent with the (no capitalization, no period) style
The format is currently inconsistent. Use the https://llvm.org/docs/CodingStandards.html#error-and-warning-messages style.

And add `error:` or `warning:` to CHECK lines wherever appropriate.
2021-06-19 14:54:25 -07:00
Fangrui Song 5540470f64 [profile][test] Delete profraw directory so that tests are immune to format version upgrade 2021-06-18 16:44:03 -07:00
Arthur Eubanks 189428c8fc [Profile] Handle invalid profile data
This mostly follows LLVM's InstrProfReader.cpp error handling.
Previously, attempting to merge corrupted profile data would result in
crashes. See https://crbug.com/1216811#c4.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D104050
2021-06-10 16:10:13 -07:00
Fangrui Song b19c0ac7dd [profile] Add -fprofile-instr-generate tests for weak definition and various linkages 2021-06-04 10:26:55 -07:00
Nico Weber 50c0aaed47 Temporarily remove another test added in one of the tests added in effb87d
This test reads the test file removed in db3e4faa4d.
2021-06-04 10:42:37 -04:00
Nico Weber db3e4faa4d Temporarily remove one of the tests added in effb87dfa8
It fails on some Linux systems. Remove the test until we've figured
out what's going on. See https://crbug.com/1216005 for details.
2021-06-04 10:03:21 -04:00
Fangrui Song 87c43f3aa9 [InstrProfiling] Delete linkage/visibility toggling for Windows
The linkage/visibility of `__profn_*` variables are derived
from the profiled functions.

    extern_weak => linkonce
    available_externally => linkonce_odr
    internal => private
    extern => private
    _ => unchanged

The linkage/visibility of `__profc_*`/`__profd_*` variables are derived from
`__profn_*` with linkage/visibility wrestling for Windows.

The changes can be folded to the following without changing semantics.

```
if (TT.isOSBinFormatCOFF() && !NeedComdat) {
  Linkage = GlobalValue::InternalLinkage;
  Visibility = GlobalValue::DefaultVisibility;
}
```

That said, I think we can just delete the code block.

An extern/internal function will now use private `__profc_*`/`__profd_*`
variables, instead of internal ones. This saves some symbol table entries.

A non-comdat {linkonce,weak}_odr function will now use hidden external
`__profc_*`/`__profd_*` variables instead of internal ones.  There is potential
object file size increase because such symbols need `/INCLUDE:` directives.
However such non-comdat functions are rare (note that non-comdat weak
definitions don't prevent duplicate definition error).

The behavior changes match ELF.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D103355
2021-06-02 16:49:54 -07:00
Fangrui Song effb87dfa8 [profile] Add -fprofile-instr-generate tests for weak definition and various linkages 2021-06-02 16:12:08 -07:00
Reid Kleckner 8f20ac9595 [PGO] Don't reference functions unless value profiling is enabled
This reduces the size of chrome.dll.pdb built with optimizations,
coverage, and line table info from 4,690,210,816 to 2,181,128,192, which
makes it possible to fit under the 4GB limit.

This change can greatly reduce binary size in coverage builds, which do
not need value profiling. IR PGO builds are unaffected. There is a minor
behavior change for frontend PGO.

PGO and coverage both use InstrProfiling to create profile data with
counters. PGO records the address of each function in the __profd_
global. It is used later to map runtime function pointer values back to
source-level function names. Coverage does not appear to use this
information.

Recording the address of every function with code coverage drastically
increases code size. Consider this program:

  void foo();
  void bar();
  inline void inlineMe(int x) {
    if (x > 0)
      foo();
    else
      bar();
  }
  int getVal();
  int main() { inlineMe(getVal()); }

With code coverage, the InstrProfiling pass runs before inlining, and it
captures the address of inlineMe in the __profd_ global. This greatly
increases code size, because now the compiler can no longer delete
trivial code.

One downside to this approach is that users of frontend PGO must apply
the -mllvm -enable-value-profiling flag globally in TUs that enable PGO.
Otherwise, some inline virtual method addresses may not be recorded and
will not be able to be promoted. My assumption is that this mllvm flag
is not popular, and most frontend PGO users don't enable it.

Differential Revision: https://reviews.llvm.org/D102818
2021-05-20 11:09:24 -07:00
Vedant Kumar 7014a10161 [profile] Skip mmap() if there are no counters
If there are no counters, an mmap() of the counters section would fail
due to the size argument being too small (EINVAL).

rdar://78175925

Differential Revision: https://reviews.llvm.org/D102735
2021-05-19 09:31:40 -07:00
Fangrui Song 0c5bfe6bb8 [profile][test] Pin Linux/instrprof-value-prof-warn.test to -fuse-ld=bfd
To work around https://sourceware.org/bugzilla/show_bug.cgi?id=27490
2021-03-05 00:52:25 -08:00
Mitch Phillips 1be97975cc Change instrprof LLVM_VP_MAX_NUM_VALS_PER_SITE threshold.
We're having flaky failures on this test on the sanitizer slow
buildbot. Not per-run flaky, but it'll be green for a while, then red
for a while. I suspect that changes in codegen are causing the
LLVM_VP_MAX_NUM_VALS_PER_SITE=150 to be above and below the limit
sporadically. The limit on my machine using lld and a non-bootstrapped
compiler is 175, but the bot uses GNU ld and ld.gold at different
points, which could be affecting behaviour.

Change this threshold to LLVM_VP_MAX_NUM_VALS_PER_SITE=130 in order to
try and get it below the failure point, at least for the foreseeable
future.

http://lab.llvm.org:8011/#/builders/37/builds/2744
2021-03-04 12:13:47 -08:00
Zequan Wu 9783e20988 Revert "Revert "[Coverage] Emit gap region between statements if first statements contains terminate statements.""
Reland with update on test case ContinuousSyncmode/basic.c.

This reverts commit fe5c2c3ca6.
2021-03-04 11:52:43 -08:00
Nico Weber fe5c2c3ca6 Revert "[Coverage] Emit gap region between statements if first statements contains terminate statements."
This reverts commit 2d7374a0c6.
Breaks ContinuousSyncMode/basic.c in check-profile on macOS.
2021-03-04 08:53:30 -05:00
Fangrui Song a84f4fc0df [InstrProfiling] Place __llvm_prf_vnodes and __llvm_prf_names in llvm.used on ELF
`__llvm_prf_vnodes` and `__llvm_prf_names` are used by runtime but not
referenced via relocation in the translation unit.

With `-z start-stop-gc` (LLD 13 (D96914); GNU ld 2.37 https://sourceware.org/bugzilla/show_bug.cgi?id=27451),
the linker does not let `__start_/__stop_` references retain their sections.

Place `__llvm_prf_vnodes` and `__llvm_prf_names` in `llvm.used` to make
them retained by the linker.

This patch changes most existing `UsedVars` cases to `CompilerUsedVars`
to reflect the ideal state - if the binary format properly supports
section based GC (dead stripping), `llvm.compiler.used` should be sufficient.

`__llvm_prf_vnodes` and `__llvm_prf_names` are switched to `UsedVars`
since we want them to be unconditionally retained by both compiler and linker.

Behaviors on COFF/Mach-O are not affected.

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D97649
2021-03-03 11:32:24 -08:00
Fangrui Song 75df61e93d [test] Improve PGO tests 2021-03-03 11:32:24 -08:00
Zequan Wu 2d7374a0c6 [Coverage] Emit gap region between statements if first statements contains terminate statements.
Differential Revision: https://reviews.llvm.org/D97101
2021-03-03 11:25:49 -08:00
Nico Weber 64f5d7e972 Revert "[InstrProfiling] Place __llvm_prf_vnodes and __llvm_prf_names in llvm.used on ELF"
This reverts commit 04c3040f41.
Breaks instrprof-value-merge.c in bootstrap builds.
2021-03-03 10:21:17 -05:00
Fangrui Song 04c3040f41 [InstrProfiling] Place __llvm_prf_vnodes and __llvm_prf_names in llvm.used on ELF
`__llvm_prf_vnodes` and `__llvm_prf_names` are used by runtime but not
referenced via relocation in the translation unit.

With `-z start-stop-gc` (D96914 https://sourceware.org/bugzilla/show_bug.cgi?id=27451),
the linker no longer lets `__start_/__stop_` references retain them.

Place `__llvm_prf_vnodes` and `__llvm_prf_names` in `llvm.used` to make
them retained by the linker.

This patch changes most existing `UsedVars` cases to `CompilerUsedVars`
to reflect the ideal state - if the binary format properly supports
section based GC (dead stripping), `llvm.compiler.used` should be sufficient.

`__llvm_prf_vnodes` and `__llvm_prf_names` are switched to `UsedVars`
since we want them to be unconditionally retained by both compiler and linker.

Behaviors on other COFF/Mach-O are not affected.

Differential Revision: https://reviews.llvm.org/D97649
2021-03-01 13:43:23 -08:00
Vedant Kumar 2fcc3f4b18 [test/profile] Disable instrprof-write-buffer-internal.c on Windows
This is failing due to:

"instrprof-write-buffer-internal.c.tmp.buf.profraw: Invalid
instrumentation profile data (file header is corrupt)"

https://lab.llvm.org/buildbot/#/builders/127/builds/6830
2021-03-01 11:06:38 -08:00
Vedant Kumar 8f7dc99647 [test/profile] Pass -w to suppress suggestion to use fopen_s 2021-03-01 11:06:38 -08:00
Vedant Kumar 18adbb86f9 [test/profile] Add test coverage for __llvm_profile_write_buffer_internal
Reviewed By: davidxl, MaskRay

Differential Revision: https://reviews.llvm.org/D97697
2021-03-01 10:46:14 -08:00
Vedant Kumar a7d4826101 [profile] Fix buffer overrun when parsing %c in filename string
Fix a buffer overrun that can occur when parsing '%c' at the end of a
filename pattern string.

rdar://74571261

Reviewed By: kastiglione

Differential Revision: https://reviews.llvm.org/D97239
2021-02-24 14:49:45 -08:00
Petr Hosek b6876ddc82 [CMake][profile] Don't use `TARGET lld` to avoid ordering issues
Depending on the order in which lld and compiler-rt projects are
processed by CMake, `TARGET lld` might evaluate to `TRUE` or `FALSE`
even though `lld-available` lit stanza is always set because lld is
being built. We check whether lld project is enabled instead which
is used by other compiler-rt tests.

The ideal solution here would be to use CMake generator expressions,
but those cannot be used for dependencies yet, see:
https://gitlab.kitware.com/cmake/cmake/-/issues/19467

Differential Revision: https://reviews.llvm.org/D97256
2021-02-22 23:33:21 -08:00
Petr Hosek c24b7a16b1 [InstrProfiling] Use ELF section groups for counters, data and values
__start_/__stop_ references retain C identifier name sections such as
__llvm_prf_*. Putting these into a section group disables this logic.

The ELF section group semantics ensures that group members are retained
or discarded as a unit. When a function symbol is discarded, this allows
allows linker to discard counters, data and values associated with that
function symbol as well.

Note that `noduplicates` COMDAT is lowered to zero-flag section group in
ELF. We only set this for functions that aren't already in a COMDAT and
for those that don't have available_externally linkage since we already
use regular COMDAT groups for those.

Differential Revision: https://reviews.llvm.org/D96757
2021-02-22 14:00:02 -08:00
Petr Hosek 4827492d9f Revert "[InstrProfiling] Use ELF section groups for counters, data and values"
This reverts commits:
5ca21175e0
97184ab99c

The instrprof-gc-sections.c is failing on AArch64 LLD bot.
2021-02-22 11:13:55 -08:00
Petr Hosek 97184ab99c [InstrProfiling] Fix instrprof-gc-sections.c test
After D97110 __llvm_prof_cnts has the nobits type so it's empty.
2021-02-21 23:47:18 -08:00