Coverage mapping data may reference names of functions
that are skipped by FE (e.g, unused inline functions). Since
those functions are skipped, normal instr-prof function lowering
pass won't put those names in the right section, so special
handling is needed to walk through coverage mapping structure
and recollect the references.
With this patch, only names that are skipped are processed. This
simplifies the lowering code and it no longer needs to make
assumptions coverage mapping data layout. It should also be
more efficient.
llvm-svn: 257092
NFC. These hints are only used for inlining and the inliner now uses
the same criteria to identify hot and cold callees and set appropriate
thresholds without relying on these hints. Hence this removed code is
superfluous.
Differential Revision: http://reviews.llvm.org/D15726
llvm-svn: 256793
Constructors and destructors may be represented by several functions
in IR. Only base structors correspond to source code, others are
small pieces of code and eventually call the base variant. In this
case instrumentation of non-base structors has little sense, this
fix remove it. Now profile data of a declaration corresponds to
exactly one function in IR, it agrees with the current logic of the
profile data loading.
This change fixes PR24996.
Differential Revision: http://reviews.llvm.org/D15158
llvm-svn: 254876
The patch is generated using this command:
$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang
To reduce churn, not touching namespaces spanning less than 10 lines.
llvm-svn: 240270
The fact that PGO has a say in how these branch weights are determined
isn't interesting to most of CodeGen, so it makes more sense for this
API to be accessible via CodeGenFunction rather than CodeGenPGO.
llvm-svn: 236380
No functional change. This just makes it more obvious that the logic
in ComputeRegionCounts only depends on the counter map and local
state.
llvm-svn: 236370
This removes the RegionCounter class, which is only used as a helper
in teh ComputeRegionCounts stmt visitor. This class is just an extra
layer of abstraction that makes the code harder to follow at this
point, and removing it makes the logic quite a bit more direct.
llvm-svn: 236364
We were assigning the counter for the body of the loop to the loop
variable initialization for some reason here, but our tests completely
lacked coverage for range-for loops. This fixes that and makes the
logic generally more similar to the logic for a regular for.
llvm-svn: 236277
The RegionCounter type does a lot of legwork, but most of it is only
meaningful within the implementation of CodeGenPGO. The uses elsewhere
in CodeGen generally just want to increment or read counters, so do
that directly.
llvm-svn: 235664
We try to use the member variable "FuncName" here, but we've also used
that name as a parameter. This ends with us getting the length of the
function name wrong when we generate the coverage data.
llvm-svn: 235565
When we instrument a program for profiling, we copy the linkage of an
instrumented function so that our datastructures merge in the same way
as the function. This avoids redundant copies for things like
linkonce, but ends up emitting names we never need to reference for
normal and internal symbols. Promoting internal and external linkage
to private for these variables reduces the size overhead of profiling
drastically.
llvm-svn: 232799
Somehow this file ended up with a strange hybrid of the old "indent
inside a namespace" style and the new "don't", giving us a wonderful
two-space indent starting halfway through a namespace. Fix it.
llvm-svn: 230244
We don't emit any coverage mapping for uncovered functions that come
from system headers, but we were creating a GlobalVariable with each
of their names. This is wasteful since the linker will need to dead
strip the unused symbols, and it can lead to issues when merging
coverage with others TUs that do have coverage for those functions.
llvm-svn: 226764
We were previously emitting counter increments even if we didn't have
an insertion point, which would result in a CallInst with no
parent. This leads to a crash, as in pr22166, if we try to do
GlobalDCE.
llvm-svn: 225495
The logic for lowering profiling counters has been moved to an LLVM
pass. Emit the intrinsics rather than duplicating the whole pass in
clang.
llvm-svn: 223683
There's no need to use different names for the local variables than we
use in the profile itself, and it's a bit simpler and easier to debug
if we're consistent.
llvm-svn: 223173
It doesn't make much sense to have std::unique_ptrs of std::string and
std::vector. Avoid some useless indirection by using these types
directly.
llvm-svn: 223166
The profile data format was recently updated and the new indexing api
requires the code coverage tool to know the function's hash as well
as the function's name to get the execution counts for a function.
Differential Revision: http://reviews.llvm.org/D4995
llvm-svn: 216208
This patch adds the '-fcoverage-mapping' option which
allows clang to generate the coverage mapping information
that can be used to provide code coverage analysis using
the execution counts obtained from the instrumentation
based profiling (-fprofile-instr-generate).
llvm-svn: 214752
We've added support for a multiple functions with the same name in
LLVM's profile data, so the lookup returning the function hash it
found doesn't make sense anymore. Update to pass in the hash we
expect.
This also adds a test that the version 1 format is still readable,
since the new API is expected to handle that.
llvm-svn: 214586
Improve the warning when building with -fprofile-instr-use and a file
appears not to have been profiled at all. This keys on whether a
function is defined in the main file or not to avoid false negatives
when one includes a header with functions that have been profiled.
llvm-svn: 211760
Shared objects are fairly broken for InstrProf right now -- a follow-up
commit in compiler-rt will fix the rest of this.
The main problem here is that at link time, profile data symbols in the
shared object might get used instead of symbols from the main
executable, creating invalid profile data sections.
<rdar://problem/16918688>
llvm-svn: 208939
We don't assign counters for implicit Decls, but we were emitting code
to increment the (non-existent) counters and adding empty counter
lists in the output. This fixes the checks in assignRegionCounters and
emitInstrumentationData to do the right thing, and adds an assert for
the pathological case of emitting zero counters.
llvm-svn: 207203
Update clang to use the InstrProfReader from LLVM to read
instrumentation based profile data. This also switches us from the
naive text format to the binary format, since that's what's
implemented in the reader.
llvm-svn: 206658
The function hash should change when control flow changes. This patch
hashes the type of each AST node that affects counters, rather than just
counting how many there are. These types are combined into a small
enumerator that currently has 16 values.
The new hash algorithm packs the enums for consecutively visited types
into a `uint64_t`. In order to save space for new types, the types are
assumed to be 6-bit values (instead of 4-bit). In order to minimize
overhead for functions with little control flow, the `uint64_t` is used
directly as a hash if it never fills up; if it does, it's passed through
an MD5 context.
<rdar://problem/16435801>
llvm-svn: 206397
This adds a warning that triggers when profile data doesn't match for
the source that's being compiled with -fprofile-instr-use=. This fires
only once per translation unit, as warning on every mismatched
function would be quite noisy.
llvm-svn: 206322
CapturedStmt was being ignored by instrumentation based profiling, and
its counters attributed to the containing function. Instead, we need
to treat this as a top level entity, like we do with blocks.
llvm-svn: 206231
Until now we were generating duplicate counters for lambdas: one set
in the function where the lambda was declared and another for the
lambda itself. Instead, we should skip over the bodies of lambdas in
their containing contexts.
llvm-svn: 206081
Fixes a bug where unsigned numbers are read using strtol and strtoll.
I don't have a testcase because this bug is effectively unobservable
right now. To expose the problem in the hash, we would need a function
with greater than INT64_MAX counters, which we don't handle anyway. To
expose the problem in the function count, we'd need a function with
greater than INT32_MAX counters; this is theoretically observable, but
it isn't a practical testcase to check in.
An upcoming commit changes the hash to be non-trivial, so we'll get some
coverage eventually.
<rdar://problem/16435801>
llvm-svn: 206001
Emitting the PGO initialization in EmitGlobalFunctionDefinition is
inefficient, since this only has an effect once per module. We move
this to Release() with the rest of the once-per-module logic.
llvm-svn: 205977
If all of our weights are zero when calculating branch weights, it
means we haven't profiled the code in question. Avoid creating a
metadata node that says all branches are equally likely in this case.
The test also checks constructs that hit the other createBranchWeights
overload. These were already working.
llvm-svn: 205606
-u behaviour is apparently not portable between linkers (see cfe-commits
discussions for r204379 and r205012). I've moved the logic to IRGen,
where it should have been in the first place.
I don't have a Linux system to test this on, so it's possible this logic
*still* doesn't pull in the instrumented profiling runtime on Linux.
I'm in the process of getting tests going on the compiler-rt side
(llvm-commits "[PATCH] InstrProf: Add initial compiler-rt test"). Once
we have tests for the full flow there, the runtime logic should get a
whole lot less brittle.
<rdar://problem/16458307>
llvm-svn: 205023
Variables with available_externally linkage can be dropped at will.
This causes link errors, since there are still references to the
instrumentation! linkonce_odr is almost equivalent, so use that
instead.
As a drive-by fix (I don't have an Elf system, so I'm not sure how to
write a testcase), use linkonce linkage for the instrumentation of
extern_weak functions.
<rdar://problem/15943240>
llvm-svn: 204408
The variable is used to set the linkage for variables, and will become
different from function linkage in a follow-up commit.
<rdar://problem/15943240>
llvm-svn: 204407
These functions are in the profile runtime. PGO comes later.
Unfortunately, there's only room for 16 characters in a Darwin section,
so use __llvm_prf_ instead of __llvm_profile_ for section names.
<rdar://problem/15943240>
llvm-svn: 204390
Remove the remaining explicit static initialization from translation
units, at least on Darwin. Instead, create a use of __llvm_pgo_runtime,
which will pull in required code from compiler-rt.
After this commit (and its pair in compiler-rt), a user can define their
own __llvm_pgo_runtime to satisfy this undefined symbol and call the
functions in compiler-rt directly.
<rdar://problem/15943240>
llvm-svn: 204379
The hash itself is still the number of counters, which isn't all that
useful, but this separates the API changes from the actual
implementation of the hash and will make it easier to transition to
the ProfileData library once it's implemented.
llvm-svn: 204186
In instrumentation-based profiling, we need a set of data structures to
represent the counters. Previously, these were built up during static
initialization. Now, they're shoved into a specially-named section so
that they show up as an array.
As a consequence of the reorganizing symbols, instrumentation data
structures for linkonce functions are now correctly coalesced.
This is the first step in a larger project to minimize runtime overhead
and dependencies in instrumentation-based profilng. The larger picture
includes removing all initialization overhead and making the dependency
on libc optional.
<rdar://problem/15943240>
llvm-svn: 204080
This updates CodeGenPGO to use the ProfileDataReader introduced to
llvm in r203703 and the new API for writing out the profile introduced
to compiler-rt in r203710.
llvm-svn: 203711
PGO counters are 64-bit and branch weights are 32-bit. Scale them down
when necessary, instead of just taking the lower 32 bits.
<rdar://problem/16276448>
llvm-svn: 203592
In addition, for all functions, use the name from the llvm::Function to
identify the function in the profile data. Compute that "function name",
including the file name for local functions, once when assigning the PGO
counters and store it in the CodeGenPGO class.
Move the code to add InlineHint and Cold attributes out of StartFunction(),
because the "function name" string isn't available at that point.
llvm-svn: 203075
For C++ functions, we will continue to use the mangled name to identify
functions in the PGO profile data, but this name is confusing for things like
Objective-C methods. For functions with local linkage, we're also going to
include the file name to help distinguish those functions, so this changes to
use more generic variable names.
No functional changes.
llvm-svn: 203074
Move the PGO.assignRegionCounters() call out of StartFunction, because that
function is called from many places where it does not make sense to do PGO
instrumentation (e.g., compiler-generated helper functions). Change several
functions to take a StringRef argument for the unique name associated with
a function, so that the name can be set differently for things like Objective-C
methods and block literals.
llvm-svn: 203073
I hit this while debugging another issue where my sources were in an
inconsistent state, so I don't have a testcase. Regardless, this check is
simpler and more direct than checking if the option is enabled.
llvm-svn: 203072
Previously, we made one traversal of the AST prior to codegen to assign
counters to the ASTs and then propagated the count values during codegen. This
patch now adds a separate AST traversal prior to codegen for the
-fprofile-instr-use option to propagate the count values. The counts are then
saved in a map from which they can be retrieved during codegen.
This new approach has several advantages:
1. It gets rid of a lot of extra PGO-related code that had previously been
added to codegen.
2. It fixes a serious bug. My original implementation (which was mailed to the
list but never committed) used 3 counters for every loop. Justin improved it to
move 2 of those counters into the less-frequently executed breaks and continues,
but that turned out to produce wrong count values in some cases. The solution
requires visiting a loop body before the condition so that the count for the
condition properly includes the break and continue counts. Changing codegen to
visit a loop body first would be a fairly invasive change, but with a separate
AST traversal, it is easy to control the order of traversal. I've added a
testcase (provided by Justin) to make sure this works correctly.
3. It improves the instrumentation overhead, reducing the number of counters for
a loop from 3 to 1. We no longer need dedicated counters for breaks and
continues, since we can just use the propagated count values when visiting
breaks and continues.
To make this work, I needed to make a change to the way we count case
statements, going back to my original approach of not including the fall-through
in the counter values. This was necessary because there isn't always an AST node
that can be used to record the fall-through count. Now case statements are
handled the same as default statements, with the fall-through paths branching
over the counter increments. While I was at it, I also went back to using this
approach for do-loops -- omitting the fall-through count into the loop body
simplifies some of the calculations and make them behave the same as other
loops. Whenever we start using this instrumentation for coverage, we'll need
to add the fall-through counts into the counter values.
llvm-svn: 201528
When a function has a single counter, we will offset the pointer by 1 when
parsing the next function. If a function has multiple counters, we are
okay after skipping rest of the counters.
llvm-svn: 201456
We collect a maximal function count among all functions in the pgo data file.
For functions that are hot, we set its InlineHint attribute. For functions that
are cold, we set its Cold attribute.
We currently treat functions with >= 30% of the maximal function count as hot
and functions with <= 1% of the maximal function count are treated as cold.
These two numbers are from preliminary tuning on SPEC.
This commit should not affect non-PGO builds and should boost performance on
instrumentation based PGO.
llvm-svn: 200874
Replace the last incorrect uses and templatize the function to require a
compile-time constant string preventing further misuse.
The diagnostic formatter expects well-formed input and has undefined behaviour
with arbitrary input or crafted user strings in source files. Accepting user
input would also have caused unbounded generation of new diagnostic IDs which
can be problematic in long-running sessions or language bindings.
This completes the work to fix several incorrect callers that passed user
input or raw messages to the diagnostics engine where a constant format string
was expected.
llvm-svn: 200132
Xinliang David Li