We've added support for a multiple functions with the same name in
LLVM's profile data, so the lookup returning the function hash it
found doesn't make sense anymore. Update to pass in the hash we
expect.
This also adds a test that the version 1 format is still readable,
since the new API is expected to handle that.
llvm-svn: 214586
Improve the warning when building with -fprofile-instr-use and a file
appears not to have been profiled at all. This keys on whether a
function is defined in the main file or not to avoid false negatives
when one includes a header with functions that have been profiled.
llvm-svn: 211760
Shared objects are fairly broken for InstrProf right now -- a follow-up
commit in compiler-rt will fix the rest of this.
The main problem here is that at link time, profile data symbols in the
shared object might get used instead of symbols from the main
executable, creating invalid profile data sections.
<rdar://problem/16918688>
llvm-svn: 208939
We don't assign counters for implicit Decls, but we were emitting code
to increment the (non-existent) counters and adding empty counter
lists in the output. This fixes the checks in assignRegionCounters and
emitInstrumentationData to do the right thing, and adds an assert for
the pathological case of emitting zero counters.
llvm-svn: 207203
Update clang to use the InstrProfReader from LLVM to read
instrumentation based profile data. This also switches us from the
naive text format to the binary format, since that's what's
implemented in the reader.
llvm-svn: 206658
The function hash should change when control flow changes. This patch
hashes the type of each AST node that affects counters, rather than just
counting how many there are. These types are combined into a small
enumerator that currently has 16 values.
The new hash algorithm packs the enums for consecutively visited types
into a `uint64_t`. In order to save space for new types, the types are
assumed to be 6-bit values (instead of 4-bit). In order to minimize
overhead for functions with little control flow, the `uint64_t` is used
directly as a hash if it never fills up; if it does, it's passed through
an MD5 context.
<rdar://problem/16435801>
llvm-svn: 206397
This adds a warning that triggers when profile data doesn't match for
the source that's being compiled with -fprofile-instr-use=. This fires
only once per translation unit, as warning on every mismatched
function would be quite noisy.
llvm-svn: 206322
CapturedStmt was being ignored by instrumentation based profiling, and
its counters attributed to the containing function. Instead, we need
to treat this as a top level entity, like we do with blocks.
llvm-svn: 206231
Until now we were generating duplicate counters for lambdas: one set
in the function where the lambda was declared and another for the
lambda itself. Instead, we should skip over the bodies of lambdas in
their containing contexts.
llvm-svn: 206081
Fixes a bug where unsigned numbers are read using strtol and strtoll.
I don't have a testcase because this bug is effectively unobservable
right now. To expose the problem in the hash, we would need a function
with greater than INT64_MAX counters, which we don't handle anyway. To
expose the problem in the function count, we'd need a function with
greater than INT32_MAX counters; this is theoretically observable, but
it isn't a practical testcase to check in.
An upcoming commit changes the hash to be non-trivial, so we'll get some
coverage eventually.
<rdar://problem/16435801>
llvm-svn: 206001
Emitting the PGO initialization in EmitGlobalFunctionDefinition is
inefficient, since this only has an effect once per module. We move
this to Release() with the rest of the once-per-module logic.
llvm-svn: 205977
If all of our weights are zero when calculating branch weights, it
means we haven't profiled the code in question. Avoid creating a
metadata node that says all branches are equally likely in this case.
The test also checks constructs that hit the other createBranchWeights
overload. These were already working.
llvm-svn: 205606
-u behaviour is apparently not portable between linkers (see cfe-commits
discussions for r204379 and r205012). I've moved the logic to IRGen,
where it should have been in the first place.
I don't have a Linux system to test this on, so it's possible this logic
*still* doesn't pull in the instrumented profiling runtime on Linux.
I'm in the process of getting tests going on the compiler-rt side
(llvm-commits "[PATCH] InstrProf: Add initial compiler-rt test"). Once
we have tests for the full flow there, the runtime logic should get a
whole lot less brittle.
<rdar://problem/16458307>
llvm-svn: 205023
Variables with available_externally linkage can be dropped at will.
This causes link errors, since there are still references to the
instrumentation! linkonce_odr is almost equivalent, so use that
instead.
As a drive-by fix (I don't have an Elf system, so I'm not sure how to
write a testcase), use linkonce linkage for the instrumentation of
extern_weak functions.
<rdar://problem/15943240>
llvm-svn: 204408
The variable is used to set the linkage for variables, and will become
different from function linkage in a follow-up commit.
<rdar://problem/15943240>
llvm-svn: 204407
These functions are in the profile runtime. PGO comes later.
Unfortunately, there's only room for 16 characters in a Darwin section,
so use __llvm_prf_ instead of __llvm_profile_ for section names.
<rdar://problem/15943240>
llvm-svn: 204390
Remove the remaining explicit static initialization from translation
units, at least on Darwin. Instead, create a use of __llvm_pgo_runtime,
which will pull in required code from compiler-rt.
After this commit (and its pair in compiler-rt), a user can define their
own __llvm_pgo_runtime to satisfy this undefined symbol and call the
functions in compiler-rt directly.
<rdar://problem/15943240>
llvm-svn: 204379
The hash itself is still the number of counters, which isn't all that
useful, but this separates the API changes from the actual
implementation of the hash and will make it easier to transition to
the ProfileData library once it's implemented.
llvm-svn: 204186
In instrumentation-based profiling, we need a set of data structures to
represent the counters. Previously, these were built up during static
initialization. Now, they're shoved into a specially-named section so
that they show up as an array.
As a consequence of the reorganizing symbols, instrumentation data
structures for linkonce functions are now correctly coalesced.
This is the first step in a larger project to minimize runtime overhead
and dependencies in instrumentation-based profilng. The larger picture
includes removing all initialization overhead and making the dependency
on libc optional.
<rdar://problem/15943240>
llvm-svn: 204080
This updates CodeGenPGO to use the ProfileDataReader introduced to
llvm in r203703 and the new API for writing out the profile introduced
to compiler-rt in r203710.
llvm-svn: 203711
PGO counters are 64-bit and branch weights are 32-bit. Scale them down
when necessary, instead of just taking the lower 32 bits.
<rdar://problem/16276448>
llvm-svn: 203592
In addition, for all functions, use the name from the llvm::Function to
identify the function in the profile data. Compute that "function name",
including the file name for local functions, once when assigning the PGO
counters and store it in the CodeGenPGO class.
Move the code to add InlineHint and Cold attributes out of StartFunction(),
because the "function name" string isn't available at that point.
llvm-svn: 203075
For C++ functions, we will continue to use the mangled name to identify
functions in the PGO profile data, but this name is confusing for things like
Objective-C methods. For functions with local linkage, we're also going to
include the file name to help distinguish those functions, so this changes to
use more generic variable names.
No functional changes.
llvm-svn: 203074
Move the PGO.assignRegionCounters() call out of StartFunction, because that
function is called from many places where it does not make sense to do PGO
instrumentation (e.g., compiler-generated helper functions). Change several
functions to take a StringRef argument for the unique name associated with
a function, so that the name can be set differently for things like Objective-C
methods and block literals.
llvm-svn: 203073
I hit this while debugging another issue where my sources were in an
inconsistent state, so I don't have a testcase. Regardless, this check is
simpler and more direct than checking if the option is enabled.
llvm-svn: 203072
Previously, we made one traversal of the AST prior to codegen to assign
counters to the ASTs and then propagated the count values during codegen. This
patch now adds a separate AST traversal prior to codegen for the
-fprofile-instr-use option to propagate the count values. The counts are then
saved in a map from which they can be retrieved during codegen.
This new approach has several advantages:
1. It gets rid of a lot of extra PGO-related code that had previously been
added to codegen.
2. It fixes a serious bug. My original implementation (which was mailed to the
list but never committed) used 3 counters for every loop. Justin improved it to
move 2 of those counters into the less-frequently executed breaks and continues,
but that turned out to produce wrong count values in some cases. The solution
requires visiting a loop body before the condition so that the count for the
condition properly includes the break and continue counts. Changing codegen to
visit a loop body first would be a fairly invasive change, but with a separate
AST traversal, it is easy to control the order of traversal. I've added a
testcase (provided by Justin) to make sure this works correctly.
3. It improves the instrumentation overhead, reducing the number of counters for
a loop from 3 to 1. We no longer need dedicated counters for breaks and
continues, since we can just use the propagated count values when visiting
breaks and continues.
To make this work, I needed to make a change to the way we count case
statements, going back to my original approach of not including the fall-through
in the counter values. This was necessary because there isn't always an AST node
that can be used to record the fall-through count. Now case statements are
handled the same as default statements, with the fall-through paths branching
over the counter increments. While I was at it, I also went back to using this
approach for do-loops -- omitting the fall-through count into the loop body
simplifies some of the calculations and make them behave the same as other
loops. Whenever we start using this instrumentation for coverage, we'll need
to add the fall-through counts into the counter values.
llvm-svn: 201528
When a function has a single counter, we will offset the pointer by 1 when
parsing the next function. If a function has multiple counters, we are
okay after skipping rest of the counters.
llvm-svn: 201456
We collect a maximal function count among all functions in the pgo data file.
For functions that are hot, we set its InlineHint attribute. For functions that
are cold, we set its Cold attribute.
We currently treat functions with >= 30% of the maximal function count as hot
and functions with <= 1% of the maximal function count are treated as cold.
These two numbers are from preliminary tuning on SPEC.
This commit should not affect non-PGO builds and should boost performance on
instrumentation based PGO.
llvm-svn: 200874
Replace the last incorrect uses and templatize the function to require a
compile-time constant string preventing further misuse.
The diagnostic formatter expects well-formed input and has undefined behaviour
with arbitrary input or crafted user strings in source files. Accepting user
input would also have caused unbounded generation of new diagnostic IDs which
can be problematic in long-running sessions or language bindings.
This completes the work to fix several incorrect callers that passed user
input or raw messages to the diagnostics engine where a constant format string
was expected.
llvm-svn: 200132
Justin Bogner