Part 4 of CSPGO changes:
(1) add support in cmake for cspgo build.
(2) fix an issue in big endian.
(3) test cases.
Differential Revision: https://reviews.llvm.org/D54175
llvm-svn: 355541
Part 2 of CSPGO changes (mostly related to ProfileSummary).
Note that I use a default parameter in setProfileSummary() and getSummary().
This is to break the dependency in clang. I will make the parameter explicit
after changing clang in a separated patch.
Differential Revision: https://reviews.llvm.org/D54175
llvm-svn: 355131
Current PGO profile counts are not context sensitive. The branch probabilities
for the inlined functions are kept the same for all call-sites, and they might
be very different from the actual branch probabilities. These suboptimal
profiles can greatly affect some downstream optimizations, in particular for
the machine basic block placement optimization.
In this patch, we propose to have a post-inline PGO instrumentation/use pass,
which we called Context Sensitive PGO (CSPGO). For the users who want the best
possible performance, they can perform a second round of PGO instrument/use on
the top of the regular PGO. They will have two sets of profile counts. The
first pass profile will be manly for inline, indirect-call promotion, and
CGSCC simplification pass optimizations. The second pass profile is for
post-inline optimizations and code-gen optimizations.
A typical usage:
// Regular PGO instrumentation and generate pass1 profile.
> clang -O2 -fprofile-generate source.c -o gen
> ./gen
> llvm-profdata merge default.*profraw -o pass1.profdata
// CSPGO instrumentation.
> clang -O2 -fprofile-use=pass1.profdata -fcs-profile-generate -o gen2
> ./gen2
// Merge two sets of profiles
> llvm-profdata merge default.*profraw pass1.profdata -o profile.profdata
// Use the combined profile. Pass manager will invoke two PGO use passes.
> clang -O2 -fprofile-use=profile.profdata -o use
This change touches many components in the compiler. The reviewed patch
(D54175) will committed in phrases.
Differential Revision: https://reviews.llvm.org/D54175
llvm-svn: 354930
Summary:
The llvm-cov tool needs to be able to find coverage names in the
executable, so the .lprfn and .lcovmap sections cannot be merged into
.rdata.
Also, the linker merges .lprfn$M into .lprfn, so llvm-cov needs to
handle that when looking up sections. It has to support running on both
relocatable object files and linked PE files.
Lastly, when loading .lprfn from a PE file, llvm-cov needs to skip the
leading zero byte added by the profile runtime.
Reviewers: vsk
Subscribers: hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58661
llvm-svn: 354840
The patch adds support for --hash-filenames to llvm-cov. This option adds md5
hash of the source path to the name of the generated .gcov file. The option is
crucial for cases where you have multiple files with the same name but can't
use --preserve-paths as resulting filenames exceed the limit.
from gcov(1):
```
-x
--hash-filenames
By default, gcov uses the full pathname of the source files to to
create an output filename. This can lead to long filenames that
can overflow filesystem limits. This option creates names of the
form source-file##md5.gcov, where the source-file component is
the final filename part and the md5 component is calculated from
the full mangled name that would have been used otherwise.
```
Patch by Igor Ignatev!
Differential Revision: https://reviews.llvm.org/D58370
llvm-svn: 354379
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
In LTO or Thin-lto mode (though linker plugin), the module
names are of temp file names which are different for
different compilations. Using SourceFileName avoids the issue.
This should not change any functionality for current PGO as
all the current callers of getPGOFuncName() is before LTO.
Differential Revision: https://reviews.llvm.org/D56327
llvm-svn: 350671
In LTO or Thin-lto mode (though linker plugin), the module
names are of temp file names which are different for
different compilations. Using SourceFileName avoids the issue.
This should not change any functionality for current PGO as
all the current callers of getPGOFuncName() is before LTO.
llvm-svn: 350442
Summary:
The goal of this patch is to have the same behaviour than gcc-gcov.
Currently the hit counts for a line is the sum of the counts for each block on that line.
The idea is to detect the cycles in the graph of blocks in using the algorithm by Hawick & James.
The count for a cycle is the min of the counts for each edge in the cycle.
Once we've the count for each cycle, we can sum them and add the transition counts of those cycles.
Fix both https://bugs.llvm.org/show_bug.cgi?id=38065 and https://bugs.llvm.org/show_bug.cgi?id=38066
Reviewers: marco-c, davidxl
Reviewed By: marco-c
Subscribers: vsk, lebedev.ri, sylvestre.ledru, dblaikie, llvm-commits
Differential Revision: https://reviews.llvm.org/D49659
llvm-svn: 342657
The patch saves a function offset table which maps function name index to the
offset of its function profile to the start of the binary profile. By using
the function offset table, for those function profiles which will not be used
when compiling a module, the profile reader does't have to read them. For
profile size around 10~20M, it saves ~10% compile time.
Differential Revision: https://reviews.llvm.org/D51863
llvm-svn: 342283
The patch tries to make sample profile loader independent of profile format
change. It moves compact format related code into FunctionSamples and
SampleProfileReader classes, and sample profile loader only has to interact
with those two classes and will be unaware of profile format changes.
The cleanup also contain some fixes to further remove the difference between
compactbinary format and binary format. After the cleanup using different
formats originated from the same profile will generate the same binaries,
which we verified by compiling two large server benchmarks w/wo thinlto.
Differential Revision: https://reviews.llvm.org/D51643
llvm-svn: 341591
Frontends emit 'unused' coverage mapping records for functions which are
provably unused in a TU. These unused records contain a single counter
with CounterKind::Zero. However, a function may be unused in one TU and
used in another. When this happens, prefer the records with a full set
of counters instead of arbitrarily picking the first loaded record.
There is no impact on the single-TU case. In the multiple-TU case, this
resolves issues causing a function to appear unused when it's not.
Testing: check-{llvm,clang,compiler-rt}
rdar://42981322
llvm-svn: 339194
Summary:
r262157 added ELF-specific logic to put a comdat on the __profc_*
globals created for available_externally functions. We should be able to
generalize that logic to all object file formats that support comdats,
i.e. everything other than MachO. This fixes duplicate symbol errors,
since on COFF, linkonce_odr doesn't make the symbol weak.
Fixes PR38251.
Reviewers: davidxl, xur
Subscribers: hiraditya, dmajor, llvm-commits, aheejin
Differential Revision: https://reviews.llvm.org/D49882
llvm-svn: 338082
The standard library functions ::isprint/std::isprint have platform-
and locale-dependent behavior which makes LLVM's output less
predictable. In particular, regression tests my fail depending on the
implementation of these functions.
Implement llvm::isPrint in StringExtras.h with a standard behavior and
replace all uses of ::isprint/std::isprint by a call it llvm::isPrint.
The function is inlined and does not look up language settings so it
should perform better than the standard library's version.
Such a replacement has already been done for isdigit, isalpha, isxdigit
in r314883. gtest does the same in gtest-printers.cc using the following
justification:
// Returns true if c is a printable ASCII character. We test the
// value of c directly instead of calling isprint(), which is buggy on
// Windows Mobile.
inline bool IsPrintableAscii(wchar_t c) {
return 0x20 <= c && c <= 0x7E;
}
Similar issues have also been encountered by Julia:
https://github.com/JuliaLang/julia/issues/7416
I noticed the problem myself when on Windows isprint('\t') started to
evaluate to true (see https://stackoverflow.com/questions/51435249) and
thus caused several unit tests to fail. The result of isprint doesn't
seem to be well-defined even for ASCII characters. Therefore I suggest
to replace isprint by a platform-independent version.
Differential Revision: https://reviews.llvm.org/D49680
llvm-svn: 338034
Name table occupies a big chunk of size in current binary format sample profile.
In order to reduce its size, the patch changes the sample writer/reader to
save/restore MD5Hash of names in the name table. Sample annotation phase will
also use MD5Hash of name to query samples accordingly.
Experiment shows compact binary format can reduce the size of sample profile by
2/3 compared with binary format generally.
Differential Revision: https://reviews.llvm.org/D47955
llvm-svn: 334447
Provide some free functions to reduce verbosity of endian-writing
a single value, and replace the endianness template parameter with
a field.
Part of PR37466.
Differential Revision: https://reviews.llvm.org/D47032
llvm-svn: 332757
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.
In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.
Differential Revision: https://reviews.llvm.org/D43624
llvm-svn: 332240
Summary:
Don't skip functions with the same name but from different files.
That change makes it possible to generate code coverage reports from
different binaries compiled from different sources even if there are functions
with non-unique names. Without that change, code coverage for such functions is
missing except of the first function processed.
Reviewers: vsk, morehouse
Reviewed By: vsk
Subscribers: llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D46478
llvm-svn: 331801
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46290
llvm-svn: 331272
See r331124 for how I made a list of files missing the include.
I then ran this Python script:
for f in open('filelist.txt'):
f = f.strip()
fl = open(f).readlines()
found = False
for i in xrange(len(fl)):
p = '#include "llvm/'
if not fl[i].startswith(p):
continue
if fl[i][len(p):] > 'Config':
fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
found = True
break
if not found:
print 'not found', f
else:
open(f, 'w').write(''.join(fl))
and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.
No intended behavior change.
llvm-svn: 331184
Summary:
r327219 added wrappers to std::sort which randomly shuffle the container before sorting.
This will help in uncovering non-determinism caused due to undefined sorting
order of objects having the same key.
To make use of that infrastructure we need to invoke llvm::sort instead of std::sort.
Note: This patch is one of a series of patches to replace *all* std::sort to llvm::sort.
Refer the comments section in D44363 for a list of all the required patches.
Reviewers: bogner, vsk, eraman, ruiu
Reviewed By: ruiu
Subscribers: ruiu, llvm-commits
Differential Revision: https://reviews.llvm.org/D45139
llvm-svn: 330057
Summary:
This was originally part of rL328132, and led to the discovery
of the issues addressed in rL328987. Re-landing.
Reviewers: xur, davidxl, bkramer
Reviewed By: bkramer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D45545
llvm-svn: 330029
Summary:
This reverts commit 364eb09576a7667bc6d3ff80c52a83014ccac976 and separates out
the portion that was fixing binary reader error propagation - turns out, there
are production cases where that causes a regression.
Will re-introduce the error propagation fix separately.
The fix to the text reader error propagation is still "in".
Reviewers: bkramer
Reviewed By: bkramer
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44807
llvm-svn: 328244
Summary:
This fixes a unittest failure introduced by D44717
D44717 introduced lazy sorting of the internal data structures of the
symbol table. The AddrToMD5Map getter was potentially exposing
inconsistent (unsorted) state. We could sort in the accessor, however,
a client may store the pointer and thus bypass the internal state
management of the symbol table. The alternative in this CL blocks
direct access to the state, thus ensuring consistent
externally-observable state.
Reviewers: davidxl, xur, eraman
Reviewed By: xur
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44757
llvm-svn: 328163
Summary:
External functions appearing as indirect call targets could not be
found in the SymTab, and the value:counter record was represented,
in the text format, using an empty string for the name. This would
then cause a silent parsing error when reading.
This CL:
- adds explicit support for such functions
- fixes the places where we would not propagate errors when reading
- addresses a performance issue due to eager resorting of the SymTab.
Reviewers: xur, eraman, davidxl
Reviewed By: davidxl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D44717
llvm-svn: 328132
SampleProfReader assumes function names in the profile are all mangled names.
However, there are cases that few demangled names are somehow contained in
the profile (usually because of debug info problems), which may trigger parsing
error in SampleProfReader and cause the whole profile to be unusable. The patch
extends SampleProfReader to handle profiles with demangled names, so that those
profiles can still be useful.
Differential revision: https://reviews.llvm.org/D44161
llvm-svn: 326905
Summary:
Exposing getOffset and findFunctionSamples as members of
SampleProfile. They are intimately tied to design choices of the
sample profile format - using offsets instead of line numbers, and
traversing inlined functions stack, respectively.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D43605
llvm-svn: 325747
Most of the -Wsign-compare warnings are due to the fact that
enums are signed by default in the MS ABI, while the
tautological comparison warnings trigger on x86 builds where
sizeof(size_t) is 4 bytes, so N > numeric_limits<unsigned>::max()
is always false.
Differential Revision: https://reviews.llvm.org/D41256
llvm-svn: 320750
This extends r319391. It teaches the segment builder to emit the right
completed segment when more than one region ends at the same location.
Fixes PR35495.
llvm-svn: 319990
These command line options are not intended for public use, and often
don't even make sense in the context of a particular tool anyway. About
90% of them are already hidden, but when people add new options they
forget to hide them, so if you were to make a brand new tool today, link
against one of LLVM's libraries, and run tool -help you would get a
bunch of junk that doesn't make sense for the tool you're writing.
This patch hides these options. The real solution is to not have
libraries defining command line options, but that's a much larger effort
and not something I'm prepared to take on.
Differential Revision: https://reviews.llvm.org/D40674
llvm-svn: 319505
This is a fix for the coverage segment builder.
If multiple regions must be popped off the active stack at once, and
more than one of them end at the same location, emit a segment using the
count from the most-recent completed region.
Fixes PR35437, rdar://35760630
Testing: invoked llvm-cov on a stage2 build of clang, additional unit
tests, check-profile
llvm-svn: 319391
We've worked around bugs in the frontend by ignoring the count from
wrapped segments when a line has at least one region entry segment.
Those frontend bugs are now fixed, so it's time to regenerate the
checked-in covmapping files and remove the workaround.
llvm-svn: 317761
This class was split between libIR and libSupport, which breaks under
modular code generation. Move it into the one library that uses it,
ProfileData, to resolve this issue.
llvm-svn: 317366
Rong Xu