forked from OSchip/llvm-project
[docs] Document the source-based code coverage feature
Differential Revision: http://reviews.llvm.org/D20715 llvm-svn: 271454
This commit is contained in:
Vedant Kumar
parent
858aba0666
commit
a530a3615e
|
|
@ -0,0 +1,186 @@
|
|||
==========================
|
||||
Source-based Code Coverage
|
||||
==========================
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
This document explains how to use clang's source-based code coverage feature.
|
||||
It's called "source-based" because it operates on AST and preprocessor
|
||||
information directly. This allows it to generate very precise coverage data.
|
||||
|
||||
Clang ships two other code coverage implementations:
|
||||
|
||||
* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
|
||||
various sanitizers. It can provide up to edge-level coverage.
|
||||
|
||||
* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
|
||||
|
||||
From this point onwards "code coverage" will refer to the source-based kind.
|
||||
|
||||
The code coverage workflow
|
||||
==========================
|
||||
|
||||
The code coverage workflow consists of three main steps:
|
||||
|
||||
1. Compiling with coverage enabled.
|
||||
|
||||
2. Running the instrumented program.
|
||||
|
||||
3. Creating coverage reports.
|
||||
|
||||
The next few sections work through a complete, copy-'n-paste friendly example
|
||||
based on this program:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
% cat <<EOF > foo.cc
|
||||
#define BAR(x) ((x) || (x))
|
||||
template <typename T> void foo(T x) {
|
||||
for (unsigned I = 0; I < 10; ++I) { BAR(I); }
|
||||
}
|
||||
int main() {
|
||||
foo<int>(0);
|
||||
foo<float>(0);
|
||||
return 0;
|
||||
}
|
||||
EOF
|
||||
|
||||
Compiling with coverage enabled
|
||||
===============================
|
||||
|
||||
To compile code with coverage enabled pass ``-fprofile-instr-generate
|
||||
-fcoverage-mapping`` to the compiler:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# Step 1: Compile with coverage enabled.
|
||||
% clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
|
||||
|
||||
Note that linking together code with and without coverage instrumentation is
|
||||
supported: any uninstrumented code simply won't be accounted for.
|
||||
|
||||
Running the instrumented program
|
||||
================================
|
||||
|
||||
The next step is to run the instrumented program. When the program exits it
|
||||
will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
|
||||
environment variable. If that variable does not exist the profile is written to
|
||||
``default.profraw`` in the current directory of the program.
|
||||
|
||||
If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
|
||||
missing directory structure will be created. Additionally, the following
|
||||
special **pattern strings** are replaced:
|
||||
|
||||
* "%p" expands out to the process ID.
|
||||
|
||||
* "%h" expands out to the hostname of the machine running the program.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# Step 2: Run the program.
|
||||
% LLVM_PROFILE_FILE="foo.profraw" ./foo
|
||||
|
||||
Creating coverage reports
|
||||
=========================
|
||||
|
||||
Raw profiles have to be **indexed** before they can be used to generated
|
||||
coverage reports. This is done using the "merge" tool in ``llvm-profdata``, so
|
||||
named because it can combine and index profiles at the same time:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# Step 3(a): Index the raw profile.
|
||||
% llvm-profdata merge -sparse foo.profraw -o foo.profdata
|
||||
|
||||
There are multiple different ways to render coverage reports. One option is to
|
||||
generate a line-oriented report:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# Step 3(b): Create a line-oriented coverage report.
|
||||
% llvm-cov show ./foo -instr-profile=foo.profdata
|
||||
|
||||
To demangle any C++ identifiers in the ouput, use:
|
||||
|
||||
.. code-block:: console
|
||||
% llvm-cov show ./foo -instr-profile=foo.profdata | c++filt -n
|
||||
|
||||
This report includes a summary view as well as dedicated sub-views for
|
||||
templated functions and their instantiations. For our example program, we get
|
||||
distinct views for ``foo<int>(...)`` and ``foo<float>(...)``. If
|
||||
``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays sub-line
|
||||
region counts (even in macro expansions):
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
20| 1|#define BAR(x) ((x) || (x))
|
||||
^20 ^2
|
||||
2| 2|template <typename T> void foo(T x) {
|
||||
22| 3| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
|
||||
^22 ^20 ^20^20
|
||||
2| 4|}
|
||||
------------------
|
||||
| void foo<int>(int):
|
||||
| 1| 2|template <typename T> void foo(T x) {
|
||||
| 11| 3| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
|
||||
| ^11 ^10 ^10^10
|
||||
| 1| 4|}
|
||||
------------------
|
||||
| void foo<float>(int):
|
||||
| 1| 2|template <typename T> void foo(T x) {
|
||||
| 11| 3| for (unsigned I = 0; I < 10; ++I) { BAR(I); }
|
||||
| ^11 ^10 ^10^10
|
||||
| 1| 4|}
|
||||
------------------
|
||||
|
||||
It's possible to generate a file-level summary of coverage statistics (instead
|
||||
of a line-oriented report) with:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# Step 3(c): Create a coverage summary.
|
||||
% llvm-cov report ./foo -instr-profile=foo.profdata
|
||||
Filename Regions Miss Cover Functions Executed
|
||||
-----------------------------------------------------------------------
|
||||
/tmp/foo.cc 13 0 100.00% 3 100.00%
|
||||
-----------------------------------------------------------------------
|
||||
TOTAL 13 0 100.00% 3 100.00%
|
||||
|
||||
A few final notes:
|
||||
|
||||
* The ``-sparse`` flag is optional but can result in dramatically smaller
|
||||
indexed profiles. This option should not be used if the indexed profile will
|
||||
be reused for PGO.
|
||||
|
||||
* Raw profiles can be discarded after they are indexed. Advanced use of the
|
||||
profile runtime library allows an instrumented program to merge profiling
|
||||
information directly into an existing raw profile on disk. The details are
|
||||
out of scope.
|
||||
|
||||
* The ``llvm-profdata`` tool can be used to merge together multiple raw or
|
||||
indexed profiles. To combine profiling data from multiple runs of a program,
|
||||
try e.g:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
% llvm-profdata merge -sparse foo1.profraw foo2.profdata -o foo3.profdata
|
||||
|
||||
Format compatibility guarantees
|
||||
===============================
|
||||
|
||||
* There are no backwards or forwards compatibility guarantees for the raw
|
||||
profile format. Raw profiles may be dependent on the specific compiler
|
||||
revision used to generate them. It's inadvisable to store raw profiles for
|
||||
long periods of time.
|
||||
|
||||
* Tools must retain **backwards** compatibility with indexed profile formats.
|
||||
These formats are not forwards-compatible: i.e, a tool which uses format
|
||||
version X will not be able to understand format version (X+k).
|
||||
|
||||
* There is a third format in play: the format of the coverage mappings emitted
|
||||
into instrumented binaries. Tools must retain **backwards** compatibility
|
||||
with these formats. These formats are not forwards-compatible.
|
||||
|
|
@ -33,6 +33,7 @@ Using Clang as a Compiler
|
|||
ControlFlowIntegrity
|
||||
LTOVisibility
|
||||
SafeStack
|
||||
SourceBasedCodeCoverage
|
||||
Modules
|
||||
MSVCCompatibility
|
||||
CommandGuide/index
|
||||
|
|
|
|||
Loading…
Reference in New Issue