This is an enhancement to LLVM Source-Based Code Coverage in clang to track how
many times individual branch-generating conditions are taken (evaluate to TRUE)
and not taken (evaluate to FALSE). Individual conditions may comprise larger
boolean expressions using boolean logical operators. This functionality is
very similar to what is supported by GCOV except that it is very closely
anchored to the ASTs.
Differential Revision: https://reviews.llvm.org/D84467
As reported here: https://reviews.llvm.org/D75153#1987272
Before, each instance of llvm-cov was creating one thread per hardware core, which wasn't needed probably because the number of inputs were small. This was probably causing a thread rlimit issue on large core count systems.
After this patch, the previous behavior is restored (to what was before rG8404aeb5):
If --num-threads is not specified, we create one thread per input, up to num.cores.
When specified, --num-threads indicates any number of threads, with no upper limit.
Differential Revision: https://reviews.llvm.org/D78408
The goal of this patch is to maximize CPU utilization on multi-socket or high core count systems, so that parallel computations such as LLD/ThinLTO can use all hardware threads in the system. Before this patch, on Windows, a maximum of 64 hardware threads could be used at most, in some cases dispatched only on one CPU socket.
== Background ==
Windows doesn't have a flat cpu_set_t like Linux. Instead, it projects hardware CPUs (or NUMA nodes) to applications through a concept of "processor groups". A "processor" is the smallest unit of execution on a CPU, that is, an hyper-thread if SMT is active; a core otherwise. There's a limit of 32-bit processors on older 32-bit versions of Windows, which later was raised to 64-processors with 64-bit versions of Windows. This limit comes from the affinity mask, which historically is represented by the sizeof(void*). Consequently, the concept of "processor groups" was introduced for dealing with systems with more than 64 hyper-threads.
By default, the Windows OS assigns only one "processor group" to each starting application, in a round-robin manner. If the application wants to use more processors, it needs to programmatically enable it, by assigning threads to other "processor groups". This also means that affinity cannot cross "processor group" boundaries; one can only specify a "preferred" group on start-up, but the application is free to allocate more groups if it wants to.
This creates a peculiar situation, where newer CPUs like the AMD EPYC 7702P (64-cores, 128-hyperthreads) are projected by the OS as two (2) "processor groups". This means that by default, an application can only use half of the cores. This situation could only get worse in the years to come, as dies with more cores will appear on the market.
== The problem ==
The heavyweight_hardware_concurrency() API was introduced so that only *one hardware thread per core* was used. Once that API returns, that original intention is lost, only the number of threads is retained. Consider a situation, on Windows, where the system has 2 CPU sockets, 18 cores each, each core having 2 hyper-threads, for a total of 72 hyper-threads. Both heavyweight_hardware_concurrency() and hardware_concurrency() currently return 36, because on Windows they are simply wrappers over std:🧵:hardware_concurrency() -- which can only return processors from the current "processor group".
== The changes in this patch ==
To solve this situation, we capture (and retain) the initial intention until the point of usage, through a new ThreadPoolStrategy class. The number of threads to use is deferred as late as possible, until the moment where the std::threads are created (ThreadPool in the case of ThinLTO).
When using hardware_concurrency(), setting ThreadCount to 0 now means to use all the possible hardware CPU (SMT) threads. Providing a ThreadCount above to the maximum number of threads will have no effect, the maximum will be used instead.
The heavyweight_hardware_concurrency() is similar to hardware_concurrency(), except that only one thread per hardware *core* will be used.
When LLVM_ENABLE_THREADS is OFF, the threading APIs will always return 1, to ensure any caller loops will be exercised at least once.
Differential Revision: https://reviews.llvm.org/D71775
Summary:
Counters are stored as uint64_t in the coverage mapping, but
exporting in JSON requires signed integers. Clamp the values to the
smaller range to make the conversion safe.
Reviewers: Dor1s, vsk
Reviewed By: Dor1s
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D70200
Add the `-whitelist-filename-regex` option to restrict coverage
reporting to file paths that match a whitelist regex.
Patch by Michael Daniels!
rdar://56720320
Summary:
CoverageExporterJson::renderFiles accounts for most of the execution time given a large profdata file with multiple binaries.
Proposed solution is to generate JSON for each file in parallel and sort at the end to preserve deterministic output. Also added flags to skip generating parts of the output to trim the output size.
Patch by Sajjad Mirza (@sajjadm).
Reviewers: Dor1s, vsk
Reviewed By: Dor1s, vsk
Subscribers: liaoyuke, mgrang, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59277
llvm-svn: 356178
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46290
llvm-svn: 331272
Summary:
The option is helpful for large projects where it's not feasible to specify sources which
user would like to see in the report. Instead, it allows to black-list specific sources via
regular expressions (e.g. now it's possible to skip all files that have "test" in its name).
This also partially fixes https://bugs.llvm.org/show_bug.cgi?id=34277
Reviewers: vsk, morehouse, liaoyuke
Reviewed By: vsk
Subscribers: kcc, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D43907
llvm-svn: 329581
Summary:
This speeds up export "summary-only" execution by an order of magnitude or two,
depending on number of threads used for prepareFileReports execution.
Also includes minor refactoring for splitting render of summary and detailed data
in two independent methods.
Reviewers: vsk, morehouse
Reviewed By: vsk
Subscribers: llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D42000
llvm-svn: 322397
Summary:
That allows to get the same data as produced by "llvm-cov report",
but in JSON format, which is better for further processing by end users.
Reviewers: vsk
Reviewed By: vsk
Differential Revision: https://reviews.llvm.org/D41085
llvm-svn: 320435
There's a bug in the way the line and region summary objects are merged.
It would have been less likely to occur if those objects kept some data
private.
llvm-svn: 313416
The "NotCovered" fields in the region and line summary structs are
redundant. We should remove them to make the code clearer.
As a follow-up, the "NotCovered" entries should be removed from the
reports as well.
llvm-svn: 313415
When we load coverage data from multiple objects, we don't have a way to
attribute a source object to a function record. Printing out the object
filename next to the source filename is already not very useful: soon,
it'll actually become misleading. Stop printing out the filename now.
llvm-svn: 285043
We used to append filenames into a vector of std::string, and then
append a reference to each string into a separate vector. This made it
easier to work with the getUniqueSourceFiles API. But it's buggy.
std::string has a small-string optimization, so you can't expect to
capture a reference to one if you're copying it into a growing vector.
Add a test that triggers this invalid reference to std::string scenario,
and kill the issue with fire by just using ArrayRef<std::string>
everywhere.
llvm-svn: 282281
This enables users to export coverage information as portable JSON for use by
analysis tools and storage in document based databases.
The export sub-command is invoked just like the others:
llvm-cov export -instr-profile path/to/foo.profdata path/to/foo.binary
The resulting JSON contains a list of files and functions. Every file object
contains a list of segments, expansions, and a summary of the file's region,
function, and line coverage. Every function object contains the function's name
and regions. There is also a total summary for the entire object file.
Changes since the initial commit (r276813):
- Fixed the regexes in the tests to handle Windows filepaths.
Patch by Eddie Hurtig!
Differential Revision: https://reviews.llvm.org/D22651
llvm-svn: 276818
This enables users to export coverage information as portable JSON for use by
analysis tools and storage in document based databases.
The export sub-command is invoked just like the others:
llvm-cov export -instr-profile path/to/foo.profdata path/to/foo.binary
The resulting JSON contains a list of files and functions. Every file object
contains a list of segments, expansions, and a summary of the file's region,
function, and line coverage. Every function object contains the function's name
and regions. There is also a total summary for the entire object file.
Patch by Eddie Hurtig!
Differential Revision: https://reviews.llvm.org/D22651
llvm-svn: 276813
Kazu Hirata