Commit Graph

382 Commits

Author SHA1 Message Date
Kazu Hirata f44473ec4e [llvm] Remove redundant member initialization (NFC)
Identified with readability-redundant-member-init.
2022-01-08 11:56:44 -08:00
Kazu Hirata 435a5a3652 [llvm] Fix bugprone argument comments (NFC)
Identified with bugprone-argument-comment.
2022-01-08 11:56:38 -08:00
Kazu Hirata e5947760c2 Revert "[llvm] Remove redundant member initialization (NFC)"
This reverts commit fd4808887e.

This patch causes gcc to issue a lot of warnings like:

  warning: base class ‘class llvm::MCParsedAsmOperand’ should be
  explicitly initialized in the copy constructor [-Wextra]
2022-01-03 11:28:47 -08:00
Kazu Hirata fd4808887e [llvm] Remove redundant member initialization (NFC)
Identified with readability-redundant-member-init.
2022-01-01 16:18:18 -08:00
Zarko Todorovski e714394ab8 [LLVM][llvm-cov] Inclusive language: rename option -name-whitelist to -name-allowlist
Renamed the option for llvm-cov and changed variable names to use more
inclusive terms. Also changed the binary for the test.

Reviewed By: alanphipps

Differential Revision: https://reviews.llvm.org/D112816
2021-11-26 11:08:01 -05:00
Kazu Hirata 4e3eebc6bd [tools, utils] Use StringRef::contains (NFC) 2021-10-22 17:22:13 -07:00
Keith Smiley e972e49b11 [llvm-cov] Add error for invalid -path-equivalence format
Differential Revision: https://reviews.llvm.org/D109042
2021-09-10 18:34:37 -07:00
Christian Fetzer 9116211d18 [Coverage][llvm-cov] Correctly export branch coverage in LCOV format
Commit 9f2967bcfe introduced support for
branch coverage including export to the LCOV format.

This commit corrects the LCOV field name for branches from BFH to BRH.
The mistake seems to have slipped in as typo because the correct field
name BRH is used in the comment section at the beginning of the file.

Differential Revision: https://reviews.llvm.org/D108358
2021-08-20 13:44:25 -05:00
Martin Storsjö 42f74e8249 [llvm] Rename StringRef _lower() method calls to _insensitive()
This is a mechanical change. This actually also renames the
similarly named methods in the SmallString class, however these
methods don't seem to be used outside of the llvm subproject, so
this doesn't break building of the rest of the monorepo.
2021-06-25 00:22:01 +03:00
Petr Hosek 8280ece0c9 [Coverage] Support overriding compilation directory
When making compilation relocatable, for example in distributed
compilation scenarios, we want to set compilation dir to a relative
value like `.` but this presents a problem when generating reports
because if the file path is relative as well, for example `..`, you
may end up writing files outside of the output directory.

This change introduces a flag that allows overriding the compilation
directory that's stored inside the profile with a different value that
is absolute.

Differential Revision: https://reviews.llvm.org/D100232
2021-05-11 15:26:45 -07:00
Petr Hosek 489a3531a4 [llvm-cov] Support for v4 format in convert-for-testing
v4 moves function records to a dedicated section so we need to write
and read it separately.

https://reviews.llvm.org/D100535
2021-05-11 14:41:55 -07:00
Alan Phipps eccb925147 Reland "[Coverage] Fix branch coverage merging in FunctionCoverageSummary::get() for instantiation""
Originally landed in: 6400905a61
Reverted in: 668dccc396

Fix branch coverage merging in FunctionCoverageSummary::get() for instantiation
groups.

This change corrects the implementation for the branch coverage summary to do
the same thing for branches that is done for lines and regions.  That is,
across function instantiations in an instantiation group, the maximum branch
coverage found in any of those instantiations is returned, with the total
number of branches being the same across instantiations.

Differential Revision: https://reviews.llvm.org/D102193
2021-05-11 11:48:23 -05:00
Alan Phipps 668dccc396 Revert "Fix branch coverage merging in FunctionCoverageSummary::get() for instantiation"
This reverts commit 6400905a61.
2021-05-11 11:26:19 -05:00
Alan Phipps 6400905a61 Fix branch coverage merging in FunctionCoverageSummary::get() for instantiation
groups.

This change corrects the implementation for the branch coverage
summary to do the same thing for branches that is done for lines and regions.
That is, across function instantiations in an instantiation group, the maximum
branch coverage found in any of those instantiations is returned, with the
total number of branches being the same across instantiations.

Differential Revision: https://reviews.llvm.org/D102193
2021-05-11 10:42:40 -05:00
Nico Weber ba7a92c01e [Support] Don't include VirtualFileSystem.h in CommandLine.h
CommandLine.h is indirectly included in ~50% of TUs when building
clang, and VirtualFileSystem.h is large.

(Already remarked by jhenderson on D70769.)

No behavior change.

Differential Revision: https://reviews.llvm.org/D100957
2021-04-21 10:19:01 -04:00
Abhina Sreeskantharajan c83cd8feef [NFC] Reordering parameters in getFile and getFileOrSTDIN
In future patches I will be setting the IsText parameter frequently so I will refactor the args to be in the following order. I have removed the FileSize parameter because it is never used.

```
  static ErrorOr<std::unique_ptr<MemoryBuffer>>
  getFile(const Twine &Filename, bool IsText = false,
          bool RequiresNullTerminator = true, bool IsVolatile = false);

  static ErrorOr<std::unique_ptr<MemoryBuffer>>
  getFileOrSTDIN(const Twine &Filename, bool IsText = false,
                 bool RequiresNullTerminator = true);

 static ErrorOr<std::unique_ptr<MB>>
 getFileAux(const Twine &Filename, uint64_t MapSize, uint64_t Offset,
            bool IsText, bool RequiresNullTerminator, bool IsVolatile);

  static ErrorOr<std::unique_ptr<WritableMemoryBuffer>>
  getFile(const Twine &Filename, bool IsVolatile = false);
```

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D99182
2021-03-25 09:47:49 -04:00
Zequan Wu dd388ba3e0 [llvm-cov] Check path emptyness in path-equivalence after removing dots. 2021-03-24 17:54:38 -07:00
Choongwoo Han 9d8a3e75b4 [llvm-cov] Cache file status information
Currently, getSourceFile accesses file system to check if two paths are
the same file with a thread lock, which is a huge performance bottleneck
in some cases. Currently, it's accessing file system size(files) * size(files) times.

Thus, cache file status information, which reduces file system access to size(files) times.

When I tested it with two binaries and 16 cpu cores,
it saved over 70% of time.

Binary 1: 56 secs -> 3 secs
Binary 2: 17 hours -> 4 hours

Differential Revision: https://reviews.llvm.org/D97061
2021-03-03 10:04:07 -08:00
Kazu Hirata 7728cc003a [llvm] Use append_range (NFC) 2021-01-29 23:23:34 -08:00
Kazu Hirata 352fcfc697 [llvm] Use llvm::sort (NFC) 2021-01-17 10:39:45 -08:00
Kazu Hirata 12fc9ca3a4 [llvm] Remove redundant string initialization (NFC)
Identified with readability-redundant-string-init.
2021-01-12 21:43:46 -08:00
Alan Phipps 9f2967bcfe [Coverage] Add support for Branch Coverage in LLVM Source-Based Code Coverage
This is an enhancement to LLVM Source-Based Code Coverage in clang to track how
many times individual branch-generating conditions are taken (evaluate to TRUE)
and not taken (evaluate to FALSE).  Individual conditions may comprise larger
boolean expressions using boolean logical operators.  This functionality is
very similar to what is supported by GCOV except that it is very closely
anchored to the ASTs.

Differential Revision: https://reviews.llvm.org/D84467
2021-01-05 09:51:51 -06:00
Kazu Hirata b676f2fee1 [llvm-cov, llvm-symbolizer] Use llvm::erase_if (NFC) 2020-12-26 12:06:27 -08:00
serge-sans-paille 9218ff50f9 llvmbuildectomy - replace llvm-build by plain cmake
No longer rely on an external tool to build the llvm component layout.

Instead, leverage the existing `add_llvm_componentlibrary` cmake function and
introduce `add_llvm_component_group` to accurately describe component behavior.

These function store extra properties in the created targets. These properties
are processed once all components are defined to resolve library dependencies
and produce the header expected by llvm-config.

Differential Revision: https://reviews.llvm.org/D90848
2020-11-13 10:35:24 +01:00
Keith Smiley 2319c49339 [llvm-cov][NFC] Fix typo in format comment 2020-11-09 14:47:39 -08:00
Zequan Wu c75a0a1e9d [llvm-cov] Fix missing slash in -path-equivalence 2020-11-06 14:54:11 -08:00
Zequan Wu e92eeaf3c2 [llvm-cov] don't include all source files when provided source files are filtered out
When all provided source files are filtered out either due to `--ignore-filename-regex` or not part of binary, don't generate coverage reults for all source files. Because if users want to generate coverage results for all source files, they don't even need to provid selected source files or `--ignore-filename-regex`.

Differential Revision: https://reviews.llvm.org/D89359
2020-10-23 19:32:16 -07:00
Jeremy Morse 0a7f41739f Revert "[llvm-cov] don't include all source files when provided source files are filtered out"
This reverts commit c2bd20ef652 and the follow up fix 16605bba6fb.

The tools/llvm-cov/warnings.h continues to fail on Windows platforms even
after the follow up, for example on the llvm-clang-win-x-armv7l builder:

    http://lab.llvm.org:8011/#/builders/60/builds/94
2020-10-16 13:46:12 +01:00
Zequan Wu cd3090819c [llvm-cov] don't include all source files when provided source files are filtered out
When all provided source files are filtered out either due to `--ignore-filename-regex` or not part of binary, don't generate coverage reults for all source files. Because if users want to generate coverage results for all source files, they don't even need to provid selected source files or `--ignore-filename-regex`.

Differential Revision: https://reviews.llvm.org/D89359
2020-10-15 18:11:01 -07:00
Vedant Kumar 3c731ba5f1 [llvm-cov] Allow commas in filenames passed to `-object` flag
Currently, -object takes a comma separated list of objects as an
argument, which prevents it working with path names that contain a
comma. Drop comma-separated support, which requires to set pass the
-object flag multiple times to set multiple objects.

Patch by Andrew Gallagher!

Differential Revision: https://reviews.llvm.org/D87003
2020-09-18 13:46:29 -07:00
Fangrui Song c16417f65f [llvm-cov gcov] Add --demangled-names (-m)
gcov 4.9 introduced the option.
2020-09-16 23:18:50 -07:00
Fangrui Song 4d7b194543 [llvm-cov gcov] Refactor counting and reporting
The current organization of FileInfo and its referenced utility functions of
(GCOVFile, GCOVFunction, GCOVBlock) is messy. Some members of FileInfo are just
copied from GCOVFile. FileInfo::print (.gcov output and --intermediate output)
is interleaved with branch statistics and computation of line execution counts.
--intermediate has to do redundant .gcov output to gather branch statistics.

This patch deletes lots of code and introduces a clearer work flow:

```
fn collectFunction
  for each block b
    for each line lineNum
      let line be LineInfo of the file on lineNum
      line.exists = 1
      increment function's lines & linesExec if necessary
      increment line.count
      line.blocks.push_back(&b)

fn collectSourceLine
  compute cycle counts
  count = incoming_counts + cycle_counts
  if line.exists
    ++summary->lines
    if line.count
      ++summary->linesExec

fn collectSource
  for each line
    call collectSourceLine

fn main
  for each function
    call collectFunction
    print function summary
  for each source file
    call collectSource
    print file summary
    annotate the source file with line execution counts
  if -i
    print intermediate file
```

The output order of functions and files now follows the original order in
.gcno files.
2020-09-13 23:00:59 -07:00
Fangrui Song b2c32c90ba [llvm-cov gcov] Add -r (--relative-only) && -s (--source-prefix)
gcov 4.7 introduced the two options.
https://sourceware.org/pipermail/gcc-patches/2011-November/328782.html

-r only dumps files with relative paths or absolute paths with the prefix
specified by -s. The two options are useful filtering out system header files.
2020-09-13 14:54:20 -07:00
Fangrui Song 2e74b6d80f [llvm-cov gcov] Don't require NUL terminator when reading files
.gcno, .gcda and source files can be modified while we are reading them. If the
concurrent modification of a file being read nullifies the NUL terminator
assumption, llvm-cov can trip over an assertion failure in MemoryBuffer::init.
This is not so rare - the source files can be in an editor and .gcda can be
written by an running process (if the process forks, when .gcda gets written is
probably more unpredictable).

There is no accompanying test because an assertion failure requires data
races with some involved setting.
2020-07-19 00:31:52 -07:00
Fangrui Song 7f7cb79b57 [llvm-cov gcov] Don't suppress .gcov output if .gcda is corrupted
If .gcda is corrupted, gcov continues to produce a .gcov and just
assumes execution counts are zeros. This is reasonable, because the
program can corrupt its .gcda output. The code path should be similar to
the code path without .gcda.
2020-06-16 14:55:38 -07:00
Fangrui Song def2156389 [gcov] Add -i --intermediate-format
Between gcov 4.9~8, `gcov -i $file` prints coverage information to
$file.gcov in an intermediate text format (single file, instead of
$source.gcov for each source file).

lcov newer than 2019-05-24 detects -i support and uses it to increase
processing speed.  gcov 9 (GCC r265587) removed --intermediate-format
and -i was changed to mean --json-format. However, we consider this
format still useful and support it. geninfo (part of lcov) supports this
format even if we announce that we are compatible with gcov 9.0.0
2020-06-16 14:14:28 -07:00
Fangrui Song 4cd7ba7eca [gcov] Refactor llvm-cov gcov and add SourceInfo 2020-06-16 14:14:26 -07:00
Michael Liao 49688b3c30 Fix `-Wpedantic` warning. NFC. 2020-05-27 15:57:03 -04:00
Simon Pilgrim 6022efb0e9 CoverageFilters.h - reduce unnecessary includes to forward declarations. NFC. 2020-05-27 16:57:47 +01:00
Fangrui Song f081150c4d [gcov] Implement --stdout -t
gcov by default prints to a .gcov file. With --stdout, stdout is used.
Some summary information is omitted. There is no separator for multiple
source files.
2020-05-10 21:02:38 -07:00
Fangrui Song 2d00eb17ca [gcov] Fix .gcda decoding and support GCC 8, 9 and 10
GCDAProfiling.c unnecessarily writes function names to .gcda files.
GCC 4.2 gcc/libgcov.c (now renamed to libgcc/libgcov*) did not write function
names. gcov-7 (compatible) crashes on .gcda produced by libclang_rt.profile
rL176173 realized the problem and introduced a mode to remove function
names.

llvm-cov code apparently takes GCDAProfiling.c output format as truth
and tries to decode function names.  Additionally, llvm-cov tries to
decode tags in certain order which does not match libgcov emitted .gcda
files.

This patch fixes the .gcda decoder and makes it work with GCC 8 and 9
(10 is compatible with 9). Note, line statistics are broken and not
fixed by this patch.

Add test/tools/llvm-cov/gcov-{4.7,8,9}.c to test compatibility.
2020-05-10 09:55:23 -07:00
Alexandre Ganea 0e13a0331f [llvm-cov] Prevent llvm-cov from using too many threads
As reported here: https://reviews.llvm.org/D75153#1987272

Before, each instance of llvm-cov was creating one thread per hardware core, which wasn't needed probably because the number of inputs were small. This was probably causing a thread rlimit issue on large core count systems.

After this patch, the previous behavior is restored (to what was before rG8404aeb5):

If --num-threads is not specified, we create one thread per input, up to num.cores.
When specified, --num-threads indicates any number of threads, with no upper limit.

Differential Revision: https://reviews.llvm.org/D78408
2020-04-24 15:28:25 -04:00
Benjamin Kramer 52dcbcbfe0 Simplify string joins. NFCI. 2020-04-11 17:20:11 +02:00
Eli Friedman 9eb1b41811 [llvm-cov] Improve error message for missing profdata
I got a report recently that a user was having trouble interpreting the
meaning of the error message.  Hopefully this is more readable; produces
something like the following:

error: No such file or directory: Could not read profile data!

Differential Revision: https://reviews.llvm.org/D76796
2020-03-30 12:54:07 -07:00
Dennis Felsing aa0be69e74 Export Segment.IsGapRegion to JSON
Summary:
So that external tools can make use of that information and not display such lines as uncovered.

Fixes https://bugs.llvm.org/show_bug.cgi?id=45300

Reviewers: vsk

Reviewed By: vsk

Differential Revision: https://reviews.llvm.org/D76763
2020-03-27 18:05:01 +01:00
Alexandre Ganea 8404aeb56a [Support] On Windows, ensure hardware_concurrency() extends to all CPU sockets and all NUMA groups
The goal of this patch is to maximize CPU utilization on multi-socket or high core count systems, so that parallel computations such as LLD/ThinLTO can use all hardware threads in the system. Before this patch, on Windows, a maximum of 64 hardware threads could be used at most, in some cases dispatched only on one CPU socket.

== Background ==
Windows doesn't have a flat cpu_set_t like Linux. Instead, it projects hardware CPUs (or NUMA nodes) to applications through a concept of "processor groups". A "processor" is the smallest unit of execution on a CPU, that is, an hyper-thread if SMT is active; a core otherwise. There's a limit of 32-bit processors on older 32-bit versions of Windows, which later was raised to 64-processors with 64-bit versions of Windows. This limit comes from the affinity mask, which historically is represented by the sizeof(void*). Consequently, the concept of "processor groups" was introduced for dealing with systems with more than 64 hyper-threads.

By default, the Windows OS assigns only one "processor group" to each starting application, in a round-robin manner. If the application wants to use more processors, it needs to programmatically enable it, by assigning threads to other "processor groups". This also means that affinity cannot cross "processor group" boundaries; one can only specify a "preferred" group on start-up, but the application is free to allocate more groups if it wants to.

This creates a peculiar situation, where newer CPUs like the AMD EPYC 7702P (64-cores, 128-hyperthreads) are projected by the OS as two (2) "processor groups". This means that by default, an application can only use half of the cores. This situation could only get worse in the years to come, as dies with more cores will appear on the market.

== The problem ==
The heavyweight_hardware_concurrency() API was introduced so that only *one hardware thread per core* was used. Once that API returns, that original intention is lost, only the number of threads is retained. Consider a situation, on Windows, where the system has 2 CPU sockets, 18 cores each, each core having 2 hyper-threads, for a total of 72 hyper-threads. Both heavyweight_hardware_concurrency() and hardware_concurrency() currently return 36, because on Windows they are simply wrappers over std:🧵:hardware_concurrency() -- which can only return processors from the current "processor group".

== The changes in this patch ==
To solve this situation, we capture (and retain) the initial intention until the point of usage, through a new ThreadPoolStrategy class. The number of threads to use is deferred as late as possible, until the moment where the std::threads are created (ThreadPool in the case of ThinLTO).

When using hardware_concurrency(), setting ThreadCount to 0 now means to use all the possible hardware CPU (SMT) threads. Providing a ThreadCount above to the maximum number of threads will have no effect, the maximum will be used instead.
The heavyweight_hardware_concurrency() is similar to hardware_concurrency(), except that only one thread per hardware *core* will be used.

When LLVM_ENABLE_THREADS is OFF, the threading APIs will always return 1, to ensure any caller loops will be exercised at least once.

Differential Revision: https://reviews.llvm.org/D71775
2020-02-14 10:24:22 -05:00
Bill Wendling c55cf4afa9 Revert "Remove redundant "std::move"s in return statements"
The build failed with

  error: call to deleted constructor of 'llvm::Error'

errors.

This reverts commit 1c2241a793.
2020-02-10 07:07:40 -08:00
Bill Wendling 1c2241a793 Remove redundant "std::move"s in return statements 2020-02-10 06:39:44 -08:00
Benjamin Kramer bd31243a34 Fix more implicit conversions. Getting closer to having clang working with gcc 5 again 2020-01-29 02:57:59 +01:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00