Commit Graph

438 Commits

Author SHA1 Message Date
Douglas Yung 5435c5def2 Add shim for fork() on PS4 as it is not supported there.
Reviewers: probinson

Subscribers: #sanitizers llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D79839
2020-05-13 18:47:01 +00:00
KAWASHIMA Takahiro 7d4167430c [gcov] Fix simultaneous .gcda creation/lock
Fixes PR45673

The commit 9180c14fe4 (D76206) resolved only a part of the problem
of concurrent .gcda file creation. It ensured that only one process
creates the file but did not ensure that the process locks the
file first. If not, the process which created the file may clobber
the contents written by a process which locked the file first.
This is the cause of PR45673.

This commit prevents the clobbering by revising the assumption
that a process which creates the file locks the file first.
Regardless of file creation, a process which locked the file first
uses fwrite (new_file==1) and other processes use mmap (new_file==0).

I also tried to keep the creation/first-lock process same by using
mkstemp/link/unlink but the code gets long. This commit is more
simple.

Note: You may be confused with other changes which try to resolve
concurrent file access. My understanding is (may not be correct):

D76206:   Resolve race of .gcda file creation (but not lock)
This one: Resolve race of .gcda file creation and lock
D54599:   Same as D76206 but abandoned?
D70910:   Resolve race of multi-threaded counter flushing
D74953:   Resolve counter sharing between parent/children processes
D78477:   Revision of D74953

Differential Revision: https://reviews.llvm.org/D79556
2020-05-13 13:03:03 +09:00
Fangrui Song f98709a982 [gcov] Fix big-endian problems
In a big-endian .gcda file, the first four bytes are "gcda" instead of "adcg".
All 32-bit values are in big-endian.

With this change, libclang_rt.profile can hopefully produce gcov
compatible output.
2020-05-11 22:36:46 -07:00
Fangrui Song 4c684b91d5 Revert part of D49132 "[gcov] Fix gcov profiling on big-endian machines"
D49132 is partially correct. For 64-bit values, the lower 32-bit part comes
before the higher 32-bit part (in a little-endian manner).

For 32-bit values, libgcov reads/writes 32-bit values in native endianness.
2020-05-11 22:27:01 -07:00
Fangrui Song 013f06703e [gcov] Emit GCOV_TAG_OBJECT_SUMMARY/GCOV_TAG_PROGRAM_SUMMARY correctly and fix llvm-cov's decoding of runcount
gcov 9 (r264462) started to use GCOV_TAG_OBJECT_SUMMARY. Before,
GCOV_TAG_PROGRAM_SUMMARY was used.
libclang_rt.profile should emit just one tag according to the version.

Another bug introduced by rL194499 is that the wrong runcount field was
selected.

Fix the two bugs so that gcov can correctly decode "Runs:" from
libclang_rt.profile produced .gcda files, and llvm-cov gcov can
correctly decode "Runs:" from libgcov produced .gcda files.
2020-05-11 21:53:53 -07:00
Fangrui Song 25544ce2df [gcov] Default coverage version to '407*' and delete CC1 option -coverage-cfg-checksum
Defaulting to -Xclang -coverage-version='407*' makes .gcno/.gcda
compatible with gcov [4.7,8)

In addition, delete clang::CodeGenOptionsBase::CoverageExtraChecksum and GCOVOptions::UseCfgChecksum.
We can infer the information from the version.

With this change, .gcda files produced by `clang --coverage a.o` linked executable can be read by gcov 4.7~7.
We don't need other -Xclang -coverage* options.
There may be a mismatching version warning, though.

(Note, GCC r173147 "split checksum into cfg checksum and line checksum"
 made gcov 4.7 incompatible with previous versions.)
2020-05-10 16:14:07 -07:00
Fangrui Song 13a633b438 [gcov] Delete CC1 option -coverage-no-function-names-in-data
rL144865 incorrectly wrote function names for GCOV_TAG_FUNCTION
(this might be part of the reasons the header says
"We emit files in a corrupt version of GCOV's "gcda" file format").

rL176173 and rL177475 realized the problem and introduced -coverage-no-function-names-in-data
to work around the issue. (However, the description is wrong.
libgcov never writes function names, even before GCC 4.2).

In reality, the linker command line has to look like:

clang --coverage -Xclang -coverage-version='407*' -Xclang -coverage-cfg-checksum -Xclang -coverage-no-function-names-in-data

Failing to pass -coverage-no-function-names-in-data can make gcov 4.7~7
either produce wrong results (for one gcov-4.9 program, I see "No executable lines")
or segfault (gcov-7).
(gcov-8 uses an incompatible format.)

This patch deletes -coverage-no-function-names-in-data and the related
function names support from libclang_rt.profile
2020-05-10 12:37:44 -07:00
Calixte Denizet bec223a9bc [profile] Don't crash when forking in several threads
Summary:
When forking in several threads, the counters were written out in using the same global static variables (see GCDAProfiling.c): that leads to crashes.
So when there is a fork, the counters are resetted in the child process and they will be dumped at exit using the interprocess file locking.
When there is an exec, the counters are written out and in case of failures they're resetted.

Reviewers: jfb, vsk, marco-c, serge-sans-paille

Reviewed By: marco-c, serge-sans-paille

Subscribers: llvm-commits, serge-sans-paille, dmajor, cfe-commits, hiraditya, dexonsmith, #sanitizers, marco-c, sylvestre.ledru

Tags: #sanitizers, #clang, #llvm

Differential Revision: https://reviews.llvm.org/D78477
2020-05-07 14:13:11 +02:00
Petr Hosek 485862ecda [profile] Avoid duplicating or leaking VMO
Now that write data continously into the memory mapping, we don't need
to keep the VMO handle around after it has been mapped. This change also
ensures that the VMO is always closed on error.

Differential Revision: https://reviews.llvm.org/D76963
2020-04-17 00:54:12 -07:00
KAWASHIMA Takahiro 9180c14fe4 Fix simultaneous .gcda creation
The intent of the `llvm_gcda_start_file` function is that only
one process create the .gcda file and initialize it to be updated
by other processes later.

Before this change, if multiple processes are started simultaneously,
some of them may initialize the file because both the first and
second `open` calls may succeed in a race condition and `new_file`
becomes 1 in those processes. This leads incorrect coverage counter
values. This often happens in MPI (Message Passing Interface) programs.
The test program added in this change is a simple reproducer.

This change ensures only one process creates/initializes the file by
using the `O_EXCL` flag.

Differential Revision: https://reviews.llvm.org/D76206
2020-04-01 10:29:50 +09:00
Petr Hosek ba1f4405c6 [profile] Move RuntimeCounterRelocation and ProfileDumped into a separate file
This avoids the test failure that was introduced in rG32bddad where
this function pulls in the rest of InstrProfilingFile.c which is
undesirable in use cases when profile runtime is being used without
the rest of libc.

This also allows additional cleanup by eliminating another variable
from platforms that don't need it.

Differential Revision: https://reviews.llvm.org/D76750
2020-03-24 20:27:14 -07:00
Petr Hosek 32bddad37b [profile] Make atexit hook a no-op on Fuchsia
On Fuchsia, we always use the continuous mode with runtime counter
relocation, so there's no need for atexit hook or support for dumping
the profile manually.

Differential Revision: https://reviews.llvm.org/D76556
2020-03-24 18:47:38 -07:00
Petr Hosek d6fc61b7e8 [profile] Record the profile size as a property of the VMO
While the VMO size is always page aligned, we can record the content
size as a property and then use this metadata when writing the profile
to a file.

Differential Revision: https://reviews.llvm.org/D76402
2020-03-19 16:22:19 -07:00
Hans Wennborg 546918cbb4 Revert "[compiler-rt] Add a critical section when flushing gcov counters"
See discussion on PR44792.

This reverts commit 02ce9d8ef5.

It also reverts the follow-up commits
8f46269f0 "[profile] Don't dump counters when forking and don't reset when calling exec** functions"
62c7d8402 "[profile] gcov_mutex must be static"
2020-02-26 13:27:44 +01:00
Calixte Denizet 62c7d84026 [profile] gcov_mutex must be static
Summary: Forget static keyword for gcov_mutex in https://reviews.llvm.org/D74953 and that causes test failure on mac.

Reviewers: erik.pilkington, vsk

Reviewed By: vsk

Subscribers: vsk, dexonsmith, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D75080
2020-02-25 10:54:52 +01:00
Calixte Denizet 8f46269f0c [profile] Don't dump counters when forking and don't reset when calling exec** functions
Summary:
There is no need to write out gcdas when forking because we can just reset the counters in the parent process.
Let say a counter is N before the fork, then fork and this counter is set to 0 in the child process.
In the parent process, the counter is incremented by P and in the child process it's incremented by C.
When dump is ran at exit, parent process will dump N+P for the given counter and the child process will dump 0+C, so when the gcdas are merged the resulting counter will be N+P+C.
About exec** functions, since the current process is replaced by an another one there is no need to reset the counters but just write out the gcdas since the counters are definitely lost.
To avoid to have lists in a bad state, we just lock them during the fork and the flush (if called explicitely) and lock them when an element is added.

Reviewers: marco-c

Reviewed By: marco-c

Subscribers: hiraditya, cfe-commits, #sanitizers, llvm-commits, sylvestre.ledru

Tags: #clang, #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D74953
2020-02-24 10:38:33 +01:00
Petr Hosek d3db13af7e [profile] Support counter relocation at runtime
This is an alternative to the continous mode that was implemented in
D68351. This mode relies on padding and the ability to mmap a file over
the existing mapping which is generally only available on POSIX systems
and isn't suitable for other platforms.

This change instead introduces the ability to relocate counters at
runtime using a level of indirection. On every counter access, we add a
bias to the counter address. This bias is stored in a symbol that's
provided by the profile runtime and is initially set to zero, meaning no
relocation. The runtime can mmap the profile into memory at abitrary
location, and set bias to the offset between the original and the new
counter location, at which point every subsequent counter access will be
to the new location, which allows updating profile directly akin to the
continous mode.

The advantage of this implementation is that doesn't require any special
OS support. The disadvantage is the extra overhead due to additional
instructions required for each counter access (overhead both in terms of
binary size and performance) plus duplication of counters (i.e. one copy
in the binary itself and another copy that's mmapped).

Differential Revision: https://reviews.llvm.org/D69740
2020-01-17 15:02:23 -08:00
Fangrui Song c5b94ea265 [profile] Support merge pool size >= 10
The executable acquires an advisory record lock (`fcntl(fd, F_SETLKW, *)`) on a profile file.
Merge pool size >= 10 may be beneficial when the concurrency is large.

Also fix a small problem about snprintf. It can cause the filename to be truncated after %m.

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D71970
2020-01-12 00:27:18 -08:00
Calixte Denizet 02ce9d8ef5 [compiler-rt] Add a critical section when flushing gcov counters
Summary:
Counters can be flushed in a multi-threaded context for example when the process is forked in different threads (https://github.com/llvm/llvm-project/blob/master/llvm/lib/Transforms/Instrumentation/GCOVProfiling.cpp#L632-L663).
In order to avoid pretty bad things, a critical section is needed around the flush.
We had a lot of crashes in this code in Firefox CI when we switched to clang for linux ccov builds and those crashes disappeared with this patch.

Reviewers: marco-c, froydnj, dmajor, davidxl, vsk

Reviewed By: marco-c, dmajor

Subscribers: ahatanak, froydnj, dmajor, dberris, jfb, #sanitizers, llvm-commits, sylvestre.ledru

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70910
2019-12-12 09:23:32 +01:00
Vedant Kumar 5a486e0f15 [profile] Avoid allocating a page on the stack, NFC
When writing out a profile, avoid allocating a page on the stack for the
purpose of writing out zeroes, as some embedded environments do not have
enough stack space to accomodate this.

Instead, use a small, fixed-size zero buffer that can be written
repeatedly.

For a synthetic file with >100,000 functions, I did not measure a
significant difference in profile write times. We are removing a
page-length zero-fill `memset()` in favor of several smaller buffered
`fwrite()` calls: in practice, I am not sure there is much of a
difference. The performance impact is only expected to affect the
continuous sync mode (%c) -- zero padding is less than 8 bytes in all
other cases.

rdar://57810014

Differential Revision: https://reviews.llvm.org/D71323
2019-12-11 10:04:00 -08:00
Akira Hatanaka 78a7af456d Revert "[compiler-rt] Add a critical section when flushing gcov counters"
This reverts commit 88f5bf77f9 as it broke
green dragon bots.

http://lab.llvm.org:8080/green/job/clang-stage1-RA/4401/
2019-12-09 19:58:26 -08:00
Calixte Denizet 88f5bf77f9 [compiler-rt] Add a critical section when flushing gcov counters
Summary:
Counters can be flushed in a multi-threaded context for example when the process is forked in different threads (https://github.com/llvm/llvm-project/blob/master/llvm/lib/Transforms/Instrumentation/GCOVProfiling.cpp#L632-L663).
In order to avoid pretty bad things, a critical section is needed around the flush.
We had a lot of crashes in this code in Firefox CI when we switched to clang for linux ccov builds and those crashes disappeared with this patch.

Reviewers: marco-c, froydnj, dmajor, davidxl

Reviewed By: marco-c, dmajor

Subscribers: froydnj, dmajor, dberris, jfb, #sanitizers, llvm-commits, sylvestre.ledru

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70910
2019-12-09 10:39:55 +01:00
Hans Wennborg 900d8a9a3b [profile] Fix file contention causing dropped counts on Windows under -fprofile-generate
See PR43425:
https://bugs.llvm.org/show_bug.cgi?id=43425

When writing profile data on Windows we were opening profile file with
exclusive read/write access.

In case we are trying to write to the file from multiple processes
simultaneously, subsequent calls to CreateFileA would return
INVALID_HANDLE_VALUE.

To fix this, I changed to open without exclusive access and then take a
lock.

Patch by Michael Holman!

Differential revision: https://reviews.llvm.org/D70330
2019-11-27 15:55:13 +01:00
Petr Hosek f35032e03d Reland "[CMake] Support installation of InstrProfData.inc"
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.

Differential Revision: https://reviews.llvm.org/D64532
2019-11-22 14:09:46 -08:00
Petr Hosek 262b10ba1f Revert "[CMake] Support installation of InstrProfData.inc"
This reverts commit f11bc1776f since it's
failing to build on some bots.
2019-11-22 12:00:23 -08:00
Petr Hosek f11bc1776f [CMake] Support installation of InstrProfData.inc
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.

Differential Revision: https://reviews.llvm.org/D64532
2019-11-22 11:52:50 -08:00
Vedant Kumar 0d4211f4e7 [profile] Address unused function warnings on Windows after D69586
This '#ifdef's out two functions which are unused on Windows, to prevent
-Wunused-function warnings.
2019-11-19 12:51:12 -08:00
Vedant Kumar 1aacf58819 [profile] Unbreak Fuchsia/Windows after D68351
Continuous mode is not yet supported on Fuchsia/Windows, however an
error should not be reported unless the user attempted to actually
enable continuous mode.
2019-11-19 12:29:57 -08:00
Vedant Kumar 2492b5a125 [profile] Support online merging with continuous sync mode
Make it possible to use online profile merging ("%m" mode) with
continuous sync ("%c" mode).

To implement this, the merged profile is locked in the runtime
initialization step and either a) filled out for the first time or b)
checked for compatibility. Then, the profile can simply be mmap()'d with
MAP_SHARED set. With the mmap() in place, counter updates from every
process which uses an image are mapped onto the same set of physical
pages assigned by the filesystem cache. After the mmap() is set up, the
profile is unlocked.

Differential Revision: https://reviews.llvm.org/D69586
2019-11-18 12:56:58 -08:00
Vedant Kumar e7aab320b1 [profile] Factor out logic for mmap'ing merged profile, NFC
Split out the logic to get the size of a merged profile and to do a
compatibility check. This can be shared with both the continuous+merging
mode implementation, as well as the runtime-allocated counters
implementation planned for Fuchsia.

Lifted out of D69586.

Differential Revision: https://reviews.llvm.org/D70135
2019-11-13 07:52:15 -08:00
Douglas Yung 7ebde1bf67 Add a shim for setenv on PS4 since it does not exist.
A few years back a similar change was made for getenv since neither function is supported on the PS4 platform.

Recently, commit d889d1e added a call to setenv in compiler-rt which was causing linking errors because the symbol was not found. This fixes that issue by putting in a shim similar to how we previously dealt with the lack of getenv.

Differential Revision: https://reviews.llvm.org/D70033
2019-11-12 15:05:45 -08:00
Martin Storsjö ec630194fe [compiler-rt] [profile] Fix building for MinGW after d889d1efef
This commit added use of a Windows API in InstrProfilingPort.h.
When _MSC_VER is defined (for MSVC), windows.h is already included
earlier in the same header (for atomics), but MinGW, the gcc
atomics builtins are used instead. Therefore explicitly include
windows.h here, where the API is used.
2019-11-01 09:20:51 +02:00
Vedant Kumar 197bad5089 [profile] Fifth speculative fix for Android after D68351
Use the printf macros from inttypes.h to sidestep -Wformat issues:

/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:425:14: error: format specifies type 'long long' but the argument has type 'off_t' (aka 'long') [-Werror,-Wformat]
             CurrentFileOffset, PageSize);
             ^~~~~~~~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:114:50: note: expanded from macro 'PROF_ERR'
  fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
                                         ~~~~~~  ^~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:461:41: error: format specifies type 'unsigned long long' but the argument has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
        strerror(errno), CountersBegin, PageAlignedCountersLength, Fileno,
                                        ^~~~~~~~~~~~~~~~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:114:50: note: expanded from macro 'PROF_ERR'
  fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
                                         ~~~~~~  ^~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:462:9: error: format specifies type 'unsigned long long' but the argument has type 'uint64_t' (aka 'unsigned long') [-Werror,-Wformat]
        FileOffsetToCounters);
        ^~~~~~~~~~~~~~~~~~~~
/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:114:50: note: expanded from macro 'PROF_ERR'
  fprintf(stderr, "LLVM Profile Error: " Format, __VA_ARGS__);
2019-10-31 17:03:52 -07:00
Vedant Kumar a49795d8ab [profile] Third speculative fix for Windows after D68351
_putenv on Windows takes 1 argument, whereas setenv elsewhere takes 3.
Just treat the two platforms differently.

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/53547
2019-10-31 16:35:58 -07:00
Vedant Kumar 6dfe8e590e [profile] Second speculative fix for Windows
VLAs in C appear to not work on Windows, so use COMPILER_RT_ALLOCA:

C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingWriter.c(264): error C2057: expected constant expression
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingWriter.c(264): error C2466: cannot allocate an array of constant size 0
C:\b\slave\sanitizer-windows\llvm-project\compiler-rt\lib\profile\InstrProfilingWriter.c(264): error C2133: 'Zeroes': unknown size
2019-10-31 16:25:25 -07:00
Vedant Kumar aa7f3a405d [profile] Speculative fix for Windows after D68351
setenv() appears to not be available on Windows:

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/53545/steps/stage%201%20build/logs/stdio
2019-10-31 16:23:39 -07:00
Vedant Kumar d889d1efef [profile] Add a mode to continuously sync counter updates to a file
Add support for continuously syncing profile counter updates to a file.

The motivation for this is that programs do not always exit cleanly. On
iOS, for example, programs are usually killed via a signal from the OS.
Running atexit() handlers after catching a signal is unreliable, so some
method for progressively writing out profile data is necessary.

The approach taken here is to mmap() the `__llvm_prf_cnts` section onto
a raw profile. To do this, the linker must page-align the counter and
data sections, and the runtime must ensure that counters are mapped to a
page-aligned offset within a raw profile.

Continuous mode is (for the moment) incompatible with the online merging
mode. This limitation is lifted in https://reviews.llvm.org/D69586.

Continuous mode is also (for the moment) incompatible with value
profiling, as I'm not sure whether there is interest in this and the
implementation may be tricky.

As I have not been able to test extensively on non-Darwin platforms,
only Darwin support is included for the moment. However, continuous mode
may "just work" without modification on Linux and some UNIX-likes. AIUI
the default value for the GNU linker's `--section-alignment` flag is set
to the page size on many systems. This appears to be true for LLD as
well, as its `no_nmagic` option is on by default. Continuous mode will
not "just work" on Fuchsia or Windows, as it's not possible to mmap() a
section on these platforms. There is a proposal to add a layer of
indirection to the profile instrumentation to support these platforms.

rdar://54210980

Differential Revision: https://reviews.llvm.org/D68351
2019-10-31 16:04:09 -07:00
Vedant Kumar 937241b0d9 [profile] Do not cache __llvm_profile_get_filename result
When the %m filename pattern is used, the filename is unique to each
image, so the cached value is wrong.

It struck me that the full filename isn't something that's recomputed
often, so perhaps it doesn't need to be cached at all. David Li pointed
out we can go further and just hide lprofCurFilename. This may regress
workflows that depend on using the set-filename API to change filenames
across all loaded DSOs, but this is expected to be very rare.

rdar://55137071

Differential Revision: https://reviews.llvm.org/D69137

llvm-svn: 375301
2019-10-18 23:33:40 +00:00
Dmitri Gribenko 57256af307 Revert "clang-misexpect: Profile Guided Validation of Performance Annotations in LLVM"
This reverts commit r371584. It introduced a dependency from compiler-rt
to llvm/include/ADT, which is problematic for multiple reasons.

One is that it is a novel dependency edge, which needs cross-compliation
machinery for llvm/include/ADT (yes, it is true that right now
compiler-rt included only header-only libraries, however, if we allow
compiler-rt to depend on anything from ADT, other libraries will
eventually get used).

Secondly, depending on ADT from compiler-rt exposes ADT symbols from
compiler-rt, which would cause ODR violations when Clang is built with
the profile library.

llvm-svn: 371598
2019-09-11 09:16:17 +00:00
Petr Hosek 394a8ed8f1 clang-misexpect: Profile Guided Validation of Performance Annotations in LLVM
This patch contains the basic functionality for reporting potentially
incorrect usage of __builtin_expect() by comparing the developer's
annotation against a collected PGO profile. A more detailed proposal and
discussion appears on the CFE-dev mailing list
(http://lists.llvm.org/pipermail/cfe-dev/2019-July/062971.html) and a
prototype of the initial frontend changes appear here in D65300

We revised the work in D65300 by moving the misexpect check into the
LLVM backend, and adding support for IR and sampling based profiles, in
addition to frontend instrumentation.

We add new misexpect metadata tags to those instructions directly
influenced by the llvm.expect intrinsic (branch, switch, and select)
when lowering the intrinsics. The misexpect metadata contains
information about the expected target of the intrinsic so that we can
check against the correct PGO counter when emitting diagnostics, and the
compiler's values for the LikelyBranchWeight and UnlikelyBranchWeight.
We use these branch weight values to determine when to emit the
diagnostic to the user.

A future patch should address the comment at the top of
LowerExpectIntrisic.cpp to hoist the LikelyBranchWeight and
UnlikelyBranchWeight values into a shared space that can be accessed
outside of the LowerExpectIntrinsic pass. Once that is done, the
misexpect metadata can be updated to be smaller.

In the long term, it is possible to reconstruct portions of the
misexpect metadata from the existing profile data. However, we have
avoided this to keep the code simple, and because some kind of metadata
tag will be required to identify which branch/switch/select instructions
are influenced by the use of llvm.expect

Patch By: paulkirth
Differential Revision: https://reviews.llvm.org/D66324

llvm-svn: 371584
2019-09-11 01:09:16 +00:00
Nico Weber 98f800dabe compiler-rt: Fix warning if COMPILER_RT_HAS_FCNTL_LCK is 0
Fixes "warning: implicit declaration of function 'flock' is invalid in C99"
for flock().

llvm-svn: 369534
2019-08-21 15:24:25 +00:00
Petr Hosek edf504f503 [Fuchsia] Create the VMO during initialization, not during exit
We want to avoid doing expensive work during atexit since the process
might be terminated before we can publish the VMO and write out the
symbolizer markup, so move the VMO creation to the initialization
phase and only write data during the atexit phase.

Differential Revision: https://reviews.llvm.org/D66323

llvm-svn: 369180
2019-08-17 00:54:22 +00:00
Nico Weber 62a0585530 compiler-rt: Rename .cc file in lib/profile to .cpp
See https://reviews.llvm.org/D58620 for discussion.

Note how the comment in the file already said ".cpp" :)

llvm-svn: 367460
2019-07-31 18:21:08 +00:00
Nico Weber b206c3e3e4 gn build: Add build files for compiler-rt/lib/profile
Differential Revision: https://reviews.llvm.org/D65518

llvm-svn: 367450
2019-07-31 17:15:32 +00:00
Pirama Arumuga Nainar 6caa8da072 [profile] In Android, do not mkdir() dirs in GCOV_PREFIX
Summary:
In Android, attempting to mkdir() or even stat() top-level directories
like /data causes noisy selinux denials.  During whole-system coverage
instrumentation, this causes a deluge of noisy messages that drown out
legitimate selinux denials, that should be audited and fixed.

To avoid this, skip creating any directory in GCOV_PREFIX (thereby
assuming that it exists).

- Android platform ensures that the GCOV_PREFIX used in Android is
created and read/writable by all processes.
- This only affects the Android platform (by checking against
__ANDROID_API_FUTURE__) and for apps built with Clang coverage, the
runtime will still create any non-existent parent directories for the
coverage files.

Reviewers: srhines, davidxl

Subscribers: krytarowski, #sanitizers, danalbert, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D65245

llvm-svn: 367064
2019-07-25 22:10:56 +00:00
Sajjad Mirza 6694b2b36b (Reland with changes) Adding a function for setting coverage output file.
Summary:
User code can open a file on its own and pass it to the runtime, rather than
specifying a name and having the runtime open the file. This supports the use
case where a process cannot open a file on its own but can receive a file
descriptor from another process.

Relanding https://reviews.llvm.org/D62541. The original revision unlocked
the file before calling flush, this revision fixes that.

Reviewers: Dor1s, davidxl

Reviewed By: Dor1s

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63581

llvm-svn: 364231
2019-06-24 21:32:50 +00:00
Hans Wennborg 05d44139ee Revert r362676 "[Profile]: Add runtime interface to specify file handle for profile data."
This caused instrumented Clang to become crashy. See llvm-commits thread
for repro steps.

This also reverts follow-up r362716 which added test cases.

> Author: Sajjad Mirza
>
> Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 363134
2019-06-12 08:44:32 +00:00
Xinliang David Li c1867557d9 [Profile]: Add runtime interface to specify file handle for profile data.
Author: Sajjad Mirza

Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 362676
2019-06-06 06:35:18 +00:00
Max Moroz 52fa90a348 This change adds an API to allow setting the flag to indicate that the profile data has been dumped to the file.
Summary:
The main use is for users to disable dumping profile data to the file
for certain processes in case the processes don't have permission to
write to the disks, and trying to do so would result in side effects
such as crashes.

Patch by Yuke Liao (@liaoyuke).

Additional context (Chromium use case):
- https://bugs.chromium.org/p/chromium/issues/detail?id=842424
- https://bugs.chromium.org/p/chromium/issues/detail?id=957655
- https://chromium-review.googlesource.com/c/chromium/src/+/1610093

Reviewers: Dor1s, vsk, davidxl

Reviewed By: Dor1s, davidxl

Subscribers: delcypher, davidxl, sajjadm, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62078

llvm-svn: 361194
2019-05-20 20:02:20 +00:00
Nico Weber 37dc0ced7a Fix some gcc warnings in compiler-rt
- Several "warning: extra ';' [-Wpedantic]"
- One "C++ style comments are not allowed in ISO C90 [enabled by default]"
  in a file that uses C style comments everywhere but in one place

llvm-svn: 360430
2019-05-10 14:15:13 +00:00