This reverts commit r371584. It introduced a dependency from compiler-rt
to llvm/include/ADT, which is problematic for multiple reasons.
One is that it is a novel dependency edge, which needs cross-compliation
machinery for llvm/include/ADT (yes, it is true that right now
compiler-rt included only header-only libraries, however, if we allow
compiler-rt to depend on anything from ADT, other libraries will
eventually get used).
Secondly, depending on ADT from compiler-rt exposes ADT symbols from
compiler-rt, which would cause ODR violations when Clang is built with
the profile library.
llvm-svn: 371598
This patch contains the basic functionality for reporting potentially
incorrect usage of __builtin_expect() by comparing the developer's
annotation against a collected PGO profile. A more detailed proposal and
discussion appears on the CFE-dev mailing list
(http://lists.llvm.org/pipermail/cfe-dev/2019-July/062971.html) and a
prototype of the initial frontend changes appear here in D65300
We revised the work in D65300 by moving the misexpect check into the
LLVM backend, and adding support for IR and sampling based profiles, in
addition to frontend instrumentation.
We add new misexpect metadata tags to those instructions directly
influenced by the llvm.expect intrinsic (branch, switch, and select)
when lowering the intrinsics. The misexpect metadata contains
information about the expected target of the intrinsic so that we can
check against the correct PGO counter when emitting diagnostics, and the
compiler's values for the LikelyBranchWeight and UnlikelyBranchWeight.
We use these branch weight values to determine when to emit the
diagnostic to the user.
A future patch should address the comment at the top of
LowerExpectIntrisic.cpp to hoist the LikelyBranchWeight and
UnlikelyBranchWeight values into a shared space that can be accessed
outside of the LowerExpectIntrinsic pass. Once that is done, the
misexpect metadata can be updated to be smaller.
In the long term, it is possible to reconstruct portions of the
misexpect metadata from the existing profile data. However, we have
avoided this to keep the code simple, and because some kind of metadata
tag will be required to identify which branch/switch/select instructions
are influenced by the use of llvm.expect
Patch By: paulkirth
Differential Revision: https://reviews.llvm.org/D66324
llvm-svn: 371584
We want to avoid doing expensive work during atexit since the process
might be terminated before we can publish the VMO and write out the
symbolizer markup, so move the VMO creation to the initialization
phase and only write data during the atexit phase.
Differential Revision: https://reviews.llvm.org/D66323
llvm-svn: 369180
Summary:
In Android, attempting to mkdir() or even stat() top-level directories
like /data causes noisy selinux denials. During whole-system coverage
instrumentation, this causes a deluge of noisy messages that drown out
legitimate selinux denials, that should be audited and fixed.
To avoid this, skip creating any directory in GCOV_PREFIX (thereby
assuming that it exists).
- Android platform ensures that the GCOV_PREFIX used in Android is
created and read/writable by all processes.
- This only affects the Android platform (by checking against
__ANDROID_API_FUTURE__) and for apps built with Clang coverage, the
runtime will still create any non-existent parent directories for the
coverage files.
Reviewers: srhines, davidxl
Subscribers: krytarowski, #sanitizers, danalbert, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D65245
llvm-svn: 367064
Summary:
User code can open a file on its own and pass it to the runtime, rather than
specifying a name and having the runtime open the file. This supports the use
case where a process cannot open a file on its own but can receive a file
descriptor from another process.
Relanding https://reviews.llvm.org/D62541. The original revision unlocked
the file before calling flush, this revision fixes that.
Reviewers: Dor1s, davidxl
Reviewed By: Dor1s
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D63581
llvm-svn: 364231
This caused instrumented Clang to become crashy. See llvm-commits thread
for repro steps.
This also reverts follow-up r362716 which added test cases.
> Author: Sajjad Mirza
>
> Differential Revision: http://reviews.llvm.org/D62541
llvm-svn: 363134
- Several "warning: extra ';' [-Wpedantic]"
- One "C++ style comments are not allowed in ISO C90 [enabled by default]"
in a file that uses C style comments everywhere but in one place
llvm-svn: 360430
Currently VMO in Zircon create using the zx_vmo_create is resizable
by default, but we'll be changing this in the future, requiring an
explicit flag to make the VMO resizable.
Prepare for this change by passing ZX_VMO_RESIZABLE option to all
zx_vmo_create calls that need resizable VMO.
Differential Revision: https://reviews.llvm.org/D61450
llvm-svn: 359803
r355343 was landed and was reverted in r355363 due to build breakage.
This patch adds Linux/Windows support on top of r355343.
In this patch, Darwin should be working with testing case. Linux should be working,
I will enable the testing case in a follwup diff. Windows/Other should be building.
Correct implementation for Other platforms will be added.
Thanks David for reviewing the original diff, helping me with issues on Linux, and
giving suggestions for adding support for Other platforms.
llvm-svn: 355701
Summary:
The llvm-cov tool needs to be able to find coverage names in the
executable, so the .lprfn and .lcovmap sections cannot be merged into
.rdata.
Also, the linker merges .lprfn$M into .lprfn, so llvm-cov needs to
handle that when looking up sections. It has to support running on both
relocatable object files and linked PE files.
Lastly, when loading .lprfn from a PE file, llvm-cov needs to skip the
leading zero byte added by the profile runtime.
Reviewers: vsk
Subscribers: hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58661
llvm-svn: 354840
Summary:
The changes in https://reviews.llvm.org/D44847 cause load time failure
due to lprofMergeValueProfData in Android libs enabled with profile generation:
"dlopen failed: cannot locate symbol "lprofMergeValueProfData" referenced by..."
Marking lprofMergeValueProfData as hidden so the correct in-module definition
is picked by the linker.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: efriedma, xur, davidxl, llvm-commits
Differential Revision: https://reviews.llvm.org/D55893
llvm-svn: 354064
This function doesn't use anything MSVC specific but works fine
for any _WIN32 target.
Differential Revision: https://reviews.llvm.org/D58106
llvm-svn: 353918
Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.
Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.
Now, Windows uses static registration and is in line with all the other
platforms.
Reviewers: davidxl, wmi, inglorion, void, calixte
Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57929
llvm-svn: 353547
There is no reason for these sections to remain separate in the final
DLL or EXE.
I have not yet added a InstrProfilingPlatformWindows.c for these, since
avoiding dynamic profile data registration is a larger project for
later.
llvm-svn: 353221
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
NetBSD uses typical UNIX interfaces.
All tests pass except instrprof-dlopen-dlclose-gcov.test, as there
is not supported semantics of atexit(3) in dlopen(3)ed+dlclose(3)d
DSO.
NetBSD also ships an older version of LLVM profile (ABI v.2 predating
ABI v.4 in upstream version) inside libc. That copy has been manually
removed during the porting and testing process of the upstream version
to NetBSD. Otherwise there were conflicts between them two.
Reviewers: joerg, vitalybuka, vsk
Subscribers: srhines, fedor.sergeev, llvm-commits, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D55997
llvm-svn: 349994
Summary: This is just a minor cleanup to the allocateOneNode interface. The formals are no-longer used, so I just removed them.
Reviewers: davidxl, void
Reviewed By: davidxl
Subscribers: dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D52509
llvm-svn: 344073
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase. This header actually also
is named with a lowercase name in the Windows SDK as well.
Differential Revision: https://reviews.llvm.org/D51877
llvm-svn: 341857
Summary:
This patch introduces `llvm_profile_set_dir_mode` and `llvm_profile_get_dir_mode` to
the compiler-rt profile API.
Originally, profile data was placed into a directory that was created with a hard-coded
mode value of 0755 (for non-win32 builds). In certain cases, it can be helpful to create
directories with a different mode other than 0755. This patch introduces set/get
routines to allow users to specify a desired mode. The default remains at 0755.
Reviewers: void, probinson
Reviewed By: probinson
Subscribers: probinson, dberris, cfe-commits
Differential Revision: https://reviews.llvm.org/D49953
llvm-svn: 338456
This ports the profiling runtime on Fuchsia and enables the
instrumentation. Unlike on other platforms, Fuchsia doesn't use
files to dump the instrumentation data since on Fuchsia, filesystem
may not be accessible to the instrumented process. We instead use
the data sink to pass the profiling data to the system the same
sanitizer runtimes do.
Differential Revision: https://reviews.llvm.org/D47208
llvm-svn: 337881
Until now, our code preferred backslashes to slashes, whereas Windows
allows using both types of directory separators in one path string.
Differential Revision: https://reviews.llvm.org/D49664
llvm-svn: 337826
Summary:
Add __llvm_profile_get_filename interface to get the profile filename,
which can be used for identifying which profile file belongs to an app
when multiple binaries are instrumented and dumping profiles into the
same directory. The filename includes the path.
Reviewers: davidxl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49529
llvm-svn: 337482
Summary:
The write buffer contains signed chars, which means the shift operations caused values such as the arc tag value (0x01a10000) to be read incorrectly (0xffa10000).
This fixes a regression from https://reviews.llvm.org/D49132.
Reviewers: uweigand, davidxl
Reviewed By: uweigand
Subscribers: llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49161
llvm-svn: 336775
Two fixes required to handle big-endian systems:
- 64-bit counter values are stored in a mixed-endian format in the
gcov files: a 32-bit low-part followed by a 32-bit high part. Note that
this is already implemented correctly on the LLVM side, see
GCOVBuffer::readInt64.
- The tag values (e.g. arcs tag, object summary tag, ...) are aways
written as the same sequence of bytes independent of byte order. But
when *reading* them back in, the code reads them as 32-bit values in
host byte order. For the comparisons to work correctly, this should
instead always read them as little-endian values.
Fixes PR 38121.
Reviewed By: marco-c
Differential Revision: https://reviews.llvm.org/D49132
llvm-svn: 336693
when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.
To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.
The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.
The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).
The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:
* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
avoid because it would include a bunch of stuff we don't need.
Differential Revision: https://reviews.llvm.org/D48422
llvm-svn: 336663
Summary:
This will make the behavior of __gcov_flush match the GCC behavior.
I would like to rename __gcov_flush to __llvm_gcov_flush (in case of programs linking to libraries built with different compilers), but I guess we can't for compatibility reasons.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: samsonov, vitalybuka, pcc, kcc, junbuml, glider, fhahn, eugenis, dvyukov, davidxl, srhines, chh, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48538
llvm-svn: 336365
__gcov_flush is hidden.
For applications to dump profiling data of selected .so files,
they can use dlsym to find and call llvm_gcov_flush in each .so file.
Differential Revision: https://reviews.llvm.org/D45454
llvm-svn: 336019
Fuchsia is no longer treated as UNIX which means we need to explicitly
enable building of shared versions of runtimes.
Differential Revision: https://reviews.llvm.org/D46609
llvm-svn: 331922
The profile library was missing some includes and was erroneously using
ftruncate. WinASan was using `= {0}` to initialize structs, which
creates -Wmissing-field-initializers and -Wmissing-braces warnings with
clang. Use `= {}` instead, since this is C++.
llvm-svn: 330616
This patch fixes the following issues:
(1) The strong definition of the merge hook function was not working which
breaks the online value profile merging. This patch removes the weak
attribute of VPMergeHook and assigns the value dynamically.
(2) Truncate the proifle file so that we don't have garbage data at the end of
the file.
(3) Add new __llvm_profile_instrument_target_value() interface to do the value
profile update in batch. This is needed as the original incremental by 1
in __llvm_profile_instrument_target() is too slow for online merge.
Differential Revision: https://reviews.llvm.org/D44847
llvm-svn: 328987
This includes a few nice bits of refactoring (e.g splitting out the
exclusive locking code into a common utility).
Hopefully the Windows support is fixed now.
Patch by Rainer Orth!
Differential Revision: https://reviews.llvm.org/D40944
llvm-svn: 320731
This includes a few nice bits of refactoring (e.g splitting out the
exclusive locking code into a common utility).
Patch by Rainer Orth!
Differential Revision: https://reviews.llvm.org/D40944
llvm-svn: 320726
Summary:
This patch implements flock for Windows, needed to make gcda writing work in a multiprocessing scenario.
Fixes https://bugs.llvm.org/show_bug.cgi?id=34923.
Reviewers: zturner
Reviewed By: zturner
Subscribers: rnk, zturner, llvm-commits
Differential Revision: https://reviews.llvm.org/D38891
llvm-svn: 317705
Summary:
Fixes https://bugs.llvm.org/show_bug.cgi?id=34922.
Apparently, the mode in **fdopen** gets simply ignored and Windows only cares about the mode of the original **open**.
I have verified this both with the simple case from bug 34922 and with a full Firefox build.
Reviewers: zturner
Reviewed By: zturner
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38984
llvm-svn: 316048
The buildbots have shown that -Wstrict-prototypes behaves differently in GCC
and Clang so we should keep it disabled until Clang follows GCC's behaviour
llvm-svn: 312246
Clang 5 supports -Wstrict-prototypes. We should use it to catch any C
declarations that declare a non-prototype function.
rdar://33705313
Differential Revision: https://reviews.llvm.org/D36669
llvm-svn: 312240
Summary: In the current implementation, the defaul number of values per site tracked by value profiler is 8, which is too small and could introduce inaccuracies to profile. Changing it to 16 will be able to gain more accurate value profiler.
Reviewers: davidxl, tejohnson
Reviewed By: tejohnson
Subscribers: sanjoy, llvm-commits
Differential Revision: https://reviews.llvm.org/D35964
llvm-svn: 309388
Users can specify the path a raw profile is written to by passing
-fprofile-instr-generate=<path>, but this functionality broke on Darwin
after __llvm_profile_filename was made weak [1], resulting in profiles
being written to "default.profraw" even when <path> is specified.
The situation is that instrumented programs provide a weak definition of
__llvm_profile_filename, which conflicts with a weak redefinition
provided by the profiling runtime.
The linker appears to pick the 'winning' definition arbitrarily: on
Darwin, it usually prefers the larger definition, which is probably why
the instrprof-override-filename.c test has been passing.
The fix is to move the runtime's definition into a separate object file
within the archive. This means that the linker won't "see" the runtime's
definition unless the user program has not provided one. I couldn't
think of a great way to test this other than to mimic the Darwin
failure: use -fprofile-instr-generate=<some-small-path>.
Testing: check-{clang,profile}, modified instrprof-override-filename.c.
[1] [Profile] deprecate __llvm_profile_override_default_filename
https://reviews.llvm.org/D22613https://reviews.llvm.org/D22614
Differential Revision: https://reviews.llvm.org/D34797
llvm-svn: 306710
Introduces a 'owner' struct to include the overridable write
method and the write context in C.
This allows easy introdution of new member API to help reduce
profile merge time in the follow up patch.
llvm-svn: 306432
And also r295364 [PGO] remove unintended debug trace. NFC
I removed the test case: it's hard to write synchronized test b/w processes
in this framework. I will revisit the test-case later.
llvm-svn: 298113
This patch adds profile run time support to profile a range of values.
This interface will be used in profiling the size of memory intrinsic calls.
Differential Revision: http://reviews.llvm.org/D28964
llvm-svn: 297895
Summary:
We found a nondeterministic behavior when doing online profile merging
for multi-process applications. The application forks a sub-process and
sub-process sets to get SIGKILL when the parent process exits,
The first process gets the lock, and dumps the profile. The second one
will mmap the file, do the merge and write out the file. Note that before
the merged write, we truncate the profile.
Depending on the timing, the child process might be terminated
abnormally when the parent exits first. If this happens:
(1) before the truncation, we will get the profile for the main process
(2) after the truncation, and before write-out the profile, we will get
0 size profile.
(3) after the merged write, we get merged profile.
This patch temporarily suspend the SIGKILL for PR_SET_PDEATHSIG
before profile-write and restore it after the write.
This patch only applies to Linux system.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: xur, llvm-commits
Differential Revision: https://reviews.llvm.org/D29954
llvm-svn: 295364
Summary: In profile data paths, we replace "%h" with the hostname of the machine the program is running on. On Windows, we used gethostname() to obtain the hostname. This requires linking with ws2_32. With this change, we instead get the hostname from GetComputerNameExW(), which does not require ws2_32.
Reviewers: rnk, vsk, amccarth
Subscribers: zturner, ruiu, hans
Differential Revision: https://reviews.llvm.org/D27178
llvm-svn: 288146
This makes __llvm_profile_set_filename() work across dylib boundaries on
Darwin.
This functionality was originally meant to work on all platforms, but
was moved to a Linux-only directory with r272404. The root cause of the
test failure on Darwin was that lprofCurFilename was not marked weak.
Each dylib maintained its own copy of the variable due to the two-level
namespace.
Tested with check-profile (on Darwin). I don't expect this to regress
other platforms.
Differential Revision: https://reviews.llvm.org/D25707
llvm-svn: 284440
This reverts commit r282294. It breaks a Linux bot:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-42vma/builds/12180
It looks like the test checks that __llvm_profile_set_filename() alters the raw
profile filename in both the dylib and the main program. Now that
lprofCurFilename is hidden, this can't work, and we get two profiles (one for
the call to "main" and one for "func").
Back this change out so that we don't affect external users.
llvm-svn: 282304
This patch builds on LLVM r279776.
In this patch I've done some cleanup and abstracted three common steps runtime components have in their CMakeLists files, and added a fourth.
The three steps I abstract are:
(1) Add a top-level target (i.e asan, msan, ...)
(2) Set the target properties for sorting files in IDE generators
(3) Make the compiler-rt target depend on the top-level target
The new step is to check if a command named "runtime_register_component" is defined, and to call it with the component name.
The runtime_register_component command is defined in llvm/runtimes/CMakeLists.txt, and presently just adds the component to a list of sub-components, which later gets used to generate target mappings.
With this patch a new workflow for runtimes builds is supported. The new workflow when building runtimes from the LLVM runtimes directory is:
> cmake [...]
> ninja runtimes-configure
> ninja asan
The "runtimes-configure" target builds all the dependencies for configuring the runtimes projects, and runs CMake on the runtimes projects. Running the runtimes CMake generates a list of targets to bind into the top-level CMake so subsequent build invocations will have access to some of Compiler-RT's targets through the top-level build.
Note: This patch does exclude some top-level targets from compiler-rt libraries because they either don't install files (sanitizer_common), or don't have a cooresponding `check` target (stats).
llvm-svn: 279863
The API is intended to be used by user to do fine
grained (per-region) control of profile dumping.
Differential Revision: http://reviews.llvm.org/D23106
llvm-svn: 278092
Dmitri Gribenko