Commit Graph

78 Commits

Author SHA1 Message Date
Petr Hosek 32bddad37b [profile] Make atexit hook a no-op on Fuchsia
On Fuchsia, we always use the continuous mode with runtime counter
relocation, so there's no need for atexit hook or support for dumping
the profile manually.

Differential Revision: https://reviews.llvm.org/D76556
2020-03-24 18:47:38 -07:00
Petr Hosek d3db13af7e [profile] Support counter relocation at runtime
This is an alternative to the continous mode that was implemented in
D68351. This mode relies on padding and the ability to mmap a file over
the existing mapping which is generally only available on POSIX systems
and isn't suitable for other platforms.

This change instead introduces the ability to relocate counters at
runtime using a level of indirection. On every counter access, we add a
bias to the counter address. This bias is stored in a symbol that's
provided by the profile runtime and is initially set to zero, meaning no
relocation. The runtime can mmap the profile into memory at abitrary
location, and set bias to the offset between the original and the new
counter location, at which point every subsequent counter access will be
to the new location, which allows updating profile directly akin to the
continous mode.

The advantage of this implementation is that doesn't require any special
OS support. The disadvantage is the extra overhead due to additional
instructions required for each counter access (overhead both in terms of
binary size and performance) plus duplication of counters (i.e. one copy
in the binary itself and another copy that's mmapped).

Differential Revision: https://reviews.llvm.org/D69740
2020-01-17 15:02:23 -08:00
Petr Hosek f35032e03d Reland "[CMake] Support installation of InstrProfData.inc"
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.

Differential Revision: https://reviews.llvm.org/D64532
2019-11-22 14:09:46 -08:00
Petr Hosek 262b10ba1f Revert "[CMake] Support installation of InstrProfData.inc"
This reverts commit f11bc1776f since it's
failing to build on some bots.
2019-11-22 12:00:23 -08:00
Petr Hosek f11bc1776f [CMake] Support installation of InstrProfData.inc
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.

Differential Revision: https://reviews.llvm.org/D64532
2019-11-22 11:52:50 -08:00
Vedant Kumar d889d1efef [profile] Add a mode to continuously sync counter updates to a file
Add support for continuously syncing profile counter updates to a file.

The motivation for this is that programs do not always exit cleanly. On
iOS, for example, programs are usually killed via a signal from the OS.
Running atexit() handlers after catching a signal is unreliable, so some
method for progressively writing out profile data is necessary.

The approach taken here is to mmap() the `__llvm_prf_cnts` section onto
a raw profile. To do this, the linker must page-align the counter and
data sections, and the runtime must ensure that counters are mapped to a
page-aligned offset within a raw profile.

Continuous mode is (for the moment) incompatible with the online merging
mode. This limitation is lifted in https://reviews.llvm.org/D69586.

Continuous mode is also (for the moment) incompatible with value
profiling, as I'm not sure whether there is interest in this and the
implementation may be tricky.

As I have not been able to test extensively on non-Darwin platforms,
only Darwin support is included for the moment. However, continuous mode
may "just work" without modification on Linux and some UNIX-likes. AIUI
the default value for the GNU linker's `--section-alignment` flag is set
to the page size on many systems. This appears to be true for LLD as
well, as its `no_nmagic` option is on by default. Continuous mode will
not "just work" on Fuchsia or Windows, as it's not possible to mmap() a
section on these platforms. There is a proposal to add a layer of
indirection to the profile instrumentation to support these platforms.

rdar://54210980

Differential Revision: https://reviews.llvm.org/D68351
2019-10-31 16:04:09 -07:00
Vedant Kumar 937241b0d9 [profile] Do not cache __llvm_profile_get_filename result
When the %m filename pattern is used, the filename is unique to each
image, so the cached value is wrong.

It struck me that the full filename isn't something that's recomputed
often, so perhaps it doesn't need to be cached at all. David Li pointed
out we can go further and just hide lprofCurFilename. This may regress
workflows that depend on using the set-filename API to change filenames
across all loaded DSOs, but this is expected to be very rare.

rdar://55137071

Differential Revision: https://reviews.llvm.org/D69137

llvm-svn: 375301
2019-10-18 23:33:40 +00:00
Nico Weber 62a0585530 compiler-rt: Rename .cc file in lib/profile to .cpp
See https://reviews.llvm.org/D58620 for discussion.

Note how the comment in the file already said ".cpp" :)

llvm-svn: 367460
2019-07-31 18:21:08 +00:00
Sajjad Mirza 6694b2b36b (Reland with changes) Adding a function for setting coverage output file.
Summary:
User code can open a file on its own and pass it to the runtime, rather than
specifying a name and having the runtime open the file. This supports the use
case where a process cannot open a file on its own but can receive a file
descriptor from another process.

Relanding https://reviews.llvm.org/D62541. The original revision unlocked
the file before calling flush, this revision fixes that.

Reviewers: Dor1s, davidxl

Reviewed By: Dor1s

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63581

llvm-svn: 364231
2019-06-24 21:32:50 +00:00
Hans Wennborg 05d44139ee Revert r362676 "[Profile]: Add runtime interface to specify file handle for profile data."
This caused instrumented Clang to become crashy. See llvm-commits thread
for repro steps.

This also reverts follow-up r362716 which added test cases.

> Author: Sajjad Mirza
>
> Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 363134
2019-06-12 08:44:32 +00:00
Xinliang David Li c1867557d9 [Profile]: Add runtime interface to specify file handle for profile data.
Author: Sajjad Mirza

Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 362676
2019-06-06 06:35:18 +00:00
Max Moroz 52fa90a348 This change adds an API to allow setting the flag to indicate that the profile data has been dumped to the file.
Summary:
The main use is for users to disable dumping profile data to the file
for certain processes in case the processes don't have permission to
write to the disks, and trying to do so would result in side effects
such as crashes.

Patch by Yuke Liao (@liaoyuke).

Additional context (Chromium use case):
- https://bugs.chromium.org/p/chromium/issues/detail?id=842424
- https://bugs.chromium.org/p/chromium/issues/detail?id=957655
- https://chromium-review.googlesource.com/c/chromium/src/+/1610093

Reviewers: Dor1s, vsk, davidxl

Reviewed By: Dor1s, davidxl

Subscribers: delcypher, davidxl, sajjadm, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62078

llvm-svn: 361194
2019-05-20 20:02:20 +00:00
Dmitry Mikulin 312b5f86b7 The error message for mismatched value sites is very cryptic.
Make it more readable for an average user.

Differential Revision: https://reviews.llvm.org/D60896

llvm-svn: 359043
2019-04-23 22:26:55 +00:00
Manman Ren e73ae9a142 Reland compiler-rt support for order file instrumentation.
r355343 was landed and was reverted in r355363 due to build breakage.
This patch adds Linux/Windows support on top of r355343.

In this patch, Darwin should be working with testing case. Linux should be working,
I will enable the testing case in a follwup diff. Windows/Other should be building.
Correct implementation for Other platforms will be added.

Thanks David for reviewing the original diff, helping me with issues on Linux, and
giving suggestions for adding support for Other platforms.

llvm-svn: 355701
2019-03-08 15:30:56 +00:00
Manman Ren ff4bb36d7c Revert compiler-rt diffs for order file instrumentation to get bot green!
This caused issues on Linux/Windows and other platforms.

r355343 355350 355350

llvm-svn: 355363
2019-03-05 01:21:40 +00:00
Manman Ren 4737abc71c Order File Instrumentation: dump the data in compiler-rt
The profile data will be dumped in a file default_xxx.profraw.order.

Differential Revision: https://reviews.llvm.org/D57530

llvm-svn: 355343
2019-03-04 22:28:38 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Teresa Johnson 73053b221f [profile] Add interface to get profile filename
Summary:
Add __llvm_profile_get_filename interface to get the profile filename,
which can be used for identifying which profile file belongs to an app
when multiple binaries are instrumented and dumping profiles into the
same directory. The filename includes the path.

Reviewers: davidxl

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D49529

llvm-svn: 337482
2018-07-19 19:03:50 +00:00
Rong Xu 95ab7582f0 [profile] Fix value profile runtime merging issues
This patch fixes the following issues:
(1) The strong definition of the merge hook function was not working which
breaks the online value profile merging. This patch removes the weak
attribute of VPMergeHook and assigns the value dynamically.
(2) Truncate the proifle file so that we don't have garbage data at the end of
the file.
(3) Add new __llvm_profile_instrument_target_value() interface to do the value
profile update in batch. This is needed as the original incremental by 1
in __llvm_profile_instrument_target() is too slow for online merge.

Differential Revision: https://reviews.llvm.org/D44847

llvm-svn: 328987
2018-04-02 16:57:00 +00:00
Alex Lorenz 341317fda4 Revert r312240
The buildbots have shown that -Wstrict-prototypes behaves differently in GCC
and Clang so we should keep it disabled until Clang follows GCC's behaviour

llvm-svn: 312246
2017-08-31 15:51:23 +00:00
Alex Lorenz fbac1ae062 Build LLVM with -Wstrict-prototypes enabled
Clang 5 supports -Wstrict-prototypes. We should use it to catch any C
declarations that declare a non-prototype function.

rdar://33705313

Differential Revision: https://reviews.llvm.org/D36669

llvm-svn: 312240
2017-08-31 13:23:24 +00:00
Xinliang David Li 3b2c002c6d [Profile] Implement new API __llvm_profile_dump
The API is intended to be used by user to do fine
grained (per-region) control of profile dumping.

Differential Revision: http://reviews.llvm.org/D23106

llvm-svn: 278092
2016-08-09 04:21:14 +00:00
Xinliang David Li f0e0a74f87 [Profile] cleanup: do not reference name directly of vars shared between rt and llvm
llvm-svn: 276385
2016-07-22 04:08:16 +00:00
Xinliang David Li e953933a9f [Profile] deprecate __llvm_profile_override_default_filename (part2)
This eliminates unncessary calls and init functions.

Differential Revision: http://reviews.llvm.org/D22614

llvm-svn: 276355
2016-07-21 23:19:18 +00:00
Xinliang David Li eaf238d494 [Profile] implement interface to get profile path prefix
Differential Revision:  http://reviews.llvm.org/D22546

llvm-svn: 276083
2016-07-20 04:26:09 +00:00
Vedant Kumar 17af892d6f [profile] Hide some external symbols in InstrProfData.inc
Differential Revision: http://reviews.llvm.org/D21116

llvm-svn: 272167
2016-06-08 16:39:43 +00:00
Xinliang David Li 315e49d213 [profile] clean up profile file handling code
Differentianl Revision: http://reviews.llvm.org/D20572

llvm-svn: 270617
2016-05-24 21:29:18 +00:00
Xinliang David Li 4e8754d2cb [profile] Static counter allocation for value profiling (part-2)
Differential Revision: http://reviews.llvm.org/D20460

llvm-svn: 270337
2016-05-21 22:55:45 +00:00
Sean Silva 29e434769e Small typo and whitespace fix.
llvm-svn: 270177
2016-05-20 03:25:15 +00:00
Sean Silva aa705eccbe Fix typo.
llvm-svn: 270005
2016-05-19 00:12:02 +00:00
Xinliang David Li 32173153a4 [profile] add runtime variable documentation
llvm-svn: 269964
2016-05-18 17:44:57 +00:00
Xinliang David Li f90425e73b [profile] minor code restructuring /NFC
This is one of the enabler patch to allow value profiler to 
allocate counter statically.

llvm-svn: 269689
2016-05-16 20:33:30 +00:00
Xinliang David Li e9a8574da1 Reapply r268840: [profile] Simplify value profile writing
Revert r268864 that reverted 268840 after underlying problem
is fixed for arm bot.

llvm-svn: 268992
2016-05-10 00:17:31 +00:00
Renato Golin 26642b4035 Revert "[profile] Simplify value profile writing"
This reverts commit r268840, as it breaks Thumb2 self-hosting. There is something
unstable in the profiling for Thumb2 that needs to be sorted out before we continue
implementing these changes to the profiler. See PR27667.

llvm-svn: 268864
2016-05-07 20:07:09 +00:00
Xinliang David Li 4b30c79723 [profile] Simplify value profile writing
With this patch, value data are longer pre-collected
before writing. The code is simplified and requires
less heap space for dumping.

llvm-svn: 268840
2016-05-07 02:50:11 +00:00
Xinliang David Li 31f251f1f0 [PGO] Add API to check compatibility of profile data in buffer
This is needed by client which uses in-process merge API.

llvm-svn: 262736
2016-03-04 18:58:30 +00:00
Xinliang David Li dd12e9a8c0 [PGO] Add API for profile merge from buffer
Differential Revision: http://reviews.llvm.org/D17831

llvm-svn: 262644
2016-03-03 18:54:46 +00:00
Vedant Kumar b850251d3e [profile] Compute number of data entries correctly
Compiler-rt miscalculates the number of entries in the __llvm_prf_data section
on i386 Darwin. This results in a number of test failures (which we started
catching after r261344).

The fix we attempted earlier is insufficient (r261683). It caused some tests to
start passing again, but that hid the fact that we drop some data entries.

This patch should fix the real problem. It fixes the way we compute DataSize by
taking into account the way the Darwin linker lays out __llvm_prf_data.

Differential Revision: http://reviews.llvm.org/D17623

llvm-svn: 261957
2016-02-26 02:49:41 +00:00
Xinliang David Li 54dd683726 [PGO]: Do not update Data->Value field during profile write.
The profile reader no longer depends on this field to be updated and point
to owning func's vp data. The VP data also no longer needs to be allocated
in a contiguous memory space.

Differential Revision:  http://reviews.llvm.org/D15258

llvm-svn: 256543
2015-12-29 07:13:59 +00:00
Xinliang David Li abfd553c2b [PGO] cleanup: unify prefix for portability macros
llvm-svn: 255748
2015-12-16 03:29:15 +00:00
Xinliang David Li 26bcd193a8 [PGO] Header file cleanup (NFC)
InstrProfiling.h file declares profile runtime public APIs.
It has become a dumping place for many different things, which
needs cleanups. In this change, core type declarations and 
portability macros are moved to a new file InstrProfilingPort.h.

llvm-svn: 255270
2015-12-10 18:17:01 +00:00
Xinliang David Li 55d927a4fd [PGO] Stop leaking libc function to buffer API impl
llvm-svn: 254943
2015-12-07 21:18:16 +00:00
Xinliang David Li 4c8a5e37e2 Move macro defs closer (NFC)
llvm-svn: 254761
2015-12-04 20:09:55 +00:00
Xinliang David Li 9ea3064be5 [PGO] Introduce error report macro in profile-rt
Also added a test case for runtime error reporting.

llvm-svn: 254625
2015-12-03 18:31:59 +00:00
Xinliang David Li 39f4c2ca3b [PGO] sync up target instrument decl between runtime and llvm (NFC)
llvm-svn: 253911
2015-11-23 20:07:09 +00:00
Xinliang David Li 6376db52fe [PGO] Unify raw profile header definition
Replace duplicate definition raw header with
 common definition in InstrProfData.inc.

llvm-svn: 253896
2015-11-23 18:36:40 +00:00
Xinliang David Li 66c32f0183 [PGO] Use the alignment macro newly introduced (NFC)
llvm-svn: 253895
2015-11-23 18:09:18 +00:00
Xinliang David Li a1bebf2987 [PGO] Unify per-function control data definition
Replace duplicate definition per-func profile control data
 with common definition in InstrProfData.inc.

llvm-svn: 253891
2015-11-23 17:50:53 +00:00
Xinliang David Li 5c7fa27b00 [PGO] Use common definition of value profile kind
Replace duplicate definition value profile kind definitions
 with common definition in InstrProfData.inc.

llvm-svn: 253890
2015-11-23 17:43:10 +00:00
Xinliang David Li 0ca5bd4a46 [PGO] Start use InstrProf template file in compiler-rt/lib/profile
- Replace use of __llvm_profile_value_data with common data structure.
- Remve duplicate InstrProfValueNode

llvm-svn: 253883
2015-11-23 17:09:40 +00:00