This includes a few nice bits of refactoring (e.g splitting out the
exclusive locking code into a common utility).
Patch by Rainer Orth!
Differential Revision: https://reviews.llvm.org/D40944
llvm-svn: 320726
Summary:
As discussed in https://github.com/google/oss-fuzz/issues/933,
it would be really awesome to be able to use ThinLTO for fuzzing.
However, as @kcc has pointed out, it is currently undefined (untested)
whether the sanitizers actually function properly with LLD and/or LTO.
This patch is inspired by the cfi test, which already do test with LTO
(and/or LLD), since LTO is required for CFI to function.
I started with UBSan, because it's cmakelists / lit.* files appeared
to be the cleanest. This patch adds the infrastructure to easily add
LLD and/or LTO sub-variants of the existing lit test configurations.
Also, this patch adds the LLD flavor, that explicitly does use LLD to link.
The check-ubsan does pass on my machine. And to minimize the [initial]
potential buildbot breakage i have put some restrictions on this flavour.
Please review carefully, i have not worked with lit/sanitizer tests before.
The original attempt, r319525 was reverted in r319526 due
to the failures in compiler-rt standalone builds.
Reviewers: eugenis, vitalybuka
Reviewed By: eugenis
Subscribers: #sanitizers, pcc, kubamracek, mgorny, llvm-commits, mehdi_amini, inglorion, kcc
Differential Revision: https://reviews.llvm.org/D39508
llvm-svn: 319575
This is a resubmission of r313270. It broke standalone builds of
compiler-rt because we were not correctly generating the llvm-lit
script in the standalone build directory.
The fixes incorporated here attempt to find llvm/utils/llvm-lit
from the source tree returned by llvm-config. If present, it
will generate llvm-lit into the output directory. Regardless,
the user can specify -DLLVM_EXTERNAL_LIT to point to a specific
lit.py on their file system. This supports the use case of
someone installing lit via a package manager. If it cannot find
a source tree, and -DLLVM_EXTERNAL_LIT is either unspecified or
invalid, then we print a warning that tests will not be able
to run.
Differential Revision: https://reviews.llvm.org/D37756
llvm-svn: 313407
This patch is still breaking several multi-stage compiler-rt bots.
I already know what the fix is, but I want to get the bots green
for now and then try re-applying in the morning.
llvm-svn: 313335
This patch simplifies LLVM's lit infrastructure by enforcing an ordering
that a site config is always run before a source-tree config.
A significant amount of the complexity from lit config files arises from
the fact that inside of a source-tree config file, we don't yet know if
the site config has been run. However it is *always* required to run
a site config first, because it passes various variables down through
CMake that the main config depends on. As a result, every config
file has to do a bunch of magic to try to reverse-engineer the location
of the site config file if they detect (heuristically) that the site
config file has not yet been run.
This patch solves the problem by emitting a mapping from source tree
config file to binary tree site config file in llvm-lit.py. Then, during
discovery when we find a config file, we check to see if we have a
target mapping for it, and if so we use that instead.
This mechanism is generic enough that it does not affect external users
of lit. They will just not have a config mapping defined, and everything
will work as normal.
On the other hand, for us it allows us to make many simplifications:
* We are guaranteed that a site config will be executed first
* Inside of a main config, we no longer have to assume that attributes
might not be present and use getattr everywhere.
* We no longer have to pass parameters such as --param llvm_site_config=<path>
on the command line.
* It is future-proof, meaning you don't have to edit llvm-lit.in to add
support for new projects.
* All of the duplicated logic of trying various fallback mechanisms of
finding a site config from the main config are now gone.
One potentially noteworthy thing that was required to implement this
change is that whereas the ninja check targets previously used the first
method to spawn lit, they now use the second. In particular, you can no
longer run lit.py against the source tree while specifying the various
`foo_site_config=<path>` parameters. Instead, you need to run
llvm-lit.py.
Differential Revision: https://reviews.llvm.org/D37756
llvm-svn: 313270
Users can specify the path a raw profile is written to by passing
-fprofile-instr-generate=<path>, but this functionality broke on Darwin
after __llvm_profile_filename was made weak [1], resulting in profiles
being written to "default.profraw" even when <path> is specified.
The situation is that instrumented programs provide a weak definition of
__llvm_profile_filename, which conflicts with a weak redefinition
provided by the profiling runtime.
The linker appears to pick the 'winning' definition arbitrarily: on
Darwin, it usually prefers the larger definition, which is probably why
the instrprof-override-filename.c test has been passing.
The fix is to move the runtime's definition into a separate object file
within the archive. This means that the linker won't "see" the runtime's
definition unless the user program has not provided one. I couldn't
think of a great way to test this other than to mimic the Darwin
failure: use -fprofile-instr-generate=<some-small-path>.
Testing: check-{clang,profile}, modified instrprof-override-filename.c.
[1] [Profile] deprecate __llvm_profile_override_default_filename
https://reviews.llvm.org/D22613https://reviews.llvm.org/D22614
Differential Revision: https://reviews.llvm.org/D34797
llvm-svn: 306710
Summary:
We found a nondeterministic behavior when doing online profile merging
for multi-process applications. The application forks a sub-process and
sub-process sets to get SIGKILL when the parent process exits,
The first process gets the lock, and dumps the profile. The second one
will mmap the file, do the merge and write out the file. Note that before
the merged write, we truncate the profile.
Depending on the timing, the child process might be terminated
abnormally when the parent exits first. If this happens:
(1) before the truncation, we will get the profile for the main process
(2) after the truncation, and before write-out the profile, we will get
0 size profile.
(3) after the merged write, we get merged profile.
This patch temporarily suspend the SIGKILL for PR_SET_PDEATHSIG
before profile-write and restore it after the write.
This patch only applies to Linux system.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: xur, llvm-commits
Differential Revision: https://reviews.llvm.org/D29954
llvm-svn: 295364
Summary:
The build system was inconsistent in its naming conventions for
link flags. This patch changes all uses of LINKFLAGS to LINK_FLAGS,
for consistency with cmake's LINK_FLAGS property.
This patch should make it easier to search the source code for
uses of link flags, as well as providing the benefit of improved
style and consistency.
Reviewers: compnerd, beanz
Subscribers: kubabrecka, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D28506
llvm-svn: 291539
The Clang driver on macOS decides the deployment target based on various things, like your host OS version, the SDK version and some environment variables, which makes lit tests pass or fail based on your environment. Let's make sure we run all lit tests with `-mmacosx-version-min=${SANITIZER_MIN_OSX_VERSION}` (10.9 unless overriden).
Differential Revision: https://reviews.llvm.org/D26929
llvm-svn: 288186
This makes __llvm_profile_set_filename() work across dylib boundaries on
Darwin.
This functionality was originally meant to work on all platforms, but
was moved to a Linux-only directory with r272404. The root cause of the
test failure on Darwin was that lprofCurFilename was not marked weak.
Each dylib maintained its own copy of the variable due to the two-level
namespace.
Tested with check-profile (on Darwin). I don't expect this to regress
other platforms.
Differential Revision: https://reviews.llvm.org/D25707
llvm-svn: 284440
Profile-aarch64 :: Linux/comdat_rename.test
Profile-aarch64 :: Linux/extern_template.test
Profile-aarch64 :: Linux/instrprof-comdat.test
Profile-aarch64 :: Linux/instrprof-cs.c
The issue is that the created (aarch64) binaries were attempting to run natively
instead of running through %run, which guarantees running in the proper
environment if the compilation was configured correctly.
llvm-svn: 282264
Another CFG optimisation patch (280364) has broken bad profile tests, and this
is a similar attempt to fix the test without changing the semantics.
llvm-svn: 280373
Commit r280364 has introduced some call-graph optmisations making a profiler
test "fail" due to not expecting the compiler to be "smart", and fold constants
across functions. This commit works around the issue, leaving the origial
semantics intact.
llvm-svn: 280365
The API is intended to be used by user to do fine
grained (per-region) control of profile dumping.
Differential Revision: http://reviews.llvm.org/D23106
llvm-svn: 278092
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
Test that __llvm_profile_set_filename invoked in
main program is 'visible' to shared lib (overriding
shared libary's profile path set on command line)
llvm-svn: 272375
(Part-1 merging API is in profile runtime)
This patch implements a portable file opening API
with exclusive access for the process. In-process
profile merge requires profile file update to be
atomic/fully sychronized.
llvm-svn: 271864
Make sure inline virtual function's address is properly recorded and
dumped in raw profile so that value profiling does not loss tracking.
(Second part of the test will be enabled after the lowering bug is fixed
in LLVM)
llvm-svn: 271528
With this change, dynamic memory allocation is only used
for testing purpose. This change is one of the many steps to
make instrument profiler dynamic allocation free.
llvm-svn: 269453
At the moment almost every lit.site.cfg.in contains two lines comment:
## Autogenerated by LLVM/Clang configuration.
# Do not edit!
The patch adds variable LIT_SITE_CFG_IN_HEADER, that is replaced from
configure_lit_site_cfg with the note and some useful information.
llvm-svn: 266520
This test case is meant to check whether verbose error logging works. It
does this by _failing_ to write a profile to a path corresponding to a
directory. Make this clearer by writing to a temporary directory.
Patch suggested by Sean Silva!
llvm-svn: 264817
This also makes it work on PS4 a bit better.
For running the libprofile tests on PS4, we are sneaking environment
variables through on the command line, so the extra argument for
LLVM_PROFILE_FILE was messing up the simple `argc < 2` check.
llvm-svn: 264812
On PS4, we have to fake environment variables by passing extra command
line arguments, so the dummy test `argc > 1` was failing.
The condition is just a dummy condition that the compiler can't fold
away, so the number is arbitrary as long as the condition is false.
Increase the number it compares against.
llvm-svn: 264491
This patch adds support for expanding "%h" out to the machine hostname
in the LLVM_PROFILE_FILE environment variable.
Patch by Daniel Waters!
Differential Revision: http://reviews.llvm.org/D16371
llvm-svn: 259272
The original submittion triggered a BFD linker bug (2.24) on Aarch64 only. Before
the build bot is upgraded to more recent linker, restrict the test to be
executed only with gold linker.
llvm-svn: 258437
(second try with more strict config check)
Currently, only gc-sections related tests are added. Gold
linker currently is required due to PR19161 of bfd linker.
llvm-svn: 257456
IR level instrumentation needs to override version with variant bits.
No change for FE instrumentation is needed. Test case is added to
detect version mismatch.
llvm-svn: 257230
Extract the buffered filer writer code used by value profile
writer and turn it into common/sharable buffered fileIO
interfaces. Added a test case for the buffered file writer and
rewrite the VP dumping using the new APIs.
llvm-svn: 256604
The profile reader no longer depends on this field to be updated and point
to owning func's vp data. The VP data also no longer needs to be allocated
in a contiguous memory space.
Differential Revision: http://reviews.llvm.org/D15258
llvm-svn: 256543
Add a test case to cover profile dumping of functions with no
value sites, functions with value sites but no dynamic VP data,
and functions with runtime VP data.
llvm-svn: 255327
Value profile runtime depends on libc which breaks
buffer API implemenation with current file organization.
Test case is also updated to check more symbols.
llvm-svn: 255294
and -no-integrated-as for source with static function.
The compiler generates assembly names containing \" in the
.s dump causing gnu assembler to complain. That issue will be
investigated separately.
llvm-svn: 254705
With the latest refactoring and code sharing patches landed,
it is possible to unify the value profile implementation between
raw and indexed profile. This is part in prfofile runtime.
Differential Revision: http://reviews.llvm.org/D15057
llvm-svn: 254678
- This is to handle a corner case where profile lib is linked
in but non of the modules are instrumented (On linux, since
we avoided the overhead to emit runtime hook use functions so
this is the side effect of that size optimization).
- Added a profile runtime test case to cover all scenarios of
shared library builds.
Differential Revision: http://reviews.llvm.org/D14468
llvm-svn: 253098
When the file is initialized, this patch checks whether the path
specifies a directory. If so, it creates the directory tree before
truncating the file.
Use default.profdata instead of pgo-data for default indexed profile name.
llvm-svn: 241824
Fix a couple of new tests that were reverted because they were causing
intermittent test failures since they were writing the same default
"default.profraw" file. Fixed by creating a unique directory and
running tests in that directory.
Patch by Teresa Johnson. Thanks!
llvm-svn: 236168
This partially reverts commit r236056, as the two tests were using the same
names and, when executed in parallel, were having intermittent failures.
llvm-svn: 236101
Summary:
I don't know anything about profiling but it seems to work out of the
box on PowerPC64. At least "make check-profile" works.
A few tests needed tweaking because PowerPC64 IR declares main with
"define signext i32 @main" instead of just "define i32 @main".
This also fixes the asan asan_and_llvm_coverage_test test, which
compiles with -coverage so requires that a profiling version of
libclang_rt has been built.
Reviewers: dexonsmith, kcc, samsonov
Reviewed By: samsonov
Subscribers: samsonov, llvm-commits
Differential Revision: http://reviews.llvm.org/D6233
llvm-svn: 221877
ASAN, UBSAN and profile tests that don't work with arm-linux-gnueabi and
android also don't work on armv7l-linux-gnueabihf. Some of the tests have
known causes, others not yet. In order to get a green bot, I'm disabling
those tests for now and will investigate when the priority rises.
llvm-svn: 219343
FreeBSD does not have libdl, so set it via lit.cfg instead of the test
input, as with asan. Also remove it from Darwin test runs - it's not
necessary, but harmless there.
Add FreeBSD to the list of hosts to test.
Differential Revision: http://reviews.llvm.org/D5650
llvm-svn: 219227
The naming scheme we're using for counters in profile data shares a
prefix with some fixed names we use for the runtime, notably
__llvm_profile_data_begin and _end. Embarrassingly, this means a
function called begin() can't be instrumented.
This modifies the runtime names so as not to collide with the
instrumentation.
llvm-svn: 217166
The instrprofile-write-file-only test was failing on the builtbots. The runtime
library initialization is explicitly being discarded to avoid the atexit hook.
However, this would also prevent the filename from being initialized. Thus,
when the write file was invoked, the filename would not be setup, and the test
would fail as the profiling data would never be written out. Explicitly
initialize the filename to ensure that the data is written out when requested.
This should hopefully finally get the build bots all green again.
llvm-svn: 209099
dlopen expects that the flags contains RTLD_LAZY or RTLD_NOW. RTLD_LAZY will
resolve the symbol on first use, and is generally preferred due to efficiency.
Add this to fix the test failure on the build bots.
llvm-svn: 209098
Add logging to report any failures that may occur on calls to libdl. Without
this, it is difficult to identify the actual problem if the test fails.
llvm-svn: 209097
Add an explicit link against libdl. libdl may not be indirectly pulled on some
Linux hosts. Explicitly link against it. This should hopefully improve the
state of the build bots.
llvm-svn: 209096
Shared objects are hard. After this commit, we do the right thing when
profiling two separate shared objects that have been dlopen'd with
`RTLD_LOCAL`, when the main executable is *not* being profiled.
This mainly simplifies the writer logic.
- At initialization, determine the output filename and truncate the
file. Depending on whether shared objects can see each other, this
may happen multiple times.
- At exit, each executable writes its own profile in append mode.
<rdar://problem/16918688>
llvm-svn: 209053
These tests were XPASS-ing on Linux bots creating Mach-O, which makes
sense, since the real difference is the object format.
I'm hoping a short-term fix to get these tests passing on ELF is to
create two copies of the runtime -- one built with -fPIC, and one
without. A follow-up patch will change clang's driver to pick between
them depending on whether `-shared` is specified.
llvm-svn: 208947
According to the buildbots, the new features for shared objects don't
work on ELF since it requires an -fPIC when building the profile
library. XFAIL these tests for now.
It's possible we'll have to build two versions of the profile library on
Linux (one with -fPIC and one without), but it's not clear to me exactly
how to resolve this.
llvm-svn: 208946
Change the API of the instrumented profiling library to work with shared
objects.
- Most things are now declared hidden, so that each executable gets
its own copy.
- Initialization hooks up a linked list of writers.
- The raw format with shared objects that are profiled consists of a
concatenated series of profiles. llvm-profdata knows how to deal
with that since r208938.
<rdar://problem/16918688>
llvm-svn: 208940
Check that the profile runtime works as expected. This tests the
functions that are meant to be available to advanced users.
In particular, check that the `atexit()` hook can be disabled by
defining a custom `__llvm_profile_runtime` variable, that the libc
dependencies are optional, and that the various functions for writing
out files work for basic cases.
llvm-svn: 208460
Implement magic in compiler-rt to enable llvm-lit to be invoked on the
source tree in lib/profile. This relies on a paired commit in the llvm
tree that exposes a new built-in parameter.
<rdar://problem/16458307>
llvm-svn: 205263
Add the test infrastructure for testing lib/profile and a single test.
This initial commit only enables the tests on Darwin, but they'll be
enabled on Linux soon after.
<rdar://problem/16458307>
llvm-svn: 205256
Vedant Kumar