Commit Graph

3422 Commits

Author SHA1 Message Date
Diana Picus 4152a09e55 Revert "tsan: add fiber support"
This reverts commit r353817 because we think it broke AARch64 and
PowerPC buildbots.

llvm-svn: 353939
2019-02-13 11:32:56 +00:00
Kostya Serebryany d085748484 [libFuzzer] a bit of refactoring of the fork mode
llvm-svn: 353910
2019-02-13 04:04:45 +00:00
Julian Lettner 55fb5314b3 [Sanitizer][NFC] Darwin: limit parallism for sanitizer_common tests
Many sanitizer_common tests (ASan, TSan) run with a "sanitized process
space" so we need to limit their parallism.

llvm-svn: 353909
2019-02-13 01:56:53 +00:00
Dmitry Vyukov 6e7089ad40 tsan: add fiber support
This patch adds functions for managing fibers:

__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()

See the added tests for use examples.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889

llvm-svn: 353817
2019-02-12 10:11:58 +00:00
Petr Hosek 61b513351c [CMake] Avoid internal dependencies in the runtimes build
When performing runtimes build, dependencies like clang and
clang-headers aren't available. This was accidentally omitted in D57992.

llvm-svn: 353796
2019-02-12 04:13:58 +00:00
Kostya Serebryany cdbb9dc962 [libFuzzer] teach the fork mode to ignore OOMs and timeouts
llvm-svn: 353792
2019-02-12 02:18:53 +00:00
Julian Lettner ee82db57ec [libFuzzer] Make coverage.test work on ARM64
Summary:
This test instruments the following code with coverage, runs the fuzzer
once, and asserts that there are uncovered PCs. The ARM64 backend
optimizes this code using the `csel` (Conditional select) instruction,
which removes all branching from the resulting machine code. The test
then fails because we do not have any uncovered PCs. The easiest
solution for now is to turn off optimization for the DSOs used in this
test.

```
int DSO1(int a) {
  if (a < 123456)
    return 0;
  return 1;
}
```

rdar://47646400

Reviewers: kcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D58087

llvm-svn: 353780
2019-02-12 00:37:40 +00:00
Kostya Serebryany 8cc38effe2 [libFuzzer] run the -fork=1 tests only on linux
llvm-svn: 353777
2019-02-12 00:29:51 +00:00
Kostya Serebryany 63f48717b5 [libFuzzer] extend the -fork=1 functionality. Still not fully usable, but good enough for the first unit test
llvm-svn: 353775
2019-02-12 00:12:33 +00:00
Julian Lettner 6142bf2c51 [NFC] Re-enable XFAILed fuzzer test on iOS
llvm-svn: 353737
2019-02-11 19:33:01 +00:00
Vitaly Buka 4fefb72019 [tsan] Remove debug logging and disable test on ppc64be
llvm-svn: 353624
2019-02-09 19:28:52 +00:00
Vitaly Buka 3eca7f5139 [tsan] Remove debug logging and disable test on ppc64be
llvm-svn: 353623
2019-02-09 18:43:24 +00:00
Vitaly Buka 9419b5bd04 [tsan] Debug failing test on PPC bot
llvm-svn: 353617
2019-02-09 16:28:41 +00:00
Douglas Yung b041a18bcf Temporarily disable calls to getgrnam/getgrnam_r in test due to it hitting unrelated issues in EGLIBC 2.19.
llvm-svn: 353594
2019-02-09 01:28:47 +00:00
Kostya Serebryany b1e8b8149b [libFuzzer] remove two unused experimental flags
llvm-svn: 353573
2019-02-08 22:02:37 +00:00
Reid Kleckner 987d331fab [InstrProf] Implement static profdata registration
Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.

Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.

Now, Windows uses static registration and is in line with all the other
platforms.

Reviewers: davidxl, wmi, inglorion, void, calixte

Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57929

llvm-svn: 353547
2019-02-08 19:03:50 +00:00
Kostya Serebryany 0719b3527f [libFuzzer] refactor the way we choose the element to cross-over with, NFC (expected1); add a flag -seed_inputs= to pass extra seed inputs as file paths, not dirs
llvm-svn: 353494
2019-02-08 01:20:54 +00:00
Kostya Serebryany 28e967aaa1 [libFuzzer] add a test for built-in CrossOver (there are unit tests for this, but it's worth having a full integration test like this)
llvm-svn: 353488
2019-02-08 00:41:29 +00:00
Reid Kleckner 17e2521724 [InstrProf] Port test suite to Windows
Summary:
Before this change, check-profile would run, but all tests would be
marked unsupported on Windows. This is the new status of 'check-profile'
after this change:

Testing Time: 6.66s
  Expected Passes    : 29
  Expected Failures  : 5
  Unsupported Tests  : 39

I moved many tests that exercise posix-y features like dlopen and DSOs
into the Posix subdirectory, and ran the tests on Linux to validate my
changes.

These are the remaining tests that I handled on a case by case basis:
- instrprof-path.c
  Passes, Fixed some path portability issues
- instrprof-gcov-exceptions.test
  Passes, the FileCheck actually succeeds on Windows, so I RUNX'd it
- instrprof-icall-promo.test
  XFAILed, probably due to C++ ABI differences in vtables
- instrprof-merge-match.test
- instrprof-merge.c
- instrprof-merging.cpp
  XFAILed, These seem like real bugs that need fixing
- instrprof-version-mismatch.c
  XFAILed, Overriding the weak version symbol doesn't work
- instrprof-without-libc.c
  UNSUPPORTED, test needs an executable symbol table, Windows has none

Reviewers: davidxl, wmi, void

Subscribers: fedor.sergeev, #sanitizers, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57853

llvm-svn: 353435
2019-02-07 17:52:05 +00:00
Dmitry Vyukov 17132b62e0 tsan: Implement pthread_exit() interceptor for Thread sanitizer
This change is preparation for fiber support.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57876
Context: https://reviews.llvm.org/D54889

llvm-svn: 353385
2019-02-07 10:46:02 +00:00
Vitaly Buka c41835230a [sanitizer] Fix Android tests
On Android some fields can be null

llvm-svn: 353377
2019-02-07 07:40:33 +00:00
Vitaly Buka 2b93ef7ef6 [sanitizer] Re-enabled getpw_getgr.cc on Android
Reviewers: eugenis

Subscribers: srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57785

llvm-svn: 353366
2019-02-07 01:55:07 +00:00
Vitaly Buka eb7031ecf6 [sanitizer] Don't unpoison buffer in getpw/getgr functions
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.

This fixes TSAN performance issue made us to disable this interceptors.

Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57731

llvm-svn: 353351
2019-02-07 00:08:14 +00:00
Evgeniy Stepanov 443c034391 [sanitizer] Decorate /proc/self/maps better.
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.

Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.

Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.

The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D57720

llvm-svn: 353255
2019-02-06 01:14:50 +00:00
Peter Collingbourne 6141b037a9 gn build: Upgrade to NDK r19.
NDK r19 includes a sysroot that can be used directly by the compiler
without creating a standalone toolchain, so we just need a handful
of flags to point Clang there.

Differential Revision: https://reviews.llvm.org/D57733

llvm-svn: 353139
2019-02-05 05:10:19 +00:00
Kostya Kortchinsky 47f0d136f1 [scudo] Initial standalone skeleton check-in
Summary:
This is the initial check-in for the Standalone version of Scudo.

The project is initially going to live in scudo/standalone then will
replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html
for details.

This initial CL is meant to lay out the project structure, of both
code & tests, providing a minimal amount of functionalities, namely
various definitions, some atomic helpers and an intrusive list.
(empty.cc is just here to have a compilation unit, but will go away
in the upcoming CLs).

Initial support is restricted to Linux i386 & x86_64 in make files
and will be extended once things land & work.

We will grow organically from here, adding functionalities in limited
amounts.

Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack

Reviewed By: morehouse, vitalybuka

Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D57412

llvm-svn: 353055
2019-02-04 16:25:40 +00:00
Jonathan Metzman e2469b11a5 [libFuzzer][Windows] Reenable passing tests
Summary:
Enable tests that were previously disabled because they didn't work on
Windows.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: morehouse

Differential Revision: https://reviews.llvm.org/D57563

llvm-svn: 353000
2019-02-03 16:53:32 +00:00
Reid Kleckner 33706e3cbf [ubsan] Make suppressions.cpp test pass for me on Windows
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.

While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.

llvm-svn: 352921
2019-02-01 21:35:17 +00:00
Kostya Serebryany ae667c49d2 [libFuzzer] make a test for exploding dfsan labels more agressive
llvm-svn: 352819
2019-01-31 23:43:54 +00:00
Jonathan Metzman 6f94a033a7 [libFuzzer][Windows] Temporarily disable value-profile-cmp2.test on Win
Summary:
Temporarily disable value-profile-cmp2.test on Win.
https://reviews.llvm.org/D57465 causes the test to fail on Win.
However, it seems that the behavior of libFuzzer on Win was broken
before that patch. It crashes in the exit handler when not used with
ASAN. Prior to the patch, the crash handler would run, tricking the
test into thinking libFuzzer on Win had exited properly.

Reviewers: morehouse, vitalybuka

Reviewed By: morehouse

Subscribers: yln

Differential Revision: https://reviews.llvm.org/D57551

llvm-svn: 352815
2019-01-31 23:34:56 +00:00
Jonathan Metzman fc7faecb40 [fuzzer] Use RawPrint instead of Printf for instrumentation warning
Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D57531

llvm-svn: 352789
2019-01-31 20:32:20 +00:00
Kostya Serebryany 02a86e6c7e [libFuzzer] relax a flaky test
llvm-svn: 352778
2019-01-31 18:40:12 +00:00
Julian Lettner 7e0219d979 [libFuzzer] Update Darwin test
Support for -fsanitize-coverage=trace-pc[-guard] was removed from
libFuzzer, which makes this currently fail.

This commit aligns this Darwin-specific test with its Linux counterpart
which changed in this commit:
3a94519a77

llvm-svn: 352721
2019-01-31 03:03:06 +00:00
Kostya Serebryany 23482e1d07 [libFuzzer] set libFuzzer's own SEGV handler even one is already present, but call that handler from ours (unless we are unprotecting lazy counters). Call ProtectLazyCounters later, so that it runs after the initialization code in the target.
llvm-svn: 352713
2019-01-31 01:40:14 +00:00
Julian Lettner 15df273eb4 [libFuzzer] Set default sanitizer options in fuzzer tests
Summary:
Set default `ASAN_OPTIONS` when running libFuzzer tests. This allows us
to remove special casing in code for Darwin where we usually pass
`abort_on_error=0` to override platform defaults for tests.

A previous commit changed the code to make the tests pass:
7764a04af0

Adapted a few tests to use `%env_asan_opts=` instead of directly setting
the environment variable.

rdar://problem/47515276

Reviewers: kcc, george.karpenkov

Differential Revision: https://reviews.llvm.org/D57465

llvm-svn: 352711
2019-01-31 01:24:01 +00:00
Kostya Serebryany 8da9479e40 [libFuzzer] experimental performance optimization -lazy_counters, off by default. Posix-only for now, tested on Linux
llvm-svn: 352700
2019-01-31 00:09:43 +00:00
Kostya Serebryany 6fd4d8ab9c [libFuzzer] refactor the handling of instrumentation counters so that they are grouped in regions one full page each. Needed for future optimization. NFC
llvm-svn: 352603
2019-01-30 06:15:52 +00:00
Kostya Serebryany 3a94519a77 [libFuzzer] update a test
llvm-svn: 352577
2019-01-30 00:36:28 +00:00
Kostya Serebryany 50a1c69712 [libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]
llvm-svn: 352564
2019-01-29 23:37:20 +00:00
Jeremy Morse b0d7cf5df4 Add 'REQUIRES: zlib' to a fuzzer test
Fixes tests on build environments that don't have zlib. See also r352483.

llvm-svn: 352487
2019-01-29 13:06:17 +00:00
Jeremy Morse 2601aa5406 Add zlib feature test to lit configuration
Some new tests in libfuzzer have dependencies on zlib: add a feature test
for zlib so that we can add a REQUIRES field to the relevant tests.

Patch by Matthew Voss.

Differential Revision: https://reviews.llvm.org/D57366

llvm-svn: 352483
2019-01-29 12:00:01 +00:00
Jonathan Metzman 3720e2b39e [fuzzer][afl] Remove AFL_DRIVER_EXTRA_STATS_FILENAME
Summary:
Remove this feature as it is unused, buggy, and not worth correcting
since the forkserver makes it difficult.

Reviewers: morehouse, jfb

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D57308

llvm-svn: 352392
2019-01-28 17:15:49 +00:00
Stefan Granitz 6163039ce0 [CMake] Quick-Fix FileCheck target does not exist when building against LLVM install-tree with COMPILER_RT_INCLUDE_TESTS=ON
The issue came up during release testing for LLVM 8: https://bugs.llvm.org/show_bug.cgi?id=40443

Differential Revision: https://reviews.llvm.org/D57224

llvm-svn: 352381
2019-01-28 16:14:57 +00:00
Kostya Serebryany 71ab4acb16 [libFuzzer] add CompressedTest.cpp, a real-life-ish test for a custom mutator
llvm-svn: 352265
2019-01-26 02:38:56 +00:00
Kostya Serebryany bb01a098b1 [libFuzzer] print uncovered functions when doing -print_coverage=1
llvm-svn: 352263
2019-01-26 01:33:09 +00:00
Kostya Kortchinsky a04584b095 [scudo] Delay allocations in the RSS check test
Summary:
D57116 fails on the armv7 bots, which is I assume due to the timing of
the RSS check on the platform. While I don't have a platform to test
that change on, I assume this would do.

The test could be made more reliable by either delaying more the
allocations, or allocating more large-chunks, but both those options
have a somewhat non negligible impact (more memory used, longer test).

Hence me trying to keep the additional sleeping/allocating to a
minimum.

Reviewers: eugenis, yroux

Reviewed By: yroux

Subscribers: javed.absar, kristof.beyls, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D57241

llvm-svn: 352220
2019-01-25 17:23:29 +00:00
Evgeniy Stepanov 9ea001401c [hwasan] Implement print_module_map flag.
Reviewers: kcc, pcc

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D57130

llvm-svn: 352150
2019-01-25 02:05:25 +00:00
Kostya Serebryany 360bf5ff58 [libFuzzer] more agressive value profiling and CMP tracing for switch statements
llvm-svn: 352107
2019-01-24 21:08:54 +00:00
Julian Lettner b62e9dc46b Revert "[Sanitizers] UBSan unreachable incompatible with ASan in the presence of `noreturn` calls"
This reverts commit cea84ab93a.

llvm-svn: 352069
2019-01-24 18:04:21 +00:00
Kostya Kortchinsky f0fbeaf44a [scudo] Tuning changes based on feedback from current use
Summary:
This tunes several of the default parameters used within the allocator:
- disable the deallocation type mismatch on Android by default; this
  was causing too many issues with third party libraries;
- change the default `SizeClassMap` to `Dense`, it caches less entries
  and is way more memory efficient overall;
- relax the timing of the RSS checks, 10 times per second was too much,
  lower it to 4 times (every 250ms), and update the test so that it
  passes with the new default.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: srhines, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D57116

llvm-svn: 352057
2019-01-24 15:56:54 +00:00