Commit Graph

350 Commits

Author SHA1 Message Date
Kostya Serebryany 02a86e6c7e [libFuzzer] relax a flaky test
llvm-svn: 352778
2019-01-31 18:40:12 +00:00
Julian Lettner 7e0219d979 [libFuzzer] Update Darwin test
Support for -fsanitize-coverage=trace-pc[-guard] was removed from
libFuzzer, which makes this currently fail.

This commit aligns this Darwin-specific test with its Linux counterpart
which changed in this commit:
3a94519a77

llvm-svn: 352721
2019-01-31 03:03:06 +00:00
Kostya Serebryany 23482e1d07 [libFuzzer] set libFuzzer's own SEGV handler even one is already present, but call that handler from ours (unless we are unprotecting lazy counters). Call ProtectLazyCounters later, so that it runs after the initialization code in the target.
llvm-svn: 352713
2019-01-31 01:40:14 +00:00
Julian Lettner 15df273eb4 [libFuzzer] Set default sanitizer options in fuzzer tests
Summary:
Set default `ASAN_OPTIONS` when running libFuzzer tests. This allows us
to remove special casing in code for Darwin where we usually pass
`abort_on_error=0` to override platform defaults for tests.

A previous commit changed the code to make the tests pass:
7764a04af0

Adapted a few tests to use `%env_asan_opts=` instead of directly setting
the environment variable.

rdar://problem/47515276

Reviewers: kcc, george.karpenkov

Differential Revision: https://reviews.llvm.org/D57465

llvm-svn: 352711
2019-01-31 01:24:01 +00:00
Kostya Serebryany 8da9479e40 [libFuzzer] experimental performance optimization -lazy_counters, off by default. Posix-only for now, tested on Linux
llvm-svn: 352700
2019-01-31 00:09:43 +00:00
Kostya Serebryany 6fd4d8ab9c [libFuzzer] refactor the handling of instrumentation counters so that they are grouped in regions one full page each. Needed for future optimization. NFC
llvm-svn: 352603
2019-01-30 06:15:52 +00:00
Kostya Serebryany 3a94519a77 [libFuzzer] update a test
llvm-svn: 352577
2019-01-30 00:36:28 +00:00
Kostya Serebryany 50a1c69712 [libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]
llvm-svn: 352564
2019-01-29 23:37:20 +00:00
Jeremy Morse b0d7cf5df4 Add 'REQUIRES: zlib' to a fuzzer test
Fixes tests on build environments that don't have zlib. See also r352483.

llvm-svn: 352487
2019-01-29 13:06:17 +00:00
Jonathan Metzman 3720e2b39e [fuzzer][afl] Remove AFL_DRIVER_EXTRA_STATS_FILENAME
Summary:
Remove this feature as it is unused, buggy, and not worth correcting
since the forkserver makes it difficult.

Reviewers: morehouse, jfb

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D57308

llvm-svn: 352392
2019-01-28 17:15:49 +00:00
Kostya Serebryany 71ab4acb16 [libFuzzer] add CompressedTest.cpp, a real-life-ish test for a custom mutator
llvm-svn: 352265
2019-01-26 02:38:56 +00:00
Kostya Serebryany bb01a098b1 [libFuzzer] print uncovered functions when doing -print_coverage=1
llvm-svn: 352263
2019-01-26 01:33:09 +00:00
Kostya Serebryany 360bf5ff58 [libFuzzer] more agressive value profiling and CMP tracing for switch statements
llvm-svn: 352107
2019-01-24 21:08:54 +00:00
Vitaly Buka d733430c19 [fuzzer] Fix test checks broken after license header update
llvm-svn: 351717
2019-01-21 02:22:06 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Jonathan Metzman 9e14cccf6f [libFuzzer] Remove unstable edge handling
Summary:
Remove code for handling unstable edges from libFuzzer since
it has not been found useful.

Differential Revision: https://reviews.llvm.org/D56730

llvm-svn: 351262
2019-01-15 22:12:51 +00:00
Roman Lebedev d8bc31a849 [compiler-rt] fuzzer-alignment-assumption.test: fixup the expected column number
llvm-svn: 351181
2019-01-15 10:25:14 +00:00
Roman Lebedev a06ad18669 [compiler-rt][UBSan] Sanitization for alignment assumptions.
Summary:
This is the compiler-rt part.
The clang part is D54589.

This is a second commit, the original one was r351106,
which was mass-reverted in r351159 because 2 compiler-rt tests were failing.

Now, i have fundamentally changed the testing approach:
i malloc a few bytes, intentionally mis-align the pointer
(increment it by one), and check that. Also, i have decreased
the expected alignment. This hopefully should be enough to pacify
all the bots. If not, i guess i might just drop the two 'bad' tests.

Reviewers: filcab, vsk, #sanitizers, vitalybuka, rsmith, morehouse

Reviewed By: morehouse

Subscribers: rjmccall, krytarowski, rsmith, kcc, srhines, kubamracek, dberris, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54590

llvm-svn: 351178
2019-01-15 09:44:27 +00:00
Vlad Tsyrklevich 86e68fda3b Revert alignment assumptions changes
Revert r351104-6, r351109, r351110, r351119, r351134, and r351153. These
changes fail on the sanitizer bots.

llvm-svn: 351159
2019-01-15 03:38:02 +00:00
Roman Lebedev cc10d54432 [compiler-rt][UBSan] Sanitization for alignment assumptions.
Summary:
This is the compiler-rt part.
The clang part is D54589.

Reviewers: filcab, vsk, #sanitizers, vitalybuka, rsmith, morehouse

Reviewed By: morehouse

Subscribers: rjmccall, krytarowski, rsmith, kcc, srhines, kubamracek, dberris, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D54590

llvm-svn: 351106
2019-01-14 19:09:29 +00:00
Jonathan Metzman 39b6ba9f33 [fuzzer] Read files as binary
Summary: Read corpus files as binary to avoid automatic conversions

Reviewers: Dor1s, morehouse

Reviewed By: Dor1s, morehouse

Differential Revision: https://reviews.llvm.org/D54180

llvm-svn: 346279
2018-11-06 23:25:25 +00:00
George Karpenkov fd68e3e2e3 [libFuzzer] Disable value-profile-cmp.test and fuzzer-oom.test on ARM64
value-profile-cmp was always flaky, and OOM fails to work in some environments.
counters.test fails when vectorization is enabled, and it is more likely
to kick in when compiling for ARM.

Differential Revision: https://reviews.llvm.org/D54004

llvm-svn: 346193
2018-11-06 01:15:51 +00:00
Roman Lebedev a0457c02b3 [NFC][compiler-rt] Cleanup Implicit Conversion Sanitizer tests to use sized types
Summary: As requested by @filcab in https://reviews.llvm.org/D50251#1280267

Reviewers: filcab, vsk, #sanitizers, vitalybuka

Reviewed By: filcab, #sanitizers, vitalybuka

Subscribers: vitalybuka, kubamracek, dberris, llvm-commits, filcab

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D53869

llvm-svn: 345661
2018-10-30 21:59:09 +00:00
Roman Lebedev 320e9af309 [compiler-rt][ubsan] Implicit Conversion Sanitizer - integer sign change - compiler-rt part
Summary:
This is a compiler-rt part.
The clang part is D50250.

See [[ https://bugs.llvm.org/show_bug.cgi?id=21530 | PR21530 ]], https://github.com/google/sanitizers/issues/940.

Reviewers: vsk, filcab, #sanitizers

Reviewed By: filcab, #sanitizers

Subscribers: mclow.lists, srhines, kubamracek, dberris, rjmccall, rsmith, llvm-commits, regehr

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D50251

llvm-svn: 345659
2018-10-30 21:58:54 +00:00
Jonathan Metzman 0b94e88007 [SanitizerCoverage] Prevent /OPT:REF from stripping constructors
Summary:
Linking with the /OPT:REF linker flag when building COFF files causes
the linker to strip SanitizerCoverage's constructors. Prevent this by
giving the constructors WeakODR linkage and by passing the linker a
directive to include sancov.module_ctor.

Include a test in compiler-rt to verify libFuzzer can be linked using
/OPT:REF

Reviewers: morehouse, rnk

Reviewed By: morehouse, rnk

Subscribers: rnk, morehouse, hiraditya

Differential Revision: https://reviews.llvm.org/D52119

llvm-svn: 344391
2018-10-12 18:11:47 +00:00
Max Moroz 6cbb3ca456 [libFuzzer] Add test for SanitizerCoverage working on Mac even with -Wl,-dead_strip.
Summary:
The corresponding asncov change: https://reviews.llvm.org/D53113.

Bug: https://bugs.chromium.org/p/chromium/issues/detail?id=892167

Reviewers: morehouse, kcc, george.karpenkov

Reviewed By: morehouse, george.karpenkov

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D53114

llvm-svn: 344346
2018-10-12 13:59:56 +00:00
Roman Lebedev d32c0d1466 [compiler-rt][ubsan] Split Implicit Integer Truncation Sanitizer into unsigned and signed checks
Summary:
This is compiler-rt part.
clang part is D50901.

Reviewers: rsmith, vsk, filcab, Sanitizers

Reviewed by: filcab

Differential Revision: https://reviews.llvm.org/D50902

llvm-svn: 344231
2018-10-11 09:09:52 +00:00
George Karpenkov d2f6590285 [libFuzzer] Disable value profiling tests on ARM
Some seem fragile, some fail, and some just take a really long time to run.
It does not seem to make sense to support some subset of value profiling tests.

Differential Revision: https://reviews.llvm.org/D53047

llvm-svn: 344105
2018-10-10 00:58:03 +00:00
George Karpenkov f28523bb3f [libFuzzer] Generalize the code for getting the previous offset for different architectures
Without this change, tests in coverage.test and dump_coverage.test are
failing on non-x86_64 platforms.
The diff is copied from sanitizer_common library, an alternative would
be to link it together with libFuzzer.

Differential Revision: https://reviews.llvm.org/D53040

llvm-svn: 344104
2018-10-10 00:57:44 +00:00
Kamil Rytarowski a4be178def Reflect the current reality and disable lsan tests on NetBSD
LSan/NetBSD is still in development (Stop-The-World routine missed).

llvm-svn: 344035
2018-10-09 10:34:36 +00:00
Jeremy Morse b52bd9eed6 [libfuzzer] Fix a test's run line
r342698 removed the 'UNSUPPORTED: windows' flags from a number of fuzzer
tests, however too many lines were removed from 'fuzzer-flags.test',
including the run-line that generates the test binary, which breaks that
test for me (and the clang-x64-ninja-win7 buildbot). This patch just
re-adds that line.

llvm-svn: 342720
2018-09-21 10:50:15 +00:00
Jonathan Metzman 0744d3c5a1 [fuzzer] Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
Summary:
Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
to get externally defined functions (eg: LLVMFuzzerInitialize,
LLVMFuzzerCustomMutator, etc) working again.

Also enable tests that depended on these functions (on windows)

Reviewers: rnk, morehouse

Reviewed By: rnk, morehouse

Subscribers: rnk, morehouse, mgorny

Differential Revision: https://reviews.llvm.org/D51700

llvm-svn: 342698
2018-09-20 23:24:48 +00:00
Matt Morehouse 0039792d7b [libFuzzer] Disable value-profile-strncmp.test on aarch64.
Test no longer finds the BINGO on clang-cmake-aarch64-full bot, and I
can't reproduce on our ARM machine.

llvm-svn: 342255
2018-09-14 18:24:02 +00:00
Jonathan Metzman 3ab0ee4d9b [libfuzzer] Enable trace-malloc-unballanced.test on Windows.
Summary:
Enables trace-malloc-unbalanced.test on Windows, fixing two problems it had with Windows before.
The first fix is specifying python instead of relying on a script's shebang since they can't be used on Windows.
The second fix is making the regex tolerate windows' implementation of the "%p" format string.

Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: morehouse

Differential Revision: https://reviews.llvm.org/D51760

llvm-svn: 341632
2018-09-07 04:28:54 +00:00
Jonathan Metzman 6167c4ee74 [libfuzzer] Use cl driver mode for tests and enable another test on Windows
Summary:
When targeting MSVC: compile using clang's cl driver mode (this is needed for
libfuzzer's exit_on_src_pos feature). Don't use -lstdc++ when linking,
it isn't needed and causes a warning.
On Windows: Fix exit_on_src_pos.test by making sure debug info isn't
overwritten during compilation of second binary by using .exe extension.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: aprantl, JDevlieghere

Differential Revision: https://reviews.llvm.org/D51757

llvm-svn: 341622
2018-09-07 01:11:31 +00:00
Jonathan Metzman c239482207 [libfuzzer] Correct typo in comment (test commit).
Differential Revision: https://reviews.llvm.org/D51739

llvm-svn: 341569
2018-09-06 17:50:01 +00:00
Max Moroz a39da2f8ca [libfuzzer] Fix fuzzer-oom.test on windows and reenable it.
Summary: Patch by Jonathan Metzman (@metzman).

Reviewers: Dor1s, morehouse

Reviewed By: Dor1s

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D51735

llvm-svn: 341562
2018-09-06 16:48:18 +00:00
Max Moroz 48d1398b36 [libfuzzer] Temporarily disable unittests failing on Windows.
Summary:
Patch by Jonathan Metzman (@metzman).


Reviewers: morehouse, Dor1s

Reviewed By: Dor1s

Subscribers: kcc, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D51712

llvm-svn: 341514
2018-09-06 02:54:47 +00:00
Matt Morehouse 02f3ad81c3 [libfuzzer] Replace memmem with strstr.
Summary: Memmem is not available on Windows.

Patch By: metzman

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: george.karpenkov, morehouse

Differential Revision: https://reviews.llvm.org/D51692

llvm-svn: 341495
2018-09-05 21:03:43 +00:00
Max Moroz 651abaf536 Enable symbolize-deadlock.test on Windows
Summary:
Remove unneeded #include of <unistd.h> from SymbolizeDeadlock.cpp to
enable compilation on Windows and enable symbolize-deadlock.test on
Windows.

Patch by Jonathan Metzman (@metzman).

Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: kcc, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D51685

llvm-svn: 341473
2018-09-05 14:43:38 +00:00
Matt Morehouse 24568789c4 [libFuzzer] Enable tests on Windows
Summary:
Enable tests on Windows and make check-fuzzer pass on it.  Make
check-fuzzer pass on Windows by fixing libFuzzer, fixing tests, and by
disabling tests on Windows. Most of these are disabled temporarily as
support for the tests and the features they test will be added
incrementally.  Other tests will not be enabled since they require
things that are not on Windows (eg: afl_driver tests).  Every test
that was explicitly disabled on Windows has a comment explaining why
(unless obvious like merge-posix.test).

The lit.cfg file was modified to support running tests on windows.
fuzzer-dirs.test was fixed by making the Windows implementation print
the same error message as the posix version.
merge-control-file.test was fixed by making the test binary end with
the ".exe" extension (on all platforms).

Patch By: metzman

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: srhines, mgorny

Differential Revision: https://reviews.llvm.org/D51549

llvm-svn: 341385
2018-09-04 17:08:47 +00:00
Max Moroz 8c95b48ba2 [libFuzzer] Remove mutation stats and weighted mutation selection.
Summary:
This was an experimental feature. After evaluating it with:

1) https://github.com/google/fuzzer-test-suite/tree/master/engine-comparison

2) enabling on real world fuzz targets running at ClusterFuzz and OSS-Fuzz

The following conclusions were made:

1) With fuzz targets that have reached a code coverage plateau, the feature does
   not improve libFuzzer's ability to discover new coverage and may actually
   negatively impact it.

2) With fuzz targets that have not yet reached a code coverage plateau, the
   feature might speed up new units discovery in some cases, but it is quite
   rare and hard to confirm with a high level on confidence.

Revert of https://reviews.llvm.org/D48054 and https://reviews.llvm.org/D49621.

Reviewers: metzman, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D51455

llvm-svn: 340976
2018-08-29 21:53:15 +00:00
Roman Lebedev 630fa0ee87 [NFC] Some small test updates for Implicit Conversion sanitizer.
Split off from D50251.

llvm-svn: 339996
2018-08-17 07:33:38 +00:00
Matt Morehouse 0f22fac274 [SanitizerCoverage] Add associated metadata to PC guards.
Summary:
Without this metadata LLD strips unused PC table entries
but won't strip unused guards.  This metadata also seems
to influence the linker to change the ordering in the PC
guard section to match that of the PC table section.

The libFuzzer runtime library depends on the ordering
of the PC table and PC guard sections being the same.  This
is not generally guaranteed, so we may need to redesign
PC tables/guards/counters in the future.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kcc, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D50483

llvm-svn: 339733
2018-08-14 22:04:34 +00:00
George Karpenkov 67d858d5e4 [libFuzzer] Increase the iteration limit in shrink.test
After https://reviews.llvm.org/D48800, shrink.test started failing on
x86_64h architecture.
Looking into this, the optimization pass is too eager to unroll the loop
on x86_64h, possibly leading to worse coverage data.

Alternative solutions include not unrolling the loop when fuzzing, or
disabling this test on that architecture.

Differential Revision: https://reviews.llvm.org/D50484

llvm-svn: 339303
2018-08-09 00:41:22 +00:00
Max Moroz e03993e6c7 [libFuzzer] Disable print_unstable_stats.test for aarch64.
Summary:
Follow-up for https://reviews.llvm.org/D50264. Reported by testbots:
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/5592

Reviewers: morehouse, kevinwkt, metzman, javed.absar, Dor1s

Reviewed By: Dor1s

Subscribers: kristof.beyls, delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D50386

llvm-svn: 339143
2018-08-07 14:22:00 +00:00
Max Moroz 84a48271d4 [libFuzzer] Add unstable function printing to print_unstable_stats flag
Summary:
There may be cases in which a user wants to know which part of their code is unstable.
We use ObservedFuncs and UnstableCounters to print at exit which of the ObservedFunctions
are unstable under the -print_unstable_stats flag.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, metzman, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D50264

llvm-svn: 339081
2018-08-06 23:14:13 +00:00
Max Moroz 5a9baa330c [libFuzzer] Initial implementation of weighted mutation leveraging during runtime.
Summary:
Added functions that calculate stats while fuzz targets are running and give
mutations weight based on how much new coverage they provide, and choose better
performing mutations more often.

Patch by Kodé Williams (@kodewilliams).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, morehouse

Subscribers: delcypher, kcc, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49621

llvm-svn: 338776
2018-08-02 22:30:03 +00:00
Kostya Serebryany cedebd5940 [libFuzzer] use absolute distance in addition to the hamming distance in value profiling; our A/B testing have (somewhat weak) indication that this provides an additional signal for corpus expansion
llvm-svn: 338661
2018-08-02 00:24:49 +00:00
Roman Lebedev eb4a9bc343 [compiler-rt][ubsan] Implicit Conversion Sanitizer - integer truncation - compiler-rt part
Summary:
This is a compiler-rt part.
The clang part is D48958.

See [[ https://bugs.llvm.org/show_bug.cgi?id=21530 | PR21530 ]], https://github.com/google/sanitizers/issues/940.

Reviewers: #sanitizers, samsonov, vsk, rsmith, pcc, eugenis, kcc, filcab

Reviewed By: #sanitizers, vsk, filcab

Subscribers: llvm-commits, eugenis, filcab, kubamracek, dberris, #sanitizers, regehr

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D48959

llvm-svn: 338287
2018-07-30 18:58:30 +00:00
Max Moroz f55b8c6e91 [libFuzzer] Disable handle-unstable.test for ARM and delete duplicated test file.
Summary:
This change should fix the failures mentioned in
https://reviews.llvm.org/D49684#1175245

Reviewers: kevinwkt, morehouse, metzman

Reviewed By: kevinwkt, morehouse

Subscribers: kristof.beyls, delcypher, chrib, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49810

llvm-svn: 337949
2018-07-25 18:34:20 +00:00
Max Moroz 8db0befc6d [libFuzzer] Handle unstable edges by disregarding unstable edges
Summary:
Added a new mode within flag -handle_unstable for new unstable handling algorithm that does the following:
    When an edge is shown as unstable, copy to UnstableCounters the value 0.
    During ApplyUnstableCounters we copy back the value 0 to ModuleInline8bitCounters if the edge was unstable.

This way we would be ignoring completely features that were collected through non-determinism.
Unstable hits would be counted as if it never hit.

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49684

llvm-svn: 337853
2018-07-24 21:02:44 +00:00
Max Moroz 1e954f78d1 [libFuzzer] Handle unstable edges by using minimum hit counts
Summary:
Created unstable_handle flag that takes 1 or 2, depending on the handling type.
Modified RunOne to accommodate the following heuristic:
    Use the first CollectFeatures to count how many features there are.
    If no new features, CollectFeatures like before.
    If there is new feature, we run CB 2 more times,
        Check which edges are unstable per input and we store the least amount of hit counts for each edge.
        Apply these hit counts back to inline8bitcounters so that CollectFeatures can work as intended.
Modified UnstableCounters to 8int_t and created a bitset UnstableSet to tell which edges are unstable.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49525

llvm-svn: 337696
2018-07-23 14:20:52 +00:00
Joel E. Denny a47cb644c7 [libFuzzer] Use separate test directory for each config
Previously, check-all failed many tests for me.  It was running the
X86_64DefaultLinuxConfig, X86_64LibcxxLinuxConfig, and
X86_64StaticLibcxxLinuxConfig configs out of
llvm-build/projects/compiler-rt/test/fuzzer.  Now, it runs them out of
separate subdirectories there, and most tests pass.

Reviewed By: morehouse, george.karpenkov

Differential Revision: https://reviews.llvm.org/D49249

llvm-svn: 337521
2018-07-20 02:39:01 +00:00
Kostya Serebryany 44edc281d9 [libFuzzer] when -print_coverage=1 is given, print more stats (the number of seeds that hit every given function)
llvm-svn: 337501
2018-07-19 22:00:48 +00:00
Kostya Serebryany 79bd14772f [libFuzzer] quick hack to fix the bot
llvm-svn: 337436
2018-07-19 01:54:28 +00:00
Kostya Serebryany 6b87e0c18f [libFuzzer] first experimental attempt at DFT-based mutations (DFT=data-flow-trace)
llvm-svn: 337434
2018-07-19 01:23:32 +00:00
Max Moroz 061b4af998 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337324
2018-07-17 20:37:40 +00:00
Matt Morehouse 43a2296976 libFuzzer: prevent irrelevant strings from leaking into auto-dictionary
This is a fix for bug 37047.

https://bugs.llvm.org/show_bug.cgi?id=37047

Implemented by basically reversing the logic. Previously all strings
were considered, with some operations excluded. Now strings are excluded
by default, and only strings during the CB considered.

Patch By: pdknsk

Differential Revision: https://reviews.llvm.org/D48800

llvm-svn: 337296
2018-07-17 16:12:00 +00:00
Matt Morehouse ef521ffe24 [libFuzzer] Avoid STL in MSan test.
Summary:
STL can cause MSan false positives if lib[std]c++ isn't instrumented
with MSan.

Reviewers: kcc

Reviewed By: kcc

Subscribers: Dor1s, llvm-commits

Differential Revision: https://reviews.llvm.org/D49404

llvm-svn: 337224
2018-07-16 23:22:54 +00:00
Max Moroz 5697c59c7f Revert r337194 (https://reviews.llvm.org/D48891) due to compilation errors.
llvm-svn: 337206
2018-07-16 20:05:18 +00:00
Alex Lorenz 88feedd7c7 [cmake][libFuzzer] fixup r337193 to ensure msan/dfsan are not added
to test deps for libfuzzer when they're not supported by the platform

llvm-svn: 337203
2018-07-16 19:41:49 +00:00
Max Moroz 8a5083df53 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337194
2018-07-16 17:50:46 +00:00
Matt Morehouse 0876a889f7 [libFuzzer] Add msan and dfsan to test deps.
Required now that we have tests using MSan and DFSan.

llvm-svn: 337193
2018-07-16 17:37:15 +00:00
Max Moroz 08dad54924 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337187
2018-07-16 16:01:31 +00:00
Max Moroz 1d369a5d01 Revert r337175 (https://reviews.llvm.org/D49212) due to unintentional format changes.
llvm-svn: 337180
2018-07-16 15:15:34 +00:00
Max Moroz 2156d885e0 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337175
2018-07-16 14:54:23 +00:00
George Karpenkov 01c762b9fd [fuzzer] [tests] Increase the number of iterations for three-bytes.test
The test is flaky otherwise on some of our macOS machines in the test fleet.

Differential Revision: https://reviews.llvm.org/D49277

llvm-svn: 336966
2018-07-13 01:21:50 +00:00
Matt Morehouse 4543816150 [SanitizerCoverage] Add associated metadata to 8-bit counters.
Summary:
This allows counters associated with unused functions to be
dead-stripped along with their functions.  This approach is the same one
we used for PC tables.

Fixes an issue where LLD removes an unused PC table but leaves the 8-bit
counter.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, hiraditya, kcc

Differential Revision: https://reviews.llvm.org/D49264

llvm-svn: 336941
2018-07-12 20:24:58 +00:00
Matt Morehouse 24d617cbee [libFuzzer] If LLD available, require it to build first.
Since we now have a test that requires LLD, make sure it is built before
that test runs.

llvm-svn: 336932
2018-07-12 18:52:10 +00:00
Matt Morehouse d507c96ff1 [libFuzzer] Use lld-available for gc-sections.test.
The lld feature is never available for libFuzzer tests, so
gc-sections.test never actually runs.

llvm-svn: 336926
2018-07-12 18:09:03 +00:00
Joel E. Denny a09b9317f5 [FileCheck] Add -allow-deprecated-dag-overlap to another compiler-rt test
See https://reviews.llvm.org/D47106 for details.

llvm-svn: 336859
2018-07-11 22:07:31 +00:00
Matt Morehouse 79d55d30c3 [libFuzzer] Disable dataflow.test on AArch64.
Summary:
After my recent change to allow MSan + libFuzzer, the
ExplodeDFSanLabelsTest.cpp test started to overflow the stack with
recursive function SetBytesForLabel() on an AArch64 bot.  Perhaps that
bot has a smaller stack size, or maybe AArch64 has larger stack frames
for this particular function.

Reviewers: kcc, javed.absar

Reviewed By: kcc

Subscribers: kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D49150

llvm-svn: 336725
2018-07-10 20:45:36 +00:00
Matt Morehouse 0e904e8806 Revert "[Fuzzer] Afl driver changing iterations handling"
This reverts rL334510 due to breakage of afl_driver's command line
interface.

Patch By: Jonathan Metzman

Differential Revision: https://reviews.llvm.org/D49141

llvm-svn: 336719
2018-07-10 19:58:42 +00:00
Dan Liew 9af77974e2 [LibFuzzer] Disable MSan test on Darwin which was added by r336619. The MemorySanitizer is
not supported on Darwin currently and so Clang refuses to compile with
`-fsanitize=memory`.

llvm-svn: 336669
2018-07-10 13:35:35 +00:00
Matt Morehouse a34c65e845 [libFuzzer] Make -fsanitize=memory,fuzzer work.
This patch allows libFuzzer to fuzz applications instrumented with MSan
without recompiling libFuzzer with MSan instrumentation.

Fixes https://github.com/google/sanitizers/issues/958.

Differential Revision: https://reviews.llvm.org/D48891

llvm-svn: 336619
2018-07-09 23:51:08 +00:00
Matt Morehouse a5bb6d53f2 Revert "[libFuzzer] Mutation tracking and logging implemented"
This reverts r336597 due to bot breakage.

llvm-svn: 336616
2018-07-09 22:31:26 +00:00
Matt Morehouse d153d46884 [libFuzzer] Mutation tracking and logging implemented
Code now exists to track number of mutations that are used in fuzzing in
total and ones that produce new coverage. The stats are currently being
dumped to the command line.

Patch By: Kode Williams

Differntial Revision: https://reviews.llvm.org/D48054

llvm-svn: 336597
2018-07-09 20:17:52 +00:00
Yvan Roux 19d28309af Recommit "[CMake] Run libFuzzer tests with check-all."
Since problematic tests on AArch64 were disabled at r336446.

llvm-svn: 336449
2018-07-06 17:22:02 +00:00
Yvan Roux becbb87553 [libFuzzer] Disable hanging tests on AArch64
Disable problematic tests which broke AArch64 bots.
Details available in Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=38034

Differential Revision: https://reviews.llvm.org/D49011

llvm-svn: 336446
2018-07-06 17:06:01 +00:00
Yvan Roux 01c0d860f3 Revert "[CMake] Run libFuzzer tests with check-all."
Revert due to AArch64 bots breakage, upstream PR raised to track the
issue: https://bugs.llvm.org/show_bug.cgi?id=38034

llvm-svn: 336341
2018-07-05 12:19:03 +00:00
Kostya Serebryany 51ddb88300 [libFuzzer] add one more value profile metric, under a flag (experimental)
llvm-svn: 336234
2018-07-03 22:33:09 +00:00
Kostya Serebryany 0bc9143de8 [libFuzzer] add a tiny and surprisingly hard puzzle
llvm-svn: 336229
2018-07-03 21:17:55 +00:00
Petr Hosek 887f26d470 Support for multiarch runtimes layout
This change adds a support for multiarch style runtimes layout, so in
addition to the existing layout where runtimes get installed to:

lib/clang/$version/lib/$os

Clang now allows runtimes to be installed to:

lib/clang/$version/$target/lib

This also includes libc++, libc++abi and libunwind; today those are
assumed to be in Clang library directory built for host, with the
new layout it is possible to install libc++, libc++abi and libunwind
into the runtime directory built for different targets.

The use of new layout is enabled by setting the
LLVM_ENABLE_RUNTIME_TARGET_DIR CMake variable and is supported by both
projects and runtimes layouts. The runtimes CMake build has been further
modified to use the new layout when building runtimes for multiple
targets.

Differential Revision: https://reviews.llvm.org/D45604

llvm-svn: 335809
2018-06-28 03:11:52 +00:00
George Karpenkov 348dd9ddd1 [libFuzzer] [Tests] [NFC] Change seed for reduce_inputs.test
Differential Revision: https://reviews.llvm.org/D48686

llvm-svn: 335802
2018-06-28 00:39:50 +00:00
George Karpenkov 0cad9cb9a4 [libFuzzer] [NFC] [Tests] Mark signal handling tests as UNSUPPORTED on Darwin
Under load, these tests tend to fail sporadically on our bots.
In my understanding, the signal handling is not guaranteed to happen
within 2 seconds, and the test is inherently flaky.

llvm-svn: 335792
2018-06-27 23:11:24 +00:00
Matt Morehouse 0ab788e271 [libFuzzer] Mark several tests UNSUPPORTED for aarch64.
Now that check-fuzzer runs as part of check-all, some aarch64 bots had
tests failing.

llvm-svn: 335639
2018-06-26 18:37:09 +00:00
Matt Morehouse 3077c82498 [CMake] Run libFuzzer tests with check-all.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D48200

llvm-svn: 335519
2018-06-25 20:30:03 +00:00
George Karpenkov b085ca5095 [libFuzzer] Filter architectures for testing on Apple platforms.
This is done in all other sanitizers, and was missing on libFuzzer.

llvm-svn: 335290
2018-06-21 21:19:43 +00:00
George Karpenkov e1a50ca120 [libFuzzer] Provide more descriptive names for testing targets.
llvm-svn: 335289
2018-06-21 21:19:24 +00:00
Kuba Mracek badaa15c88 [sanitizer] Unify and generalize Apple platforms in CMake and lit test configs
There's more platforms than just "ios" and "iossim" that we should support, and adding more lit config variables for each platform isn't great. Let's generalize and have a single value that determines what the platform under test is.

Differential Revision: https://reviews.llvm.org/D48309

llvm-svn: 335123
2018-06-20 13:33:42 +00:00
Matt Morehouse 668dd03c8d [libFuzzer] Avoid -fuse-ld=lld on gc-sections.
The bot doesn't recognize lld as a linker even though it has the
property lld-available.

llvm-svn: 334864
2018-06-15 21:01:56 +00:00
Matt Morehouse 0ea9a90b3d [SanitizerCoverage] Add associated metadata to pc-tables.
Summary:
Using associated metadata rather than llvm.used allows linkers to
perform dead stripping with -fsanitize-coverage=pc-table.  Unfortunately
in my local tests, LLD was the only linker that made use of this metadata.

Partially addresses https://bugs.llvm.org/show_bug.cgi?id=34636 and fixes
https://github.com/google/sanitizers/issues/971.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: Dor1s, hiraditya, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48203

llvm-svn: 334858
2018-06-15 20:12:58 +00:00
George Karpenkov 9afe1fa7e2 [libFuzzer] [NFC] XFAIL one of the tests on iOS.
llvm-svn: 334775
2018-06-14 22:18:18 +00:00
George Karpenkov d7e1a9488b [libFuzzer] [NFC] Remaining minor fixes to support testing on devices.
XFAIL's and adding %run commands.

llvm-svn: 334774
2018-06-14 22:13:36 +00:00
George Karpenkov d1e0365ce3 [libFuzzer] [NFC] Support multi-arch and multi-OS building and testing
Differential Revision: https://reviews.llvm.org/D47296

llvm-svn: 334768
2018-06-14 20:46:07 +00:00
George Karpenkov 46b0df649f [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 334766
2018-06-14 20:30:04 +00:00
George Karpenkov 077c75bfd1 [libFuzzer] [NFC] Do not use hardcoded relative paths in tests.
For paths without "%t" inside uniqueness is not guaranteed, and
potential collisions might be present.

Differential Revision: https://reviews.llvm.org/D47288

llvm-svn: 334546
2018-06-12 21:14:33 +00:00
George Karpenkov 43a0f71aa4 [libFuzzer] [NFC] Make compiler command generation more readable.
Use config.clang as a more general option than config.c_compiler.

Differential Revision: https://reviews.llvm.org/D47295

llvm-svn: 334545
2018-06-12 21:14:11 +00:00
David Carlier 301855fb0d [Fuzzer] Afl driver changing iterations handling
Handling differently the iterations with the type limit and eventually an error message.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D47880

llvm-svn: 334510
2018-06-12 15:47:58 +00:00
Kostya Serebryany 67af99235f [libFuzzer] make the corpus elements aware of their data flow traces
llvm-svn: 334158
2018-06-07 01:40:20 +00:00
Kostya Serebryany 7a88f19099 [libFuzzer] simplify a test, hopefully to fix the bot
llvm-svn: 334156
2018-06-07 01:18:43 +00:00
Kostya Serebryany 1fd005f552 [libFuzzer] initial implementation of -data_flow_trace. It parses the data flow trace and prints the summary, but doesn't use the information in any other way yet
llvm-svn: 334058
2018-06-06 01:23:29 +00:00
Kostya Serebryany 980e45fe55 [libFuzzer] add collect_data_flow.py that allows to run the data-flow tracer several times on subsets of inputs bytes, to overcome DFSan out-of-label failures
llvm-svn: 333616
2018-05-31 01:27:07 +00:00
Vitaly Buka 540d7e528d [libFuzzer] Avoid optimization of "abs(x) < 0"
llvm-svn: 333295
2018-05-25 18:27:12 +00:00
Vitaly Buka b161db099d Revert "[libFuzzer] [NFC] Generalize DSO tests to work even when files are moved."
Breaks libFuzzer tests.

This reverts commit r333243.

llvm-svn: 333257
2018-05-25 07:18:29 +00:00
George Karpenkov 698b0a674d [libFuzzer] [NFC] Generalize fdmask test.
The ordering of stderr/stdout writes might not be guaranteed,
and is irrelevant for this test.

Differential Revision: https://reviews.llvm.org/D47294

llvm-svn: 333245
2018-05-24 23:55:54 +00:00
George Karpenkov b4ccb3ab08 [libFuzzer] [NFC] Do not remove SanitizerLintCheck from libFuzzer tests
With %run commands being present now, we want to make sure that they
stay present.

Differential Revision: https://reviews.llvm.org/D47293

llvm-svn: 333244
2018-05-24 23:55:54 +00:00
George Karpenkov fe44eb4cf1 [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 333243
2018-05-24 23:55:53 +00:00
George Karpenkov 464d1415b2 [libFuzzer] Fix multi-arch build support for fuzzer tests.
The bug was not previously exposed as only one architecture is currently
supported for libFuzzer.

Differential Revision: https://reviews.llvm.org/D47287

llvm-svn: 333242
2018-05-24 23:55:53 +00:00
George Karpenkov 0dcb6902f1 [libFuzzer] XFAIL a few libFuzzer tests on iOS.
NFC for now.

Differential Revision: https://reviews.llvm.org/D47284

llvm-svn: 333240
2018-05-24 23:55:52 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00
Kostya Serebryany 500ca8713c [libFuzzer] fix two off-by-ones (!!) in the data flow tracer
llvm-svn: 333142
2018-05-23 23:55:54 +00:00
Kostya Serebryany 4925392897 [libFuzzer] change the output format for the DataFlow tracer
llvm-svn: 333122
2018-05-23 20:57:11 +00:00
Kostya Serebryany 28fe54fac8 [libFuzzer] add a stress test for the DataFlow tracer
llvm-svn: 333119
2018-05-23 20:23:33 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00
Kostya Serebryany a52d30b2bc [libFuzzer] rename a test from .c to .cpp
llvm-svn: 332558
2018-05-16 23:38:53 +00:00
Kostya Serebryany e9c6f06cce [libFuzzer] add an experimental flag -focus_function: libFuzzer will try to focus on inputs that trigger that function
llvm-svn: 332554
2018-05-16 23:26:37 +00:00
Max Moroz d63396097d [libFuzzer] add a symbolic execution puzzle (difficult for today's libFuzzer).
Summary:
This can be solved just in seconds with KLEE. Current libFuzzer
is able to satistfy 101 constraints out of 410 constraints presented during
the first hour of running with -use_value_profile=1 and -max_len=20.
During the next 3 hours, libFuzzer is able to generate ~50 NEW inputs,
bot none of those solve any new constraint.
During the next 20 hours, it didn't find any NEW inputs.

This test might be interesting for experimenting with the data flow tracing
approach started in https://reviews.llvm.org/D46666.

For the solution with KLEE and other information, see
https://github.com/Dor1s/codegate2017-quals-angrybird

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D46924

llvm-svn: 332507
2018-05-16 18:19:30 +00:00
Kostya Serebryany 2f7edaeb39 [libFuzzer] deprecate equivalence_server
llvm-svn: 332316
2018-05-15 01:15:47 +00:00
George Karpenkov 669375814c [libFuzzer] [NFC] Split fuzzer.test into smaller tests
Keeping fuzzer.test together as a gigantic test has no advantages and multiple disadvantages:

 - Worse test parallelization (fuzzer.test is always blocking the test
run on a machine with many cores)
 - Debugging test failures is made more difficult (not clear what fails
from fuzzer.test failing)
 - Makes porting tests to new platforms more difficult (whenever
fuzzer.test fails have to inspect the output to figure out what is going
on, and then restart all of it)
 - Hides dead code (in our case, "Done1000000" FileCheck variable was
never used, DISABLED: not %t-UninstrumentedTest-Uninstrumented was never
compiled, and there was small amount of duplication vs. simple-cmp.test)
 - Makes correspondence between LIT .test files and .cpp files less
straightforward

Differential Revision: https://reviews.llvm.org/D46557

llvm-svn: 332145
2018-05-11 21:29:53 +00:00
George Karpenkov fa57519e31 [libFuzzer] [NFC] Prefix all run lines with %run
This patch does not do anything, but paves the way for future changes,
where %run command will be expanded into a script performing the testing
on device.

Differential Revision: https://reviews.llvm.org/D46553

llvm-svn: 332144
2018-05-11 21:29:51 +00:00
Kostya Serebryany 6a6e690d24 [libFuzzer] refactor the implementation of -print_coverage
llvm-svn: 332073
2018-05-11 01:17:52 +00:00
Kostya Serebryany 5d95f2782d [libFuzzer] simplify tests, remove one redundant test; NFC
llvm-svn: 332037
2018-05-10 20:37:08 +00:00
Kostya Serebryany d80e821646 [libFuzzer] remove the dump_coverage flag, it hasn't been working with the inline sanitizer coverage anyway
llvm-svn: 332036
2018-05-10 20:24:39 +00:00
Kostya Serebryany 59bf1e8b36 [libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage
llvm-svn: 332034
2018-05-10 20:12:15 +00:00
Kostya Serebryany f489e2bfef [libFuzzer] Experimental data flow tracer for fuzz targets.
Summary:
Experimental data flow tracer for fuzz targets.
Allows to tell which bytes of the input affect which functions of the fuzz target.

We previously attempted to use DFSan directly in the libFuzzer process,
and that didn't work nicely.
Now we will try to collect the data flow information for the seed corpus
in a separate process (using this tracer), and then use it in the regular libFuzzer runs.

Reviewers: morehouse, pcc, Dor1s

Reviewed By: morehouse, Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D46666

llvm-svn: 332029
2018-05-10 19:59:01 +00:00
Kamil Rytarowski aee7b0f209 wrong usages of sem_open in the libFuzzer
Summary:
Fixed two non-standard usages of sem_open in the libFuzzer library and
one NetBSD-related modification with test script.


  - The return value to indicate error should be SEM_FAILED instead of
    (void *)-1 (please refer to "RETURN VALUE" section in this [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | page ]]). Actually, SEM_FAILED != (void *)-1 holds in NetBSD.

  - The SharedMemoryRegion::SemName function should return name
    starting with slash. Because the behaviour of name which does not
    start with slash is unspecified as the [[
    http://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html
    | "DESCRIPTION" section ]] specified:

> If name does not begin with the <slash> character, the effect is implementation-defined.

  - The length of name is limited to 14 in NetBSD, it is suggested to
    reduce the length of equivalence server name in the test script.

Patch by: Yang Zheng

Reviewers: vitalybuka, krytarowski, kcc

Reviewed By: kcc

Subscribers: kcc, #sanitizers, llvm-commits, joerg

Differential Revision: https://reviews.llvm.org/D46622

llvm-svn: 332003
2018-05-10 17:31:06 +00:00
Kostya Serebryany d790effacb [libFuzzer] add a simple puzzle that is difficult for today's libFuzzer
llvm-svn: 331951
2018-05-10 02:02:41 +00:00
Matt Morehouse 0a4d213176 [libFuzzer] Disable print-func.test on Darwin.
The try-lock guard change seems to be making this test fail on Mac, but
I haven't been able to reproduce the failure.  Disabling the test on Mac
to fix build bot.

llvm-svn: 331894
2018-05-09 17:05:07 +00:00
Matt Morehouse 14cf71a3a5 [libFuzzer] Guard symbolization with try-lock.
Summary:
When out-of-memory or timeout occurs, threads can be stopped during
symbolization, thereby causing a deadlock when the OOM/TO handlers
attempt symbolization.  We avoid this deadlock by skipping symbolization
if another thread is symbolizing.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D46605

llvm-svn: 331825
2018-05-08 23:45:05 +00:00
Matt Morehouse 52fd169035 [libFuzzer] Report at most one crash per input.
Summary:
Fixes https://github.com/google/sanitizers/issues/788/, a deadlock
caused by multiple crashes happening at the same time.  Before printing
a crash report, we now test and set an atomic flag.  If the flag was
already set, the crash handler returns immediately.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D46277

llvm-svn: 331310
2018-05-01 21:01:53 +00:00
Dan Liew b7a3bc99c9 [LibFuzzer] Report when custom counters are available.
This upstreams a feature from the JFS solver's fork of LibFuzzer.

Differential Revision: https://reviews.llvm.org/D45675

llvm-svn: 330391
2018-04-20 06:46:19 +00:00
Matt Morehouse fcc97b29c1 [libFuzzer] Print a correct error message when a directory can't be
opened.

Summary:
Currently if the directory cannot be opened for a reason other than
non-existence (e.g. too many open file descriptors) the error message
printed is incredibly confusing.

Patch By: Alex Gaynor

Reviewers: kcc, morehouse

Reviewed By: morehouse

Subscribers: delcypher, llvm-commits, Sanitizers

Differential Revision: https://reviews.llvm.org/D45322

llvm-svn: 329438
2018-04-06 18:15:24 +00:00
Matt Morehouse 86db6dd2ed [libFuzzer] Disable optimization for exit_on_src_pos.test.
Test fails on Darwin with -O2.

llvm-svn: 328634
2018-03-27 16:40:34 +00:00
Matt Morehouse 22a1afd99f [libFuzzer] Place volatile after pointer types.
For a few tests, volatile was placed before the '*' in pointer
declarations, resulting in it applying to the underlying data rather
than the pointer itself.  Placing volatile after the '*' allows us to
switch those tests to -O2.

llvm-svn: 328633
2018-03-27 16:40:20 +00:00
Matt Morehouse 93ce24838c [libFuzzer] Do not optimize minimize_two_crashes.test.
Speculative fix for build bot breakage on Mac.

llvm-svn: 328576
2018-03-26 19:36:20 +00:00
Matt Morehouse 5317f2e4c9 [libFuzzer] Use OptForFuzzing attribute with -fsanitize=fuzzer.
Summary:
Disables certain CMP optimizations to improve fuzzing signal under -O1
and -O2.

Switches all fuzzer tests to -O2 except for a few leak tests where the
leak is optimized out under -O2.

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: cfe-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D44798

llvm-svn: 328384
2018-03-23 23:35:28 +00:00
George Karpenkov 430aa7405f [libFuzzer] Disable equivalence-signals test on Darwin due to persistent failures
llvm-svn: 328125
2018-03-21 17:55:36 +00:00
Petr Hosek 2d19f9036e [Fuzzer] Avoid the unnecessary rebuild of the custom libc++
This changes the add_custom_libcxx macro to resemble the
llvm_ExternalProject_Add. The primary motivation is to avoid
unnecessary libFuzzer rebuilds that are being done on every
Ninja/Make invocation. The libc++ should be only rebuilt whenever
the libc++ source itself changes.

Differential Revision: https://reviews.llvm.org/D43213

llvm-svn: 326921
2018-03-07 18:14:09 +00:00
Hiroshi Inoue 93eaad7dac [NFC] fix trivial typos in comments
"the the" -> "the"

llvm-svn: 323080
2018-01-22 07:51:37 +00:00
Petr Hosek eac2b47b9f Reland "[libFuzzer] Support using libc++"
This is needed in case the users of libFuzzer use libc++ in their
code, which the fuzz target (libFuzzer) will be linked against.
When libc++ source is available, we build a private version of it
and link it against libFuzzer which allows using the same static
library against codebases which use both libc++ and libstdc++.

Differential Revision: https://reviews.llvm.org/D37631

llvm-svn: 322755
2018-01-17 20:39:14 +00:00
Petr Hosek 68bc4bd6e5 Revert "[libFuzzer] Support using libc++"
This reverts commit r322604: test is failing for standalone compiler-rt.

llvm-svn: 322689
2018-01-17 17:24:56 +00:00
Petr Hosek a1b57e694e [libFuzzer] Support using libc++
This is needed in case the users of libFuzzer use libc++ in their
code, which the fuzz target (libFuzzer) will be linked against.
When libc++ source is available, we build a private version of it
and link it against libFuzzer which allows using the same static
library against codebases which use both libc++ and libstdc++.

Differential Revision: https://reviews.llvm.org/D37631

llvm-svn: 322604
2018-01-17 00:42:48 +00:00
Matt Morehouse 59e4e400c3 [libFuzzer] Fix bug introduced in r322380 that always links -lc++.
llvm-svn: 322409
2018-01-12 22:10:47 +00:00
Kamil Rytarowski e81e944199 lib Fuzzer FreeBSD support
Summary: Patch by David CARLIER

Reviewers: vitalybuka, kcc, dim, emaste, davide, morehouse, george.karpenkov

Reviewed By: morehouse

Subscribers: george.karpenkov, kubamracek, srhines, mgorny, emaste, krytarowski

Differential Revision: https://reviews.llvm.org/D41642

llvm-svn: 322380
2018-01-12 17:15:05 +00:00
Kostya Serebryany b99a7102c1 [libFuzzer] make the tests depend on ubsan
llvm-svn: 320787
2017-12-15 01:39:52 +00:00
Kostya Serebryany e9ed2327b6 [libFuzzer] change the strategy for -experimental_len_control to grow max_len slower
llvm-svn: 320531
2017-12-12 23:11:28 +00:00
Kostya Serebryany de891a1946 [libFuzzer] even less aggressive step function for stack depth. Also don't use asan in one test to speed it up
llvm-svn: 320259
2017-12-09 19:18:10 +00:00
Kostya Serebryany de9bafb162 [libFuzzer] add a flag -malloc_limit_mb
llvm-svn: 319590
2017-12-01 22:12:04 +00:00
Matt Morehouse 6497193cfe [libFuzzer] Fix flaky merge.test.
llvm-svn: 318475
2017-11-17 00:11:12 +00:00
Matt Morehouse 12615ec49b [libFuzzer] Make merge-sigusr.test work on Mac.
llvm-svn: 318465
2017-11-16 23:06:17 +00:00
Kostya Serebryany dc3c9eb0cf [libFuzzer] add more tests for merge
llvm-svn: 318453
2017-11-16 20:25:45 +00:00
Kostya Serebryany 0e194faeea [libFuzzer] disable test/fuzzer/merge-sigusr.test on Mac while investigating the bot failure
llvm-svn: 318376
2017-11-16 04:52:05 +00:00
Kostya Serebryany 8eb9ca2580 [libFuzzer] respect max_len during merge
llvm-svn: 318302
2017-11-15 16:42:52 +00:00
Kostya Serebryany c183e2c33b [libFuzzer] trying to make sigusr tests more reliable
llvm-svn: 317852
2017-11-10 00:26:23 +00:00
Kostya Serebryany e8637f45ea [libFuzzer] split a test into two
llvm-svn: 317851
2017-11-10 00:18:13 +00:00
Kostya Serebryany 533cbf749c [libFuzzer] use pkill instead of killall in a test since the bots don't have killall
llvm-svn: 317840
2017-11-09 22:01:06 +00:00
Kostya Serebryany 54a8456005 [libFuzzer] make sure to flush IO when done merging one file
llvm-svn: 317835
2017-11-09 21:30:33 +00:00
Matt Morehouse 947838c9c5 [libFuzzer] Don't add leaking inputs to corpus.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39850

llvm-svn: 317831
2017-11-09 20:44:08 +00:00
Kostya Serebryany a2ca2dcc46 [libFuzzer] handle SIGUSR1/SIGUSR2 and try to exit grafully on these signals
llvm-svn: 317829
2017-11-09 20:30:19 +00:00
Kostya Serebryany 3ac31d4453 [libFuzzer] allow merge to resume after being preempted
llvm-svn: 317767
2017-11-09 05:49:28 +00:00
Kostya Serebryany e1d121d715 [libFuzzer] mechanically simplify a test, NFC
llvm-svn: 317756
2017-11-09 02:13:43 +00:00
Kostya Serebryany 8a94f220aa [libFuzzer] fix a test (and hopefully, the bot)
llvm-svn: 317749
2017-11-09 01:45:59 +00:00
Kostya Serebryany 68fdef1f7a [libFuzzer] allow user to specify the merge control file
llvm-svn: 317747
2017-11-09 01:05:29 +00:00
Vitaly Buka ca9075272b [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317119
2017-11-01 20:27:06 +00:00
Vitaly Buka 7dbc1d8433 [fuzzer] Fix threaded stack printing
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317071
2017-11-01 03:02:59 +00:00
Vitaly Buka df00e89cdc Revert "[fuzzer] Fix threaded stack printing and nested mallocs"
Fails on darwin

Revert "[fuzzer] Script to detect unbalanced allocation in -trace_malloc output"
Needs previous one.

This reverts commit r317034, r317036.

llvm-svn: 317061
2017-11-01 00:55:52 +00:00
Vitaly Buka 0747700ce1 [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317036
2017-10-31 20:50:07 +00:00
Vitaly Buka 58da33e35c [fuzzer] Fix threaded stack printing and nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317034
2017-10-31 20:49:48 +00:00
Kostya Serebryany 3c9e59d0f6 [libFuzzer] trying to make a test more stable on Mac
llvm-svn: 316627
2017-10-25 23:24:45 +00:00
Kostya Serebryany 51823d3aae [libFuzzer] tweam use_feature_frequency to be less aggressive; run a dummy input before the seed corpus
llvm-svn: 315657
2017-10-13 01:12:23 +00:00
Max Moroz 330496c3fc [libFuzzer] Disable experimental clang coverage support by default.
Summary:
It can be enabled via "-use_clang_coverage=1" flag. Reason for disabling:
libFuzzer resets Clang Counters and makes it impossible to generate coverage
report for a regular fuzz target (i.e. not standalone build).

Reviewers: kcc

Reviewed By: kcc

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38604

llvm-svn: 315029
2017-10-05 22:41:03 +00:00
Kostya Serebryany 80e9a6ccd1 [libFuzzer] add linux-specific test for gc-sections
llvm-svn: 313421
2017-09-15 23:07:18 +00:00
Kostya Serebryany ad66fefdca [libFuzzer] test fix
llvm-svn: 313411
2017-09-15 22:29:20 +00:00
Max Moroz 1a33da2401 [libFuzzer] Fix lit files to make running tests more straightforward on Mac OS.
Summary:
Current implementation does not work if CMAKE_OSX_SYSROOT is not specified.

It silently generates invalid command with the following flags:

`-std=c++11 -lc++ -gline-tables-only -isysroot  -fsanitize=address,fuzzer`

and then fails with the following error:

```
warning: no such sysroot directory: '-fsanitize=address,fuzzer' [-Wmissing-sysroot]"
<...>/RepeatedBytesTest.cpp:5:10: fatal error: 'assert.h' file not found
#include <assert.h>
         ^~~~~~~~~~
1 error generated.
```

However, if you have Command Line Tools installed, you have '/usr/include' dir.
In that case, it is not necessary to specify isysroot path.

Also, with the patch, in case of '/usr/include' does not exist, the '-sysroot'
path would be resolved automatically in compiler-rt/cmake/base-config-ix.cmake.

For more context, see the comment at `compiler-rt/cmake/base-config-ix.cmake#L76`

Reviewers: kcc, george.karpenkov

Reviewed By: kcc, george.karpenkov

Differential Revision: https://reviews.llvm.org/D37721

llvm-svn: 313033
2017-09-12 15:02:10 +00:00
Max Moroz 3f26dac416 [libfuzzer] Compare TotalNumberOfRuns with MaxNumberOfRuns when testing a memory leak.
Summary:
Fuzzer::TryDetectingAMemoryLeak may call ExecuteCallback which would
increment TotalNumberOfRuns, but it doesn't respect Options.MaxNumberOfRuns
value specified by a user.

Context: https://github.com/google/oss-fuzz/issues/822#issuecomment-328153970

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D37632

llvm-svn: 312993
2017-09-12 02:01:54 +00:00
Kostya Serebryany 754e584076 [libFuzzer] simplify CustomCrossOverTest even more
llvm-svn: 312697
2017-09-07 05:33:05 +00:00
Kostya Serebryany 57c03648e1 [libFuzzer] simplify CustomCrossOverTest a bit more
llvm-svn: 312695
2017-09-07 05:23:23 +00:00
Kostya Serebryany d0386fac26 [libFuzzer] simplify and re-enable CustomCrossOverTest
llvm-svn: 312689
2017-09-07 02:04:06 +00:00
Kostya Serebryany 79cdf36a2c [libFuzzer] remporary disable an unstable test
llvm-svn: 312593
2017-09-05 23:45:54 +00:00
Kostya Serebryany 07490f8415 [libFuzzer] use more iterations for a test
llvm-svn: 312356
2017-09-01 19:45:08 +00:00
George Karpenkov 2928375cd2 [libFuzzer] Enable 8-bit counters test on macOS
llvm-svn: 312339
2017-09-01 17:13:26 +00:00
Matt Morehouse 034126e507 [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer
Summary:
- Don't sanitize __sancov_lowest_stack.
- Don't instrument leaf functions.
- Add CoverageStackDepth to Fuzzer and FuzzerNoLink.
- Only enable on Linux.

Reviewers: vitalybuka, kcc, george.karpenkov

Reviewed By: kcc

Subscribers: kubamracek, cfe-commits, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D37156

llvm-svn: 312185
2017-08-30 22:49:31 +00:00
Matt Morehouse ba2e61b357 Revert "[SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer"
This reverts r312026 due to bot breakage.

llvm-svn: 312047
2017-08-29 21:56:56 +00:00
Kostya Serebryany 4faeb87ebe [libFUzzer] change the way we load the seed corpora: instead of loading all files and these executing all files, load and execute them one-by-one. This should reduce the memory usage in many cases
llvm-svn: 312033
2017-08-29 20:51:24 +00:00
Matt Morehouse 2ad8d948b2 [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer
Summary:
- Don't sanitize __sancov_lowest_stack.
- Don't instrument leaf functions.
- Add CoverageStackDepth to Fuzzer and FuzzerNoLink.
- Disable stack depth tracking on Mac.

Reviewers: vitalybuka, kcc, george.karpenkov

Reviewed By: kcc

Subscribers: kubamracek, cfe-commits, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D37156

llvm-svn: 312026
2017-08-29 19:48:12 +00:00
Kostya Serebryany c07008653c [libFuzzer] allow -print_funcs=N: N is the max number of new covered function printed
llvm-svn: 311945
2017-08-28 22:52:22 +00:00
George Karpenkov 52a3e48e9f [libFuzzer] Exclude a test failing on OS X.
llvm-svn: 311916
2017-08-28 20:10:30 +00:00
Matt Morehouse 6ec7595b1e Revert "[SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer"
This reverts r311801 due to a bot failure.

llvm-svn: 311803
2017-08-25 22:01:21 +00:00
Matt Morehouse f42bd31323 [SanitizeCoverage] Enable stack-depth coverage for -fsanitize=fuzzer
Summary:
- Don't sanitize __sancov_lowest_stack.
- Don't instrument leaf functions.
- Add CoverageStackDepth to Fuzzer and FuzzerNoLink.

Reviewers: vitalybuka, kcc

Reviewed By: kcc

Subscribers: cfe-commits, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D37156

llvm-svn: 311801
2017-08-25 21:18:29 +00:00
Kostya Serebryany 09c4161029 [libFuzzer] prepare tests for switching from -fsanitize-coverage=trace-pc-guard to -fsanitize-coverage=inline-8bit-counters
llvm-svn: 311798
2017-08-25 20:20:46 +00:00
Kostya Serebryany 2eef816e6e [libFuzzer] add -print_funcs=1 (on bey default): print newly discovered functions during fuzzing
llvm-svn: 311797
2017-08-25 20:09:25 +00:00
Kostya Serebryany d3e4b7e24a [sanitizer-coverage] extend fsanitize-coverage=pc-table with flags for every PC
llvm-svn: 311794
2017-08-25 19:29:47 +00:00
Matt Morehouse b1fa8255db [SanitizerCoverage] Optimize stack-depth instrumentation.
Summary:
Use the initialexec TLS type and eliminate calls to the TLS
wrapper.  Fixes the sanitizer-x86_64-linux-fuzzer bot failure.

Reviewers: vitalybuka, kcc

Reviewed By: kcc

Subscribers: hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D37026

llvm-svn: 311490
2017-08-22 21:28:29 +00:00
Kostya Serebryany 868e69091c [libFuzzer] disable a test failing on the bot
llvm-svn: 311427
2017-08-22 05:15:57 +00:00
Kostya Serebryany 33fb36c353 [libFuzzer] fix the stack-depth initialization, add a lit test for DeepRecursionTest.cpp
llvm-svn: 311421
2017-08-22 01:50:00 +00:00
George Karpenkov e4c5322992 [NFC] do not run linter on libFuzzer's tests.
llvm-svn: 311419
2017-08-22 01:19:17 +00:00
George Karpenkov 92d8000487 Remove check-fuzzer from check-all, as tests don't pass on some bots.
llvm-svn: 311415
2017-08-22 00:54:57 +00:00
George Karpenkov 10ab2ace13 Move libFuzzer to compiler_rt.
Resulting library binaries will be named libclang_rt.fuzzer*, and will
be placed in Clang toolchain, allowing redistribution.

Differential Revision: https://reviews.llvm.org/D36908

llvm-svn: 311407
2017-08-21 23:25:50 +00:00