Commit Graph

138 Commits

Author SHA1 Message Date
Max Moroz 8a5083df53 [libFuzzer] Mutation tracking and logging implemented.
Summary:
Code now exists to track number of mutations that are used in fuzzing in total
and ones that produce new coverage. The stats are currently being dumped to the
command line.

Patch by Kodé Williams (@kodewilliams).

Reviewers: metzman, Dor1s, morehouse, kcc

Reviewed By: Dor1s, morehouse, kcc

Subscribers: delcypher, kubamracek, kcc, morehouse, llvm-commits, #sanitizers, mgorny

Differential Revision: https://reviews.llvm.org/D48054

llvm-svn: 337194
2018-07-16 17:50:46 +00:00
Matt Morehouse 0876a889f7 [libFuzzer] Add msan and dfsan to test deps.
Required now that we have tests using MSan and DFSan.

llvm-svn: 337193
2018-07-16 17:37:15 +00:00
Max Moroz 08dad54924 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337187
2018-07-16 16:01:31 +00:00
Max Moroz 1d369a5d01 Revert r337175 (https://reviews.llvm.org/D49212) due to unintentional format changes.
llvm-svn: 337180
2018-07-16 15:15:34 +00:00
Max Moroz 2156d885e0 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337175
2018-07-16 14:54:23 +00:00
George Karpenkov 01c762b9fd [fuzzer] [tests] Increase the number of iterations for three-bytes.test
The test is flaky otherwise on some of our macOS machines in the test fleet.

Differential Revision: https://reviews.llvm.org/D49277

llvm-svn: 336966
2018-07-13 01:21:50 +00:00
Matt Morehouse 4543816150 [SanitizerCoverage] Add associated metadata to 8-bit counters.
Summary:
This allows counters associated with unused functions to be
dead-stripped along with their functions.  This approach is the same one
we used for PC tables.

Fixes an issue where LLD removes an unused PC table but leaves the 8-bit
counter.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, hiraditya, kcc

Differential Revision: https://reviews.llvm.org/D49264

llvm-svn: 336941
2018-07-12 20:24:58 +00:00
Matt Morehouse 24d617cbee [libFuzzer] If LLD available, require it to build first.
Since we now have a test that requires LLD, make sure it is built before
that test runs.

llvm-svn: 336932
2018-07-12 18:52:10 +00:00
Matt Morehouse d507c96ff1 [libFuzzer] Use lld-available for gc-sections.test.
The lld feature is never available for libFuzzer tests, so
gc-sections.test never actually runs.

llvm-svn: 336926
2018-07-12 18:09:03 +00:00
Joel E. Denny a09b9317f5 [FileCheck] Add -allow-deprecated-dag-overlap to another compiler-rt test
See https://reviews.llvm.org/D47106 for details.

llvm-svn: 336859
2018-07-11 22:07:31 +00:00
Matt Morehouse 79d55d30c3 [libFuzzer] Disable dataflow.test on AArch64.
Summary:
After my recent change to allow MSan + libFuzzer, the
ExplodeDFSanLabelsTest.cpp test started to overflow the stack with
recursive function SetBytesForLabel() on an AArch64 bot.  Perhaps that
bot has a smaller stack size, or maybe AArch64 has larger stack frames
for this particular function.

Reviewers: kcc, javed.absar

Reviewed By: kcc

Subscribers: kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D49150

llvm-svn: 336725
2018-07-10 20:45:36 +00:00
Matt Morehouse 0e904e8806 Revert "[Fuzzer] Afl driver changing iterations handling"
This reverts rL334510 due to breakage of afl_driver's command line
interface.

Patch By: Jonathan Metzman

Differential Revision: https://reviews.llvm.org/D49141

llvm-svn: 336719
2018-07-10 19:58:42 +00:00
Dan Liew 9af77974e2 [LibFuzzer] Disable MSan test on Darwin which was added by r336619. The MemorySanitizer is
not supported on Darwin currently and so Clang refuses to compile with
`-fsanitize=memory`.

llvm-svn: 336669
2018-07-10 13:35:35 +00:00
Matt Morehouse a34c65e845 [libFuzzer] Make -fsanitize=memory,fuzzer work.
This patch allows libFuzzer to fuzz applications instrumented with MSan
without recompiling libFuzzer with MSan instrumentation.

Fixes https://github.com/google/sanitizers/issues/958.

Differential Revision: https://reviews.llvm.org/D48891

llvm-svn: 336619
2018-07-09 23:51:08 +00:00
Matt Morehouse a5bb6d53f2 Revert "[libFuzzer] Mutation tracking and logging implemented"
This reverts r336597 due to bot breakage.

llvm-svn: 336616
2018-07-09 22:31:26 +00:00
Matt Morehouse d153d46884 [libFuzzer] Mutation tracking and logging implemented
Code now exists to track number of mutations that are used in fuzzing in
total and ones that produce new coverage. The stats are currently being
dumped to the command line.

Patch By: Kode Williams

Differntial Revision: https://reviews.llvm.org/D48054

llvm-svn: 336597
2018-07-09 20:17:52 +00:00
Yvan Roux 19d28309af Recommit "[CMake] Run libFuzzer tests with check-all."
Since problematic tests on AArch64 were disabled at r336446.

llvm-svn: 336449
2018-07-06 17:22:02 +00:00
Yvan Roux becbb87553 [libFuzzer] Disable hanging tests on AArch64
Disable problematic tests which broke AArch64 bots.
Details available in Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=38034

Differential Revision: https://reviews.llvm.org/D49011

llvm-svn: 336446
2018-07-06 17:06:01 +00:00
Yvan Roux 01c0d860f3 Revert "[CMake] Run libFuzzer tests with check-all."
Revert due to AArch64 bots breakage, upstream PR raised to track the
issue: https://bugs.llvm.org/show_bug.cgi?id=38034

llvm-svn: 336341
2018-07-05 12:19:03 +00:00
Kostya Serebryany 51ddb88300 [libFuzzer] add one more value profile metric, under a flag (experimental)
llvm-svn: 336234
2018-07-03 22:33:09 +00:00
Kostya Serebryany 0bc9143de8 [libFuzzer] add a tiny and surprisingly hard puzzle
llvm-svn: 336229
2018-07-03 21:17:55 +00:00
Petr Hosek 887f26d470 Support for multiarch runtimes layout
This change adds a support for multiarch style runtimes layout, so in
addition to the existing layout where runtimes get installed to:

lib/clang/$version/lib/$os

Clang now allows runtimes to be installed to:

lib/clang/$version/$target/lib

This also includes libc++, libc++abi and libunwind; today those are
assumed to be in Clang library directory built for host, with the
new layout it is possible to install libc++, libc++abi and libunwind
into the runtime directory built for different targets.

The use of new layout is enabled by setting the
LLVM_ENABLE_RUNTIME_TARGET_DIR CMake variable and is supported by both
projects and runtimes layouts. The runtimes CMake build has been further
modified to use the new layout when building runtimes for multiple
targets.

Differential Revision: https://reviews.llvm.org/D45604

llvm-svn: 335809
2018-06-28 03:11:52 +00:00
George Karpenkov 348dd9ddd1 [libFuzzer] [Tests] [NFC] Change seed for reduce_inputs.test
Differential Revision: https://reviews.llvm.org/D48686

llvm-svn: 335802
2018-06-28 00:39:50 +00:00
George Karpenkov 0cad9cb9a4 [libFuzzer] [NFC] [Tests] Mark signal handling tests as UNSUPPORTED on Darwin
Under load, these tests tend to fail sporadically on our bots.
In my understanding, the signal handling is not guaranteed to happen
within 2 seconds, and the test is inherently flaky.

llvm-svn: 335792
2018-06-27 23:11:24 +00:00
Matt Morehouse 0ab788e271 [libFuzzer] Mark several tests UNSUPPORTED for aarch64.
Now that check-fuzzer runs as part of check-all, some aarch64 bots had
tests failing.

llvm-svn: 335639
2018-06-26 18:37:09 +00:00
Matt Morehouse 3077c82498 [CMake] Run libFuzzer tests with check-all.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D48200

llvm-svn: 335519
2018-06-25 20:30:03 +00:00
George Karpenkov b085ca5095 [libFuzzer] Filter architectures for testing on Apple platforms.
This is done in all other sanitizers, and was missing on libFuzzer.

llvm-svn: 335290
2018-06-21 21:19:43 +00:00
George Karpenkov e1a50ca120 [libFuzzer] Provide more descriptive names for testing targets.
llvm-svn: 335289
2018-06-21 21:19:24 +00:00
Kuba Mracek badaa15c88 [sanitizer] Unify and generalize Apple platforms in CMake and lit test configs
There's more platforms than just "ios" and "iossim" that we should support, and adding more lit config variables for each platform isn't great. Let's generalize and have a single value that determines what the platform under test is.

Differential Revision: https://reviews.llvm.org/D48309

llvm-svn: 335123
2018-06-20 13:33:42 +00:00
Matt Morehouse 668dd03c8d [libFuzzer] Avoid -fuse-ld=lld on gc-sections.
The bot doesn't recognize lld as a linker even though it has the
property lld-available.

llvm-svn: 334864
2018-06-15 21:01:56 +00:00
Matt Morehouse 0ea9a90b3d [SanitizerCoverage] Add associated metadata to pc-tables.
Summary:
Using associated metadata rather than llvm.used allows linkers to
perform dead stripping with -fsanitize-coverage=pc-table.  Unfortunately
in my local tests, LLD was the only linker that made use of this metadata.

Partially addresses https://bugs.llvm.org/show_bug.cgi?id=34636 and fixes
https://github.com/google/sanitizers/issues/971.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: Dor1s, hiraditya, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D48203

llvm-svn: 334858
2018-06-15 20:12:58 +00:00
George Karpenkov 9afe1fa7e2 [libFuzzer] [NFC] XFAIL one of the tests on iOS.
llvm-svn: 334775
2018-06-14 22:18:18 +00:00
George Karpenkov d7e1a9488b [libFuzzer] [NFC] Remaining minor fixes to support testing on devices.
XFAIL's and adding %run commands.

llvm-svn: 334774
2018-06-14 22:13:36 +00:00
George Karpenkov d1e0365ce3 [libFuzzer] [NFC] Support multi-arch and multi-OS building and testing
Differential Revision: https://reviews.llvm.org/D47296

llvm-svn: 334768
2018-06-14 20:46:07 +00:00
George Karpenkov 46b0df649f [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 334766
2018-06-14 20:30:04 +00:00
George Karpenkov 077c75bfd1 [libFuzzer] [NFC] Do not use hardcoded relative paths in tests.
For paths without "%t" inside uniqueness is not guaranteed, and
potential collisions might be present.

Differential Revision: https://reviews.llvm.org/D47288

llvm-svn: 334546
2018-06-12 21:14:33 +00:00
George Karpenkov 43a0f71aa4 [libFuzzer] [NFC] Make compiler command generation more readable.
Use config.clang as a more general option than config.c_compiler.

Differential Revision: https://reviews.llvm.org/D47295

llvm-svn: 334545
2018-06-12 21:14:11 +00:00
David Carlier 301855fb0d [Fuzzer] Afl driver changing iterations handling
Handling differently the iterations with the type limit and eventually an error message.

Reviewers: morehouse, kcc

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D47880

llvm-svn: 334510
2018-06-12 15:47:58 +00:00
Kostya Serebryany 67af99235f [libFuzzer] make the corpus elements aware of their data flow traces
llvm-svn: 334158
2018-06-07 01:40:20 +00:00
Kostya Serebryany 7a88f19099 [libFuzzer] simplify a test, hopefully to fix the bot
llvm-svn: 334156
2018-06-07 01:18:43 +00:00
Kostya Serebryany 1fd005f552 [libFuzzer] initial implementation of -data_flow_trace. It parses the data flow trace and prints the summary, but doesn't use the information in any other way yet
llvm-svn: 334058
2018-06-06 01:23:29 +00:00
Kostya Serebryany 980e45fe55 [libFuzzer] add collect_data_flow.py that allows to run the data-flow tracer several times on subsets of inputs bytes, to overcome DFSan out-of-label failures
llvm-svn: 333616
2018-05-31 01:27:07 +00:00
Vitaly Buka 540d7e528d [libFuzzer] Avoid optimization of "abs(x) < 0"
llvm-svn: 333295
2018-05-25 18:27:12 +00:00
Vitaly Buka b161db099d Revert "[libFuzzer] [NFC] Generalize DSO tests to work even when files are moved."
Breaks libFuzzer tests.

This reverts commit r333243.

llvm-svn: 333257
2018-05-25 07:18:29 +00:00
George Karpenkov 698b0a674d [libFuzzer] [NFC] Generalize fdmask test.
The ordering of stderr/stdout writes might not be guaranteed,
and is irrelevant for this test.

Differential Revision: https://reviews.llvm.org/D47294

llvm-svn: 333245
2018-05-24 23:55:54 +00:00
George Karpenkov b4ccb3ab08 [libFuzzer] [NFC] Do not remove SanitizerLintCheck from libFuzzer tests
With %run commands being present now, we want to make sure that they
stay present.

Differential Revision: https://reviews.llvm.org/D47293

llvm-svn: 333244
2018-05-24 23:55:54 +00:00
George Karpenkov fe44eb4cf1 [libFuzzer] [NFC] Generalize DSO tests to work even when files are moved.
Differential Revision: https://reviews.llvm.org/D47292

llvm-svn: 333243
2018-05-24 23:55:53 +00:00
George Karpenkov 464d1415b2 [libFuzzer] Fix multi-arch build support for fuzzer tests.
The bug was not previously exposed as only one architecture is currently
supported for libFuzzer.

Differential Revision: https://reviews.llvm.org/D47287

llvm-svn: 333242
2018-05-24 23:55:53 +00:00
George Karpenkov 0dcb6902f1 [libFuzzer] XFAIL a few libFuzzer tests on iOS.
NFC for now.

Differential Revision: https://reviews.llvm.org/D47284

llvm-svn: 333240
2018-05-24 23:55:52 +00:00
Kostya Serebryany 4d53b744ca [libFuzzer] DataFlow tracer now tags a subset of the input. A separate script merges traces from the subsets
llvm-svn: 333149
2018-05-24 01:43:48 +00:00