Commit Graph

63 Commits

Author SHA1 Message Date
Matt Morehouse 6e294952b6 [libFuzzer] Add dummy call of LLVMFuzzerTestOneInput to afl_driver.
Summary:
Add dummy call of LLVMFuzzerTestOneInput to afl_driver before it starts
executing on actual inputs. Do this so that first time initialization
performed by LLVMFuzzerTestOneInput is not considered code covered by
a particular input.

Patch By: metzman

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: llvm-commits, Sanitizers

Differential Revision: https://reviews.llvm.org/D41193

llvm-svn: 320643
2017-12-13 22:02:44 +00:00
Kostya Serebryany e9ed2327b6 [libFuzzer] change the strategy for -experimental_len_control to grow max_len slower
llvm-svn: 320531
2017-12-12 23:11:28 +00:00
Kostya Serebryany de891a1946 [libFuzzer] even less aggressive step function for stack depth. Also don't use asan in one test to speed it up
llvm-svn: 320259
2017-12-09 19:18:10 +00:00
Kostya Serebryany f94d87aadf [libFuzzer] make the stack depth signal less agressive, otherwise we are sometimes exploding the corpus size. This still needs more analysis and tuning
llvm-svn: 320237
2017-12-09 02:45:23 +00:00
Matt Morehouse ffb1132e52 [libFuzzer] Add support for Fuchsia OS.
Summary:
This patch adds the initial support for Fuchsia.

- LIBFUZZER_FUCHSIA is added as an OS type in FuzzerDefs.h
- Fuchsia is, by design, not POSIX compliant. However, it does use ELF and
  supports common POSIX I/O functions. Thus, FuzzerExtFunctions.h and
  FuzzerIO.h are implemented by extending the header guards in
  FuzzerExtFunctionsWeak.cpp and FuzzerIOPosix.cpp to include
  LIBFUZZER_FUCHSIA.
- The platform-specific portions of FuzzerUtil.h are implemented by
  FuzzerUtilFuchsia.cpp, which makes use of exception ports, syscalls, and
  the launchpad library.
- The experimental equivalence server is not currently supported, so
  FuzzerShmem.h is implemented by stub methods in FuzzerShmemFuchsia.cpp.
  Any future implementation will likely involve VMOs.

Tested with ASAN/SanCov on Fuchsia/x86-64 with the canonical toy fuzzer.

Patch By: aarongreen

Reviewers: kcc, morehouse, flowerhack, phosek

Reviewed By: kcc, phosek, Eugene.Zelenko

Subscribers: srhines, mgorny, Eugene.Zelenko

Differential Revision: https://reviews.llvm.org/D40974

llvm-svn: 320210
2017-12-08 22:54:44 +00:00
Kostya Serebryany 2659c63e2e [libFuzzer] honor -use_counters, sligntly change the meaning of -experimental_len_control, call UpdateFeatureFrequency only if instructed by the flag
llvm-svn: 320205
2017-12-08 22:21:42 +00:00
Kostya Serebryany a97bd9a7d3 [libFuzzer] Decrease stack usage in unit tests
Summary: With 3 Dictionary objects, each containing space of ~16k DictionaryEntry objects, the MutationDispatcher object is fairly memory heavy.  On platforms with a lower default stack size, this can cause panics in FuzzerUnittest as those tests stack-allocate the MutationDispatcher.  This may be especially problematic for platforms that do not (yet) have a way to programmatically change their stack size, aside from link-time flags.  In general, it seems more prudent to use the heap for an object of this size.

Reviewers: kcc, morehouse

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D40926

llvm-svn: 319988
2017-12-06 23:35:02 +00:00
Kostya Serebryany 7ac58ee3e1 [libFuzzer] fix a minor regression in printing
llvm-svn: 319975
2017-12-06 22:12:24 +00:00
Matt Morehouse 056774d13b [libFuzzer] Make redirects happen in proper sequence.
"> file" must come before "2>&1" to have redirection occur correctly in
all cases.  Fixes a regression on minimize_two_crashes.test.

llvm-svn: 319792
2017-12-05 17:13:17 +00:00
Matt Morehouse e911a5d356 [libFuzzer] Remove const from ignoreRemainingArgs return value.
In this case const does nothing but trigger a warning.

llvm-svn: 319685
2017-12-04 20:06:52 +00:00
Matt Morehouse dc81fd6cec [libFuzzer] Remove FuzzerCommand.cpp from build sources.
FuzzerCommand.cpp was not introduced in r319680.  Instead, it was
implemented in the header file.

llvm-svn: 319682
2017-12-04 19:54:14 +00:00
Matt Morehouse 04304d129b [libFuzzer] Encapsulate commands in a class.
Summary:
To be more portable (especially w.r.t. platforms without system()),
commands should be managed programmatically rather than via string
manipulation on the command line. This change introduces
Fuzzer::Command, with methods to manage arguments and flags, set output
options, and execute the command.

Patch By: aarongreen

Reviewers: kcc, morehouse

Reviewed By: kcc, morehouse

Subscribers: llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D40103

llvm-svn: 319680
2017-12-04 19:25:59 +00:00
Kostya Serebryany de9bafb162 [libFuzzer] add a flag -malloc_limit_mb
llvm-svn: 319590
2017-12-01 22:12:04 +00:00
Kostya Serebryany 6afa7a54be [libFuzzer] remove stale flags; NFC
llvm-svn: 319572
2017-12-01 19:24:06 +00:00
Kostya Serebryany ad05ee0512 [libFuzzer] add an experimental search heuristic flag -reduce_depth
llvm-svn: 319571
2017-12-01 19:18:38 +00:00
Dan Liew 37516b527b [LibFuzzer] Improve comments on `CounterToFeature()` function.
This is based on discussion in https://reviews.llvm.org/D40376 .

The comments try to explain the reason for the current implementation
and note that it might change in the future, so clients should not
rely on this particular implementation.

Differential Revision: https://reviews.llvm.org/D40565

llvm-svn: 319190
2017-11-28 17:41:58 +00:00
Vitaly Buka 065421f0cc [fuzzer] Initialize PcDescr buffer before calling __sanitizer_symbolize_pc
__sanitizer_symbolize_pc is not instrumented so msan assumes that PcDescr
was not initialized.

llvm-svn: 318488
2017-11-17 00:46:59 +00:00
Kostya Serebryany cb74c65f5e libfuzzer: Fix file listing on some filesystems
Summary:
For some filesystems, readdir will not populate dirent::d_type with valuable information. This causes libfuzzer to proceed with an empty corpus, instead of the file it contains.

This has been tested on a server using XFS.

It should fix https://bugs.llvm.org//show_bug.cgi?id=25991

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D40028

llvm-svn: 318303
2017-11-15 16:45:17 +00:00
Kostya Serebryany 8eb9ca2580 [libFuzzer] respect max_len during merge
llvm-svn: 318302
2017-11-15 16:42:52 +00:00
Kostya Serebryany 54a8456005 [libFuzzer] make sure to flush IO when done merging one file
llvm-svn: 317835
2017-11-09 21:30:33 +00:00
Matt Morehouse 947838c9c5 [libFuzzer] Don't add leaking inputs to corpus.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39850

llvm-svn: 317831
2017-11-09 20:44:08 +00:00
Kostya Serebryany a2ca2dcc46 [libFuzzer] handle SIGUSR1/SIGUSR2 and try to exit grafully on these signals
llvm-svn: 317829
2017-11-09 20:30:19 +00:00
Kostya Serebryany 3ac31d4453 [libFuzzer] allow merge to resume after being preempted
llvm-svn: 317767
2017-11-09 05:49:28 +00:00
Kostya Serebryany 68fdef1f7a [libFuzzer] allow user to specify the merge control file
llvm-svn: 317747
2017-11-09 01:05:29 +00:00
Vitaly Buka 7d22324b87 [fuzzer] Fix nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317186
2017-11-02 04:12:10 +00:00
Vitaly Buka ca9075272b [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317119
2017-11-01 20:27:06 +00:00
Vitaly Buka 7dbc1d8433 [fuzzer] Fix threaded stack printing
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317071
2017-11-01 03:02:59 +00:00
Vitaly Buka df00e89cdc Revert "[fuzzer] Fix threaded stack printing and nested mallocs"
Fails on darwin

Revert "[fuzzer] Script to detect unbalanced allocation in -trace_malloc output"
Needs previous one.

This reverts commit r317034, r317036.

llvm-svn: 317061
2017-11-01 00:55:52 +00:00
Vitaly Buka 0747700ce1 [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317036
2017-10-31 20:50:07 +00:00
Vitaly Buka 58da33e35c [fuzzer] Fix threaded stack printing and nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317034
2017-10-31 20:49:48 +00:00
Alex Shlyapnikov 5ded0701a9 [Sanitizers-libFuzzer] Addressing coding style issues.
Summary: The result of clang-format and few manual changes (as prompted on D39155).

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39211

llvm-svn: 316395
2017-10-23 23:24:33 +00:00
Alex Shlyapnikov 6f1c26f222 [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.
Summary:
Fuzzing targets that allocate/deallocate a lot of memory tend to consume
a lot of RSS when ASan quarantine is enabled. Purging quarantine between
iterations and returning memory to OS keeps RSS down and should not
reduce the quarantine effectiveness provided the fuzz target does not
preserve state between iterations (in this case this feature can be turned off).

Based on D39153.

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39155

llvm-svn: 316382
2017-10-23 22:04:30 +00:00
Kostya Serebryany fc3079963f [libFuzzer] print a better warning if we hit the ld bug
llvm-svn: 315778
2017-10-14 00:07:11 +00:00
Alex Shlyapnikov 29d3ba7576 [Sanitizers] Always build libFuzzer with frame pointers.
Lacking frame pointers, sanitized fuzzers collect bogus stack
traces and the set of stack traces grows indefinitely, leading to OOMs.

llvm-svn: 315770
2017-10-13 21:57:43 +00:00
Kostya Serebryany 51823d3aae [libFuzzer] tweam use_feature_frequency to be less aggressive; run a dummy input before the seed corpus
llvm-svn: 315657
2017-10-13 01:12:23 +00:00
Kostya Serebryany 75480e3871 [libFuzzer] disable use_feature_frequency as it degrades some of the benchmarks too much :(
llvm-svn: 315503
2017-10-11 20:31:01 +00:00
Kostya Serebryany 198d385a77 [libFuzzer] make -use_feature_frequency less aggressive and enable by default
llvm-svn: 315490
2017-10-11 19:01:35 +00:00
Kostya Serebryany 4083d54675 [libFuzzer] experimental flag to tweak the corpus distribution. Seems to improve the situation dramatically on the png benchmark and make things worse on a number of micro-puzzles. Needs more A/B testing
llvm-svn: 315407
2017-10-11 01:44:26 +00:00
Max Moroz 330496c3fc [libFuzzer] Disable experimental clang coverage support by default.
Summary:
It can be enabled via "-use_clang_coverage=1" flag. Reason for disabling:
libFuzzer resets Clang Counters and makes it impossible to generate coverage
report for a regular fuzz target (i.e. not standalone build).

Reviewers: kcc

Reviewed By: kcc

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38604

llvm-svn: 315029
2017-10-05 22:41:03 +00:00
Kostya Serebryany bcd78491ef [libFuzzer] minor refactoring, NFC
llvm-svn: 313406
2017-09-15 22:10:36 +00:00
Kostya Serebryany 97b8b2daff [libFuzzer] reduce the size of the merge control file by not dumping redundant features into it
llvm-svn: 313403
2017-09-15 22:02:26 +00:00
Kostya Serebryany 93679be037 [libFuzzer] factor out some code into GetSizedFilesFromDir; NFC
llvm-svn: 313081
2017-09-12 21:58:07 +00:00
Max Moroz 3f26dac416 [libfuzzer] Compare TotalNumberOfRuns with MaxNumberOfRuns when testing a memory leak.
Summary:
Fuzzer::TryDetectingAMemoryLeak may call ExecuteCallback which would
increment TotalNumberOfRuns, but it doesn't respect Options.MaxNumberOfRuns
value specified by a user.

Context: https://github.com/google/oss-fuzz/issues/822#issuecomment-328153970

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D37632

llvm-svn: 312993
2017-09-12 02:01:54 +00:00
Kostya Serebryany 25d6b95679 [libFuzzer] remove a couple of reduntant includes
llvm-svn: 312848
2017-09-09 00:56:34 +00:00
Kostya Serebryany 082e9a7528 [libFuzzer] tolerate missing files when loading the seed corpus
llvm-svn: 312269
2017-08-31 19:17:15 +00:00
Kamil Rytarowski 8da2722a9f Add preliminary NetBSD support in libfuzzer
Summary:
This code already works and passes some number of tests.

There is need to finish remaining sanitizers to get better coverage.

Many tests fail due to overly long file names of executables (>31).
This is a current shortcoming of the NetBSD 8(beta) kernel, as
certain functions can fail (like retrieving file name of executable).

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, george.karpenkov

Reviewed By: kcc

Subscribers: mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D37304

llvm-svn: 312183
2017-08-30 22:44:11 +00:00
Kostya Serebryany 4faeb87ebe [libFUzzer] change the way we load the seed corpora: instead of loading all files and these executing all files, load and execute them one-by-one. This should reduce the memory usage in many cases
llvm-svn: 312033
2017-08-29 20:51:24 +00:00
Kostya Serebryany 3a8e3c8336 [libFuzzer] refactoring: move reading the seed corpus closer to where it's consumed; NFC
llvm-svn: 311972
2017-08-29 02:05:01 +00:00
Kostya Serebryany c07008653c [libFuzzer] allow -print_funcs=N: N is the max number of new covered function printed
llvm-svn: 311945
2017-08-28 22:52:22 +00:00
George Karpenkov dacfd66dfd [libFuzzer] Fix libFuzzer flag propagation for standalone builds.
Under the previous configurations, flags from SANITIZER_COMMON were not
propagated for standalone builds.

Differential Revision: https://reviews.llvm.org/D37225

llvm-svn: 311912
2017-08-28 19:44:19 +00:00