Commit Graph

37 Commits

Author SHA1 Message Date
Kostya Serebryany 77cbc62544 [libFuzzer] print new functions as they are discovered in the fork mode
llvm-svn: 354092
2019-02-15 01:22:00 +00:00
Kostya Serebryany 96f81bc679 [libFuzzer] when doing the merge, keep track of the coveraged edges, not just features
llvm-svn: 354076
2019-02-14 23:12:33 +00:00
Kostya Serebryany 8da9479e40 [libFuzzer] experimental performance optimization -lazy_counters, off by default. Posix-only for now, tested on Linux
llvm-svn: 352700
2019-01-31 00:09:43 +00:00
Kostya Serebryany 6fd4d8ab9c [libFuzzer] refactor the handling of instrumentation counters so that they are grouped in regions one full page each. Needed for future optimization. NFC
llvm-svn: 352603
2019-01-30 06:15:52 +00:00
Kostya Serebryany 2891b257c2 [libFuzzer] remove stale code
llvm-svn: 352571
2019-01-29 23:53:28 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Jonathan Metzman b795c31dff [libFuzzer][MSVC] Make attribute-use compatible with MSVC
Summary:
Replace attributes with macros that use equivalent declspecs
for MSVC.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D56512

llvm-svn: 351456
2019-01-17 16:36:05 +00:00
Jonathan Metzman 9e14cccf6f [libFuzzer] Remove unstable edge handling
Summary:
Remove code for handling unstable edges from libFuzzer since
it has not been found useful.

Differential Revision: https://reviews.llvm.org/D56730

llvm-svn: 351262
2019-01-15 22:12:51 +00:00
Max Moroz cd02f3147b [libFuzzer] Optimize handle unstable checks by reducing iterations
Summary:
We only run the 3rd check if 2nd check finds unstable edges. 
3rd UpdateUnstableCounters is now merged with ApplyUnstableCounters to only run 1 iteration.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D50411

llvm-svn: 339249
2018-08-08 14:32:46 +00:00
Max Moroz 8db0befc6d [libFuzzer] Handle unstable edges by disregarding unstable edges
Summary:
Added a new mode within flag -handle_unstable for new unstable handling algorithm that does the following:
    When an edge is shown as unstable, copy to UnstableCounters the value 0.
    During ApplyUnstableCounters we copy back the value 0 to ModuleInline8bitCounters if the edge was unstable.

This way we would be ignoring completely features that were collected through non-determinism.
Unstable hits would be counted as if it never hit.

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49684

llvm-svn: 337853
2018-07-24 21:02:44 +00:00
Max Moroz 1e954f78d1 [libFuzzer] Handle unstable edges by using minimum hit counts
Summary:
Created unstable_handle flag that takes 1 or 2, depending on the handling type.
Modified RunOne to accommodate the following heuristic:
    Use the first CollectFeatures to count how many features there are.
    If no new features, CollectFeatures like before.
    If there is new feature, we run CB 2 more times,
        Check which edges are unstable per input and we store the least amount of hit counts for each edge.
        Apply these hit counts back to inline8bitcounters so that CollectFeatures can work as intended.
Modified UnstableCounters to 8int_t and created a bitset UnstableSet to tell which edges are unstable.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, morehouse

Reviewed By: Dor1s, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D49525

llvm-svn: 337696
2018-07-23 14:20:52 +00:00
Kostya Serebryany 44edc281d9 [libFuzzer] when -print_coverage=1 is given, print more stats (the number of seeds that hit every given function)
llvm-svn: 337501
2018-07-19 22:00:48 +00:00
Max Moroz 42b54e8117 [libFuzzer] Create single template for visiting Inline8bitCounters
Summary:
Created IterateInline8bitCounters, a single template for visiting  Inline8bitCounters (nested for loop)
Made InitializeUnstableCounters and UpdateUnstableCounters both send a lambda to IterateInline8bitCounters.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: Dor1s, metzman, kcc, morehouse

Reviewed By: metzman, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D49453

llvm-svn: 337403
2018-07-18 17:03:27 +00:00
Matt Morehouse 43a2296976 libFuzzer: prevent irrelevant strings from leaking into auto-dictionary
This is a fix for bug 37047.

https://bugs.llvm.org/show_bug.cgi?id=37047

Implemented by basically reversing the logic. Previously all strings
were considered, with some operations excluded. Now strings are excluded
by default, and only strings during the CB considered.

Patch By: pdknsk

Differential Revision: https://reviews.llvm.org/D48800

llvm-svn: 337296
2018-07-17 16:12:00 +00:00
Max Moroz 08dad54924 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337187
2018-07-16 16:01:31 +00:00
Max Moroz 1d369a5d01 Revert r337175 (https://reviews.llvm.org/D49212) due to unintentional format changes.
llvm-svn: 337180
2018-07-16 15:15:34 +00:00
Max Moroz 2156d885e0 [libFuzzer] Implement stat::stability_rate based on the percentage of unstable edges.
Summary:
Created a -print_unstable_stats flag.
When -print_unstable_stats=1, we run it 2 more times on interesting inputs poisoning unstable edges in an array.
On program termination, we run PrintUnstableStats() which will print a line with a stability percentage like AFL does.

Patch by Kyungtak Woo (@kevinwkt).

Reviewers: metzman, Dor1s, kcc, morehouse

Reviewed By: metzman, Dor1s, morehouse

Subscribers: delcypher, llvm-commits, #sanitizers, kcc, morehouse, Dor1s

Differential Revision: https://reviews.llvm.org/D49212

llvm-svn: 337175
2018-07-16 14:54:23 +00:00
Kostya Serebryany 51ddb88300 [libFuzzer] add one more value profile metric, under a flag (experimental)
llvm-svn: 336234
2018-07-03 22:33:09 +00:00
Kostya Serebryany 69c2b71a51 [libFuzzer] reinstate -dump_coverage, which is still in use (reverts r332036)
llvm-svn: 332876
2018-05-21 19:47:00 +00:00
Kostya Serebryany e9c6f06cce [libFuzzer] add an experimental flag -focus_function: libFuzzer will try to focus on inputs that trigger that function
llvm-svn: 332554
2018-05-16 23:26:37 +00:00
Kostya Serebryany 6a6e690d24 [libFuzzer] refactor the implementation of -print_coverage
llvm-svn: 332073
2018-05-11 01:17:52 +00:00
Kostya Serebryany d80e821646 [libFuzzer] remove the dump_coverage flag, it hasn't been working with the inline sanitizer coverage anyway
llvm-svn: 332036
2018-05-10 20:24:39 +00:00
Kostya Serebryany 59bf1e8b36 [libFuzzer] remove the experimental support for clang coverage instrumentation. This mode has not been used and our experiments with https://github.com/google/fuzzer-test-suite show that this signal is weaker than the SanitizerCoverage
llvm-svn: 332034
2018-05-10 20:12:15 +00:00
Max Moroz a40ce7bc36 [libfuzzer] Fix UB when calculating Log(0) in StackDepthStepFunction().
Summary:
__builtin_clz used for Log calculation returns an undefined result
when argument is 0. I noticed that issue when was testing some fuzzers:

```
/src/libfuzzer/FuzzerTracePC.h:282:33: runtime error: shift exponent 450349 is too large for 32-bit type 'uint32_t' (aka 'unsigned int')
  #0 0x43d83f in operator() /src/libfuzzer/FuzzerTracePC.h:283:33
  #1 0x43d83f in void fuzzer::TracePC::CollectFeatures<fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*)::$_1>(fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*)::$_1) const /src/libfuzzer/FuzzerTracePC.h:290
  #2 0x43cbd4 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/libfuzzer/FuzzerLoop.cpp:445:7
  #3 0x43e5f1 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:706:5
  #4 0x43e9e1 in fuzzer::Fuzzer::Loop(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /src/libfuzzer/FuzzerLoop.cpp:739:3
  #5 0x432f8c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/libfuzzer/FuzzerDriver.cpp:754:6
  #6 0x42ee18 in main /src/libfuzzer/FuzzerMain.cpp:20:10
  #7 0x7f17ffeb182f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
  #8 0x407838 in _start (/out/rotate_fuzzer+0x407838)

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41457

llvm-svn: 321211
2017-12-20 19:31:51 +00:00
Kostya Serebryany e9ed2327b6 [libFuzzer] change the strategy for -experimental_len_control to grow max_len slower
llvm-svn: 320531
2017-12-12 23:11:28 +00:00
Kostya Serebryany de891a1946 [libFuzzer] even less aggressive step function for stack depth. Also don't use asan in one test to speed it up
llvm-svn: 320259
2017-12-09 19:18:10 +00:00
Kostya Serebryany f94d87aadf [libFuzzer] make the stack depth signal less agressive, otherwise we are sometimes exploding the corpus size. This still needs more analysis and tuning
llvm-svn: 320237
2017-12-09 02:45:23 +00:00
Kostya Serebryany 2659c63e2e [libFuzzer] honor -use_counters, sligntly change the meaning of -experimental_len_control, call UpdateFeatureFrequency only if instructed by the flag
llvm-svn: 320205
2017-12-08 22:21:42 +00:00
Dan Liew 37516b527b [LibFuzzer] Improve comments on `CounterToFeature()` function.
This is based on discussion in https://reviews.llvm.org/D40376 .

The comments try to explain the reason for the current implementation
and note that it might change in the future, so clients should not
rely on this particular implementation.

Differential Revision: https://reviews.llvm.org/D40565

llvm-svn: 319190
2017-11-28 17:41:58 +00:00
Max Moroz 330496c3fc [libFuzzer] Disable experimental clang coverage support by default.
Summary:
It can be enabled via "-use_clang_coverage=1" flag. Reason for disabling:
libFuzzer resets Clang Counters and makes it impossible to generate coverage
report for a regular fuzz target (i.e. not standalone build).

Reviewers: kcc

Reviewed By: kcc

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38604

llvm-svn: 315029
2017-10-05 22:41:03 +00:00
Kostya Serebryany bcd78491ef [libFuzzer] minor refactoring, NFC
llvm-svn: 313406
2017-09-15 22:10:36 +00:00
Kostya Serebryany c07008653c [libFuzzer] allow -print_funcs=N: N is the max number of new covered function printed
llvm-svn: 311945
2017-08-28 22:52:22 +00:00
George Karpenkov bebcbfb46d [libFuzzer] Use custom allocators for STL containers in libFuzzer.
Avoids ODR violations causing spurious ASAN warnings.

Differential Revision: https://reviews.llvm.org/D37086

llvm-svn: 311866
2017-08-27 23:20:09 +00:00
Kostya Serebryany 2eef816e6e [libFuzzer] add -print_funcs=1 (on bey default): print newly discovered functions during fuzzing
llvm-svn: 311797
2017-08-25 20:09:25 +00:00
Kostya Serebryany d3e4b7e24a [sanitizer-coverage] extend fsanitize-coverage=pc-table with flags for every PC
llvm-svn: 311794
2017-08-25 19:29:47 +00:00
Kostya Serebryany f65cf64fa2 [libFuzzer] apply changes lost during the migration to compiler-rt
llvm-svn: 311420
2017-08-22 01:28:32 +00:00
George Karpenkov 10ab2ace13 Move libFuzzer to compiler_rt.
Resulting library binaries will be named libclang_rt.fuzzer*, and will
be placed in Clang toolchain, allowing redistribution.

Differential Revision: https://reviews.llvm.org/D36908

llvm-svn: 311407
2017-08-21 23:25:50 +00:00