Commit Graph

608 Commits

Author SHA1 Message Date
Kostya Serebryany d216922a80 [libFuzzer] implement the -shrink=1 option that tires to make elements of the corpus smaller, off by default
llvm-svn: 282995
2016-10-01 01:04:29 +00:00
Kostya Serebryany 90f8f36bca [libFuzzer] remove some experimental code
llvm-svn: 282983
2016-09-30 23:29:27 +00:00
Kostya Serebryany 7022b94687 [libFuzzer] fix openssl fuzzer tests when running on a machine w/o openssl installed
llvm-svn: 282972
2016-09-30 22:35:08 +00:00
Kostya Serebryany e7e790bad6 [libFuzzer] remove unused option
llvm-svn: 282971
2016-09-30 22:29:57 +00:00
Kostya Serebryany b7e7a5473d [libFuzzer] move common parts of shell scripts into a separate file
llvm-svn: 282954
2016-09-30 21:12:30 +00:00
Kostya Serebryany cfa31b6307 [libFuzzer] add a fuzzer test that finds CVE-2015-3193
llvm-svn: 282892
2016-09-30 18:16:16 +00:00
Kostya Serebryany cad612a472 [libfuzzer] test for c-ares CVE-2016-5180
llvm-svn: 282839
2016-09-30 05:15:45 +00:00
Kostya Serebryany b3949ef885 [libFuzzer] remove the code for -print_pcs=1 with the old coverage. It still works with the new one (trace-pc-guard)
llvm-svn: 282831
2016-09-30 01:24:57 +00:00
Kostya Serebryany 2c55613a08 [libFuzzer] more the feature set to InputCorpus; on feature update, change the feature counter of the old best input
llvm-svn: 282829
2016-09-30 01:19:56 +00:00
Kostya Serebryany a9b0dd0e51 [sanitizer-coverage/libFuzzer] make the guards for trace-pc 32-bit; create one array of guards per function, instead of one guard per BB. reorganize the code so that trace-pc-guard does not create unneeded globals
llvm-svn: 282735
2016-09-29 17:43:24 +00:00
Kostya Serebryany a9a135b4f5 [libFuzzer] initialize ValueBitMap::NumBits
llvm-svn: 282721
2016-09-29 15:51:28 +00:00
Kostya Serebryany 3ee6c213d6 [libFuzzer] speedup TracePC::FinalizeTrace
llvm-svn: 282562
2016-09-28 01:16:24 +00:00
Kostya Serebryany 7d6935c184 [libFuzzer] run re2 test in 8 threads by default
llvm-svn: 282469
2016-09-27 03:33:57 +00:00
Kostya Serebryany 45c144754b [sanitizer-coverage] fix a bug in trace-gep
llvm-svn: 282467
2016-09-27 01:55:08 +00:00
Kostya Serebryany 53543af036 [libFuzzer] add a test based on openssl-1.0.1f (finds heartbleed)
llvm-svn: 282460
2016-09-27 00:27:40 +00:00
Kostya Serebryany 5ff481fd9e [libFuzzer] add -exit_on_src_pos to test libFuzzer itself, add a test script for RE2 that uses this flag
llvm-svn: 282458
2016-09-27 00:10:20 +00:00
Kostya Serebryany 273d767215 [libFuzzer] add a standalone build script
llvm-svn: 282321
2016-09-24 04:00:00 +00:00
Kostya Serebryany 0800b81a21 [libFuzzer] simplify HandleTrace again, start re-running interesting units and collecting their features.
llvm-svn: 282316
2016-09-23 23:51:58 +00:00
Kostya Serebryany 2d1d944f7e [libFuzzer] first steps in adding a proper automated test suite based on real-life code: add a script to build RE2 at a revision that has known bugs
llvm-svn: 282292
2016-09-23 20:43:22 +00:00
Kostya Serebryany 0d26de3922 [libFuzzer] reset Counters (trace-pc-guard) before every run
llvm-svn: 282284
2016-09-23 20:04:13 +00:00
Kostya Serebryany ce1cab169f [libFuzzer] be more precise about what we reset in TracePC
llvm-svn: 282225
2016-09-23 02:18:59 +00:00
Kostya Serebryany 16a145fd0f [libFuzzer] fix merging with trace-pc-guard
llvm-svn: 282224
2016-09-23 01:58:51 +00:00
Kostya Serebryany 87a598e19f [libFuzzer] simplify the TracePC logic
llvm-svn: 282222
2016-09-23 01:20:07 +00:00
Kostya Serebryany ab73c6924f [libFuzzer] move value profiling logic into TracePC
llvm-svn: 282219
2016-09-23 00:46:18 +00:00
Kostya Serebryany d28099de5d [libFuzzer] change ValueBitMap to remember the number of bits in it
llvm-svn: 282216
2016-09-23 00:22:46 +00:00
Kostya Serebryany be0ed59cdc [libFuzzer] simplify the crash minimizer; split MaxLen into two: MaxInputLen and MaxMutationLen, allow MaxMutationLen to be less than MaxInputLen
llvm-svn: 282211
2016-09-22 23:16:36 +00:00
Kostya Serebryany 624f59f4d8 [libFuzzer] add 'features' to the corpus elements, allow mutations with Size > MaxSize, fix sha1 in corpus stats; various refactorings
llvm-svn: 282129
2016-09-22 01:34:58 +00:00
Kostya Serebryany c9e3de35ed [libFuzzer] one more test
llvm-svn: 282127
2016-09-22 00:57:29 +00:00
Kostya Serebryany 29bb664075 [libFuzzer] add stats to the corpus; more refactoring
llvm-svn: 282121
2016-09-21 22:42:17 +00:00
Kostya Serebryany 20801e1b8a [libFuzzer] more refactoring; don't compute sha1sum every time we mutate a unit from the corpus, use the stored one.
llvm-svn: 282115
2016-09-21 21:41:48 +00:00
Kostya Serebryany 8658618ea0 [libFuzzer] more refactoring
llvm-svn: 282113
2016-09-21 21:17:23 +00:00
Kostya Serebryany 225d8e45d4 [libFuzzer] fix libc++ build
llvm-svn: 282050
2016-09-21 03:50:37 +00:00
Kostya Serebryany 556894fb10 [libFuzzer] more refactoring; NFC
llvm-svn: 282047
2016-09-21 02:05:39 +00:00
Kostya Serebryany 6f5a804cdb [libFuzzer] refactoring: split the large header into many; NFC
llvm-svn: 282044
2016-09-21 01:50:50 +00:00
Kostya Serebryany 09aa01a6f8 [libFuzzer] refactoring: move the Corpus into a separate class; delete two unused experimental features
llvm-svn: 282042
2016-09-21 01:04:43 +00:00
Kostya Serebryany 3750c04f7e [libFuzzer] use sleep() instead of std::this_thread::sleep_for to avoid coverage from instrumented libc++
llvm-svn: 281933
2016-09-19 20:32:34 +00:00
Kostya Serebryany b706b481ba [libFuzzer] add -print_coverage=1 flag to print coverage directly from libFuzzer
llvm-svn: 281866
2016-09-18 21:47:08 +00:00
Kostya Serebryany 8e781a888a [libFuzzer] use 'if guard' instead of 'if guard >= 0' with trace-pc; change the guard type to intptr_t; use separate array for 8-bit counters
llvm-svn: 281845
2016-09-18 04:52:23 +00:00
Kostya Serebryany bc3789a919 [libFuzzer] properly reset the guards when reseting the coverage. Also try to fix check-fuzzer on the bot
llvm-svn: 281814
2016-09-17 06:01:55 +00:00
Kostya Serebryany 3e36ec1d18 [libFuzzer] change trace-pc to use 8-byte guards
llvm-svn: 281810
2016-09-17 05:04:47 +00:00
Kostya Serebryany 0984517021 [libFuzzer] make caller-callee feedback work with trace-pc-guard
llvm-svn: 281667
2016-09-15 22:16:15 +00:00
Kostya Serebryany 21c3573733 [libFuzzer] fix the build for AFLDriverTest
llvm-svn: 281633
2016-09-15 18:10:38 +00:00
Kostya Serebryany 09e416615e [libFuzzer] disable test that requires debug info -- it fails on the bot
llvm-svn: 281584
2016-09-15 05:46:58 +00:00
Kostya Serebryany 0b47fbcb30 [libFuzzer] move the AFL driver build rule test into the uninstrumented dir
llvm-svn: 281583
2016-09-15 05:17:39 +00:00
Kostya Serebryany 33a497abf4 [libFuzzer] fix print_pcs test
llvm-svn: 281580
2016-09-15 04:43:06 +00:00
Kostya Serebryany 5350178487 [libFuzzer] implement print_pcs with trace-pc-guard. Change the trace-pc-guard heuristic for 8-bit counters to look more like in AFL (not that it's provable better, but the existin test preferes this heuristic)
llvm-svn: 281577
2016-09-15 04:36:45 +00:00
Kostya Serebryany a5277d59d0 [libFuzzer] add 8-bit counters to trace-pc-guard handler
llvm-svn: 281568
2016-09-15 01:30:18 +00:00
Kostya Serebryany a00b243c75 [libFuzzer] start using trace-pc-guard as an alternative source of coverage
llvm-svn: 281435
2016-09-14 02:13:06 +00:00
Kostya Serebryany 8c537c556a [libFuzzer] print a failed-merge warning only in the merge mode
llvm-svn: 281130
2016-09-10 02:17:22 +00:00
Kostya Serebryany 4529960a3b [libFuzzer] don't print help for internal flags
llvm-svn: 281124
2016-09-10 00:35:30 +00:00
Kostya Serebryany b991cc1f0e [libFuzzer] print a visible message if merge fails due to a crash
llvm-svn: 281122
2016-09-10 00:15:41 +00:00
Kostya Serebryany 1837152a34 [libFuzzer] use sizeof() in tests instead of 4 and 8
llvm-svn: 281111
2016-09-09 22:21:16 +00:00
Kostya Serebryany 4b17a331ae [libFuzzer] one more puzzle for value profile
llvm-svn: 281106
2016-09-09 21:58:42 +00:00
Kostya Serebryany 00ef27112e [libFuzzer] one more puzzle, value_profile cracks it in a second
llvm-svn: 281066
2016-09-09 18:00:04 +00:00
Kostya Serebryany b76a2a5503 [libFuzzer] improve -print_pcs to not print new PCs coming from libFuzzer itself
llvm-svn: 281016
2016-09-09 02:38:28 +00:00
Kostya Serebryany 8ea4f9873b [libFuzzer] remove unneeded call
llvm-svn: 281014
2016-09-09 01:57:38 +00:00
Kostya Serebryany 5c04bd250e [libFuzzer] remove use_traces=1 since use_value_profile seems to be strictly better
llvm-svn: 281007
2016-09-09 01:17:03 +00:00
Kostya Serebryany e2d0f63654 [libFuzzer] add -minimize_crash flag (to minimize crashers). also add two tests that I failed to commit last time
llvm-svn: 280332
2016-09-01 01:22:27 +00:00
Mike Aizatsky b077d3fef2 [libfuzzer] simplified unit truncation; do not write trunc items to disc
Differential Revision: https://reviews.llvm.org/D24049

llvm-svn: 280153
2016-08-30 20:49:07 +00:00
Kostya Serebryany a016a45d60 [libFuzzer] fix a bug when running a single unit of N bytes with -max_len=M, M<N, caused a buffer overflow
llvm-svn: 280098
2016-08-30 14:52:05 +00:00
Kostya Serebryany 248d11519a [libFuzzer] stop using bits for memcmp's value profile -- seems to blow up the corpus too much
llvm-svn: 280096
2016-08-30 14:39:33 +00:00
Kostya Serebryany d4492f8101 [libFuzzer] use bits instead of bytes for memcmp/strcmp value profile -- the fuzzer reaches the goal much faster, at least on the simple puzzles
llvm-svn: 280054
2016-08-30 03:05:50 +00:00
Kostya Serebryany 4d22e4fcb9 [libFuzzer] use trace-div and trace-gep for guided fuzzing, add tests
llvm-svn: 280046
2016-08-30 01:30:14 +00:00
Kostya Serebryany 3e5991e540 [libFuzzer] simplify a test to make it pass on the bot
llvm-svn: 279796
2016-08-26 00:18:16 +00:00
Kostya Serebryany 1426f59a76 [libFuzzer] make sure we have symbols on fuzzer tests
llvm-svn: 279792
2016-08-25 23:30:02 +00:00
Kostya Serebryany 0f0fa4faf2 [libFizzer] rename -print_new_cov_pcs=1 into -print_pcs=1 and make it more useful: print PCs only after the initial corpus has been read and symbolize them
llvm-svn: 279787
2016-08-25 22:35:08 +00:00
Kostya Serebryany f67357c671 [libFuzzer] simplify the code, NFC
llvm-svn: 279697
2016-08-25 01:25:03 +00:00
Kostya Serebryany 41bcb830af [libFuzzer] make a test more deterministic
llvm-svn: 279686
2016-08-24 23:10:17 +00:00
Kostya Serebryany bceadcf1cd [libFuzzer] use __attribute__((target("popcnt"))) only on x86_64
llvm-svn: 279601
2016-08-24 01:38:42 +00:00
Kostya Serebryany ac524cfcce [libFuzzer] collect 64 states for value profile, not 65
llvm-svn: 279588
2016-08-23 23:37:37 +00:00
Kostya Serebryany a533e514b8 [libFuzzer] fix the non-debug build warnings
llvm-svn: 279321
2016-08-19 20:57:09 +00:00
Kostya Serebryany 32661f9d66 [libFuzzer] add more __attribute__((visibility("default")))
llvm-svn: 279143
2016-08-18 20:52:52 +00:00
Kostya Serebryany 524c3f32e7 [sanitizer-coverage/libFuzzer] instrument comparisons with __sanitizer_cov_trace_cmp[1248] instead of __sanitizer_cov_trace_cmp, don't pass the comparison type to save a bit performance. Use these new callbacks in libFuzzer
llvm-svn: 279027
2016-08-18 01:25:28 +00:00
Kostya Serebryany 5a5d5548f0 [libFuzzer] force proper popcnt instruction
llvm-svn: 279002
2016-08-17 23:09:57 +00:00
Kostya Serebryany e72774dd69 [libFuzzer] given 0 and 255 more preference when inserting repeated bytes
llvm-svn: 278986
2016-08-17 21:50:54 +00:00
Kostya Serebryany 0c537b124c [libFuzzer] one more mutation: ChangeBinaryInteger; also fix the breakage from r278970
llvm-svn: 278982
2016-08-17 21:30:30 +00:00
Kostya Serebryany a9a548049a [libFuzzer] when printing the reproducer input, also print the base input and the mutation sequence
llvm-svn: 278975
2016-08-17 20:45:23 +00:00
Justin Bogner cd1d5aaf2e Replace a few more "fall through" comments with LLVM_FALLTHROUGH
Follow up to r278902. I had missed "fall through", with a space.

llvm-svn: 278970
2016-08-17 20:30:52 +00:00
Kostya Serebryany a7398ba024 [libFuzzer] more mutations
llvm-svn: 278950
2016-08-17 18:10:42 +00:00
Kostya Serebryany 3044390af1 [libFuzzer] minor speed improvement
llvm-svn: 278856
2016-08-16 21:28:05 +00:00
Kostya Serebryany d46a59fac4 [libFuzzer] new experimental feature: value profiling. Profiles values that affect control flow and treats new values as new coverage.
llvm-svn: 278839
2016-08-16 19:33:51 +00:00
Kostya Serebryany c98ef718ea [libFuzzer] refactoring around PCMap, NFC
llvm-svn: 278825
2016-08-16 17:37:13 +00:00
Kostya Serebryany bdb220c7a0 [libFuzzer] print a verbose message after executing inputs in non-fuzzing mode
llvm-svn: 278724
2016-08-15 19:44:04 +00:00
Kostya Serebryany a0d40a21e7 [libFuzzer] fix the bot
llvm-svn: 278721
2016-08-15 19:36:13 +00:00
Kostya Serebryany dfbe59b03d [libFuzzer] add InsertRepeatedBytes and EraseBytes.
New mutation: InsertRepeatedBytes.
Updated mutation: EraseByte => EraseBytes.

This helps https://github.com/google/sanitizers/issues/710
where libFuzzer was not able to find a known bug.
Now it finds it in minutes.

Hopefully, the change is general enough to help other targets.

llvm-svn: 278687
2016-08-15 17:48:28 +00:00
Dan Liew ed3c9cae49 [LibFuzzer] Fix `-jobs=<N>` where <N> > 1 and the number of workers is > 1 on macOS.
The original `ExecuteCommand()` called `system()` from the C library.
The C library implementation of this on macOS contains a mutex which
serializes calls to `system()`. This prevented the `-jobs=` flag
from running copies of the fuzzing binary in parallel which is
the opposite of what is intended.

To fix this on macOS an alternative implementation of `ExecuteCommand()`
is provided that can be used concurrently. This is provided in
`FuzzerUtilDarwin.cpp` which is guarded to only compile code on Apple
platforms. The existing implementation has been moved to a new file
`FuzzerUtilLinux.cpp` which is guarded to only compile code on Linux.

This commit includes a simple test to check that LibFuzzer is being
executed in parallel when requested.

Differential Revision: https://reviews.llvm.org/D22742

llvm-svn: 278544
2016-08-12 18:29:36 +00:00
Kostya Serebryany 728447bd3b [libFuzzer] make libFuzzer work with a bit older clang versions
llvm-svn: 277941
2016-08-06 21:28:56 +00:00
Kostya Serebryany ff1f2107ec [libFuzzer] don't print bogus error message
llvm-svn: 277940
2016-08-06 21:23:29 +00:00
Mike Aizatsky a8e84b9b37 [libfuzzer] do not warn about missing pcbuffer functions: they are new.
llvm-svn: 277927
2016-08-06 17:03:22 +00:00
Mike Aizatsky b4bbc3bb7a [sanitizers] trace buffer API to use user-allocated buffer.
Differential Revision: https://reviews.llvm.org/D23185

llvm-svn: 277859
2016-08-05 20:09:53 +00:00
Kostya Serebryany 0ccf06f467 [libFuzzer] extend the messages printed by afl_driver
llvm-svn: 276052
2016-07-19 23:18:28 +00:00
Kostya Serebryany 6b08be9279 [libFuzzer] properly intercept memmem
llvm-svn: 276006
2016-07-19 18:29:06 +00:00
Kostya Serebryany c135b55ae0 [libFuzzer] add hooks for strstr, strcasestr, strcasecmp, strncasecmp
llvm-svn: 275648
2016-07-15 23:27:19 +00:00
Kostya Serebryany 5d9a17742e [libFuzzer] add ThreadedLeakTest
llvm-svn: 275582
2016-07-15 17:19:43 +00:00
Dan Liew 30ca81f1f7 [LibFuzzer] Unbreak the build on macOS which was broken by r272858.
``afl_driver.cpp`` currently relies on weak symbols which doesn't
work properly under macOS. For now fix the build by providing a
dummy implementation of ``LLVMFuzzerInitialize(...)``. This is just
a temporary measure until we fix ``afl_driver.cpp`` for macOS.

llvm-svn: 274778
2016-07-07 18:14:11 +00:00
Mike Aizatsky 8ba86a5a48 [libFuzzer] Let user specify extra stats file.
Summary: If AFL_DRIVER_EXTRA_STATS_FILENAME is set and valid, write to it peak_rss_mb and slowest_unit_time_sec. These are both stats that libFuzzer can print but afl cannot.

Reviewers: kcc, aizatsky, metzman

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D21742

llvm-svn: 274273
2016-06-30 20:43:06 +00:00
Mike Aizatsky f0b3e85f4e [libfuzzer] moving is_ascii handler inside mutation dispatcher.
Summary: It also fixes a bug, when first random might not be ascii.

Differential Revision: http://reviews.llvm.org/D21573

llvm-svn: 273611
2016-06-23 20:44:48 +00:00
Vitaly Buka 0618cbacb6 [libFuzzer] Add standard license info and comment header to AFLDriverTest.cpp
Summary: Add license info and brief description of file to AFLDriverTest.cpp.

Reviewers: kcc, aizatsky

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D21487

llvm-svn: 273527
2016-06-23 02:19:36 +00:00
Kostya Serebryany 042d1a7b04 [libFuzzer] make the single-run output more reliable
llvm-svn: 272998
2016-06-17 13:07:06 +00:00
Kostya Serebryany fd6ad5bba9 [libFuzzer] use the new chainable malloc hooks instead of the old un-chainable ones, use atomic for malloc/free counters instead of a thread local counter in the main thread. This should make on-the-spot leak detection in libFuzzer more reliable
llvm-svn: 272948
2016-06-16 20:17:41 +00:00
Vitaly Buka 7bd4720e6b Fix test from D21194
Bot sets ASAN_OPTIONS=handle_abort=1 which prevents expected crash.

llvm-svn: 272866
2016-06-16 01:52:48 +00:00
Vitaly Buka d6da8f6612 Debugging D21194 issues on bot
llvm-svn: 272863
2016-06-16 01:26:46 +00:00
Vitaly Buka d01720d46d Enable libFuzzer's afl_driver to append stderr to a file.
Summary:
[libFuzzer] Enable afl_driver to append stderr to a user specified file.

Append stderr of afl_driver to the file specified by the environmental variable
AFL_DRIVER_STDERR_DUPLICATE_FILENAME if it is set. This lets users see outputs
on crashes without rerunning crashing test cases (which won't work for crashes
that are difficult to reproduce). Before this patch, stderr would only be sent to afl-fuzz
and users would have no way of seeing it.

Reviewers: llvm-commits, aizatsky, kcc, vitalybuka

Subscribers: vitalybuka

Differential Revision: http://reviews.llvm.org/D21194

llvm-svn: 272858
2016-06-16 00:14:42 +00:00
Dan Liew c4989d27b5 [LibFuzzer] Fix ``FuzzerMutate.ShuffleBytes2`` unit test on OSX.
The ``FuzzerMutate.ShuffleBytes2`` unit test was failing on
OSX due to the implementation of ``std::random_shuffle()``
being different between libcxx and libstdc++.

@kcc has decided (see http://reviews.llvm.org/D21218) it is acceptable
for there to be different mutation behavior on different platforms so
this commit just adjusts the test to perform the minimum number of
iterations (that is a power of 2) to see all the mutations the unit test
is looking for.

Differential Revision: http://reviews.llvm.org/D21359

llvm-svn: 272743
2016-06-15 01:40:02 +00:00
Vitaly Buka dca553d021 Revert "Enable libFuzzer's afl_driver to append stderr to a file."
Crashes with AddressSanitizer: SEGV on unknown address
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/9924/steps/annotate/logs/stdio

This reverts commit r272706.

llvm-svn: 272726
2016-06-14 22:09:00 +00:00
Vitaly Buka 4b73cc88bf Enable libFuzzer's afl_driver to append stderr to a file.
Summary:
[libFuzzer] Enable afl_driver to append stderr to a user specified file.

Append stderr of afl_driver to the file specified by the environmental variable
AFL_DRIVER_STDERR_DUPLICATE_FILENAME if it is set. This lets users see outputs
on crashes without rerunning crashing test cases (which won't work for crashes
that are difficult to reproduce). Before this patch, stderr would only be sent to afl-fuzz
and users would have no way of seeing it.

Reviewers: llvm-commits, aizatsky, kcc, vitalybuka

Subscribers: vitalybuka

Differential Revision: http://reviews.llvm.org/D21194

llvm-svn: 272706
2016-06-14 20:42:05 +00:00
Dan Liew 916bc2b024 [LibFuzzer] Disable the ``fuzzer-trace-pc.test`` test on non-linux platforms.
On OSX this test sometimes fails due to the
``LLVMFuzzer-FullCoverageSetTest-TracePC`` program going over the
default 2GiB memory limit. This shouldn't be happening and needs
investigating. For now just disable the test so we can set up an
OSX buildbot.

Differential Revision: http://reviews.llvm.org/D21319

llvm-svn: 272696
2016-06-14 19:15:13 +00:00
Dan Liew 02432480c7 [LibFuzzer] Move tests in ``fuzzer-traces.test`` that require hooks to their own test.
The tests in ``fuzzer-traces-hooks.test`` only work on Linux because calls to hooks
(e.g. ``__sanitizer_weak_hook_memcmp()``) from inside the sanitizer
runtime are only implemented on Linux. Therefore these tests are set to
only run on Linux.

Differential Revision: http://reviews.llvm.org/D21253

llvm-svn: 272600
2016-06-14 00:11:34 +00:00
Dan Liew 0617f15897 [LibFuzzer] Fix some unit test crashes on OSX.
This fixes the following unit tests:

FuzzerDictionary.ParseOneDictionaryEntry
FuzzerDictionary.ParseDictionaryFile

The issue appears to be mixing non-ASan-ified code (LibFuzzer) and
ASan-ified code (the unittest) as the tests would pass fine if
everything was built with ASan enabled.

I believe the issue is that different implementations of std::vector<>
are being used in LibFuzzer and outside LibFuzzer (in the unittests).
For Libcxx (I've not seen the issue manifest for libstdc++) we can disable
the ASanified std::vector<> by definining the ``_LIBCPP_HAS_NO_ASAN`` macro.
Doing this fixes the tests on OSX.

Differential Revision: http://reviews.llvm.org/D21049

llvm-svn: 272374
2016-06-10 05:33:07 +00:00
Kostya Serebryany f7798526b9 [libFuzzer] add one more OOM test, which we currently don't handle very well
llvm-svn: 272240
2016-06-09 01:20:35 +00:00
Kostya Serebryany 53b7b3ca5f [libFuzzer] add 'weak' back to __sanitizer_malloc_hook and __sanitizer_free_hook
llvm-svn: 272116
2016-06-08 04:49:29 +00:00
Kostya Serebryany 76f425211e [libFuzzer] add a test that is built w/o coverage instrumentation but has the coverage rt (it should now fail with a descriptive message)
llvm-svn: 272090
2016-06-08 01:46:13 +00:00
Dan Liew 1873a496e2 [LibFuzzer] Declare and use sanitizer functions in ``fuzzer::ExternalFunctions``
This fixes linking problems on OSX.

Unfortunately it turns out we need to use an instance of the
``fuzzer::ExternalFunctions`` object in several places so this
commit also replaces all instances with a single global instance.

It also turns out initializing a global ``fuzzer::ExternalFunctions``
before main is entered (i.e. letting the object be initialised by the
global initializers) is not safe (on OSX the call to ``Printf()`` in the
CTOR crashes if it is called from a global initializer) so we instead
have a global ``fuzzer::ExternalFunctions*`` and initialize it inside
``FuzzerDriver()``.

Multiple unit tests depend also depend on the
``fuzzer::ExternalFunctions*`` global so a ``main()`` function has been
added that initializes it before running any tests.

Differential Revision: http://reviews.llvm.org/D20943

llvm-svn: 272072
2016-06-07 23:32:50 +00:00
Dan Liew 1d0a9fd089 [LibFuzzer] Split the fuzzer-oom.test into two tests.
This is necessary because the existing fuzzer-oom.test was Linux
specific due to its use of __sanitizer_print_memory_profile() which
is only available on Linux right now and so the test would fail on OSX.

Differential Revision: http://reviews.llvm.org/D20977

llvm-svn: 272061
2016-06-07 21:23:30 +00:00
Mike Aizatsky 41d6683c39 [libfuzzer] custom crossover interface function.
Differential Revision: http://reviews.llvm.org/D21089

llvm-svn: 272054
2016-06-07 20:22:15 +00:00
Mike Aizatsky 1f88b12272 [libfuzzer] prune_corpus option for disabling pruning during the load.
Summary:
The option is very useful for testing, plus I intend to measure
its effect on fuzzer effectiveness.

Differential Revision: http://reviews.llvm.org/D21084

llvm-svn: 272035
2016-06-07 18:16:32 +00:00
Dan Liew d15c106c9b [LibFuzzer] s/dataflow sanitizer/DataflowSanitizer/
llvm-svn: 271980
2016-06-07 04:44:49 +00:00
Dan Liew 0d09f14554 [LibFuzzer] Disable building and running LSan tests on Apple platforms because LSan is not currently supported.
Differential Revision: http://reviews.llvm.org/D20947

llvm-svn: 271979
2016-06-07 04:44:39 +00:00
Dan Liew 5914407185 [LibFuzzer] Provide stub implementation of __sanitizer_cov_trace_pc_indir
Calls to this function are currently injected by the
``SanitizerCoverageModule`` pass when the both the ``indirect-calls``
and ``trace-pc`` sanitizer coverage options are enabled and the code
being instrumented has indirect calls. Previously because LibFuzzer did
not define this function this would lead to link errors when building
some of the tests on OSX.

Differential Revision: http://reviews.llvm.org/D20946

llvm-svn: 271938
2016-06-06 20:27:09 +00:00
Mike Aizatsky 70fd3e412a [libfuzzer] hiding custom mutator handling in MutationDispatcher.
Summary: Refactoring, no functional changes.

Differential Revision: http://reviews.llvm.org/D20975

llvm-svn: 271740
2016-06-03 21:34:29 +00:00
Mike Aizatsky 3c1782f23b [libfuzzer] splitting fuzzer.test
llvm-svn: 271697
2016-06-03 18:05:22 +00:00
Dan Liew fba4a67a32 [LibFuzzer] Disable compiling and running the LibFuzzer dataflow sanitizer tests on Apple platforms.
This fixes a broken part of the build on OSX as the dataflow sanitizer is not supported
on OSX yet.

Differential Revision: http://reviews.llvm.org/D20894

llvm-svn: 271492
2016-06-02 05:48:09 +00:00
Dan Liew d3c33116fd [LibFuzzer] Reimplement how the optional user functions are called.
The motivation for this change is to fix linking issues on OSX.
However this only partially fixes linking issues (the uninstrumented
tests and a few others  won't succesfully link yet).

This change introduces a struct of function pointers
(``fuzzer::ExternalFuntions``) which when initialised will point to the
optional functions if they are available.  Currently these
``LLVMFuzzerInitialize`` and ``LLVMFuzzerCustomMutator`` functions.

Two implementations of ``fuzzer::ExternalFunctions`` constructor are
provided one for Linux and one for OSX.

The OSX implementation uses ``dlsym()`` because the prior implementation
using weak symbols does not work unless the additional flags are passed
to the linker.

The Linux implementation continues to use weak symbols because the
``dlsym()`` approach does not work unless additional flags are passed
to the linker.

Differential Revision: http://reviews.llvm.org/D20741

llvm-svn: 271491
2016-06-02 05:48:02 +00:00
Kostya Serebryany f6414426f2 [libFuzzer] use __sanitizer_print_memory_profile to print the memory profile on OOM
llvm-svn: 271465
2016-06-02 01:33:11 +00:00
Kostya Serebryany f817731a19 [libFuzzer] when an invalid flag is given, warn, but don't crash
llvm-svn: 271404
2016-06-01 16:41:12 +00:00
Kostya Serebryany 4795210f9c [libFuzzer] fix a use-after-free (!) in libFuzzer caused by r270905: that CL caused a push_back in the main corpus invalidating the vector<> iterators in rare cases.
llvm-svn: 271186
2016-05-29 15:58:57 +00:00
Kostya Serebryany 311cc8378e [libFuzzer] fix a failure that occurs when running individual inputs
llvm-svn: 271095
2016-05-28 04:19:46 +00:00
Kostya Serebryany 470d04400e [libFuzzer] use __sanitizer_set_report_fd with -close_fd_mask. This allows us to keep asan reports when closing target's stderr
llvm-svn: 271053
2016-05-27 21:46:22 +00:00
Richard Smith b62e7e31f7 Fix compilation with GCC, which treats this as a constructor name not a type
name. (GCC is correct here per the latest language DRs.)

llvm-svn: 271044
2016-05-27 21:05:35 +00:00
Dan Liew 8c11fce707 [LibFuzzer] Refactor declaration of tests in CMake.
Add a new CMake function (``add_libfuzzer_test()``) to simplify
declaration of executables for testing LibFuzzer and use it to
reorganise how tests are declared.

Note that configuration of the lit configuration files has been moved
as late as possible because we are going to need to disable some tests
for some platforms and we will need to propagate this information into
the lit configuration.

Note the code for custom mains was removed because no tests are
currently written for this and Kostya seems happy to remove this.

Differential Revision: http://reviews.llvm.org/D20706

llvm-svn: 270958
2016-05-27 03:14:40 +00:00
Kostya Serebryany 76dacb4ba9 [libFuzzer] make check-fuzzer a bit faster
llvm-svn: 270947
2016-05-27 01:12:21 +00:00
Kostya Serebryany 0edb563f27 [libFuzzer] make OOM-handling more portable. Instead of sending a signal to the main fuzzing thread, print the message in the getrusage thread and exit.
llvm-svn: 270945
2016-05-27 00:54:15 +00:00
Kostya Serebryany 8fc3a27c5c [libFuzzer] more refactoring: make sure CurrentUnitData is awlays a valid pointer to read from
llvm-svn: 270942
2016-05-27 00:21:33 +00:00
Kostya Serebryany d8384122a3 [libFuzzer] more refactoring around CurrentUnit. Also add a threading test on which we currently have a race (when reporting bugs from multiple threads)
llvm-svn: 270929
2016-05-26 22:17:32 +00:00
Dan Liew 9551fdd517 [LibFuzzer] Add missing #include<string>
This partially fixes the compilation of the LibFuzzer unit test
on OSX using AppleClang.

llvm-svn: 270926
2016-05-26 21:54:25 +00:00
Kostya Serebryany f26017baf9 [libFuzzer] refactor: hide CurrentUnitData inside an interface function. NFC
llvm-svn: 270922
2016-05-26 21:32:30 +00:00
Dan Liew c752a26428 [LibFuzzer] Emit error if LLVM_USE_SANITIZER is not correctly set.
Previously CMake would successfully configure and compile (with warnings
about ``-fsanitize-coverage=...`` being unused) but the tests LibFuzzer
tests would fail.

Differential Revision: http://reviews.llvm.org/D20662

llvm-svn: 270913
2016-05-26 20:55:09 +00:00
Dan Liew 676892a5d5 [LibFuzzer] Allow LibFuzzer to be built in modes other than RELEASE.
Previously the flags were only being set correctly when the
build type was "Release". Now the build should work properly
for all the supported build types. When building libFuzzer
the optimization level respects whatever is used for the
rest of LLVM but for the LibFuzzer tests we force -O0.

Differential Revision: http://reviews.llvm.org/D20558

llvm-svn: 270912
2016-05-26 20:55:05 +00:00
Kostya Serebryany 4b92326b17 [libFuzzer] when there is a leak in the existing corpus report the reproducer properly
llvm-svn: 270905
2016-05-26 20:25:49 +00:00
Kostya Serebryany f1f3f93c9e [libFuzzer] reimplement the way we do -only_ascii to allow more 'const' in function declarations. Add a test for -only_ascii. NFC intended
llvm-svn: 270900
2016-05-26 20:03:02 +00:00
Mike Aizatsky e66846ab32 [libfuzzer] replacing unittest for truncate_units with functional test.
Differential Revision: http://reviews.llvm.org/D20641

llvm-svn: 270755
2016-05-25 21:00:17 +00:00
Kostya Serebryany ff2e6badbd [libFuzzer] print stats if we crash on empty input
llvm-svn: 270639
2016-05-25 00:15:36 +00:00
Mike Aizatsky af432a45e3 [libfuzzer] Trying random unit prefixes during corpus load.
Differential Revision: http://reviews.llvm.org/D20301

llvm-svn: 270632
2016-05-24 23:14:29 +00:00
Kostya Serebryany c8bc8821cb [libFuzzer] add a license header to afl/afl_driver.cpp
llvm-svn: 270598
2016-05-24 19:05:25 +00:00
Dan Liew 11565444ca [LibFuzzer] Fix implementation of ``GetPeakRSSMb()`` on Mac OSX.
On Linux ``rusage.ru_maxrss`` is in KiB but on Mac OSX it is in bytes.

Differential Revision: http://reviews.llvm.org/D20410

llvm-svn: 270173
2016-05-20 01:37:54 +00:00
Dan Liew e6ac1fd089 [LibFuzzer] Fix ``NumberOfCpuCores()`` on Mac OSX.
The ``nprocs`` command does not exist under Mac OSX so use
``sysctl`` instead on that platform.

Whilst I'm here

* Use ``pclose()`` instead of ``fclose()`` which the ``popen()``
  documentation says should be used.
* Check for errors that were previously unhandled.

Differential Revision: http://reviews.llvm.org/D20409

llvm-svn: 270172
2016-05-20 01:30:36 +00:00
Dan Liew 3868e468fe [LibFuzzer]
Work around crashes in ``__sanitizer_malloc_hook()`` under Mac OSX.

Under Mac OSX we intercept calls to malloc before thread local
storage is initialised leading to a crash when accessing
``AllocTracer``. To workaround this ``AllocTracer`` is only accessed
in the hook under Linux. For symmetry ``__sanitizer_free_hook()``
is also modified in the same way.

To support this change a set of new macros
LIBFUZZER_LINUX and LIBFUZZER_APPLE has been defined which can be
used to check the target being compiled for.

Differential Revision: http://reviews.llvm.org/D20402

llvm-svn: 270145
2016-05-19 22:00:33 +00:00
Kostya Serebryany a0788e7dd4 [libFuzzer] do the merge faster and a bit less precise
llvm-svn: 269497
2016-05-13 22:11:23 +00:00
Kostya Serebryany 6289536922 [libFuzzer] print the file name before executing the input so that if there is a crash we know which files has caused it
llvm-svn: 269450
2016-05-13 18:10:33 +00:00
Kostya Serebryany 8b0d90a6d4 [libFuzzer] simplify FuzzerInterface.h
llvm-svn: 269448
2016-05-13 18:04:35 +00:00
Kostya Serebryany 781e8da888 [libFuzzer] add a driver file to use AFL on LLVMFuzzerTestOneInput
llvm-svn: 269141
2016-05-10 23:46:50 +00:00
Mike Aizatsky 1aa501e7e8 [libfuzzer] Refactoring coverage state-management code.
It is now less state-dependent and will allow easier comparing of
coverages of different units.

Differential Revision: http://reviews.llvm.org/D20085

llvm-svn: 269140
2016-05-10 23:43:15 +00:00
Kostya Serebryany ca9694ba2c [libFuzzer] add a test for libFuzzer+ubsan, extend the docs on using libFuzzer+ubsan
llvm-svn: 268968
2016-05-09 21:02:36 +00:00
Kostya Serebryany 61b07ac75b [libFuzzer] better document the -merge=1 flag
llvm-svn: 268957
2016-05-09 19:11:36 +00:00
Kostya Serebryany 8b8f7a3cda [libFuzzer] enhance -rss_limit_mb and enable by default. Now it will print the OOM reproducer.
llvm-svn: 268821
2016-05-06 23:38:07 +00:00
Kostya Serebryany 52b394e981 [libFuzzer] add exeprimental -rss_limit_mb flag to fight against OOMs
llvm-svn: 268807
2016-05-06 21:58:35 +00:00
Kostya Serebryany baf7fd0b16 [libFuzzer] print stats after running individual inputs
llvm-svn: 268547
2016-05-04 20:44:50 +00:00
Kostya Serebryany 2fe9304d62 [libFuzzer] enable detect_leaks=1, add proper docs
llvm-svn: 268088
2016-04-29 18:49:55 +00:00
Kostya Serebryany 0e0bcc4bdb [libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak [part 2]
llvm-svn: 267771
2016-04-27 19:52:56 +00:00
Kostya Serebryany 7018a1aaa4 [libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak
llvm-svn: 267770
2016-04-27 19:52:34 +00:00
Kostya Serebryany 9ba19182be [libFuzzer] remove dead code
llvm-svn: 267455
2016-04-25 19:41:45 +00:00
Kostya Serebryany 1bfd583d82 [libFuzzer] added -detect_leaks flag (0 by default for now). When enabled, it will help finding leaks while fuzzing
llvm-svn: 266838
2016-04-20 00:24:21 +00:00
Kostya Serebryany ebb932d060 [libFuzzer] try to print correct time in seconds when reporting a timeout. Don't report timeouts while still loading the corpus.
llvm-svn: 266693
2016-04-18 22:50:39 +00:00
Kostya Serebryany 09087bba4d [libFuzzer] warn if the corpus is empty
llvm-svn: 266670
2016-04-18 21:14:11 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Kostya Serebryany b60397f54c [libFuzzer] add a better warning for command line flags with -- (two dashes)
llvm-svn: 266480
2016-04-15 21:56:29 +00:00
Hans Wennborg e631996350 Remove redundant .c_str(), as suggested by PR25633
llvm-svn: 265988
2016-04-11 20:35:17 +00:00
Mike Aizatsky 94e29668b0 [libfuzzer] defensive assert
llvm-svn: 265866
2016-04-08 23:32:24 +00:00
Mike Aizatsky f13cbee12e [libfuzzer] adding license headers to cpp files
Differential Revision: http://reviews.llvm.org/D18705

llvm-svn: 265174
2016-04-01 18:38:58 +00:00
Kostya Serebryany 9e1a238357 [libFuzzer] more docs
llvm-svn: 264803
2016-03-29 23:07:36 +00:00
Kostya Serebryany f3ab6d9e10 [libFuzzer] use fflush after every Printf
llvm-svn: 264459
2016-03-25 20:31:26 +00:00
Kostya Serebryany f389ae12c1 [libFuzzer] handle SIGTERM
llvm-svn: 264338
2016-03-24 21:03:58 +00:00
Kostya Serebryany 315167339e [libFuzzer] don't report memory leaks if we are dying due to a timeout (just use _Exit instead of exit in the timeout callback)
llvm-svn: 264237
2016-03-24 01:32:08 +00:00
Kostya Serebryany 6278f933a8 [libFuzzer] use fdopen+vfprintf instead of fsnprintf+write
llvm-svn: 264230
2016-03-24 00:57:32 +00:00
Kostya Serebryany 49e409068a [libFuzzer] add a flag close_fd_mask so that we can silence spammy targets by closing stderr/stdout
llvm-svn: 263831
2016-03-18 20:58:29 +00:00
Benjamin Kramer d96b0c14fb [Fuzzer] Guard no_sanitize_memory attributes behind __has_feature.
Otherwise GCC fails to build it because it doesn't know the attribute.

llvm-svn: 263787
2016-03-18 14:19:19 +00:00
Kostya Serebryany c43b584c1c [libFuzzer] read corpus dirs recursively
llvm-svn: 263773
2016-03-18 01:36:00 +00:00
Kostya Serebryany 945761b8c2 [libFuzzer] improve -merge functionality
llvm-svn: 263769
2016-03-18 00:23:29 +00:00
Kostya Serebryany c5575aabd6 [libFuzzer] deprecate several flags
llvm-svn: 263739
2016-03-17 19:59:39 +00:00
Kostya Serebryany 23dbc390af [libFuzzer] add __attribute__((no_sanitize_memory)) to two functions that may be called from signal handler(s) or from msan. This will hopefully avoid msan false reports which I can't reproduce
llvm-svn: 263737
2016-03-17 19:42:35 +00:00
Mike Aizatsky 298516ffa9 [libfuzzer] speeding up corpus load
llvm-svn: 263591
2016-03-15 21:47:21 +00:00
Kostya Serebryany 0c5e3af862 [libFuzzer] use max_len exactly equal to the max size of input. Fix 32-bit build
llvm-svn: 263518
2016-03-15 01:28:00 +00:00
Kostya Serebryany 64d24578d8 [libFuzzer] try to use max_len based on the items of the corpus instead of blindly defaulting to 64 bytes.
llvm-svn: 263323
2016-03-12 01:57:04 +00:00
Mike Aizatsky 243fe2b3a0 [libfuzzer] adding std:string to allowed adaptable argument.
llvm-svn: 262757
2016-03-04 23:18:01 +00:00
Kostya Serebryany 5c3701c621 [libFuzzer] log less when re-loading files; fix a silly bug: when running single files actually run all of them, not just the first one
llvm-svn: 262754
2016-03-04 22:35:40 +00:00
Mike Aizatsky b8627a89a6 [libfuzzer] arbitrary function adapter.
The adapter automates converting sequence of bytes into arbitrary
arguments.

Differential Revision: http://reviews.llvm.org/D17829

llvm-svn: 262673
2016-03-03 23:45:29 +00:00
Kostya Serebryany e483ed2825 [libFuzzer] when interrupted, call _Exit() instead of exit()
llvm-svn: 262667
2016-03-03 22:36:37 +00:00
Kostya Serebryany 4394b31e1d [libFuzzer] add -Werror for libFuzzer build rule
llvm-svn: 262517
2016-03-02 21:08:16 +00:00
Dmitry Vyukov 2eed1218e5 libfuzzer: fix compiler warnings
- unused sigaction/setitimer result (used in assert)
- unchecked fscanf return value
- signed/unsigned comparison

llvm-svn: 262472
2016-03-02 09:54:40 +00:00
Kostya Serebryany 3d95dd9149 [libFuzzer] deprecate exit_on_first flag
llvm-svn: 262417
2016-03-01 22:33:14 +00:00
Kostya Serebryany 228d5b1ce4 [libFuzzer] add generic signal handlers so that libFuzzer can report at least something if ASan is not handlig the signals for us. Remove abort_on_timeout flag.
llvm-svn: 262415
2016-03-01 22:19:21 +00:00
Kostya Serebryany f84df30e4f [libFuzzer] remove FuzzerSanitizerOptions.cpp
llvm-svn: 262354
2016-03-01 17:46:32 +00:00
Rafael Espindola 5cd721ae12 Refactor duplicated code for linking with pthread.
llvm-svn: 262344
2016-03-01 15:54:40 +00:00
Kostya Serebryany bf821db932 [libFuzzer] fixing the bot
llvm-svn: 262106
2016-02-27 03:14:23 +00:00
Kostya Serebryany 2d4f8f168b [libFuzzer] speedup path coverage handling
llvm-svn: 262102
2016-02-27 01:50:16 +00:00
Kostya Serebryany 66ff0756e4 [libFuzzer] add -print_final_stats=1 flag
llvm-svn: 262084
2016-02-26 22:42:23 +00:00
Kostya Serebryany da63c1d09a [libFuzzer] initial implementation of path coverage based on -fsanitize-coverage=trace-pc. This does not scale well yet, but already cracks FullCoverageSetTest in seconds
llvm-svn: 262073
2016-02-26 21:33:56 +00:00
Kostya Serebryany a35f7d383f [libFuzzer] only read MaxLen bytes from every file in the corpus to speedup loading the corpus
llvm-svn: 261267
2016-02-18 21:49:10 +00:00
Kostya Serebryany 90bcdb512b [libFuzzer] fix the libFuzzer bot
llvm-svn: 261184
2016-02-18 02:02:40 +00:00
Kostya Serebryany cfbcf9097d [libFuzzer] don't timeout when loading the corpus. Be a bit more verbose when loading large corpus.
llvm-svn: 261143
2016-02-17 19:42:34 +00:00
Kostya Serebryany 8a5bef0fcf [libFuzzer] remove std::vector operations from hot paths, NFC
llvm-svn: 260829
2016-02-13 17:56:51 +00:00
Kostya Serebryany 1deb0498f5 [libFuzzer] don't require seed in fuzzer::Mutate, instead use the global Fuzzer object for fuzzer::Mutate. This makes custom mutators fast
llvm-svn: 260810
2016-02-13 06:24:18 +00:00
Kostya Serebryany 29bcb9f54e [libFuzzer] remove the C++-ish variant of FuzzerDriver from the interface
llvm-svn: 260801
2016-02-13 03:59:26 +00:00
Kostya Serebryany 23194963f7 [libFuzzer] simplify CTOR of MutationDispatcher
llvm-svn: 260800
2016-02-13 03:46:26 +00:00
Kostya Serebryany 292cf0379c [libFuzzer] get rid of MutationDispatcher::Impl (simplify the code; NFC)
llvm-svn: 260799
2016-02-13 03:37:24 +00:00
Kostya Serebryany 7ec0c56e07 [libFuzzer] get rid of UserSuppliedFuzzer; NFC
llvm-svn: 260798
2016-02-13 03:25:16 +00:00
Kostya Serebryany a399221c32 [libFuzzer] simplify the code around Random. NFC
llvm-svn: 260797
2016-02-13 03:00:53 +00:00
Kostya Serebryany ecab57b3ce [libFuzzer] remove UserSuppliedFuzzer from the interface (it was a bad idea).
llvm-svn: 260796
2016-02-13 02:39:30 +00:00
Kostya Serebryany 22cc5e2375 [libFuzzer] provide a plain C interface for custom mutators (experimental)
llvm-svn: 260794
2016-02-13 02:29:38 +00:00
Kostya Serebryany 9d14e4bb15 [libFuzzer] make -runs=N flag also affect the simple runner (will execute every input N times)
llvm-svn: 260649
2016-02-12 02:32:03 +00:00
Mike Aizatsky fcb06b4aa5 [libfuzzer] Removing coverage-related flags from asan options.
Summary:
Reasons to remove are twofold:
 - we don't really need coverage=1 for libfuzzer operation
 - makes controlling coverage for fuzzer processes non-trivial.

Differential Revision: http://reviews.llvm.org/D17168

llvm-svn: 260611
2016-02-11 22:20:34 +00:00
Kostya Serebryany ce925c580e [libFuzzer] hot fix a test
llvm-svn: 259732
2016-02-04 00:12:28 +00:00
Kostya Serebryany b92602ada0 [libFuzzer] don't write the test unit when a leak is detected (since we don't know which unit causes the leak)
llvm-svn: 259731
2016-02-04 00:02:17 +00:00
Kostya Serebryany d88d1305c4 [libFuzzer] don't create too many trace-based mutations as it may be too slow
llvm-svn: 259600
2016-02-02 23:17:45 +00:00
Kostya Serebryany bfbe7fc404 [libFuzzer] allow passing 1 or more files as individual inputs
llvm-svn: 259459
2016-02-02 03:03:47 +00:00
Kostya Serebryany 078e984d8d [libFuzzer] fail if the corpus dir does not exist
llvm-svn: 259454
2016-02-02 02:07:26 +00:00
Kostya Serebryany 54a6363a8f [libFuzzer] add -timeout_exitcode option
llvm-svn: 259265
2016-01-29 23:30:07 +00:00
Kostya Serebryany 085ca4131f [libFuzzer] re-enable test for -abort_on_timeout=1, this time protecting from ASAN_OPTIONS set outside
llvm-svn: 259263
2016-01-29 23:19:00 +00:00
Ivan Krasin d7785e75f3 Temporary disable broken fuzzer/timeout tests.
Reviewers: kcc

Differential Revision: http://reviews.llvm.org/D16543

llvm-svn: 258702
2016-01-25 19:05:45 +00:00
Kostya Serebryany 9768e7f06b [libFuzzer] add -abort_on_timeout option
llvm-svn: 258631
2016-01-23 19:34:19 +00:00
Kostya Serebryany 160dcba81f [libFuzzer] add more fields to DictionaryEntry to count the number of uses and successes
llvm-svn: 258589
2016-01-22 23:55:14 +00:00
Ivan Krasin df91910bd4 Use std::piecewise_constant_distribution instead of ad-hoc binary search.
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.

Note: this is the second attempt (prev: r258473). Now, libc++ build is fixed.

Reviewers: aizatsky, kcc

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D16487

llvm-svn: 258571
2016-01-22 22:28:27 +00:00
Ivan Krasin d84f74cab7 Revert r258473 as it's breaking the build with libc++
Reviewers: kcc

Differential Revision: http://reviews.llvm.org/D16441

llvm-svn: 258479
2016-01-22 03:21:52 +00:00
Ivan Krasin b008fd4d89 Use std::piecewise_constant_distribution instead of ad-hoc binary search.
Summary:
Fix the issue with the most recently discovered unit receiving much less attention.

Note: I had to change the seed for one test to make it pass. Alternatively,
the number of runs could be increased. I believe that the average time of
'foo' discovery is not increased, just seed=1 was particularly convenient
for the previous PRNG scheme used.

Reviewers: aizatsky, kcc

Subscribers: llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D16419

llvm-svn: 258473
2016-01-22 01:32:34 +00:00
Kostya Serebryany b5e984992a [libFuzzer] don't do expensive memmem if the result will not be used
llvm-svn: 258462
2016-01-22 01:04:58 +00:00
Kostya Serebryany 2f13f223c7 [libFuzzer] don't use std::vector in one more hot path
llvm-svn: 258380
2016-01-21 01:52:14 +00:00
Mike Aizatsky e313f8f8ff [libfuzzer] use %p for printing addresses
llvm-svn: 258370
2016-01-21 00:02:09 +00:00
Kostya Serebryany 311f27c0a8 [libFuzzer] use std::mt19937 for generating random numbers by default. Fix MyStoll to handle negative values. Use std::any_of instead of std::find_if
llvm-svn: 258178
2016-01-19 20:33:57 +00:00
Kostya Serebryany 476f0ce31a [libFuzzer] replace vector with a simpler data structure in the Dictionaries to avoid memory allocations on hot path
llvm-svn: 257985
2016-01-16 03:53:32 +00:00
Kostya Serebryany aca7696f4d [libFuzzer] introduce LLVMFuzzerInitialize
llvm-svn: 257980
2016-01-16 01:23:12 +00:00
Kostya Serebryany 628bc3ec00 [libFuzzer] move some code from public interface header to a non-public header. NFC
llvm-svn: 257963
2016-01-16 00:04:36 +00:00
Kostya Serebryany ae5b9567bc [libFuzzer] do mutations based on memcmp/strcmp interceptors under a separate flag (-use_memcmp, default=1)
llvm-svn: 257873
2016-01-15 06:24:05 +00:00
Kostya Serebryany 4282d30516 [libFuzzer] use custom stol; also introduce __libfuzzer_is_present so that users can check for its presence.
llvm-svn: 257848
2016-01-15 00:17:37 +00:00
Kostya Serebryany 4b35874b2a [libFuzzer] suggest a dictionary to the user of some of the trace-based dictionary entries were successful
llvm-svn: 257736
2016-01-14 02:36:44 +00:00
Kostya Serebryany 98abb2c90a [libFuzzer] make CurrentUnit a POD object instead of vector to avoid extra allocations
llvm-svn: 257713
2016-01-13 23:46:01 +00:00
Kostya Serebryany d50a3eedb4 [libFuzzer] make sure we find buffer overflow in the input buffer. Previously, re-using the same vector object was hiding buffer overflows (unless we used annotated vector)
llvm-svn: 257701
2016-01-13 23:02:30 +00:00
Kostya Serebryany 72fdb32dac [libFuzzer] make sure to update CurrentUnit when drilling
llvm-svn: 257560
2016-01-13 01:58:27 +00:00
Kostya Serebryany 4b83a4f6fe [libFuzzer] add a macro LLVM_FUZZER_DEFINES_SANITIZER_WEAK_HOOOKS
llvm-svn: 257482
2016-01-12 16:50:18 +00:00
Kostya Serebryany 4174005622 [libFuzzer] when a new unit is discovered using a dictionary, print all used dictionary entries
llvm-svn: 257435
2016-01-12 02:36:59 +00:00
Kostya Serebryany 859e86d962 [libFuzzer] add various debug prints. Also don't mutate based on a cmp trace like (a eq a) or (a neq a)
llvm-svn: 257434
2016-01-12 02:08:37 +00:00
Kostya Serebryany e3580956ea [libFuzzer] extend the weak memcmp/strcmp/strncmp interceptors to receive the result of the computations. With that, don't do any mutations if memcmp/etc returned 0
llvm-svn: 257423
2016-01-12 00:43:42 +00:00
Kostya Serebryany 1f9c40db1d [libFuzzer] debug prints in tracing
llvm-svn: 257249
2016-01-09 03:46:08 +00:00
Kostya Serebryany b65805a939 [libFuzzer] change the way trace-based mutations are applied. Instead of a custom code just rely on the automatically created dictionary
llvm-svn: 257248
2016-01-09 03:08:58 +00:00
Kostya Serebryany c573316eee [libFuzzer] don't limit memcmp tracing with 8 bytes
llvm-svn: 257245
2016-01-09 01:39:55 +00:00
Kostya Serebryany e7583d21e3 [libFuzzer] refactor the way we collect cmp traces (don't use std::vector, don't limit with 8 bytes)
llvm-svn: 257239
2016-01-09 00:38:40 +00:00
Kostya Serebryany 152ac7ad70 [libFuzzer] add a position hint to the dictionary-based mutator
llvm-svn: 257013
2016-01-07 01:49:35 +00:00
Kostya Serebryany 80eb76abf4 [libFuzzer] extend the dictionary mutator to optionally overwrite data with the dict entry
llvm-svn: 256900
2016-01-06 02:13:04 +00:00
Mike Aizatsky 8b11f877e4 [libfuzzer] print_new_cov_pcs experimental option.
Differential Revision: http://reviews.llvm.org/D15901

llvm-svn: 256882
2016-01-06 00:21:22 +00:00
Kostya Serebryany 226b734d73 [libFuzzer] make trace-based fuzzing not crash in presence of threads
llvm-svn: 256876
2016-01-06 00:03:35 +00:00
Kostya Serebryany b0fb6e8508 [libFuzzer] add AFL-style dictionary for C++, remove the old file with tokens
llvm-svn: 256229
2015-12-22 01:50:51 +00:00