Commit Graph

287 Commits

Author SHA1 Message Date
Dan Liew fba4a67a32 [LibFuzzer] Disable compiling and running the LibFuzzer dataflow sanitizer tests on Apple platforms.
This fixes a broken part of the build on OSX as the dataflow sanitizer is not supported
on OSX yet.

Differential Revision: http://reviews.llvm.org/D20894

llvm-svn: 271492
2016-06-02 05:48:09 +00:00
Dan Liew d3c33116fd [LibFuzzer] Reimplement how the optional user functions are called.
The motivation for this change is to fix linking issues on OSX.
However this only partially fixes linking issues (the uninstrumented
tests and a few others  won't succesfully link yet).

This change introduces a struct of function pointers
(``fuzzer::ExternalFuntions``) which when initialised will point to the
optional functions if they are available.  Currently these
``LLVMFuzzerInitialize`` and ``LLVMFuzzerCustomMutator`` functions.

Two implementations of ``fuzzer::ExternalFunctions`` constructor are
provided one for Linux and one for OSX.

The OSX implementation uses ``dlsym()`` because the prior implementation
using weak symbols does not work unless the additional flags are passed
to the linker.

The Linux implementation continues to use weak symbols because the
``dlsym()`` approach does not work unless additional flags are passed
to the linker.

Differential Revision: http://reviews.llvm.org/D20741

llvm-svn: 271491
2016-06-02 05:48:02 +00:00
Kostya Serebryany f6414426f2 [libFuzzer] use __sanitizer_print_memory_profile to print the memory profile on OOM
llvm-svn: 271465
2016-06-02 01:33:11 +00:00
Kostya Serebryany f817731a19 [libFuzzer] when an invalid flag is given, warn, but don't crash
llvm-svn: 271404
2016-06-01 16:41:12 +00:00
Kostya Serebryany 4795210f9c [libFuzzer] fix a use-after-free (!) in libFuzzer caused by r270905: that CL caused a push_back in the main corpus invalidating the vector<> iterators in rare cases.
llvm-svn: 271186
2016-05-29 15:58:57 +00:00
Kostya Serebryany 311cc8378e [libFuzzer] fix a failure that occurs when running individual inputs
llvm-svn: 271095
2016-05-28 04:19:46 +00:00
Kostya Serebryany 470d04400e [libFuzzer] use __sanitizer_set_report_fd with -close_fd_mask. This allows us to keep asan reports when closing target's stderr
llvm-svn: 271053
2016-05-27 21:46:22 +00:00
Richard Smith b62e7e31f7 Fix compilation with GCC, which treats this as a constructor name not a type
name. (GCC is correct here per the latest language DRs.)

llvm-svn: 271044
2016-05-27 21:05:35 +00:00
Dan Liew 8c11fce707 [LibFuzzer] Refactor declaration of tests in CMake.
Add a new CMake function (``add_libfuzzer_test()``) to simplify
declaration of executables for testing LibFuzzer and use it to
reorganise how tests are declared.

Note that configuration of the lit configuration files has been moved
as late as possible because we are going to need to disable some tests
for some platforms and we will need to propagate this information into
the lit configuration.

Note the code for custom mains was removed because no tests are
currently written for this and Kostya seems happy to remove this.

Differential Revision: http://reviews.llvm.org/D20706

llvm-svn: 270958
2016-05-27 03:14:40 +00:00
Kostya Serebryany 76dacb4ba9 [libFuzzer] make check-fuzzer a bit faster
llvm-svn: 270947
2016-05-27 01:12:21 +00:00
Kostya Serebryany 0edb563f27 [libFuzzer] make OOM-handling more portable. Instead of sending a signal to the main fuzzing thread, print the message in the getrusage thread and exit.
llvm-svn: 270945
2016-05-27 00:54:15 +00:00
Kostya Serebryany 8fc3a27c5c [libFuzzer] more refactoring: make sure CurrentUnitData is awlays a valid pointer to read from
llvm-svn: 270942
2016-05-27 00:21:33 +00:00
Kostya Serebryany d8384122a3 [libFuzzer] more refactoring around CurrentUnit. Also add a threading test on which we currently have a race (when reporting bugs from multiple threads)
llvm-svn: 270929
2016-05-26 22:17:32 +00:00
Dan Liew 9551fdd517 [LibFuzzer] Add missing #include<string>
This partially fixes the compilation of the LibFuzzer unit test
on OSX using AppleClang.

llvm-svn: 270926
2016-05-26 21:54:25 +00:00
Kostya Serebryany f26017baf9 [libFuzzer] refactor: hide CurrentUnitData inside an interface function. NFC
llvm-svn: 270922
2016-05-26 21:32:30 +00:00
Dan Liew c752a26428 [LibFuzzer] Emit error if LLVM_USE_SANITIZER is not correctly set.
Previously CMake would successfully configure and compile (with warnings
about ``-fsanitize-coverage=...`` being unused) but the tests LibFuzzer
tests would fail.

Differential Revision: http://reviews.llvm.org/D20662

llvm-svn: 270913
2016-05-26 20:55:09 +00:00
Dan Liew 676892a5d5 [LibFuzzer] Allow LibFuzzer to be built in modes other than RELEASE.
Previously the flags were only being set correctly when the
build type was "Release". Now the build should work properly
for all the supported build types. When building libFuzzer
the optimization level respects whatever is used for the
rest of LLVM but for the LibFuzzer tests we force -O0.

Differential Revision: http://reviews.llvm.org/D20558

llvm-svn: 270912
2016-05-26 20:55:05 +00:00
Kostya Serebryany 4b92326b17 [libFuzzer] when there is a leak in the existing corpus report the reproducer properly
llvm-svn: 270905
2016-05-26 20:25:49 +00:00
Kostya Serebryany f1f3f93c9e [libFuzzer] reimplement the way we do -only_ascii to allow more 'const' in function declarations. Add a test for -only_ascii. NFC intended
llvm-svn: 270900
2016-05-26 20:03:02 +00:00
Mike Aizatsky e66846ab32 [libfuzzer] replacing unittest for truncate_units with functional test.
Differential Revision: http://reviews.llvm.org/D20641

llvm-svn: 270755
2016-05-25 21:00:17 +00:00
Kostya Serebryany ff2e6badbd [libFuzzer] print stats if we crash on empty input
llvm-svn: 270639
2016-05-25 00:15:36 +00:00
Mike Aizatsky af432a45e3 [libfuzzer] Trying random unit prefixes during corpus load.
Differential Revision: http://reviews.llvm.org/D20301

llvm-svn: 270632
2016-05-24 23:14:29 +00:00
Kostya Serebryany c8bc8821cb [libFuzzer] add a license header to afl/afl_driver.cpp
llvm-svn: 270598
2016-05-24 19:05:25 +00:00
Dan Liew 11565444ca [LibFuzzer] Fix implementation of ``GetPeakRSSMb()`` on Mac OSX.
On Linux ``rusage.ru_maxrss`` is in KiB but on Mac OSX it is in bytes.

Differential Revision: http://reviews.llvm.org/D20410

llvm-svn: 270173
2016-05-20 01:37:54 +00:00
Dan Liew e6ac1fd089 [LibFuzzer] Fix ``NumberOfCpuCores()`` on Mac OSX.
The ``nprocs`` command does not exist under Mac OSX so use
``sysctl`` instead on that platform.

Whilst I'm here

* Use ``pclose()`` instead of ``fclose()`` which the ``popen()``
  documentation says should be used.
* Check for errors that were previously unhandled.

Differential Revision: http://reviews.llvm.org/D20409

llvm-svn: 270172
2016-05-20 01:30:36 +00:00
Dan Liew 3868e468fe [LibFuzzer]
Work around crashes in ``__sanitizer_malloc_hook()`` under Mac OSX.

Under Mac OSX we intercept calls to malloc before thread local
storage is initialised leading to a crash when accessing
``AllocTracer``. To workaround this ``AllocTracer`` is only accessed
in the hook under Linux. For symmetry ``__sanitizer_free_hook()``
is also modified in the same way.

To support this change a set of new macros
LIBFUZZER_LINUX and LIBFUZZER_APPLE has been defined which can be
used to check the target being compiled for.

Differential Revision: http://reviews.llvm.org/D20402

llvm-svn: 270145
2016-05-19 22:00:33 +00:00
Kostya Serebryany a0788e7dd4 [libFuzzer] do the merge faster and a bit less precise
llvm-svn: 269497
2016-05-13 22:11:23 +00:00
Kostya Serebryany 6289536922 [libFuzzer] print the file name before executing the input so that if there is a crash we know which files has caused it
llvm-svn: 269450
2016-05-13 18:10:33 +00:00
Kostya Serebryany 8b0d90a6d4 [libFuzzer] simplify FuzzerInterface.h
llvm-svn: 269448
2016-05-13 18:04:35 +00:00
Kostya Serebryany 781e8da888 [libFuzzer] add a driver file to use AFL on LLVMFuzzerTestOneInput
llvm-svn: 269141
2016-05-10 23:46:50 +00:00
Mike Aizatsky 1aa501e7e8 [libfuzzer] Refactoring coverage state-management code.
It is now less state-dependent and will allow easier comparing of
coverages of different units.

Differential Revision: http://reviews.llvm.org/D20085

llvm-svn: 269140
2016-05-10 23:43:15 +00:00
Kostya Serebryany ca9694ba2c [libFuzzer] add a test for libFuzzer+ubsan, extend the docs on using libFuzzer+ubsan
llvm-svn: 268968
2016-05-09 21:02:36 +00:00
Kostya Serebryany 61b07ac75b [libFuzzer] better document the -merge=1 flag
llvm-svn: 268957
2016-05-09 19:11:36 +00:00
Kostya Serebryany 8b8f7a3cda [libFuzzer] enhance -rss_limit_mb and enable by default. Now it will print the OOM reproducer.
llvm-svn: 268821
2016-05-06 23:38:07 +00:00
Kostya Serebryany 52b394e981 [libFuzzer] add exeprimental -rss_limit_mb flag to fight against OOMs
llvm-svn: 268807
2016-05-06 21:58:35 +00:00
Kostya Serebryany baf7fd0b16 [libFuzzer] print stats after running individual inputs
llvm-svn: 268547
2016-05-04 20:44:50 +00:00
Kostya Serebryany 2fe9304d62 [libFuzzer] enable detect_leaks=1, add proper docs
llvm-svn: 268088
2016-04-29 18:49:55 +00:00
Kostya Serebryany 0e0bcc4bdb [libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak [part 2]
llvm-svn: 267771
2016-04-27 19:52:56 +00:00
Kostya Serebryany 7018a1aaa4 [libFuzzer] disable leak detection if we have tried it for 1000 times w/o finding a leak
llvm-svn: 267770
2016-04-27 19:52:34 +00:00
Kostya Serebryany 9ba19182be [libFuzzer] remove dead code
llvm-svn: 267455
2016-04-25 19:41:45 +00:00
Kostya Serebryany 1bfd583d82 [libFuzzer] added -detect_leaks flag (0 by default for now). When enabled, it will help finding leaks while fuzzing
llvm-svn: 266838
2016-04-20 00:24:21 +00:00
Kostya Serebryany ebb932d060 [libFuzzer] try to print correct time in seconds when reporting a timeout. Don't report timeouts while still loading the corpus.
llvm-svn: 266693
2016-04-18 22:50:39 +00:00
Kostya Serebryany 09087bba4d [libFuzzer] warn if the corpus is empty
llvm-svn: 266670
2016-04-18 21:14:11 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Kostya Serebryany b60397f54c [libFuzzer] add a better warning for command line flags with -- (two dashes)
llvm-svn: 266480
2016-04-15 21:56:29 +00:00
Hans Wennborg e631996350 Remove redundant .c_str(), as suggested by PR25633
llvm-svn: 265988
2016-04-11 20:35:17 +00:00
Mike Aizatsky 94e29668b0 [libfuzzer] defensive assert
llvm-svn: 265866
2016-04-08 23:32:24 +00:00
Mike Aizatsky f13cbee12e [libfuzzer] adding license headers to cpp files
Differential Revision: http://reviews.llvm.org/D18705

llvm-svn: 265174
2016-04-01 18:38:58 +00:00
Kostya Serebryany 9e1a238357 [libFuzzer] more docs
llvm-svn: 264803
2016-03-29 23:07:36 +00:00
Kostya Serebryany f3ab6d9e10 [libFuzzer] use fflush after every Printf
llvm-svn: 264459
2016-03-25 20:31:26 +00:00