Commit Graph

223 Commits

Author SHA1 Message Date
Kostya Serebryany d085748484 [libFuzzer] a bit of refactoring of the fork mode
llvm-svn: 353910
2019-02-13 04:04:45 +00:00
Kostya Serebryany 5c08e811de [libFuzzer] move the implementation of the fork mode into a separate file
llvm-svn: 353891
2019-02-12 22:48:55 +00:00
Kostya Serebryany 2b9a8f37a4 [libFuzzer] make the fork mode less verbose
llvm-svn: 353794
2019-02-12 03:12:40 +00:00
Kostya Serebryany cdbb9dc962 [libFuzzer] teach the fork mode to ignore OOMs and timeouts
llvm-svn: 353792
2019-02-12 02:18:53 +00:00
Kostya Serebryany 73bf6d5b35 [libFuzzer] replace slow std::mt19937 with a much faster std::minstd_rand; second attempt after failed r352732, this time with a fix for cmake
llvm-svn: 353782
2019-02-12 01:00:08 +00:00
Kostya Serebryany 56b2d57cbf [libFuzzer] simplify the code for print_coverage=1 so that it doesn't fail on broken debug info
llvm-svn: 353781
2019-02-12 00:52:11 +00:00
Kostya Serebryany 63f48717b5 [libFuzzer] extend the -fork=1 functionality. Still not fully usable, but good enough for the first unit test
llvm-svn: 353775
2019-02-12 00:12:33 +00:00
David L. Jones 6218b8e783 [Fuzzer] Fix function prototype in fuzzer::ExternalFunctions. [NFC]
The __sanitizer_acquire_crash_state function has int return type, but the
fuzzer's external function definitions give it bool.

Places where __sanitizer_acquire_crash_state is declared:

  include/sanitizer_common/sanitizer_interface_defs.h
  lib/sanitizer_common/sanitizer_interface_internal.h
  lib/sanitizer_common/sanitizer_common.cc
  lib/fuzzer/FuzzerExtFunctions.def (this is the only bool)

llvm-svn: 353596
2019-02-09 01:45:29 +00:00
Kostya Serebryany 0fda9dcbe9 [libFuzzer] more refactoring; change some of the exit codes (timeout, OOM, interrupt) so that the parent process can distinguish those
llvm-svn: 353584
2019-02-09 00:16:21 +00:00
Kostya Serebryany 114cfafe05 [libFuzzer] refactor the merging code, NFC
llvm-svn: 353576
2019-02-08 22:59:03 +00:00
Kostya Serebryany b1e8b8149b [libFuzzer] remove two unused experimental flags
llvm-svn: 353573
2019-02-08 22:02:37 +00:00
Kostya Serebryany f762a11544 [libFuzzer] introduce an experimental mode -fork=1, where fuzzing happens in a subprocess (still running multiple inputs per process), thus making the fuzzing more resilient to timeouts and OOMs. This is just a skeleton of the code, and some associated refactoring, not a fully working feature yet.
llvm-svn: 353570
2019-02-08 21:27:23 +00:00
Kostya Serebryany 0719b3527f [libFuzzer] refactor the way we choose the element to cross-over with, NFC (expected1); add a flag -seed_inputs= to pass extra seed inputs as file paths, not dirs
llvm-svn: 353494
2019-02-08 01:20:54 +00:00
Jonathan Metzman 7c907bac6f [libFuzzer][Windows] Clean up RawPrint
Summary:
Use `_write` instead of the deprecated alias `write` on Windows.
Also, remove comment saying RawPrint is untested on Windows.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: vitalybuka

Differential Revision: https://reviews.llvm.org/D57589

llvm-svn: 353108
2019-02-04 23:01:06 +00:00
Jonathan Metzman fc7faecb40 [fuzzer] Use RawPrint instead of Printf for instrumentation warning
Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D57531

llvm-svn: 352789
2019-01-31 20:32:20 +00:00
Jeremy Morse d778d4fded Revert r352732: [libFuzzer] replace slow std::mt19937 with a much faster std::minstd_rand
This causes a failure on the following bot as well as our internal ones:

http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/23103

llvm-svn: 352747
2019-01-31 12:44:14 +00:00
Kostya Serebryany 046cac61c1 [libFuzzer] replace slow std::mt19937 with a much faster std::minstd_rand
llvm-svn: 352732
2019-01-31 06:52:55 +00:00
Kostya Serebryany 23482e1d07 [libFuzzer] set libFuzzer's own SEGV handler even one is already present, but call that handler from ours (unless we are unprotecting lazy counters). Call ProtectLazyCounters later, so that it runs after the initialization code in the target.
llvm-svn: 352713
2019-01-31 01:40:14 +00:00
Julian Lettner 15df273eb4 [libFuzzer] Set default sanitizer options in fuzzer tests
Summary:
Set default `ASAN_OPTIONS` when running libFuzzer tests. This allows us
to remove special casing in code for Darwin where we usually pass
`abort_on_error=0` to override platform defaults for tests.

A previous commit changed the code to make the tests pass:
7764a04af0

Adapted a few tests to use `%env_asan_opts=` instead of directly setting
the environment variable.

rdar://problem/47515276

Reviewers: kcc, george.karpenkov

Differential Revision: https://reviews.llvm.org/D57465

llvm-svn: 352711
2019-01-31 01:24:01 +00:00
Kostya Serebryany 8da9479e40 [libFuzzer] experimental performance optimization -lazy_counters, off by default. Posix-only for now, tested on Linux
llvm-svn: 352700
2019-01-31 00:09:43 +00:00
Kostya Serebryany 5c0751ec3e [libFuzzer] remove stale code, NFC
llvm-svn: 352604
2019-01-30 06:21:20 +00:00
Kostya Serebryany 6fd4d8ab9c [libFuzzer] refactor the handling of instrumentation counters so that they are grouped in regions one full page each. Needed for future optimization. NFC
llvm-svn: 352603
2019-01-30 06:15:52 +00:00
Kostya Serebryany 2891b257c2 [libFuzzer] remove stale code
llvm-svn: 352571
2019-01-29 23:53:28 +00:00
Kostya Serebryany 035c08f73d [libFuzzer] revert an accidental commit
llvm-svn: 352567
2019-01-29 23:41:50 +00:00
Kostya Serebryany 62d7270610 [libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]
llvm-svn: 352566
2019-01-29 23:40:05 +00:00
Kostya Serebryany 50a1c69712 [libFuzzer] remove deprecated support for -fsanitize-coverage=trace-pc[-guard]
llvm-svn: 352564
2019-01-29 23:37:20 +00:00
Jonathan Metzman 8f7fc95ab7 [libFuzzer][Windows] Use dllexport for all declarations in FuzzerInterface.h
Summary:
Use dllexport for all declarations in FuzzerInterface.h Use it for clang
even though clang supports default visibility attribute to prevent a
warning from being thrown when LLVMFuzzerMutate is defined with dllexport.
This makes `FUZZER_INTERFACE_VISIBILITY` (FuzzerInterface.h) consistent with
`ATTRIBUTE_INTERFACE` (FuzzerDefs.h) when using clang on Windows.

Reviewers: vitalybuka, morehouse

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D57305

llvm-svn: 352395
2019-01-28 17:51:13 +00:00
Jonathan Metzman 3720e2b39e [fuzzer][afl] Remove AFL_DRIVER_EXTRA_STATS_FILENAME
Summary:
Remove this feature as it is unused, buggy, and not worth correcting
since the forkserver makes it difficult.

Reviewers: morehouse, jfb

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D57308

llvm-svn: 352392
2019-01-28 17:15:49 +00:00
Kostya Serebryany bb01a098b1 [libFuzzer] print uncovered functions when doing -print_coverage=1
llvm-svn: 352263
2019-01-26 01:33:09 +00:00
Jonathan Metzman 62f1871060 [libFuzzer][MSVC] Disable exceptions in MSVC headers
Summary:
Disable exceptions in MSVC headers using -D_HAS_EXCEPTIONS=0
to silence compiler warning instead of using /Ehsc.

Reviewers: rnk, morehouse, metzman

Reviewed By: rnk, morehouse, metzman

Subscribers: rnk, morehouse, mgorny

Differential Revision: https://reviews.llvm.org/D57119

llvm-svn: 352144
2019-01-25 01:10:57 +00:00
Kostya Serebryany 360bf5ff58 [libFuzzer] more agressive value profiling and CMP tracing for switch statements
llvm-svn: 352107
2019-01-24 21:08:54 +00:00
Martin Storsjo b5a5055704 Fix building sanitizers for MinGW
The /EHsc flag is MSVC specific, not generic to the windows target.

llvm-svn: 351927
2019-01-23 07:23:16 +00:00
Jonathan Metzman 3c535a60dd [libFuzzer][MSVC] Enable building libFuzzer with MSVC
Summary:
Enable building libFuzzer with MSVC.

* Don't try to include <endian.h> in FuzzerSHA1.cpp. MSVC
  doesn't have this header, and WINDOWS is always little
  endian (even on ARM)

Subscribers: srhines, mgorny, javed.absar, kristof.beyls

Differential Revision: https://reviews.llvm.org/D56510

llvm-svn: 351855
2019-01-22 18:59:25 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Jonathan Metzman b795c31dff [libFuzzer][MSVC] Make attribute-use compatible with MSVC
Summary:
Replace attributes with macros that use equivalent declspecs
for MSVC.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D56512

llvm-svn: 351456
2019-01-17 16:36:05 +00:00
Jonathan Metzman 9e14cccf6f [libFuzzer] Remove unstable edge handling
Summary:
Remove code for handling unstable edges from libFuzzer since
it has not been found useful.

Differential Revision: https://reviews.llvm.org/D56730

llvm-svn: 351262
2019-01-15 22:12:51 +00:00
Jonathan Metzman a4d569bc19 [libFuzzer][MSVC] Use alternatename for ext functions
Summary:
Use alternatename for external functions only when using
MSVC since Clang doesn't support it and MSVC doesn't support
Clang's method (weak aliases).

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: rnk, thakis, mgorny

Differential Revision: https://reviews.llvm.org/D56514

llvm-svn: 351152
2019-01-15 02:20:53 +00:00
Jonathan Metzman 55ddb2c790 [libfuzzer][MSVC] Make calls to builtin functions work with MSVC
Summary:
Replace calls to builtin functions with macros or functions that call the
Windows-equivalents when targeting windows and call the original
builtin functions everywhere else.
This change makes more parts of libFuzzer buildable with MSVC.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: mgorny, rnk, thakis

Differential Revision: https://reviews.llvm.org/D56439

llvm-svn: 350766
2019-01-09 21:46:09 +00:00
Kostya Serebryany 1879e8d3fc [libFuzzer] make len_control less aggressive
llvm-svn: 349210
2018-12-14 23:21:31 +00:00
Michal Gorny ef4b600301 [test] Add missing cmake include for building libFuzzer alone
Include CompilerRTCompile in fuzzer tests explicitly.  Otherwise, when
building only libFuzzer, CMake fails due to:

CMake Error at cmake/Modules/AddCompilerRT.cmake:395 (sanitizer_test_compile):
  Unknown CMake command "sanitizer_test_compile".
Call Stack (most recent call first):
  lib/fuzzer/tests/CMakeLists.txt:53 (generate_compiler_rt_tests)

Differential Revision: https://reviews.llvm.org/D55378

llvm-svn: 348524
2018-12-06 20:04:08 +00:00
Jonathan Metzman 39b6ba9f33 [fuzzer] Read files as binary
Summary: Read corpus files as binary to avoid automatic conversions

Reviewers: Dor1s, morehouse

Reviewed By: Dor1s, morehouse

Differential Revision: https://reviews.llvm.org/D54180

llvm-svn: 346279
2018-11-06 23:25:25 +00:00
Kamil Rytarowski 2e61186c42 Follow Windows' approach for NetBSD in AlarmCallback()
NetBSD's current behavior needs to ignore check for
InFuzzingThread() in Fuzzer::AlarmCallback().

llvm-svn: 346194
2018-11-06 01:28:01 +00:00
Petr Hosek 466f0f028a [compiler-rt][Fuzzer] Fix the fuzzer test build
This fixes the issue introduced in r345765 which changed the way in
which the embedded libc++ is being built but omitted tests.

Differential Revision: https://reviews.llvm.org/D54058

llvm-svn: 346052
2018-11-02 23:29:13 +00:00
Petr Hosek 44eba12b74 [compiler-rt][Fuzzer] Use the new C++ ABI namespace CMake support
libc++ now supports customizing the ABI namespace directly from the
CMake build so we no longer need to rely on custom CFLAGS.

Differential Revision: https://reviews.llvm.org/D53939

llvm-svn: 345765
2018-10-31 19:15:48 +00:00
Jonathan Metzman 9d0f3206ce [libfuzzer][Windows] Silence linker warning in unittest
Summary:
Silence warning when linking unittest binary by not passing
-lstdc++ to the linker since it is ignored.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: mgorny

Differential Revision: https://reviews.llvm.org/D53225

llvm-svn: 344480
2018-10-14 17:07:40 +00:00
George Karpenkov f28523bb3f [libFuzzer] Generalize the code for getting the previous offset for different architectures
Without this change, tests in coverage.test and dump_coverage.test are
failing on non-x86_64 platforms.
The diff is copied from sanitizer_common library, an alternative would
be to link it together with libFuzzer.

Differential Revision: https://reviews.llvm.org/D53040

llvm-svn: 344104
2018-10-10 00:57:44 +00:00
Petr Hosek a70ece45eb [lib/fuzzer] Fix logging for Fuchsia
This change fixes two aspects of logging on Fuchsia:

1. When logging to stdout, the output file descriptor should not be
closed in ExecuteCommand, as it prevent fdio_spawn_etc from succeeding
in subsequent calls.
2. When logging to a file in anything other than standalone Zircon, the
log file needs to be created in mutable storage. The best indicator
where this is relative to the Fuchsia component will be given by
'-artifact_prefix=...', so save the log file relative to that path.

Patch by: aarongreen
Differential Revision: https://reviews.llvm.org/D52562

llvm-svn: 343607
2018-10-02 17:21:04 +00:00
Jonathan Metzman 0744d3c5a1 [fuzzer] Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
Summary:
Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
to get externally defined functions (eg: LLVMFuzzerInitialize,
LLVMFuzzerCustomMutator, etc) working again.

Also enable tests that depended on these functions (on windows)

Reviewers: rnk, morehouse

Reviewed By: rnk, morehouse

Subscribers: rnk, morehouse, mgorny

Differential Revision: https://reviews.llvm.org/D51700

llvm-svn: 342698
2018-09-20 23:24:48 +00:00
Matt Morehouse 1e1f3c8298 [libFuzzer] Avoid fuzzer symbols being hidden.
Fixes https://bugzilla.mozilla.org/show_bug.cgi?id=1490845#c5.

Patch By: Mike Hommey

llvm-svn: 342423
2018-09-17 23:08:15 +00:00
Martin Storsjo e199a2d434 [libFuzzer] [Windows] Include windows.h and psapi.h with lowercase
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase, and matches how these
headers are included elsewhere in compiler-rt.

Also include these headers with angle brackets, as they are system
headers.

Differential Revision: https://reviews.llvm.org/D51913

llvm-svn: 341983
2018-09-11 19:11:54 +00:00