Commit Graph

600 Commits

Author SHA1 Message Date
Kostya Serebryany 32c5004cf5 [libFuzzer] increase the size of FixedWord from 27 to 64, see PR31950
llvm-svn: 295117
2017-02-14 23:02:37 +00:00
Kostya Serebryany ae579a79c0 Use "%zd" format specifier for printing number of testcases executed.
Summary:
This helps to avoid signed integer overflow after running a fast fuzz target for several hours, e.g.:

<...>
Done -1097903291 runs in 54001 second(s)



Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D29941

llvm-svn: 295112
2017-02-14 22:14:36 +00:00
Marcos Pividori e81f9cc63d [libFuzzer] Use stoull instead of stol to ensure 64 bits.
Differential revision: https://reviews.llvm.org/D29831

llvm-svn: 294769
2017-02-10 18:44:14 +00:00
Eric Fiselier 87c87f4c30 [CMake] Fix pthread handling for out-of-tree builds
LLVM defines `PTHREAD_LIB` which is used by AddLLVM.cmake and various projects
to correctly link the threading library when needed. Unfortunately
`PTHREAD_LIB` is defined by LLVM's `config-ix.cmake` file which isn't installed
and therefore can't be used when configuring out-of-tree builds. This causes
such builds to fail since `pthread` isn't being correctly linked.

This patch attempts to fix that problem by renaming and exporting
`LLVM_PTHREAD_LIB` as part of`LLVMConfig.cmake`. I renamed `PTHREAD_LIB`
because It seemed likely to cause collisions with downstream users of
`LLVMConfig.cmake`.

llvm-svn: 294690
2017-02-10 01:59:20 +00:00
Marcos Pividori a0b23b8e63 [libFuzzer] Export external functions on tests.
We need to export external functions so they are found when calling
GetProcAddress() on Windows. But we can't use `__declspec(dllexport)` because
we want the targets to be completely independent from the fuzz engines and don't
depend on other header files. Also, we don't want to include platform specific
code managed with conditional macros.
So, the solution is to add the exported symbols with linker flags in cmake.

Differential revision: https://reviews.llvm.org/D29752

llvm-svn: 294688
2017-02-10 01:40:28 +00:00
Marcos Pividori 0ae27e80b0 [libFuzzer] Use dynamic loading for External Functions on Windows.
Replace weak aliases with dynamic loading.
Weak aliases were generating some problems when linking for MT on Windows. For
MT, compiler-rt's libraries are statically linked to the main executable the
same than libFuzzer, so if we use weak aliases, we are providing two different
default implementations for the same weak function and the linker fails.

In this diff I re implement ExternalFunctions() using dynamic loading, so it
works in both cases (MD and MT). Also, dynamic loading is simpler, since we are
not defining any auxiliary external function, and we don't need to deal with
weak aliases.
This is equivalent to the implementation using dlsym(RTLD_DEFAULT, FnName) for
Posix.

Differential revision: https://reviews.llvm.org/D29751

llvm-svn: 294687
2017-02-10 01:35:46 +00:00
Marcos Pividori 5a53567620 [libFuzzer] Use long long to ensure 64 bits.
We should always use unsigned long long to ensure 64 bits. On Windows, unsigned
long is 4 bytes. This was the reason why value-profile-cmp4.test was failing on
Windows.

Differential Revision: https://reviews.llvm.org/D29617

llvm-svn: 294390
2017-02-08 00:03:31 +00:00
Marcos Pividori 2a318a18db [libFuzzer] Use custom target instead of list of binaries for tests.
Update cmake to use a custom target TestBinaries instead of a list of targets.
This simplifies cmake, and fix some errors. This way, we don't have to propagate
the values into parents directories. We only need to use add_dependencies.

Differential Revision: https://reviews.llvm.org/D29593

llvm-svn: 294389
2017-02-08 00:03:26 +00:00
Marcos Pividori 360be4d7ad [libFuzzer] Properly use Handle instead of FD on Windows.
For Windows, sanitizers work with Handles, not with posix file descriptors,
because they use the windows-specific API. So we need to convert the fds to
handles before passing them to the sanitizer library.
After this change, close_fd_mask is fixed for Windows (this fix some tests too).

Differential Revision: https://reviews.llvm.org/D29548

llvm-svn: 294388
2017-02-08 00:03:18 +00:00
Marcos Pividori e219d40191 [libFuzzer] Change Uninstrumented test name.
On Windows, executables with the word "uninst" included in their names are
associated with administrator privileges.

Differential Revision: https://reviews.llvm.org/D29549

llvm-svn: 294387
2017-02-08 00:03:13 +00:00
Marcos Pividori fd6578896d [libFuzzer] fuzzer-jobs requires posix.
llvm-svn: 294386
2017-02-08 00:03:07 +00:00
Marcos Pividori c7ac33454f [libFuzzer] Update fuzzer.test to properly set environment variables.
Use env to set environment variables, so it works on Windows and Linux.

Differential Revision: https://reviews.llvm.org/D29537

llvm-svn: 294385
2017-02-08 00:03:03 +00:00
Marcos Pividori 992e6308a7 [libFuzzer] Don't add newline character when using echo in tests.
Add the option "-n", so we don't add a new line character at the end of the file
when using echo. (on Windows this means 2 characters).

Differential Revision: https://reviews.llvm.org/D29536

llvm-svn: 294384
2017-02-08 00:02:59 +00:00
Marcos Pividori abcdcf9f86 [libFuzzer] Disable ulimit test on windows.
Differential Revision: https://reviews.llvm.org/D29535

llvm-svn: 294383
2017-02-08 00:02:54 +00:00
Marcos Pividori 7444aae914 [libFuzzer] Remove ? wildcard on tests.
We can not use the wildcard ? on Windows.

Differential Revision: https://reviews.llvm.org/D29534

llvm-svn: 294382
2017-02-08 00:02:50 +00:00
Marcos Pividori 8fcd6d8394 [libFuzzer] Update fuzzer-segv to set environment variables.
Use env to set environment variables, so it works on Linux and Windows.

Differential Revision: https://reviews.llvm.org/D29533

llvm-svn: 294381
2017-02-08 00:02:45 +00:00
Marcos Pividori 509d4f5aa9 [libFuzzer] Properly configure tests for Windows.
This configuration is necessary, and is included in all tests suites.
We need to execute: `config.test_format = lit.formats.ShTest(False)`
Otherwise, lit will try to use bash, which generates many problems.

Differential Revision: https://reviews.llvm.org/D29529

llvm-svn: 294380
2017-02-08 00:02:41 +00:00
Marcos Pividori 4a65bc35c3 [libFuzzer] Simplify dump_coverage test.
Environment variables are handled differently on Windows. In this case it is not
necessary to use environment variables. So, I simplify the test to work on
Windows.

Differential Revision: https://reviews.llvm.org/D29532

llvm-svn: 294379
2017-02-08 00:02:36 +00:00
Marcos Pividori 638ede81af [libFuzzer] Update Load test to work on 32 bits.
We should ensure the size of the variable `a` is 8 bytes. Otherwise, this
generates a stack buffer overflow inside the memcpy call in 32 bits machines.
(We write more bytes than the size of a, when it is 4 bytes)

Differential Revision: https://reviews.llvm.org/D29602

llvm-svn: 294378
2017-02-08 00:02:32 +00:00
Marcos Pividori 51f907f3a4 [libFuzzer] Update test to consider different exceptions.
Differential Revision: https://reviews.llvm.org/D29531

llvm-svn: 294377
2017-02-08 00:02:25 +00:00
Marcos Pividori c8cee28a5b [libFuzzer] Temporarily fix Shmem for Windows.
In this diff, I add stubs for shared memory on Windows. Now we can compile and
use libFuzzer without support for shared memory.

Differential Revision: https://reviews.llvm.org/D29544

llvm-svn: 294376
2017-02-08 00:02:12 +00:00
Kostya Serebryany 6ac64c3a6d [libFuzzer] replace std::random_shuffle with std::shuffle as std::random_shuffle is being deprecated in C++17. Also simplify fuzzer::Random. NFC
llvm-svn: 294366
2017-02-07 22:37:34 +00:00
Kostya Serebryany c24ec32370 [libFuzzer] make code less clever to avoid fallthrough in switch (and in turn avoid compiler warnings). NFC. Suggested by Christian Holler.
llvm-svn: 294239
2017-02-06 21:21:37 +00:00
Kostya Serebryany 9f8e47b28c [libFuzzer] properly hide the memcmp interceptor from msan
llvm-svn: 294061
2017-02-03 22:51:38 +00:00
Mike Aizatsky 1b65812267 [libfuzzer] chromium-related compilation fixes
Reviewers: kcc

Differential Revision: https://reviews.llvm.org/D29502

llvm-svn: 294035
2017-02-03 20:26:44 +00:00
Kostya Serebryany 68382d0900 [libFuzzer] reorganize the tracing code to make it easier to experiment with inlined coverage instrumentation. NFC
llvm-svn: 293928
2017-02-02 19:56:01 +00:00
Marcos Pividori d64360d935 [libFuzzer] Properly handle exceptions with UnhandledExceptionFilter.
Use SetUnhandledExceptionFilter instead of AddVectoredExceptionHandler.

According to the documentation on Structured Exception Handling, this is the
order for the Exception Dispatching:
+ If the process is being debugged, the system notifies the debugger.
+ The Vectored Exception Handler is called.
+ The system attempts to locate a frame-based exception handler by searching the
 stack frames of the thread in which the exception occurred.
+ If no frame-based handler can be found, the UnhandledExceptionFilter filter is
 called.
+ Default handling based on the exception type.

So, similar to what we do for asan, we should use SetUnhandledExceptionFilter
instead of AddVectoredExceptionHandler, so user's code that is being fuzzed can
execute frame-based exception handlers before we catch them . We want to catch
unhandled exceptions, not all the exceptions.

Differential Revision: https://reviews.llvm.org/D29462

llvm-svn: 293920
2017-02-02 19:07:53 +00:00
Marcos Pividori ba03abebfe [libFuzzer] Disable afl tests on non-posix systems.
AflDriver is not supported on non posix systems.

Differential Revision: https://reviews.llvm.org/D29422

llvm-svn: 293830
2017-02-01 22:40:50 +00:00
Marcos Pividori 36464dd6a5 [libFuzzer] Disable equivalence tests on non posix systems.
We can not run this test until we implement shared memory on Windows.

Differential Revision: https://reviews.llvm.org/D29421

llvm-svn: 293829
2017-02-01 22:40:45 +00:00
Marcos Pividori b056879700 [libFuzzer] Isolate merge tests that require posix.
Differential Revision: https://reviews.llvm.org/D29420

llvm-svn: 293828
2017-02-01 22:40:40 +00:00
Marcos Pividori 9c0244c1eb [libFuzzer] Add features `windows` and `posix` for lit tests.
Add 2 features: posix and windows.
Sometimes we want some specific tests only for posix and we use:
 REQUIRES: posix
Sometimes we want some specific tests only for windows and we use:
 REQUIRES: windows

Differential Revision: https://reviews.llvm.org/D29418

llvm-svn: 293827
2017-02-01 22:40:34 +00:00
Marcos Pividori 477d153045 [libFuzzer] Accept different extensions.
Differential Revision: https://reviews.llvm.org/D29417

llvm-svn: 293826
2017-02-01 22:40:29 +00:00
Marcos Pividori b340471ff5 [libFuzzer] Fix test because cmd prompt does not expand wildcard.
Commands should expand the wildcards on Windows, the cmd prompt doesn't.
Because of that sancov was not finding the needed file.
To deal with this, we use ls and xargs from gnu win utils.

Differential Revision: https://reviews.llvm.org/D29374

llvm-svn: 293825
2017-02-01 22:39:55 +00:00
Marcos Pividori 460886e3cf [libFuzzer] Do not use llvm-objdump for disassembling a DSO.
When disassembling a DSO, for calls to functions from the PLT, llvm-objdump only
prints the offset from the PLT, like: <.plt+0x30>.

While objdump and dumpbin print the function name, like:
<__sanitizer_cov_trace_pc_guard@plt>

When analyzing the coverage in libFuzzer we dissasemble and look for the calls
to __sanitizer_cov_trace_pc_guard.

So, this fails when using llvm-objdump on a DSO.

Differential Revision: https://reviews.llvm.org/D29372

llvm-svn: 293791
2017-02-01 17:59:23 +00:00
Marcos Pividori 7a3a390afb [libFuzzer] Properly check if we can use dumpbin.
The flag "/sumary" is necessary, otherwise it returns a non-zero value.

Differential Revision: https://reviews.llvm.org/D29371

llvm-svn: 293790
2017-02-01 17:59:19 +00:00
Kostya Serebryany 5c76e3d034 [libFuzzer] increase the default size for shmem
llvm-svn: 293722
2017-02-01 00:07:47 +00:00
Marcos Pividori d2406ea900 [libFuzzer] Implement TmpDir() for Windows.
Differential Revision: https://reviews.llvm.org/D28977

llvm-svn: 293516
2017-01-30 18:14:53 +00:00
Kostya Serebryany ac2a633467 [libfuzzer] include errno.h. On Ubuntu 14.04 we got away w/o it, but other systems seem to require it
llvm-svn: 293389
2017-01-28 18:56:05 +00:00
Kostya Serebryany 6d58dbb62f [libFuzzer] make shmem more robust in the presence of signals
llvm-svn: 293339
2017-01-27 22:41:30 +00:00
Kostya Serebryany 70182deaae [libFuzzer] simplify the value profiling callback further: don't use (idx MOD prime) on the hot path where it is useless anyway
llvm-svn: 293239
2017-01-27 00:39:12 +00:00
Kostya Serebryany 8e9ac42742 [libFuzzer] make sure (again) that __builtin_popcountl is compiled into popcnt
llvm-svn: 293237
2017-01-27 00:20:55 +00:00
Kostya Serebryany 7f058972ee [libFuzzer] simplify the value profile code and disable asan/msan on it
llvm-svn: 293236
2017-01-27 00:09:59 +00:00
Kostya Serebryany 419634bdb8 [libFuzzer] remove a bit of stale code
llvm-svn: 293129
2017-01-26 01:45:54 +00:00
Kostya Serebryany 7856fb36b0 [libFuzzer] further simplify __sanitizer_cov_trace_pc_guard
llvm-svn: 293128
2017-01-26 01:34:58 +00:00
Kostya Serebryany d0ecb4c69e [libFuzzer] simplify the code for __sanitizer_cov_trace_pc_guard and make sure it is not asan/msan-instrumented
llvm-svn: 293125
2017-01-26 01:04:54 +00:00
Kostya Serebryany 7c021afef2 [libFuzzer] don't call GetPreviousInstructionPc on the hot path -- only when dumping the PCs
llvm-svn: 293117
2017-01-26 00:22:08 +00:00
Kostya Serebryany b7a0086db1 [libFuzzer] mutate empty input using the regular mutators (instead of a custom dummy one). This way when we mutate an empty input there is a chance we will get a dictionary word
llvm-svn: 292843
2017-01-23 22:52:13 +00:00
Kostya Serebryany 6bdd8fc5b6 [libFuzzer] make sure we use the feedback from std::string operator ==
llvm-svn: 292835
2017-01-23 22:11:04 +00:00
Kostya Serebryany 3230b24ea1 [libFuzzer] deflake a test
llvm-svn: 292813
2017-01-23 18:44:40 +00:00
Marcos Pividori 0f84c7aef6 [libFuzzer] Add missing dependency for tests.
Dependency on TestBinaries was erroneously removed on r292735.

llvm-svn: 292765
2017-01-22 21:55:46 +00:00