Summary:
This helps to avoid signed integer overflow after running a fast fuzz target for several hours, e.g.:
<...>
Done -1097903291 runs in 54001 second(s)
Reviewers: kcc
Reviewed By: kcc
Differential Revision: https://reviews.llvm.org/D29941
llvm-svn: 295112
LLVM defines `PTHREAD_LIB` which is used by AddLLVM.cmake and various projects
to correctly link the threading library when needed. Unfortunately
`PTHREAD_LIB` is defined by LLVM's `config-ix.cmake` file which isn't installed
and therefore can't be used when configuring out-of-tree builds. This causes
such builds to fail since `pthread` isn't being correctly linked.
This patch attempts to fix that problem by renaming and exporting
`LLVM_PTHREAD_LIB` as part of`LLVMConfig.cmake`. I renamed `PTHREAD_LIB`
because It seemed likely to cause collisions with downstream users of
`LLVMConfig.cmake`.
llvm-svn: 294690
We need to export external functions so they are found when calling
GetProcAddress() on Windows. But we can't use `__declspec(dllexport)` because
we want the targets to be completely independent from the fuzz engines and don't
depend on other header files. Also, we don't want to include platform specific
code managed with conditional macros.
So, the solution is to add the exported symbols with linker flags in cmake.
Differential revision: https://reviews.llvm.org/D29752
llvm-svn: 294688
Replace weak aliases with dynamic loading.
Weak aliases were generating some problems when linking for MT on Windows. For
MT, compiler-rt's libraries are statically linked to the main executable the
same than libFuzzer, so if we use weak aliases, we are providing two different
default implementations for the same weak function and the linker fails.
In this diff I re implement ExternalFunctions() using dynamic loading, so it
works in both cases (MD and MT). Also, dynamic loading is simpler, since we are
not defining any auxiliary external function, and we don't need to deal with
weak aliases.
This is equivalent to the implementation using dlsym(RTLD_DEFAULT, FnName) for
Posix.
Differential revision: https://reviews.llvm.org/D29751
llvm-svn: 294687
We should always use unsigned long long to ensure 64 bits. On Windows, unsigned
long is 4 bytes. This was the reason why value-profile-cmp4.test was failing on
Windows.
Differential Revision: https://reviews.llvm.org/D29617
llvm-svn: 294390
Update cmake to use a custom target TestBinaries instead of a list of targets.
This simplifies cmake, and fix some errors. This way, we don't have to propagate
the values into parents directories. We only need to use add_dependencies.
Differential Revision: https://reviews.llvm.org/D29593
llvm-svn: 294389
For Windows, sanitizers work with Handles, not with posix file descriptors,
because they use the windows-specific API. So we need to convert the fds to
handles before passing them to the sanitizer library.
After this change, close_fd_mask is fixed for Windows (this fix some tests too).
Differential Revision: https://reviews.llvm.org/D29548
llvm-svn: 294388
On Windows, executables with the word "uninst" included in their names are
associated with administrator privileges.
Differential Revision: https://reviews.llvm.org/D29549
llvm-svn: 294387
Add the option "-n", so we don't add a new line character at the end of the file
when using echo. (on Windows this means 2 characters).
Differential Revision: https://reviews.llvm.org/D29536
llvm-svn: 294384
This configuration is necessary, and is included in all tests suites.
We need to execute: `config.test_format = lit.formats.ShTest(False)`
Otherwise, lit will try to use bash, which generates many problems.
Differential Revision: https://reviews.llvm.org/D29529
llvm-svn: 294380
Environment variables are handled differently on Windows. In this case it is not
necessary to use environment variables. So, I simplify the test to work on
Windows.
Differential Revision: https://reviews.llvm.org/D29532
llvm-svn: 294379
We should ensure the size of the variable `a` is 8 bytes. Otherwise, this
generates a stack buffer overflow inside the memcpy call in 32 bits machines.
(We write more bytes than the size of a, when it is 4 bytes)
Differential Revision: https://reviews.llvm.org/D29602
llvm-svn: 294378
In this diff, I add stubs for shared memory on Windows. Now we can compile and
use libFuzzer without support for shared memory.
Differential Revision: https://reviews.llvm.org/D29544
llvm-svn: 294376
Use SetUnhandledExceptionFilter instead of AddVectoredExceptionHandler.
According to the documentation on Structured Exception Handling, this is the
order for the Exception Dispatching:
+ If the process is being debugged, the system notifies the debugger.
+ The Vectored Exception Handler is called.
+ The system attempts to locate a frame-based exception handler by searching the
stack frames of the thread in which the exception occurred.
+ If no frame-based handler can be found, the UnhandledExceptionFilter filter is
called.
+ Default handling based on the exception type.
So, similar to what we do for asan, we should use SetUnhandledExceptionFilter
instead of AddVectoredExceptionHandler, so user's code that is being fuzzed can
execute frame-based exception handlers before we catch them . We want to catch
unhandled exceptions, not all the exceptions.
Differential Revision: https://reviews.llvm.org/D29462
llvm-svn: 293920
Add 2 features: posix and windows.
Sometimes we want some specific tests only for posix and we use:
REQUIRES: posix
Sometimes we want some specific tests only for windows and we use:
REQUIRES: windows
Differential Revision: https://reviews.llvm.org/D29418
llvm-svn: 293827
Commands should expand the wildcards on Windows, the cmd prompt doesn't.
Because of that sancov was not finding the needed file.
To deal with this, we use ls and xargs from gnu win utils.
Differential Revision: https://reviews.llvm.org/D29374
llvm-svn: 293825
When disassembling a DSO, for calls to functions from the PLT, llvm-objdump only
prints the offset from the PLT, like: <.plt+0x30>.
While objdump and dumpbin print the function name, like:
<__sanitizer_cov_trace_pc_guard@plt>
When analyzing the coverage in libFuzzer we dissasemble and look for the calls
to __sanitizer_cov_trace_pc_guard.
So, this fails when using llvm-objdump on a DSO.
Differential Revision: https://reviews.llvm.org/D29372
llvm-svn: 293791
We need to set BINARY_DIR to: ${CMAKE_BINARY_DIR}/lib/Fuzzer/test , so the dll
is placed in the same directory than the test LLVMFuzzer-DSOTest, and is found
when executing that test.
As we are using CMAKE_CXX_CREATE_SHARED_LIBRARY to link the dll, we can't modify
the output directory for the import library. It will be created in the same
directory than the dll (in BINARY_DIR), no matter which value we set to
LIBRARY_DIR. So, if we set LIBRARY_DIR to a different directory than BINARY_DIR,
when linking LLVMFuzzer-DSOTest, cmake will look for the import library
LLVMFuzzer-DSO1.lib in LIBRARY_DIR, and won't find it, since it was created in
BINARY_DIR. So, for Windows, we need that LIBRARY_DIR and BINARY_DIR are the
same directory.
Differential Revision: https://reviews.llvm.org/D27870
llvm-svn: 292748
Don't check for InFuzzingThread() on Windows, since the AlarmHandler() is
always executed by a different thread from a thread pool.
If we don't add these changes, the alarm handler will never execute.
Note that we decided to ignore possible problem in the synchronization.
Differential Revision: https://reviews.llvm.org/D28723
llvm-svn: 292746
I add 2 changes to make the tests work on 32 bits and on 64 bits.
I change the size allocated to 0x20000000 and add the flag: -rss_limit_mb=300.
Otherwise the output for 32 bits and 64 bits is different.
For 64 bits the value 0xff000000 doesn't exceed kMaxAllowedMallocSize.
For 32 bits, kMaxAllowedMallocSize is set to 0xc0000000, so the call to
Allocate() will fail earlier printing "WARNING: AddressSanitizer failed to
allocate ..." , and wont't call malloc hooks.
So, we need to consider a size smaller than 2GB (so malloc doesn't fail on
32bits) and greater that the value provided by -rss_limit_mb.
Because of that I use: 0x20000000.
Differential Revision: https://reviews.llvm.org/D28706
llvm-svn: 292744
Fix libFuzzer when setting -close_fd_mask to a non-zero value.
In previous implementation, libFuzzer closes the file descriptors for
stdout/stderr. This has some disavantages:
For `fuzzer-fdmask.test`, we write directly to stdout and stderr using the
file streams stdout and stderr, after the file descriptors are closed, which is
undefined behavior. In Windows, in particular, this was making the test fail.
Also, if we close stdout and we open a new file in libFuzzer, we get the file
descriptor 1, which could generate problem if some code assumes file descriptors
refers to stdout and works directly writing to the file descriptor 1, but it
will be writing to the opened file (for example using std::cout).
Instead of closing the file descriptors, I redirect the output to /dev/null on
linux and nul on Windows.
Differential Revision: https://reviews.llvm.org/D28718
llvm-svn: 292743
This changes is necessary on Windows, where libraries doesn't include the prefix
"lib".
Differential Revision: https://reviews.llvm.org/D28710
llvm-svn: 292742
Update `ListFilesInDirRecursive` implementation on Windows to have the same
behavior than for Posix, when the directory doesn't exists and when it is empty.
Differential Revision: https://reviews.llvm.org/D28711
llvm-svn: 292741
Instead of directly using objdump, which is not present on Windows, we consider
different tools depending on the platform.
For Windows, we consider dumpbin and llvm-objdump.
Differential Revision: https://reviews.llvm.org/D28635
llvm-svn: 292739
We need to build all the tests with -O0, otherwise optimizations may merge some
basic blocks and the tests will fail.
In this diff, I simplify the cmake implementation and I remove the flags for
Windows too (/O[123s]).
Differential Revision: https://reviews.llvm.org/D28632
llvm-svn: 292737
We need to expose Sanitizer Coverage's functions that are rewritten with a
different implementation, so compiler-rt's libraries have access to it.
Differential Revision: https://reviews.llvm.org/D28618
llvm-svn: 292736
Remove dependency on FileCheck, sancov and not for tests on Windows.
If LLVM_USE_SANITIZER=Address and LLVM_USE_SANITIZE_COVERAGE=YES, this will
trigger the building of dependencies with sanitizer instrumentation.
This will fail in Windows, since cmake will use link.exe for linking and won't
include compiler-rt libraries.
Differential Revision: https://reviews.llvm.org/D27993
llvm-svn: 292735
In order to use sanitizers on Windows, we need to link against many runtime
libraries which will depend on the target being created (executable or dll) and
the c runtime library used (MT/MD).
By default, cmake uses link.exe for linking, which fails because we don't
specify the appropiate dependencies. As we don't want to consider all of that
possible situations which depends on the implementation of the compiler-rt, the
simplest option is to change the rules for linking executables and shared
libraries, using the compiler instead of link.exe.
Clang driver will consider the sanitizer flags, and automatically provide the
required libraries to the linker.
Differential Revision: https://reviews.llvm.org/D27869
llvm-svn: 292669
Replace sleep() posix function by a more portable sleep_for() function
from std. Also, ignore memmem() and strcasestr() on Windows.
Differential Revision: https://reviews.llvm.org/D27729
llvm-svn: 289964
Windows uses some macros to replace DeleteFile() by DeleteFileA() or
DeleteFileW(). This was causing an error at link time.
DeleteFile was renamed to RemoveFile().
Differential Revision: https://reviews.llvm.org/D27577
llvm-svn: 289563
Implement DirName from scratch to avoid dependencies on external libraries.
It's based on MSDN documentation for Naming Files, Paths, and Namespaces.
The algorithm can't simply start from the end and look backwards for the
first separator, because we need to preserve the prefix that represent
the root location. We shouldn't remove anything there. In Windows we
have many different options, like:
\\Server\Share\ , \ , C: , C:\ , \\?\C:\ , \\?\UNC\Server\Share\
We remove the last separator in the rest of the path, if it exists.
It was implemented to have a similar behaviour to dirname() in linux,
removing trailing separators, returning "." when the path doesn't
contain separators, etc.
Differential Revision: https://reviews.llvm.org/D27579
llvm-svn: 289562
I added a new flag RunningCB to know if the Fuzzer's main thread is
running the CB function, instead of using (!CurrentUnitSize).
(!CurrentUnitSize) doesn't work properly. For example, in FuzzerLoop.cpp,
inside ShuffleAndMinimize() function, we execute the callback with an
empty string (size=0). Previous implementation failed to detect timeouts
in that execution.
Also, I add a regression test for that case.
Differential Revision: https://reviews.llvm.org/D27433
llvm-svn: 289561
Reorganize #includes to follow LLVM Coding Standards.
Include some missing headers. Required to use `Printf()`.
Aside from that, this patch contains no functional change.
It is purely a re-organization.
Differential Revision: https://reviews.llvm.org/D27363
llvm-svn: 289560
std:🧵:hardware_concurrency() returns an unsigned, so I modify
NumberOfCpuCores() to return unsigned too.
The number of cpus is used to define the number of workers, so I decided
to update the worker and jobs flags to be declared as unsigned too.
Differential Revision: https://reviews.llvm.org/D27685
llvm-svn: 289559
Use unsigned for PID instead of signed int. GetCurrentProcessId() returns
an unsigned (DWORD) so we must be sure we can deal with all possible values.
I use a long unsigned to be sure it can hold a 32 bit unsigned (DWORD).
Differential Revision: https://reviews.llvm.org/D27281
llvm-svn: 289558
Add new flags to FuzzingOptions to represent the different conditions
on the signal handling. These options are passed when calling
SetSignalHandler().
This changes simplify the implementation of Windows's exception
handling. Now we can define a unique handler for all the exceptions.
Differential Revision: https://reviews.llvm.org/D27238
llvm-svn: 289557
Implemented timeouts for Windows using TimerQueueTimers.
Timers are used to supervise the time of execution of the
callback function that is being fuzzed.
Differential Revision: https://reviews.llvm.org/D27237
llvm-svn: 289495
This resubmits r288529, which was resubmitted because it broke a
fuzzer bot. According to kcc@ the test that broke was flakey
and it is unlikely to be a result of this patch.
llvm-svn: 288549
Windows doesn't really support weak aliases, but with some
linker magic we can get something that's pretty close on
Windows. This introduces an interface to accessing weakly
aliased symbols that will work on any platform. Linker
magic changes to come in a separate patch.
Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27235
llvm-svn: 288530
Pave the way for separating out platform specific
utility functions into separate files.
Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27234
llvm-svn: 288529
This patch moves some posix specific file i/o code into a new
file, FuzzerIOPosix.cpp, and provides implementations for these
functions on Windows in FuzzerIOWindows.cpp. This is another
incremental step towards getting libfuzzer working on Windows,
although it still should not be expected to be fully working.
Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27233
llvm-svn: 288275
In an effort to get libfuzzer working on Windows, we need to make
a distinction between what functions require platform specific
code (e.g. different code on Windows vs Linux) and what code
doesn't. IO functions, for example, tend to be platform
specific.
This patch separates out some of the functions which will need
to have platform specific implementations into different headers,
so that we can then provide different implementations for each
platform.
Aside from that, this patch contains no functional change. It
is purely a re-organization.
Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27230
llvm-svn: 288264
This is the beginning of an effort to get libfuzzer working on
Windows. This is a NFC to just add some macros for platform
detection on Windows.
Patch by Marcos Pividori
Differential Revision: https://reviews.llvm.org/D27229
llvm-svn: 288249
Example of output:
COVERAGE:
COVERED: in DSO2(int) /pathto/DSO2.cpp:6
COVERED: in DSO2(int) /pathto/DSO2.cpp:8
COVERED: in DSO1(int) /pathto/DSO1.cpp:6
COVERED: in DSO1(int) /pathto/DSO1.cpp:8
COVERED: in LLVMFuzzerTestOneInput /pathto/DSOTestMain.cpp:16
COVERED: in LLVMFuzzerTestOneInput /pathto/DSOTestMain.cpp:19
COVERED: in LLVMFuzzerTestOneInput /pathto/DSOTestMain.cpp:25
COVERED: in LLVMFuzzerTestOneInput /pathto/DSOTestMain.cpp:26
MODULE_WITH_COVERAGE: /pathto/libLLVMFuzzer-DSO1.so
UNCOVERED_LINE: in DSO1(int) /pathto/DSO1.cpp:9
UNCOVERED_FUNC: in Uncovered1()
MODULE_WITH_COVERAGE: /pathto/libLLVMFuzzer-DSO2.so
UNCOVERED_LINE: in DSO2(int) /pathto/DSO2.cpp:9
UNCOVERED_FUNC: in Uncovered2()
MODULE_WITH_COVERAGE: /pathto/LLVMFuzzer-DSOTest
UNCOVERED_LINE: in LLVMFuzzerTestOneInput /pathto/DSOTestMain.cpp:21
UNCOVERED_LINE: in LLVMFuzzerTestOneInput /pathto/DSOTestMain.cpp:27
UNCOVERED_FILE: /pathto/DSOTestExtra.cpp
Several things are not perfect here:
* we are using objdump+awk instead of sancov because sancov does not support DSOs yet.
* this breaks in the presence of ASAN_OPTIONS=strip_path_prefix=...
(need to implement another API to get the module name by PC)
llvm-svn: 284554
This reverts commit r283946.
This breaks when build with GCC:
lib/Fuzzer/FuzzerTracePC.cpp:169:6: error: always_inline function might not be inlinable [-Werror=attributes]
lib/Fuzzer/FuzzerTracePC.cpp:169:6: error: inlining failed in call to always_inline 'void fuzzer::TracePC::HandleCmp(void*, T, T) [with T = long unsigned int]': target specific option mismatch
lib/Fuzzer/FuzzerTracePC.cpp:198:65: error: called from here
llvm-svn: 283979
Kostya Serebryany