Commit Graph

11583 Commits

Author SHA1 Message Date
Dmitry Mikulin 312b5f86b7 The error message for mismatched value sites is very cryptic.
Make it more readable for an average user.

Differential Revision: https://reviews.llvm.org/D60896

llvm-svn: 359043
2019-04-23 22:26:55 +00:00
Julian Lettner 4d2b9426b9 [TSan] Support fiber API on macOS
Committing on behalf of Yuri Per (yuri).

Reviewers: dvyukov, kubamracek, yln

Reviewed By: kubamracek

Authored By: yuri

Differential Revision: https://reviews.llvm.org/D58110

llvm-svn: 358802
2019-04-20 00:18:44 +00:00
Chris Bieneman 2436237895 [CMake] Add fuzzer as a component for runtime builds
Calling `add_compiler_rt_component` sets up the component connection between runtime builds and the parent CMake configuration. Adding this call allows specifying `fuzzer` as a `LLVM_RUNTIME_DISTRIBUTION_COMPONENT`.

llvm-svn: 358780
2019-04-19 20:13:25 +00:00
Petr Hosek 45fc90326a [gn] Support dots in CMake paths in the sync script
Some file paths use dots to pick up sources from parent directories.

Differential Revision: https://reviews.llvm.org/D60734

llvm-svn: 358774
2019-04-19 18:29:17 +00:00
Mitch Phillips f4878d8905 Removed regiser dump checks from FP unit test.
llvm-svn: 358770
2019-04-19 17:36:58 +00:00
Mitch Phillips aa68f946dc [HWASan] Added no-FP unit test for register dump.
Summary: Unit test for D60798.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60806

llvm-svn: 358769
2019-04-19 17:36:56 +00:00
Mitch Phillips d1a18469a8 Updated test case.
llvm-svn: 358768
2019-04-19 17:36:55 +00:00
Mitch Phillips 2c3cecef3d Added no-FP unit test for register dump.
llvm-svn: 358767
2019-04-19 17:36:53 +00:00
Kostya Serebryany 14ada6d1ad [libFuzzer] revert part of r358726 committed by mistake (oops)
llvm-svn: 358727
2019-04-19 01:40:16 +00:00
Kostya Serebryany bbb8522bde [libFuzzer] in fork mode, make sure the child stats are always collected. This should fix the flakiness in fork-ubsan.test
llvm-svn: 358726
2019-04-19 01:39:14 +00:00
Yi Kong 815a4c902d [builtins] Build x86_64 with GENERIC_TF_SOURCES
llvm-svn: 358706
2019-04-18 19:29:03 +00:00
Jonathan Metzman 139e216e66 Summary:
Add close_fd_mask functionality to AFL driver.

Summary:
Add support for env var AFL_DRIVER_CLOSE_FD_MASK which behaves
the same as libFuzzer's -close_fd_mask=1.

Also add tests.

Reviewers: kcc, vitalybuka, morehouse

Reviewed By: morehouse

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60334

llvm-svn: 358703
2019-04-18 18:49:11 +00:00
Dan Liew 16c3883b98 [asan_symbolize] Workaround bug in old Python 2 versions.
The change landed in r358657 broke some of the buildbots because they
use an older version of Python 2 that raises this error.

```
File "/Volumes/data/dev/llvm/upstream/master/src/projects/compiler-rt/lib/asan/scripts/asan_symbolize.py", line 509
  exec(f.read(), globals_space, None)
SyntaxError: unqualified exec is not allowed in function 'load_plugin_from_file' it contains a nested function with free variables
```

I can reproduce this problem when using Python 2.7.6.

To workaround this some indirection has been added to prevent the broken
(the line at fault would never be executed) SyntaxError error in old
Python versions from being raised.

rdar://problem/49476995

llvm-svn: 358682
2019-04-18 16:22:55 +00:00
Dan Liew 403d3187a7 [asan_symbolize] Add a simple plugin architecture
Summary:
This change adds a simple plugin architecture to `asan_symbolize.py`.
The motivation here is that sometimes it's necessary to perform extra
work to figure out where binaries with debug symbols can actually be
found. For example it might be the case that a remote service needs
to be queried for binaries and then copied to the local system.

This "extra work" can be extremely site-specific such that adding the
code directly into the `asan_symbolize.py` would just clutter the code
for a very niche use case. To avoid this, the `asan_symbolize.py` can
now load external code via a new `--plugins` command line option.

These plugins are loaded before main command line argument parsing so
that they can add their own command line options.

Right now the only hook into the behaviour of symbolization is the
`filter_binary_path()` function which assumes a very similar role
to the `binary_name_filter` function that was previously in the code.
We can add more hooks as necessary.

Code in the `asan_symbolize.py` script does not call plugin code
directly. Instead it uses a `AsanSymbolizerPlugInProxy` object.
This object

* Loads plugins from files.
* Manages the lifetime of the plugins.
* Provides an interface for calling into plugin functions and handles
  calling into multiple plugins.

To unify the way binary paths are filtered the old `sysroot_path_filter`
function (and associated code) has been turned into a simple plugin
(`SysRootFilterPlugIn`) that is always loaded. The plugin unloads
itself if the `-s` option is not present on the command line. Users
should not see any functional change relating to this command line
option.

Some simple tests are provided to illustrate what plugin code looks
like and also to check the functionality continues to work.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: srhines, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60529

llvm-svn: 358657
2019-04-18 11:34:31 +00:00
Yi Kong 64c32362f0 [builtins] Add __cmpsf2 for ARM version of comparesf2
The generic version of comparesf2 defines __cmpsf2 alias for libgcc
compatibility, but the ARM overlay is missing the alias.

Differential Revision: https://reviews.llvm.org/D60805

llvm-svn: 358542
2019-04-17 01:30:33 +00:00
Mitch Phillips 8f9d95d9d7 [HWASan] Fixed slow DWARF unwinding.
Summary: CFA was setup incorrectly, as there is an 8-byte gap at the top of the stack for SP 16-byte alignment purposes.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, javed.absar, #sanitizers, llvm-commits, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60798

llvm-svn: 358535
2019-04-16 22:16:01 +00:00
Matt Morehouse 57213d839a [libFuzzer] Remove tautological compare.
llvm-svn: 358510
2019-04-16 17:38:19 +00:00
Julian Lettner 2632643454 [TSan][libdispatch] Port gcd-sync-block-copy.mm to C++
Summary:
Apparently, it makes a difference on where a block lives depending on if
it's passed "inline" versus assigned and then passed via a variable.
Both tests in this commit now give a signal, if `Block_copy` is used in
`dispatch_sync`.

Since these tests use different mechanisms (Objective-C retain versus
C++ copy constructor) as proxies to observe if the block was copied, we
should keep both of them.

Commit, that first avoided the unnecessary copy:
faef7d034a

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60639

llvm-svn: 358469
2019-04-16 01:34:38 +00:00
Kostya Serebryany 5e67abd91f [libFuzzer] speedup the merge step in the fork mode by merging only the files that have unique features.
llvm-svn: 358320
2019-04-13 01:57:33 +00:00
Kostya Serebryany 4614cc3dfd [libFuzzer] add -features_dir= flag to dump unique input features on disk
llvm-svn: 358317
2019-04-13 00:20:31 +00:00
Julian Lettner c77bf89dcc [TSan][libdispatch] Move libdispatch tests out of Darwin folder
The libdispatch tests are now not tied to Darwin anymore. Move them to
their own platform-independent folder.

llvm-svn: 358316
2019-04-12 23:07:00 +00:00
Max Moroz b6e6d3c740 [libFuzzer] Fix DataFlow.cpp logic when tracing long inputs.
Summary:
1. Do not create DFSan labels for the bytes which we do not trace. This is where we run out of labels at the first place.
2. When dumping the traces on the disk, make sure to offset the label identifiers by the number of the first byte in the trace range.
3. For the last label, make sure to write it at the last position of the trace bit string, as that label represents the input size, not any particular byte.

Also fixed the bug with division in python which I've introduced when migrated the scripts to Python3 (`//` is required for integral division).

Otherwise, the scripts are wasting too much time unsuccessfully trying to
collect and process traces from the long inputs. For more context, see
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481761789

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60538

llvm-svn: 358311
2019-04-12 21:00:12 +00:00
Julian Lettner 6636978243 [TSan][libdispatch] Don't link against Foundation
Now that our tests don't depend on Foundation anymore,
don't link it in.

llvm-svn: 358309
2019-04-12 20:27:05 +00:00
Julian Lettner bfcbd677c8 [TSan][libdispatch] Replace NSTemporaryDirectory in tests
After this change, most tests don't have a dependency on Foundation.

Note: To hold the file name `tempnam` allocates a new buffer. We leak
      this buffer (omit the free), but I don't think we need to care.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60591

llvm-svn: 358308
2019-04-12 20:27:02 +00:00
Kostya Serebryany 65132e2118 [libFuzzer] support -runs=N in the fork mode. Make sure we see one-line reports from ubsan in the fork mode. Test both
llvm-svn: 358306
2019-04-12 20:20:57 +00:00
Julian Lettner 43e27eb87f [TSan][libdispatch] Fix failing test
The ThreadSanitizer-x86_64-iossim configuration (different SDK) seems to
require an additional #include.

rdar://49856637

llvm-svn: 358300
2019-04-12 18:31:09 +00:00
Julian Lettner 2edfcf9065 [TSan][libdispatch] Delete old tests
In a previous commit, I re-enabled the ported variants of these 2 tests:
tsan/Darwin/gcd-data.mm -> tsan/libdispatch/data.c
tsan/Darwin/gcd-source-serial.mm -> tsan/libdispatch/source-serial.c

So now we can delete the Darwin-only version.

llvm-svn: 358235
2019-04-11 22:27:57 +00:00
Julian Lettner 1740c6e0da [TSan][libdispatch] Re-enable disabled tests
I re-activated "broad strokes suppressions"
(ignore_noninstrumented_modules=1) in my last commit. Re-enable tests
that only fail on our bots to check if they work now.

llvm-svn: 358209
2019-04-11 18:34:47 +00:00
Julian Lettner 5b981638fb [TSan][libdispatch] Turn ignore_noninstrumented_modules=1 back on for Darwin
llvm-svn: 358208
2019-04-11 18:34:44 +00:00
Max Moroz bcb93a35c0 [libFuzzer] Make MutateWithMask work when the Mask is shorter than the input.
Summary:
Before this change, MutateWithMask used to assert that Mask should be
of sufficient length (>= Size of the input). However, in real cases we may have
inputs that are longer than the Mask they have inherited from the based inputs.

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60571

llvm-svn: 358207
2019-04-11 18:21:53 +00:00
Julian Lettner 3e58f94ced [TSan][libdispatch] Replace usage of NSMutableData with stack array
Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60477

llvm-svn: 358205
2019-04-11 18:14:19 +00:00
Julian Lettner edea731af2 [TSan][libdispatch] Change test to have two simultaneous timers
Change test semantics by waiting for both timer callbacks at the end
instead of serializing operations: start/wait timer 1 then 2.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60476

llvm-svn: 358204
2019-04-11 18:14:16 +00:00
Julian Lettner 0f7a0c71c6 [TSan][libdispatch] Replace CFRunLoop with dispatch_semaphore, pt. 2
Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60475

llvm-svn: 358203
2019-04-11 18:14:13 +00:00
Matt Morehouse abed4acc2c Revert "tsan: Update test expectations."
This reverts r357831 to go with r358113.

llvm-svn: 358202
2019-04-11 18:05:37 +00:00
Max Moroz 9d5e7ee296 [libFuzzer] Fallback to default Mutate when MutateWithMask fails.
Summary:
In case the current corpus input doesn't have bytes going into the
focus function, MutateWithMask is useless and may fail gently, allowing the
default mutation routine happen, rather than crashing on an assertion.

For more context and the initial fix suggestion, see:
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481862879

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60567

llvm-svn: 358190
2019-04-11 16:24:53 +00:00
John McCall 98da442b6d Prospective test fix in response to r358104.
llvm-svn: 358149
2019-04-11 03:04:04 +00:00
JF Bastien 2f46de8c0b [NFC] Use clearer naming for local variables
llvm-svn: 358145
2019-04-10 23:23:17 +00:00
Julian Lettner 19f52bde59 [TSan][libdispatch] Replace CFRunLoop with dispatch_semaphore, pt. 1
Remove the dependency on Foundation so we can start running those tests
on other platforms. Rename/move of tests will be done in a separate
commit.

Reviewed By: kubamracek, dvyukov

Differential Revision: https://reviews.llvm.org/D60347

llvm-svn: 358023
2019-04-09 17:51:55 +00:00
Kostya Kortchinsky 7e2b15382c [scudo][standalone] Add flags & related parsers
Summary:
As with other Sanitizers, and the current version of Scudo, we can
provide flags in differents way: at compile time, through a weak
function, through an environment variable.

This change adds support for the configuration flags, and the string
parsers. Those are fairly similar to the sanitizer_common way of doing
things.

Reviewers: morehouse, hctim, vitalybuka

Reviewed By: morehouse, vitalybuka

Subscribers: mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D59597

llvm-svn: 358011
2019-04-09 14:57:25 +00:00
Dan Liew 2316429d04 [asan_symbolize] Move argument parser epilog text into script doc comment.
Summary:
This will make it easier to expand on the documentation in the future
that avoids cluttering the code.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60398

llvm-svn: 357978
2019-04-09 08:30:43 +00:00
Peter Collingbourne df57979ba7 hwasan: Enable -hwasan-allow-ifunc by default.
It's been on in Android for a while without causing problems, so it's time
to make it the default and remove the flag.

Differential Revision: https://reviews.llvm.org/D60355

llvm-svn: 357960
2019-04-09 00:25:59 +00:00
Dan Liew 836da99af9 [asan_symbolize] Use proper logging infrastructure.
Summary:
The previous logging infrastructure had several problems:

* Debugging output was emitted to standard output which is also where
  the symbolized output would go. Interleaving these two separate
  bits of information makes inspecting the output difficult and could
  potentially break tests.

* Enabling debugging output requires modifying the script which is
  not very conveninent.

* When debugging it isn't immediately obvious where the output is
  coming from.

This patch uses the Python standard library logging infrastructure
which fixes all of the above problems. Logging is controlled using
two new options.

* `--log-level` - Sets the logging level, default is
`info`.
* `--log-dest` - Set the logging destination, default
is standard error.

Some simple test cases for the feature are included.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60343

llvm-svn: 357951
2019-04-08 22:28:27 +00:00
Pavel Labath 19cad6f089 [Sanitizer] Make wcrtomb test posix-only
wcrtomb is not intercepted on windows, so this test fails there. It's
not clear to me why we do not intercept this function there (I'll look
into that separately), but for now this should at least make the windows
sanitizer bot green again (broken by r357889, when I added this test).

I also add "UNSUPPORTED: android" as this function is also not
intercepted there.

llvm-svn: 357892
2019-04-08 09:13:13 +00:00
Pavel Labath 618dcfcaa3 [Sanitizer] Fix a possible write to freed memory in the wcrtomb interceptor
Summary:
r357240 added an interceptor for wctomb, which uses a temporary local
buffer to make sure we don't write to unallocated memory. This patch
applies the same technique to wcrtomb, and adds some additional tests
for this function.

Reviewers: vitalybuka, eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D59984

llvm-svn: 357889
2019-04-08 08:39:50 +00:00
Julian Lettner 18bc982c06 [TSan][libdispatch] Remove Darwin-only version of fully-ported tests
Remove 10 tests that already have a copy in tsan/libdispatch, without
dependencies on Darwin-specifis.

llvm-svn: 357832
2019-04-06 01:41:40 +00:00
Peter Collingbourne c0da957bb0 tsan: Update test expectations.
llvm-svn: 357831
2019-04-06 00:43:17 +00:00
Petr Hosek 40442658db [gn] Support for building compiler-rt builtins
This is support for building compiler-rt builtins, The library build
should be complete for a subset of supported platforms, but not all
CMake options have been replicated in GN.

We always use the just built compiler to build all the runtimes, which
is equivalent to the CMake runtimes build. This simplifies the build
configuration because we don't need to support arbitrary host compiler
and can always assume the latest Clang. With GN's toolchain support,
this is significantly more efficient than the CMake runtimes build.

Differential Revision: https://reviews.llvm.org/D60331

llvm-svn: 357821
2019-04-05 21:30:40 +00:00
Vitaly Buka 3db6ad2bcf Use binary write mode in WriteToFile function to avoid appended \r characters on Windows
Summary:
When using libfuzzer on Windows, in the contents of a crash sample, bytes that can be mistaken for a \n are replaced by a \r\n sequence. As a consequence, crashes are not reproducible. This patch will open files in binary mode to fix this issue. The patch does not affect POSIX systems.

Patch by tuktuk

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: dexonsmith, jdoerfert, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60008

llvm-svn: 357807
2019-04-05 20:17:03 +00:00
Rainer Orth 92db30fc00 [Sanitizers] Provide __internal_dup on Solaris
Both LLVM 8.0.0 and current trunk fail to link libclang_rt.ubsan.standalone-*.so
on Solaris 11/x86 with GCC 8.1.0:

  Undefined			first referenced
   symbol  			    in file
  Scanning dependencies of target clang_rt.asan_cxx-x86_64
  __sanitizer::internal_dup(int)      ../sanitizer_common/CMakeFiles/RTSanitizerCommon.i386.dir/sanitizer_posix.cc.o
  ld: fatal: symbol referencing errors
  collect2: error: ld returned 1 exit status
  make[2]: *** [projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan_standalone-dynamic-i386.dir/build.make:233: lib/clang/9.0.0/lib/sunos/libclang_rt.ubsan_standalone-i386.so] Error 1

Fixed by providing an implementation, which allowed the i386-pc-solaris2.11 build
to finish.

Differential Revision: https://reviews.llvm.org/D60045

llvm-svn: 357753
2019-04-05 08:42:21 +00:00
Rainer Orth a0524521ea [Sanitizers] Fix sanitizer_linux_libcdep.cc compilation on Solaris
Both LLVM 8.0.0 and current trunk fail to compile on Solaris 11/x86 with
GCC 8.1.0:

  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc: In function ‘void __sanitizer::ReExec()’:
  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc:831:14: error: ‘getexecname’ was not declared in this scope
     pathname = getexecname();
                ^~~~~~~~~~~
  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc:831:14: note: suggested alternative: ‘gethostname’
     pathname = getexecname();
                ^~~~~~~~~~~
                gethostname

This is easily fixed by including <stdlib.h> which declares that function.
With that patch, compilation continues.

Differential Revision: https://reviews.llvm.org/D60044

llvm-svn: 357751
2019-04-05 08:40:08 +00:00