Commit Graph

3824 Commits

Author SHA1 Message Date
Nico Weber 217222abea compiler-rt: Rename .cc file in lib/asan to .cpp
Like r367463, but for asan.

llvm-svn: 367558
2019-08-01 13:43:28 +00:00
Nico Weber 65492d959b compiler-rt: Rename .cc file in lib/sanitizer_common to .cpp
See https://reviews.llvm.org/D58620 for discussion, and for the commands
I ran. In addition I also ran

  for f in $(svn diff | diffstat | grep .cc | cut -f 2 -d ' '); do rg $f . ; done

and manually updated (many) references to renamed files found by that.

llvm-svn: 367463
2019-07-31 18:51:27 +00:00
Nico Weber 46ba969752 compiler-rt: Rename .cc files in lib/ubsan to .cpp.
See https://reviews.llvm.org/D58620 for discussion, and for the commands
I ran. In addition I also ran

  for f in $(svn diff | diffstat | grep .cc | cut -f 2 -d ' '); do rg $f . ; done

and manually updated references to renamed files found by that.

llvm-svn: 367452
2019-07-31 17:51:05 +00:00
Alexander Richardson a4ea27de92 [Sanitizer][ASAN][MSAN] Fix infinite recursion on FreeBSD
Summary:
MSAN was broken on FreeBSD by https://reviews.llvm.org/D55703: after this
change accesses to the key variable call __tls_get_addr, which is
intercepted. The interceptor then calls GetCurrentThread which calls
MsanTSDGet which again calls __tls_get_addr, etc...
Using the default implementation in the SANITIZER_FREEBSD case fixes MSAN
for me.

I then applied the same change to ASAN (introduced in https://reviews.llvm.org/D55596)
but that did not work yet. In the ASAN case, we get infinite recursion
again during initialization, this time because calling pthread_key_create() early on
results in infinite recursion. pthread_key_create() calls sysctlbyname()
which is intercepted but COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED returns
true, so the interceptor calls internal_sysctlbyname() which then ends up
calling the interceptor again. I fixed this issue by using dlsym() to get
the libc version of sysctlbyname() instead.

This fixes https://llvm.org/PR40761

Reviewers: vitalybuka, krytarowski, devnexen, dim, bsdjhb, #sanitizers, MaskRay

Reviewed By: MaskRay

Subscribers: MaskRay, emaste, kubamracek, jfb, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D65221

llvm-svn: 367442
2019-07-31 16:31:55 +00:00
Rainer Orth 63d9605091 [builtins][test] XFAIL two SPARC tests
Two SPARC builtins tests are currently FAILing due to codegen bugs:

  Builtins-sparc-sunos :: divtc3_test.c
  Builtins-sparcv9-sunos :: compiler_rt_logbl_test.c
  Builtins-sparcv9-sunos :: divtc3_test.c

I'd like to XFAIL them to reduce testsuite noise. 
  
Done as follows, tested on sparcv9-sun-solaris2.11 and x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D64796

llvm-svn: 367295
2019-07-30 08:05:14 +00:00
Rainer Orth 58aa6a87a6 [ASan][test] XFAIL AddressSanitizer-*-sunos :: TestCases/intercept-rethrow-exception.cc on Solaris
AddressSanitizer-*-sunos :: TestCases/intercept-rethrow-exception.cc currently FAILs
on Solaris.  This happens because std::rethrow_exception cannot be intercepted, as
detailed in Bug 42703.

To account for this and reduce testsuite noise, this patch XFAILs the test.

Tested on x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D65056

llvm-svn: 367293
2019-07-30 07:59:43 +00:00
Lei Huang 686cee0945 [NFC][ASAN] Add brackets around not command
Under certain execution conditions, the `not` command binds to the command the
output is piped to rather than the command piping the output. In this case, that
flips the return code of the FileCheck invocation, causing a failure when
FileCheck succeeds.

llvm-svn: 366805
2019-07-23 13:10:29 +00:00
Julian Lettner 2ef9ec4050 [TSan] Enable fiber tests on iOS simulator
These tests *do not* work on device, but they *do* work in the
simulator.

rdar://53403778

llvm-svn: 366738
2019-07-22 21:13:19 +00:00
Serge Guelton a30a4a35ec Fix asan infinite loop on undefined symbol
Fix llvm#39641

Recommit of r366413

Differential Revision: https://reviews.llvm.org/D63877

> llvm-svn: 366632

llvm-svn: 366638
2019-07-20 17:44:30 +00:00
Serge Guelton 7a3d4c15a7 Revert "Fix asan infinite loop on undefined symbol"
This reverts commit cbd28cd05b.

Buildbot fail: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/22434/steps/64-bit%20check-asan-dynamic/

llvm-svn: 366634
2019-07-20 13:00:12 +00:00
Serge Guelton cbd28cd05b Fix asan infinite loop on undefined symbol
Fix llvm#39641

Recommit of r366413

Differential Revision: https://reviews.llvm.org/D63877

llvm-svn: 366632
2019-07-20 12:01:18 +00:00
Matthew Voss 407e837540 Revert "Fix asan infinite loop on undefined symbol"
This reverts commit 8e46275488.

This was failing on sanitizer-x86_64-linux and our internal CI.

llvm-svn: 366618
2019-07-19 21:41:07 +00:00
Serge Guelton 8e46275488 Fix asan infinite loop on undefined symbol
Fix llvm#39641

Recommit of r366413

Differential Revision: https://reviews.llvm.org/D63877

llvm-svn: 366588
2019-07-19 15:20:36 +00:00
Matthew Voss 892758a526 [compiler-rt] Complete revert of r366413
Incomplete revert. Mea culpa. This test is failing on sanitizer-x86_64-linux
and our internal CI.

llvm-svn: 366482
2019-07-18 18:39:06 +00:00
Serge Guelton ec2a7c463e Restrict asan + dlopen testcase to x86
llvm-svn: 366436
2019-07-18 13:47:28 +00:00
Serge Guelton 6a61bea4d6 Relax regexp to detect failed interception by asan
This should fix failed detection on aarch64/ppc64/thumbv8...

llvm-svn: 366432
2019-07-18 13:13:29 +00:00
Serge Guelton 63719119c7 Fix asan infinite loop on undefined symbol
Fix llvm#39641

Differential Revision: https://reviews.llvm.org/D63877

llvm-svn: 366413
2019-07-18 08:09:31 +00:00
Peter Collingbourne 749f556bbd hwasan: Use C++ driver for cfi.cc test.
It turns out that this test was only passing by accident. It was relying on
the optimizer to remove the only reference to A's vtable by realizing that
the CFI check will always fail. The vtable contains a reference to RTTI in
libc++, which will be unresolved because the C driver won't link against it.

This was found by my prototype implementation of HWASAN for globals, which
happens to end up preserving the reference.

Differential Revision: https://reviews.llvm.org/D64890

llvm-svn: 366389
2019-07-17 23:35:15 +00:00
Julian Lettner 9a050f92bb [ASan] Support `{f}puts(NULL)` on Darwin
On Darwin, the man page states that "both fputs() and puts() print
`(null)' if str is NULL."

rdar://48227136

Reviewed By: Lekensteyn

Differential Revision: https://reviews.llvm.org/D64773

llvm-svn: 366342
2019-07-17 16:09:25 +00:00
Stephan Bergmann e215996a29 Finish "Adapt -fsanitize=function to SANITIZER_NON_UNIQUE_TYPEINFO"
i.e., recent 5745eccef54ddd3caca278d1d292a88b2281528b:

* Bump the function_type_mismatch handler version, as its signature has changed.

* The function_type_mismatch handler can return successfully now, so
  SanitizerKind::Function must be AlwaysRecoverable (like for
  SanitizerKind::Vptr).

* But the minimal runtime would still unconditionally treat a call to the
  function_type_mismatch handler as failure, so disallow -fsanitize=function in
  combination with -fsanitize-minimal-runtime (like it was already done for
  -fsanitize=vptr).

* Add tests.

Differential Revision: https://reviews.llvm.org/D61479

llvm-svn: 366186
2019-07-16 06:23:27 +00:00
Diana Picus 0bf0b8ff7c [libFuzzer] Disable fork.test on AArch64
This crashes sporadically on our AArch64 buildbots. Disable for now.

llvm-svn: 366055
2019-07-15 11:33:41 +00:00
Rainer Orth 4a9a772f44 Enable compiler-rt on SPARC
This patch enables compiler-rt on SPARC targets. Most of the changes are straightforward:

- Add 32 and 64-bit sparc to compiler-rt

- lib/builtins/fp_lib.h needed to check if the int128_t and uint128_t types exist (which they don't on sparc)

There's one issue of note: many asan tests fail to compile on Solaris/SPARC:

fatal error: error in backend: Function "_ZN7testing8internal16BoolFromGTestEnvEPKcb": over-aligned dynamic alloca not supported.

Therefore, while asan is still built, both asan and ubsan-with-asan testing is disabled. The
goal is to check if asan keeps compiling on Solaris/SPARC. This serves asan in gcc,
which doesn't have the problem above and works just fine.

With this patch, sparcv9-sun-solaris2.11 test results are pretty good:

Failing Tests (9):
    Builtins-sparc-sunos :: divtc3_test.c
    Builtins-sparcv9-sunos :: compiler_rt_logbl_test.c
    Builtins-sparcv9-sunos :: divtc3_test.c
[...]
    UBSan-Standalone-sparc :: TestCases/TypeCheck/misaligned.cpp
    UBSan-Standalone-sparcv9 :: TestCases/TypeCheck/misaligned.cpp

The builtin failures are due to Bugs 42493 and 42496. The tree contained a few additonal
patches either currently in review or about to be submitted.

Tested on sparcv9-sun-solaris2.11.

Differential Revision: https://reviews.llvm.org/D40943

llvm-svn: 365880
2019-07-12 08:30:17 +00:00
Artem Dergachev 8bd441af8b NFC: Unforget a colon in a few CHECK: directives.
Differential Revision: https://reviews.llvm.org/D64526

llvm-svn: 365863
2019-07-12 02:16:56 +00:00
Reid Kleckner 1a285c27fd Use clang driver for libfuzzer tests on Windows
Summary:
There's no real reason to use clang-cl on Windows, the clang driver
works just as well. This fixes a test which uses the -O0 flag, which was
recently removed from clang-cl to match MSVC, which lacks this flag.

While I'm here, remove the explicit -std=c++11 flag. Previously, this
flag was necessary when the default C++ standard was C++98. Now that the
default is C++14, this is no longer necessary. It's problematic on
Windows, because the Visual C++ standard library relies on C++14
features, and attempting to compile it with C++11 results in errors.
Rather than adding logic to conditionally set the standard to C++11 only
on non-Win, this flag can be removed.

See http://lab.llvm.org:8011/builders/clang-x64-windows-msvc and
https://reviews.llvm.org/D64506.

Reviewers: morehouse, thakis

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D64587

llvm-svn: 365841
2019-07-11 23:20:04 +00:00
Rainer Orth 9512c0a1d1 [profile][test] Fix Profile-* :: instrprof-merge.c etc. on SPARC
While working on https://reviews.llvm.org/D40900 (which effectively is about enabling compiler-rt on sparc these days), I came across two failing profile testcases:

  Profile-sparc :: instrprof-merge-match.test
  Profile-sparc :: instrprof-merge.c
  Profile-sparcv9 :: instrprof-merge-match.test
  Profile-sparcv9 :: instrprof-merge.c

All of them crashed with a SIGBUS in __llvm_profile_merge_from_buffer:

  Thread 2 received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 1 (LWP 1)]
  0x00012368 in __llvm_profile_merge_from_buffer (
      ProfileData=0x2384c <main.Buffer> "\377lprofR\201", ProfileSize=360)
      at /vol/llvm/src/llvm/local/projects/compiler-rt/lib/profile/InstrProfilingMerge.c:95
  95        SrcDataEnd = SrcDataStart + Header->DataSize;

where Header is insufficiently aligned for a strict-alignment target like SPARC.

Fixed by forcing the alignment to uint64_t, the members of struct __llvm_profile_header,
in the callers.

Tested on sparcv9-sun-solaris2.11.

https://reviews.llvm.org/D64498

llvm-svn: 365805
2019-07-11 18:26:24 +00:00
Kamil Rytarowski 983d7ddd0b Add NetBSD LSan support
Summary:
Combine few relatively small changes into one:

 - implement internal_ptrace() and internal_clone() for NetBSD
 - add support for stoptheworld based on the ptrace(2) API
 - define COMPILER_RT_HAS_LSAN for NetBSD
 - enable tests for NetBSD/amd64

Inspired by the original implementation by Christos Zoulas in netbsd/src for GCC.

The implementation is in theory CPU independent through well defined macros
across all NetBSD ports, however only the x86_64 version was tested.

Reviewers: mgorny, dvyukov, vitalybuka, joerg, jfb

Reviewed By: vitalybuka

Subscribers: dexonsmith, jfb, srhines, kubamracek, llvm-commits, christos

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64057

llvm-svn: 365735
2019-07-11 06:22:35 +00:00
Nico Weber e837847ec6 Change another test I missed in r365724
llvm-svn: 365725
2019-07-11 01:21:17 +00:00
Nico Weber da233838c9 clang-cl: Remove -O0 option
cl.exe doesn't understand it; there's /Od instead. See also the review
thread for r229575.

Update lots of compiler-rt tests to use -Od instead of -O0.
Ran `rg -l 'clang_cl.*O0' compiler-rt/test/ | xargs sed -i -c 's/-O0/-Od/'`

Differential Revision: https://reviews.llvm.org/D64506

llvm-svn: 365724
2019-07-11 01:18:05 +00:00
Fangrui Song 7f9a94e1f8 [ubsan][test] Restore float-divide-by-zero test
Removed by rCTE365307 to fix buildbots. It can be restored now because D64317/rC365587 brought back -fsanitize=float-divide-by-zero

llvm-svn: 365591
2019-07-10 01:53:11 +00:00
Peter Collingbourne 1366262b74 hwasan: Improve precision of checks using short granule tags.
A short granule is a granule of size between 1 and `TG-1` bytes. The size
of a short granule is stored at the location in shadow memory where the
granule's tag is normally stored, while the granule's actual tag is stored
in the last byte of the granule. This means that in order to verify that a
pointer tag matches a memory tag, HWASAN must check for two possibilities:

* the pointer tag is equal to the memory tag in shadow memory, or
* the shadow memory tag is actually a short granule size, the value being loaded
  is in bounds of the granule and the pointer tag is equal to the last byte of
  the granule.

Pointer tags between 1 to `TG-1` are possible and are as likely as any other
tag. This means that these tags in memory have two interpretations: the full
tag interpretation (where the pointer tag is between 1 and `TG-1` and the
last byte of the granule is ordinary data) and the short tag interpretation
(where the pointer tag is stored in the granule).

When HWASAN detects an error near a memory tag between 1 and `TG-1`, it
will show both the memory tag and the last byte of the granule. Currently,
it is up to the user to disambiguate the two possibilities.

Because this functionality obsoletes the right aligned heap feature of
the HWASAN memory allocator (and because we can no longer easily test
it), the feature is removed.

Also update the documentation to cover both short granule tags and
outlined checks.

Differential Revision: https://reviews.llvm.org/D63908

llvm-svn: 365551
2019-07-09 20:22:36 +00:00
Matthew G McGovern 4e636156ef [sanitizers][windows] FIX: Rtl-Heap Interception and tests
- Adds interceptors for Rtl[Allocate|Free|Size|ReAllocate]Heap
   - Adds unit tests for the new interceptors and expands HeapAlloc
     tests to demonstrate new functionality.
   Reviewed as D62927
   - adds fixes for ~win and x64 tests

> llvm-svn: 365381

llvm-svn: 365424
2019-07-09 01:55:11 +00:00
Matthew G McGovern 848a19e4eb [sanitizers][windows] Rtl-Heap Interception and tests
- Adds interceptors for Rtl[Allocate|Free|Size|ReAllocate]Heap
   - Adds unit tests for the new interceptors and expands HeapAlloc
     tests to demonstrate new functionality.
   Reviewed as D62927

llvm-svn: 365422
2019-07-09 01:47:08 +00:00
JF Bastien c5630ac641 Revert "[sanitizers][windows] Rtl-Heap Interception and tests"
Causes build failure on clang-ppc64be-linux-lnt:

compiler-rt/lib/asan/asan_malloc_win.cc:23:2: error: #error "Missing arch or unsupported platform for Windows."
 #error "Missing arch or unsupported platform for Windows."
  ^~~~~
compiler-rt/lib/asan/asan_malloc_win.cc:25:10: fatal error: heapapi.h: No such file or directory
 #include <heapapi.h>
          ^~~~~~~~~~~
compilation terminated.
[39/1151] Building CXX object projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_debugging.cc.o
[40/1151] Building CXX object projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o
FAILED: projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o
/usr/bin/c++  -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iprojects/compiler-rt/lib/asan -Icompiler-rt/lib/asan -Iinclude -I/home/buildbots/ppc64be-clang-lnt-test/clang-ppc64be-lnt/llvm/include -Icompiler-rt/lib/asan/.. -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -Wall -std=c++11 -Wno-unused-parameter -O2    -UNDEBUG  -m64 -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fvisibility=hidden -fno-lto -O3 -g -Wno-variadic-macros -Wno-non-virtual-dtor -fno-rtti -MD -MT projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o -MF projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o.d -o projects/compiler-rt/lib/asan/CMakeFiles/RTAsan.powerpc64.dir/asan_malloc_win.cc.o -c compiler-rt/lib/asan/asan_malloc_win.cc
compiler-rt/lib/asan/asan_malloc_win.cc:23:2: error: #error "Missing arch or unsupported platform for Windows."
 #error "Missing arch or unsupported platform for Windows."
  ^~~~~
compiler-rt/lib/asan/asan_malloc_win.cc:25:10: fatal error: heapapi.h: No such file or directory
 #include <heapapi.h>
          ^~~~~~~~~~~

llvm-svn: 365384
2019-07-08 20:21:09 +00:00
Matthew G McGovern c9fa99d066 [sanitizers][windows] Rtl-Heap Interception and tests
- Adds interceptors for Rtl[Allocate|Free|Size|ReAllocate]Heap
   - Adds unit tests for the new interceptors and expands HeapAlloc
     tests to demonstrate new functionality.
   Reviewed as D62927

llvm-svn: 365381
2019-07-08 19:58:50 +00:00
Julian Lettner 611c122045 Revert "[TSan] Attempt to fix iOS on-device test"
This reverts commit a2ca358291.

llvm-svn: 365375
2019-07-08 19:26:21 +00:00
Fangrui Song 1f7bd40f68 [ubsan][test] Fix cast-overflow.cpp and delete float-divide-by-zero test after D63793/rC365272
llvm-svn: 365307
2019-07-08 09:47:04 +00:00
Rainer Orth 04ea772d5a [ubsan][test] Fix several UBSan-* :: TestCases/ImplicitConversion tests on Solaris
A couple of UBSan-*  :: TestCases/ImplicitConversion testcases FAIL on Solaris/x86
(and Solaris/SPARC with https://reviews.llvm.org/D40900):

  FAIL: UBSan-AddressSanitizer-i386 :: TestCases/ImplicitConversion/signed-integer-truncation-or-sign-change-blacklist.c (49187 of 49849)
  ******************** TEST 'UBSan-AddressSanitizer-i386 :: TestCases/ImplicitConversion/signed-integer-truncation-or-sign-change-blacklist.c' FAILED ********************
  [...]
  Command Output (stderr):
  --
  /vol/llvm/src/compiler-rt/local/test/ubsan/TestCases/ImplicitConversion/signed-integer-truncation-or-sign-change-blacklist.c:53:11: error: CHECK: expected string not found in input
  // CHECK: {{.*}}signed-integer-truncation-or-sign-change-blacklist.c:[[@LINE-1]]:10: runtime error: implicit conversion from type '{{.*}}' (aka 'unsigned int') of value 4294967295 (32-bit, unsigned) to type '{{.*}}' (aka 'signed char') changed the value to -1 (8-bit, signed)
            ^
  <stdin>:1:1: note: scanning from here
  /vol/llvm/src/compiler-rt/local/test/ubsan/TestCases/ImplicitConversion/signed-integer-truncation-or-sign-change-blacklist.c:52:10: runtime error: implicit conversion from type 'uint32_t' (aka 'unsigned int') of value 4294967295 (32-bit, unsigned) to type 'int8_t' (aka 'char') changed the value to -1 (8-bit, signed)
  ^
  <stdin>:1:1: note: with "@LINE-1" equal to "52"
  /vol/llvm/src/compiler-rt/local/test/ubsan/TestCases/ImplicitConversion/signed-integer-truncation-or-sign-change-blacklist.c:52:10: runtime error: implicit conversion from type 'uint32_t' (aka 'unsigned int') of value 4294967295 (32-bit, unsigned) to type 'int8_t' (aka 'char') changed the value to -1 (8-bit, signed)
  ^
  <stdin>:1:69: note: possible intended match here
  /vol/llvm/src/compiler-rt/local/test/ubsan/TestCases/ImplicitConversion/signed-integer-truncation-or-sign-change-blacklist.c:52:10: runtime error: implicit conversion from type 'uint32_t' (aka 'unsigned int') of value 4294967295 (32-bit, unsigned) to type 'int8_t' (aka 'char') changed the value to -1 (8-bit, signed)
                                                                      ^

This is always a difference for int8_t where signed char is expected, but only
char seen.

I could trace this to <sys/int_types.h> which has

  /*
   * Basic / Extended integer types
   *
   * The following defines the basic fixed-size integer types.
   *
   * Implementations are free to typedef them to Standard C integer types or
   * extensions that they support. If an implementation does not support one
   * of the particular integer data types below, then it should not define the
   * typedefs and macros corresponding to that data type.  Note that int8_t
   * is not defined in -Xs mode on ISAs for which the ABI specifies "char"
   * as an unsigned entity because there is no way to define an eight bit
   * signed integral.
   */
  #if defined(_CHAR_IS_SIGNED)
  typedef char			int8_t;
  #else
  #if defined(__STDC__)
  typedef signed char		int8_t;
  #endif
  #endif

_CHAR_IS_SIGNED is always defined on both sparc and x86.  Since it seems ok
to have either form, I've changed the affected tests to use
'{{(signed )?}}char' instead of 'signed char'.

Tested on x86_64-pc-solaris2.11, sparcv9-sun-solaris2.11, and x86_64-pc-linux-gnu.

Differential Revision: https://reviews.llvm.org/D63984

llvm-svn: 365303
2019-07-08 09:22:05 +00:00
Rainer Orth d11df93898 [ubsan][test] Don't disable ubsan testing on 64-bit Solaris/x86
Unlike asan, which isn't supported yet on 64-bit Solaris/x86, there's no reason to disable
ubsan.  This patch does that, but keeps the 64-bit ubsan-with-asan tests disabled.

Tested on x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D63982

llvm-svn: 365302
2019-07-08 09:18:38 +00:00
Julian Lettner a2ca358291 [TSan] Attempt to fix iOS on-device test
llvm-svn: 365257
2019-07-06 00:49:44 +00:00
Julian Lettner b1ff896e92 XFAIL a few failing TSan-fiber tests for iOS
llvm-svn: 365254
2019-07-05 23:23:36 +00:00
Julian Lettner b6654319f7 Remove `XFAIL: ios` from test that passes in CI
llvm-svn: 365253
2019-07-05 22:53:18 +00:00
Mitch Phillips 7339ca278c [GWP-ASan] Add generic unwinders and structure backtrace output.
Summary:
Adds two flavours of generic unwinder and all the supporting cruft. If the
supporting allocator is okay with bringing in sanitizer_common, they can use
the fast frame-pointer based unwinder from sanitizer_common. Otherwise, we also
provide the backtrace() libc-based unwinder as well. Of course, the allocator
can always specify its own unwinder and unwinder-symbolizer.

The slightly changed output format is exemplified in the first comment on this
patch. It now better incorporates backtrace information, and displays
allocation details on the second line.

Reviewers: eugenis, vlad.tsyrklevich

Reviewed By: eugenis, vlad.tsyrklevich

Subscribers: srhines, kubamracek, mgorny, cryptoad, #sanitizers, llvm-commits, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63841

llvm-svn: 364941
2019-07-02 16:04:52 +00:00
Peter Collingbourne d11ea6518c hwasan: Teach the runtime to identify the local variable being accessed in UAR reports.
Each function's PC is recorded in the ring buffer. From there we can access
the function's local variables and reconstruct the tag of each one with the
help of the information printed by llvm-symbolizer's new FRAME command. We
can then find the variable that was likely being accessed by matching the
pointer's tag against the reconstructed tag.

Differential Revision: https://reviews.llvm.org/D63469

llvm-svn: 364607
2019-06-27 23:16:13 +00:00
Reid Kleckner 8007ff1ab1 [compiler-rt] Rename lit.*.cfg.* -> lit.*.cfg.py.*
These lit configuration files are really Python source code. Using the
.py file extension helps editors and tools use the correct language
mode. LLVM and Clang already use this convention for lit configuration,
this change simply applies it to all of compiler-rt.

Reviewers: vitalybuka, dberris

Differential Revision: https://reviews.llvm.org/D63658

llvm-svn: 364591
2019-06-27 20:56:04 +00:00
Rainer Orth e6474e682a [compiler-rt][test] Set shared_libasan_path in test/asan/lit.cfg on Solaris
While checking warnings from the Solaris buildbots, I noticed

  llvm-lit: /opt/llvm-buildbot/home/solaris11-amd64/clang-solaris11-amd64/llvm/projects/compiler-rt/test/asan/lit.cfg:119: warning: %shared_libasan substitution not set but dynamic ASan is available.

Fixed as follows.  Tested on x86_64-pc-solaris2.11.

Differential Revision: https://reviews.llvm.org/D63761

llvm-svn: 364394
2019-06-26 08:19:57 +00:00
Dan Liew d38e251bf2 Follow up fix for r364366.
When setting the parallelism group support MSan and use a syntactically
compact condition.

rdar://problem/51754620

llvm-svn: 364369
2019-06-26 00:54:06 +00:00
Dan Liew 5242fbde5a Add USan+ASan and UBSan+TSan tests to shadow-memory lit parallelism group.
Summary:
Previously we were running these tests without the "shadow-memory"
lit parallelism group even though we run the ASan and TSan tests in
this group to avoid problems with many processes using shadow memory
in parallel.

On my local machine the UBSan+TSan tests would previously timeout
if I set a 30 second per test limit. With this change I no longer
see individual test timeouts.

This change was made in response to the greendragon build bot reporting
individual test timeouts for these tests. Given that the UBSan+ASan and
UBSan+TSan tests did not have a parallelism group previously it's likely
that some other change has caused the performance degradation. However
I haven't been able to track down the cause so until we do, this change
seems reasonable and is in line with what we already do with ASan and
TSan tests.

rdar://problem/51754620

Reviewers: yln, kubamracek, vsk, samsonov

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63797

llvm-svn: 364366
2019-06-26 00:35:51 +00:00
Vitaly Buka 5b9d0205aa [sanitizer] Enabled getpw_getgr.cc on iOS
Reviewers: kubamracek, delcypher, yln

Reviewed By: delcypher

Subscribers: yln, delcypher, llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57786

llvm-svn: 364261
2019-06-25 01:01:46 +00:00
Sajjad Mirza 6694b2b36b (Reland with changes) Adding a function for setting coverage output file.
Summary:
User code can open a file on its own and pass it to the runtime, rather than
specifying a name and having the runtime open the file. This supports the use
case where a process cannot open a file on its own but can receive a file
descriptor from another process.

Relanding https://reviews.llvm.org/D62541. The original revision unlocked
the file before calling flush, this revision fixes that.

Reviewers: Dor1s, davidxl

Reviewed By: Dor1s

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63581

llvm-svn: 364231
2019-06-24 21:32:50 +00:00
Reid Kleckner 08b2bd0f30 [asan] Quote the path to the Python exe in case it has spaces
These days, Python 3 installs itself into Program Files, so it often has
spaces. At first, I resisted this, and I reinstalled it globally into
C:/Python37, similar to the location used for Python 2.7. But then I
updated VS 2019, and it uninstalled my copy of Python and installed a
new one inside "C:/Program Files (x86)/Microsoft Visual Studio/". At
this point, I gave up and switched to using its built-in version of
Python. However, now these tests fail, and have to be made aware of the
possibility of spaces in paths. :(

llvm-svn: 364077
2019-06-21 16:54:58 +00:00
Kostya Serebryany 679669a77e [libFuzzer] split DataFlow.cpp into two .cpp files, one of which can be compiled w/o dfsan to speed things up (~25% speedup)
llvm-svn: 364002
2019-06-21 01:39:35 +00:00
Kostya Serebryany 27cf743bff [libFuzzer] ensure that DFT and autofocus works for C++ (mangled) functions
llvm-svn: 363905
2019-06-20 01:48:45 +00:00
Hans Wennborg 2cf990fa27 Revert r363633 "[CMake] Fix the value of `config.target_cflags` for non-macOS Apple platforms. Attempt #2."
This caused Chromium's clang package to stop building, see comment on
https://reviews.llvm.org/D61242 for details.

> Summary:
> The main problem here is that `-*-version_min=` was not being passed to
> the compiler when building test cases. This can cause problems when
> testing on devices running older OSs because Clang would previously
> assume the minimum deployment target is the the latest OS in the SDK
> which could be much newer than what the device is running.
>
> Previously the generated value looked like this:
>
> `-arch arm64 -isysroot
> <path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`
>
> With this change it now looks like:
>
> `-arch arm64 -stdlib=libc++ -miphoneos-version-min=8.0 -isysroot
> <path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`
>
> This mirrors the setting of `config.target_cflags` on macOS.
>
> This change is made for ASan, LibFuzzer, TSan, and UBSan.
>
> To implement this a new `get_test_cflags_for_apple_platform()` function
> has been added that when given an Apple platform name and architecture
> returns a string containing the C compiler flags to use when building
> tests. This also calls a new helper function `is_valid_apple_platform()`
> that validates Apple platform names.
>
> This is the second attempt at landing the patch. The first attempt (r359305)
> had to be reverted (r359327) due to a buildbot failure. The problem was
> that calling `get_test_cflags_for_apple_platform()` can trigger a CMake
> error if the provided architecture is not supported by the current
> CMake configuration. Previously, this could be triggered by passing
> `-DCOMPILER_RT_ENABLE_IOS=OFF` to CMake. The root cause is that we were
> generating test configurations for a list of architectures without
> checking if the relevant Sanitizer actually supported that architecture.
> We now intersect the list of architectures for an Apple platform
> with `<SANITIZER>_SUPPORTED_ARCH` (where `<SANITIZER>` is a Sanitizer
> name) to iterate through the correct list of architectures.
>
> rdar://problem/50124489
>
> Reviewers: kubamracek, yln, vsk, juliehockett, phosek
>
> Subscribers: mgorny, javed.absar, kristof.beyls, #sanitizers, llvm-commits
>
> Tags: #llvm, #sanitizers
>
> Differential Revision: https://reviews.llvm.org/D61242

llvm-svn: 363779
2019-06-19 09:09:39 +00:00
Dan Liew 9216358c21 Disable recently added Darwin symbolization tests for iOS.
These tests won't necessarily work because the reported modules paths
from the device don't match what's on the host and so offline
symbolization fails.

llvm-svn: 363641
2019-06-18 01:38:03 +00:00
Dan Liew 745632c63a [NFC] Split `Darwin/asan-symbolize-partial-report-with-module-map.cc`.
Split `Darwin/asan-symbolize-partial-report-with-module-map.cc` into two
separate test cases due to them testing slightly different things.

llvm-svn: 363640
2019-06-18 01:38:02 +00:00
Dan Liew 3c9f66dccf [asan_symbolize] Teach `asan_symbolize.py` to symbolicate partially symbolicated ASan reports.
Summary:
The use case here is to be able symbolicate ASan reports that might be
partially symbolicated, in particular where the function name is known but no source
location is available. This can be caused by missing debug info. Previously we
would only try to symbolicate completely unsymbolicated reports.

The code currently contains an unfortunate quirk to handle a darwin
specific bug (rdar://problem/49784442) in the way partially symbolicated
reports are emitted when the source location is missing.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: aprantl, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60533

llvm-svn: 363639
2019-06-18 01:21:16 +00:00
Peter Collingbourne d57f7cc15e hwasan: Use bits [3..11) of the ring buffer entry address as the base stack tag.
This saves roughly 32 bytes of instructions per function with stack objects
and causes us to preserve enough information that we can recover the original
tags of all stack variables.

Now that stack tags are deterministic, we no longer need to pass
-hwasan-generate-tags-with-calls during check-hwasan. This also means that
the new stack tag generation mechanism is exercised by check-hwasan.

Differential Revision: https://reviews.llvm.org/D63360

llvm-svn: 363636
2019-06-17 23:39:51 +00:00
Dan Liew 964909e4a6 [CMake] Fix the value of `config.target_cflags` for non-macOS Apple platforms. Attempt #2.
Summary:
The main problem here is that `-*-version_min=` was not being passed to
the compiler when building test cases. This can cause problems when
testing on devices running older OSs because Clang would previously
assume the minimum deployment target is the the latest OS in the SDK
which could be much newer than what the device is running.

Previously the generated value looked like this:

`-arch arm64 -isysroot
<path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`

With this change it now looks like:

`-arch arm64 -stdlib=libc++ -miphoneos-version-min=8.0 -isysroot
<path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`

This mirrors the setting of `config.target_cflags` on macOS.

This change is made for ASan, LibFuzzer, TSan, and UBSan.

To implement this a new `get_test_cflags_for_apple_platform()` function
has been added that when given an Apple platform name and architecture
returns a string containing the C compiler flags to use when building
tests. This also calls a new helper function `is_valid_apple_platform()`
that validates Apple platform names.

This is the second attempt at landing the patch. The first attempt (r359305)
had to be reverted (r359327) due to a buildbot failure. The problem was
that calling `get_test_cflags_for_apple_platform()` can trigger a CMake
error if the provided architecture is not supported by the current
CMake configuration. Previously, this could be triggered by passing
`-DCOMPILER_RT_ENABLE_IOS=OFF` to CMake. The root cause is that we were
generating test configurations for a list of architectures without
checking if the relevant Sanitizer actually supported that architecture.
We now intersect the list of architectures for an Apple platform
with `<SANITIZER>_SUPPORTED_ARCH` (where `<SANITIZER>` is a Sanitizer
name) to iterate through the correct list of architectures.

rdar://problem/50124489

Reviewers: kubamracek, yln, vsk, juliehockett, phosek

Subscribers: mgorny, javed.absar, kristof.beyls, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61242

llvm-svn: 363633
2019-06-17 23:37:46 +00:00
Mitch Phillips 21184ec5c4 [GWP-ASan] Integration with Scudo [5].
Summary:
See D60593 for further information.

This patch adds GWP-ASan support to the Scudo hardened allocator. It also
implements end-to-end integration tests using Scudo as the backing allocator.
The tests include crash handling for buffer over/underflow as well as
use-after-free detection.

Reviewers: vlad.tsyrklevich, cryptoad

Reviewed By: vlad.tsyrklevich, cryptoad

Subscribers: kubamracek, mgorny, #sanitizers, llvm-commits, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62929

llvm-svn: 363584
2019-06-17 17:45:34 +00:00
Kostya Serebryany db88fc56b9 [libFuzzer] implement a better queue for the fork mode. Add an internal flag -stop_file to allow graceful shutdown of fuzzing. Enhance the logging in the fork mode
llvm-svn: 363470
2019-06-14 22:56:50 +00:00
Kostya Serebryany 3f39123d15 [libFuzzer] simplify the DFT trace collection using the new faster DFSan mode that traces up to 16 labels at a time and never runs out of labels. Second attempt. This time with a fix for windows (putenv instead of setenv))
llvm-svn: 363445
2019-06-14 19:54:32 +00:00
Max Moroz 0784e01a98 [libFuzzer] Disable len_control by default if LLVMFuzzerCustomMutator is used.
Summary:
Some custom mutators may not peform well when size restriction is
enforced by len_control. Because of that, it's safer to disable len_control
by default in such cases, but still allow users to enable it manually.
Bug example: https://bugs.chromium.org/p/chromium/issues/detail?id=919530.

Tested manually with LPM-based and regular fuzz targets.

Reviewers: kcc, vitalybuka, metzman

Reviewed By: kcc, metzman

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63334

llvm-svn: 363443
2019-06-14 19:34:11 +00:00
Hans Wennborg 9bc707c0e5 Revert r363326 "[libFuzzer] simplify the DFT trace collection using the new faster DFSan mode that traces up to 16 labels at a time and never runs out of labels."
It broke the Windows build:

C:\b\s\w\ir\cache\builder\src\third_party\llvm\compiler-rt\lib\fuzzer\FuzzerDataFlowTrace.cpp(243): error C3861: 'setenv': identifier not found

This also reverts the follow-up r363327.

llvm-svn: 363358
2019-06-14 07:32:22 +00:00
Kostya Serebryany a5b12be60f fix whitespaces
llvm-svn: 363327
2019-06-13 21:17:49 +00:00
Kostya Serebryany 2fa83cb7ee [libFuzzer] simplify the DFT trace collection using the new faster DFSan mode that traces up to 16 labels at a time and never runs out of labels.
llvm-svn: 363326
2019-06-13 21:17:49 +00:00
Kostya Serebryany 6b936d88a4 [dfsan] Introduce dfsan_flush().
Summary:
dfsan_flush() allows to restart tain tracking from scratch in the same process.
The primary purpose right now is to allow more efficient data flow tracing
for DFT fuzzing: https://github.com/google/oss-fuzz/issues/1632

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63037

llvm-svn: 363321
2019-06-13 20:11:06 +00:00
Amy Kwan 7eddb16fed [compiler-rt] Fix name_to_handle_at.cc test on Overlay2 (for Docker)
This patch aims to fix the test case, name_to_handle_at.cc that fails on Docker.

Overlay2 on Docker does not support the current check for the name_to_handle_at()
function call of the test case. The proposed fix is to check for /dev/null in
the test instead, as this check is supported. Checking for /dev/null has been
utilized in the past for other test cases, as well.

Differential Revision: https://reviews.llvm.org/D63094

llvm-svn: 363167
2019-06-12 14:19:24 +00:00
Hans Wennborg 05d44139ee Revert r362676 "[Profile]: Add runtime interface to specify file handle for profile data."
This caused instrumented Clang to become crashy. See llvm-commits thread
for repro steps.

This also reverts follow-up r362716 which added test cases.

> Author: Sajjad Mirza
>
> Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 363134
2019-06-12 08:44:32 +00:00
Max Moroz 10ed68189a Add FuzzedDataProvider helper class / single header library.
Summary:
This class is useful for writing fuzz target that have multiple inputs.

Current CL imports the existing `FuzzedDataProvider` from Chromium
without any modifications. Feel free to review it thoroughly, if you're
interested, but I'd prefer changing the class in a follow up CL.

The CL also introduces an exhaustive test for the library, as the behavior
of `FuzzedDataProvider` must not change over time.

In follow up CLs I'm planning on changing some implementation details
(I can share a doc with some comments to be addressed). After that, we
will document how `FuzzedDataProvider` should be used.

I have tested this on Linux, Windows and Mac platforms.

Reviewers: morehouse, metzman, kcc

Reviewed By: morehouse

Subscribers: metzman, thakis, rnk, mgorny, ormris, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62733

llvm-svn: 363071
2019-06-11 14:30:18 +00:00
Kostya Serebryany 300c0c79de Experimantal dfsan mode "fast16labels=1"
Summary:
dfsan mode "fast16labels=1".
In this mode the labels are treated as 16-bit bit masks.

Reviewers: pcc

Reviewed By: pcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62870

llvm-svn: 362859
2019-06-08 00:22:23 +00:00
Mitch Phillips 45500fcd5d [GWP-ASan] Removed unittests from Android build.
Summary:
Longstanding issues in the Android test runner means that compiler-rt unit
tests don't work on Android due to libc++ link-time issues. Looks like the
exported libc++ from the Android NDK is x86-64, even though it's part of the
ARM[64] toolchain... See similar measures for ASan and sanitizer-common that
disable unit tests for Android.

Should fully fix the Android bots (@vlad.tsyrklevich).

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, mgorny, javed.absar, kristof.beyls, #sanitizers, llvm-commits, vlad.tsyrklevich

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D63019

llvm-svn: 362842
2019-06-07 20:56:32 +00:00
Peter Collingbourne c7903b9f1e Set an output file name for the override-new-delete.cpp test.
The android_compile.py script requires one.

llvm-svn: 362764
2019-06-07 02:30:58 +00:00
Mitch Phillips e41e366ae7 Change GWP-ASan build to use '-pthread' instead of '-lpthread' in order
to try and fix android buildbot. Also make sure that the empty dummy
test contains an output file name so the android_build.py wrapper script
doesn't check fail.

llvm-svn: 362758
2019-06-06 23:43:25 +00:00
Xinliang David Li 758c08921d [Profile]: Add runtime interface to specify file handle for profile data (Part-II)
Test cases

Author: Sajjad Mirza

Differential Revision: http://reviews.llvm.org/D62541

llvm-svn: 362716
2019-06-06 16:29:44 +00:00
Eugene Leviant c3c686f5f8 [HWASAN] Make new/delete weak
This allows instrumenting programs which have their own
versions of new and delete operators.

Differential revision: https://reviews.llvm.org/D62794

llvm-svn: 362478
2019-06-04 09:20:02 +00:00
Mitch Phillips 5f0f4e3ae0 [GWP-ASan] Mutex implementation [2].
Summary:
See D60593 for further information.
This patch pulls out the mutex implementation and the required definitions file.

We implement our own mutex for GWP-ASan currently, because:

1. We must be compatible with the sum of the most restrictive elements of the supporting allocator's build system. Current targets for GWP-ASan include Scudo (on Linux and Fuchsia), and bionic (on Android).
2. Scudo specifies `-nostdlib++ -nonodefaultlibs`, meaning we can't use `std::mutex` or `mtx_t`.
3. We can't use `sanitizer_common`'s mutex, as the supporting allocators cannot afford the extra maintenance (Android, Fuchsia) and code size (Fuchsia) overheads that this would incur.

In future, we would like to implement a shared base mutex for GWP-ASan, Scudo and sanitizer_common. This will likely happen when both GWP-ASan and Scudo standalone are not in the development phase, at which point they will have stable requirements.

Reviewers: vlad.tsyrklevich, morehouse, jfb

Reviewed By: morehouse

Subscribers: dexonsmith, srhines, cfe-commits, kubamracek, mgorny, cryptoad, jfb, #sanitizers, llvm-commits, vitalybuka, eugenis

Tags: #sanitizers, #llvm, #clang

Differential Revision: https://reviews.llvm.org/D61923

llvm-svn: 362138
2019-05-30 19:45:32 +00:00
Nico Weber 4dd6a82e26 mac: Make ubsan test config look more like asan test config
In particular, don't call get_target_flags_for_arch() since that
will cause an error in some situations:

If DARWIN_iossim_ARCHS=i386;x86_64, DARWIN_osx_ARCHS=x86_64, and
DARWIN_iossym_SYSROOT isn't set (due to the simulator sysroot not being
available), then config-ix.cmake won't add i386 to COMPILER_RT_SUPPORTED_ARCH
but ubsan's test/CMakeLists.txt would call get_target_flags_for_arch()
with i386, which would then run into the error in
get_target_flags_for_arch().

Having these conditions isn't ideal. The background here is that we
configure our mac-hosted trunk bots all the same (so they all have the
same DARWIN_*_archs, and we don't easily know if a mac host bot is
targeting mac or ios at the place where we call cmake), but only the
ios-targeting bots have ios sysroots available.

This will hopefully unbreak that use case without impacting anything
else -- and it makes ubsan and asan test setup more alike.

llvm-svn: 362010
2019-05-29 18:54:28 +00:00
Kostya Serebryany 060f4b48d5 [libFuzzer] when using data-flow-trace (DFT) only load the DFT for the files present in the corpus
llvm-svn: 361579
2019-05-24 00:43:52 +00:00
Kostya Serebryany eac9a7830b [libFuzzer] remove the data-flow-trace (DFT) python scripts; their functionality is now part of libFuzzer proper; also write functions.txt to the disk only if this file doesn't exist yet
llvm-svn: 361452
2019-05-23 01:03:42 +00:00
Kostya Serebryany b7cc3d9953 [libFuzzer] automatically collect the data flow trace (DFT) in the fork mode if -collect_data_flow= is given
llvm-svn: 361448
2019-05-23 00:22:46 +00:00
Pavel Labath 269340f1cf [Sanitizer] Add interceptor for wcsdup
Summary: The wide-string equivalent of strdup. Implementation trivial.

Reviewers: vitalybuka, eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D62189

llvm-svn: 361357
2019-05-22 08:34:56 +00:00
Matt Morehouse 4d7a6142de [libFuzzer] Sleep after process exits in merge-sigusr.test.
Ensure that log file has been fully updated before trying to read it.

llvm-svn: 361339
2019-05-22 00:41:54 +00:00
Matt Morehouse 9e0f6cc3a3 [libFuzzer] Kill by session ID in merge-sigusr.test.
Ensures that parent and all child processes are killed at once.

llvm-svn: 361336
2019-05-21 23:54:39 +00:00
Matt Morehouse db62d375dc [libFuzzer] Ignore exit status of wait in merge-sigusr.test.
If process $PID has already exited, wait will give a non-zero exit
status.

llvm-svn: 361326
2019-05-21 22:48:40 +00:00
Matt Morehouse df17ddf9fc [libFuzzer] Reduce flakiness of merge-sigusr.test.
Double the number of files to merge, and use wait instead of sleep.

llvm-svn: 361313
2019-05-21 21:15:51 +00:00
Matt Morehouse ef7e4d530c [libFuzzer] Disable fork-sigusr.test on AArch64.
Test fails on the clang-cmake-aarch64-lld build and I'm not sure why.

llvm-svn: 361185
2019-05-20 18:38:58 +00:00
Matt Morehouse 2fd318e543 [libFuzzer] Dump input on failure for sigusr tests.
Should help with debugging failures on the bots.

llvm-svn: 361070
2019-05-17 19:33:31 +00:00
Evgeniy Stepanov bf161e6783 [hwasan] Limit try-catch tests to aarch64.
HWASan C++ tests do not do well on x86_64.
Fixes https://bugs.llvm.org/show_bug.cgi?id=41923

llvm-svn: 361063
2019-05-17 18:40:06 +00:00
Matt Morehouse 012ef1cca7 [libFuzzer] Use SleepOneSecondTest.cpp for fork-sigusr.test.
ShallowOOMDeepCrash.cpp may hit libFuzzer's RSS limit before the SIGUSR2
is delivered, causing the test to be flaky when bots are under load.
SleepOneSecondTest.cpp will keep running until the signal is delivered.

llvm-svn: 361048
2019-05-17 16:56:01 +00:00
Evgeniy Stepanov 7f281b2c06 HWASan exception support.
Summary:
Adds a call to __hwasan_handle_vfork(SP) at each landingpad entry.

Reusing __hwasan_handle_vfork instead of introducing a new runtime call
in order to be ABI-compatible with old runtime library.

Reviewers: pcc

Subscribers: kubamracek, hiraditya, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61968

llvm-svn: 360959
2019-05-16 23:54:41 +00:00
Julian Lettner a59dad920f [TSan][libdispatch] Use FileCheck's --implicit-check-not=...
Use FileCheck's --implicit-check-not='ThreadSanitizer' which increases
the strictness of our tests a bit. NFC.

```
CHECK: start
CHECK-NOT: ThreadSanitizer
CHECK: done
```

With --implicit-check-not, the above is turned into:

```
CHECK-NOT: ThreadSanitizer
CHECK: start
CHECK-NOT: ThreadSanitizer
CHECK: done
CHECK-NOT: ThreadSanitizer
```

llvm-svn: 360927
2019-05-16 18:57:36 +00:00
Matt Morehouse 8779b74db1 [libFuzzer] Disable merge-sigusr.test on linux.
Make buildbot green while I rethink the test.

llvm-svn: 360914
2019-05-16 16:42:45 +00:00
Yvan Roux eff622b23c [crt] Mark dso_handle test as xfailing on ARM.
This is a temporary action to fix the bots.

llvm-svn: 360873
2019-05-16 11:13:49 +00:00
Matt Morehouse f2669eebd5 [libFuzzer] Increase merge-sigusr sleep after sending signal.
Test is flaky on buildbot at least partially due to the fuzz target not
exiting before we read its output.

llvm-svn: 360848
2019-05-16 04:00:41 +00:00
Matt Morehouse d5529629fb [libFuzzer] Also kill parent process in merge-siguser.test.
llvm-svn: 360840
2019-05-16 03:04:44 +00:00
Matt Morehouse f9d382946f [libFuzzer] Fix typo in merge-sigusr.test.
llvm-svn: 360836
2019-05-16 01:56:11 +00:00
Matt Morehouse ab10de8bab [libFuzzer] Use PID to send signals rather than process name.
pkill reads the process name as a pattern, not a raw name.  This means
that if the process name contains + or other regex characters, pkill
fails.

llvm-svn: 360835
2019-05-16 01:32:39 +00:00
Matt Morehouse a05ffdbfc7 [libFuzzer] Echo fuzzer output on sigusr tests.
Improves debuggability when the fuzz target crashes.

llvm-svn: 360824
2019-05-15 22:26:48 +00:00
Kostya Serebryany 27d22b6b7a [libFuzzer] reimplement DFT's collect_data_flow inside libFuzzer so that we don't need external python scripts
llvm-svn: 360712
2019-05-14 21:47:35 +00:00
Mitch Phillips c9dd299736 [GWP-ASan] Initial build files, implementation of PRNG [1].
Summary:
See D60593 for further information.
This patch slices off the PRNG implementation and the initial build files for GWP-ASan.

Reviewers: vlad.tsyrklevich, morehouse, vitalybuka

Reviewed By: morehouse

Subscribers: srhines, kubamracek, mgorny, #sanitizers, llvm-commits, cryptoad, eugenis

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61867

llvm-svn: 360710
2019-05-14 21:43:11 +00:00
Matt Morehouse 3478494c1f [libFuzzer] Unpoison parameters before calling user callback.
Summary:
Fixes an MSan false positive when compiling with
-fsanitize=memory,fuzzer.

See https://github.com/google/oss-fuzz/issues/2369 for more details.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, metzman, eugenis

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61753

llvm-svn: 360390
2019-05-09 22:48:46 +00:00
Julian Lettner a335d85763 [TSan][libdispatch] Enable test that supposedly deadlocks on bot
Re-enable test that was disabled because it deadlocks when running on
the bot, but was never enabled again. Can't reproduce deadlock locally
so trying to investigate by re-enabling test.

llvm-svn: 360388
2019-05-09 22:47:19 +00:00
Kostya Serebryany 4a5793f7d0 [libFuzzer] perform more agressive value profiling in memcmp
llvm-svn: 360385
2019-05-09 22:09:25 +00:00
Matt Morehouse a612b5adb7 [MSan] Introduce __msan_unpoison_param().
Summary:
This allows libFuzzer to unpoison parameter shadow before calling
LLVMFuzzerTestOneInput to eliminate the false positives described
in https://github.com/google/oss-fuzz/issues/2369.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: llvm-commits, metzman, kcc

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61751

llvm-svn: 360379
2019-05-09 21:50:57 +00:00
Kostya Serebryany e9aaa5582f [libFuzzer] implement -focus_function=auto, to be used with Data Flow Traces
llvm-svn: 360378
2019-05-09 21:29:45 +00:00
Kostya Serebryany 194b1c3078 [libFuzzer] simplify value-profile-mem.test a little bit
llvm-svn: 360372
2019-05-09 20:20:36 +00:00
Kostya Serebryany e13eff293d [libFuzzer] DFT: when dumping coverage, also dump the total number of instrumented blocks in a function; update merge_data_flow.py to merge coverage
llvm-svn: 360272
2019-05-08 17:20:09 +00:00
Nico Weber a3ff5727b7 [compiler-rt] Make builtins test pass when using i386 gcc as host compiler
Just-built-clang is used to compile the test, but the library is built
with gcc, so the usual 80-bit FPU vs 32-bit SSE mismatch makes the
floating computations not bitwise identical. Fixes PR32910, see there
for details.

This uses the same technique used in all the other *c3* tests, see in
particular mulsc3_test.c.

(It might be cleaner to add compareResultCF to fp_test.h to force the
floats into 32-bit in memory, but this is the less invasive fix.)

Differential Revision: https://reviews.llvm.org/D61684

llvm-svn: 360264
2019-05-08 15:50:21 +00:00
Kostya Serebryany ba670b404e [libFuzzer] extend the test for data flow tracer and coverage; also hopefully fix it on the bot
llvm-svn: 360215
2019-05-08 01:03:05 +00:00
Kostya Serebryany 219b2b3a4a [libFuzzer] extend the data flow tracer to also produce basic block coverage for every input. An extended test coming in a separte change.
llvm-svn: 360213
2019-05-08 00:51:15 +00:00
Kostya Serebryany a27a0914d3 [libFuzzer] disable two tests on i386 that are causing timeouts on the bots
llvm-svn: 360211
2019-05-08 00:43:12 +00:00
Peter Smith 3f585ae3ce [libFuzzer] Increase timeouts on fork tests and skip one on aarch64
The tests fork.text, fork.sigusr.test and fork-ubsan.test intermittently
fail on the aarch64 buildbots. Input gathered from the fork.sigusr.test
implies that when the builder is under load the timeout value is not
sufficient. The fork-ubsan.test doesn't have a timeout and I think is not
always finding the error after 10000 runs so I've marked it as unsupported
for now.

Differential Revision: https://reviews.llvm.org/D61449

llvm-svn: 360126
2019-05-07 09:31:14 +00:00
Xing Xue 865a39d328 Add libc++ to link XRay test cases if libc++ is used to build CLANG
Summary: When libc++ is used to build CLANG, its XRay libraries libclang_rt.xray-*.a have dependencies on libc++. Therefore, libc++ is needed to link and run XRay test cases. For Linux -rpath is also needed to specify where to load libc++. This change sets macro LLVM_LIBCXX_USED to 1 if libc++ is actually used in the build. XRay tests then check the flag and add -L<llvm_shlib_dir> -lc++ and -Wl,-rpath=<llvm_shlib_dir> if needed.

Reviewers: hubert.reinterpretcast, amyk, dberris, jasonliu, sfertile, EricWF

Subscribers: dberris, mgorny, jsji, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61016

llvm-svn: 360060
2019-05-06 17:45:21 +00:00
Reid Kleckner 3961507ba1 Fix check-builtins on Windows after alias changes
llvm-svn: 359835
2019-05-02 22:11:55 +00:00
Jonathan Metzman 3d1d3ad50e [libFuzzer] Re-enable libFuzzer on i386 Linux and fix test
Summary:
Re-enable libFuzzer on i386 Linux after it was accidentally
disabled.

Also disable gc-sections.test on i386 since lld isn't
garbage collecting properly with ASAN on i386.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: srhines, mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61415

llvm-svn: 359802
2019-05-02 16:45:17 +00:00
Vitaly Buka fbcec6cad0 [tsan] Fix and re-enable user_malloc.cc test
Summary: no_sanitize_thread is not enough as it still puts some tsan instrumentation

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61393

llvm-svn: 359731
2019-05-01 21:53:39 +00:00
Evgeniy Stepanov d1a710047b [sanitizer] Implement reallocarray.
Summary:
It's a cross of calloc and realloc. Sanitizers implement calloc-like check for size
overflow.

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61108

llvm-svn: 359708
2019-05-01 17:33:01 +00:00
Peter Smith 101bf520d1 [libFuzzer] Add --dump-input-on-failure to help diagnose AArch64 failures
The fork-siguser.test and fork.test intermittently fail on the AArch64
buildbot. Unfortunately these failures are not reproducible on a similar
machine and seem to fail when the machines are under load. Before
suggesting the tests be marked unsupported for AArch64 we'd like to see
if we can get some more information about the failures to see if it helps
us reproduce. This patch adds --dump-input-on-failure to the FileCheck
commands to see if we can get some more information about the failures.

Differential Revision: https://reviews.llvm.org/D61315

llvm-svn: 359675
2019-05-01 12:30:04 +00:00
Jonathan Metzman c0806e0d24 [libFuzzer] Fix failing test: sigint.test
Summary:
Fix sigint.test by making it require msan rather
than enumerating unsupported platforms.

Reviewers: kcc

Reviewed By: kcc

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61353

llvm-svn: 359638
2019-04-30 23:46:52 +00:00
Kostya Serebryany 905e60c422 [libFuzzer] temporarily disable a test on windows, where there is no memmem in the usual place
llvm-svn: 359624
2019-04-30 22:27:38 +00:00
Kostya Serebryany c239eda8a0 [libFuzzer] add MagicSeparatorTest
llvm-svn: 359620
2019-04-30 22:05:55 +00:00
Jonathan Metzman f3ee97731e [libFuzzer] Replace -seed_corpus to better support fork mode on Win
Summary:
Pass seed corpus list in a file to get around argument length limits on Windows.
This limit was preventing many uses of fork mode on Windows.

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60980

llvm-svn: 359610
2019-04-30 20:56:18 +00:00
Amy Kwan 31dbbf1df9 [compiler-rt][builtins][sanitizers] Update compiler-rt test cases for
compatibility with system's toolchain

This patch aims to:
- Guard ompiler-rt/test/builtins/Unit/compiler_rt_logb_test.c with macros, so
the test runs on GLIBC versions >= 2.23. This is because the test relies on
comparing its computed values to libm. Oolder versions might not compute to the
same value as the compiler-rt value.
- Update compiler-rt/test/sanitizer_common/TestCases/Posix/getpw_getgr.cc
so that std::string is not used, since false positives may be detected.

Differential Revision: https://reviews.llvm.org/D60644

llvm-svn: 359606
2019-04-30 20:09:00 +00:00
Petr Hosek 999bb5ac27 Reland "[compiler-rt] Simple crtbegin.o and crtend.o implementation"
Clang relies on existence of certain symbols that are normally
provided by crtbegin.o/crtend.o. However, LLVM does not currently
provide implementation of these files, instead relying on either
libgcc or implementations provided as part of the system.

This change provides an initial implementation of crtbegin.o/crtend.o
that can be used on system that don't provide crtbegin.o/crtend.o as
part of their C library.

Differential Revision: https://reviews.llvm.org/D28791

llvm-svn: 359591
2019-04-30 18:13:22 +00:00
Jonathan Metzman 7a2ce3790b fix broken test
llvm-svn: 359590
2019-04-30 17:58:59 +00:00
Jonathan Metzman 17bd74d406 remove extra zeros
llvm-svn: 359589
2019-04-30 17:58:58 +00:00
Jonathan Metzman 5a271cd758 remove stale comment
llvm-svn: 359588
2019-04-30 17:58:58 +00:00
Jonathan Metzman 1fbc6116e1 fix comment
llvm-svn: 359586
2019-04-30 17:58:57 +00:00
Jonathan Metzman 2697664582 [libFuzzer] Enable for i386
Summary: Get libFuzzer to build on i386 and fix tests.

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61070

llvm-svn: 359585
2019-04-30 17:58:56 +00:00
Jonathan Metzman 6fa864b7a6 Fix issues with testing for i386
llvm-svn: 359584
2019-04-30 17:58:55 +00:00
Jonathan Metzman f721230831 Enable x86 builds
llvm-svn: 359583
2019-04-30 17:58:54 +00:00
Petr Hosek 9300f60c8c Revert "[compiler-rt] Simple crtbegin.o and crtend.o implementation"
This reverts commit r359576 since it's failing on Windows bots.

llvm-svn: 359579
2019-04-30 17:32:05 +00:00
Petr Hosek c8be6e670e [compiler-rt] Simple crtbegin.o and crtend.o implementation
Clang relies on existence of certain symbols that are normally
provided by crtbegin.o/crtend.o. However, LLVM does not currently
provide implementation of these files, instead relying on either
libgcc or implementations provided as part of the system.

This change provides an initial implementation of crtbegin.o/crtend.o
that can be used on system that don't provide crtbegin.o/crtend.o as
part of their C library.

Differential Revision: https://reviews.llvm.org/D28791

llvm-svn: 359576
2019-04-30 17:21:13 +00:00
Matthew G McGovern d62416dfcd [AddressSanitizer] [Windows] Fix HeapReAlloc and _recalloc bugs in asan_malloc_win.cc
HeapReAlloc should allow for 0 sized reallocations without freeing the memory block provided by the user.

_recalloc previously did not zero new memory after reallocation.
https://reviews.llvm.org/D61268

llvm-svn: 359498
2019-04-29 20:26:19 +00:00
Dan Liew 8651edf898 [CMake] Don't modify `FUZZER_SUPPORTED_ARCH` is place.
On a Darwin host we were modifying the `FUZZER_SUPPORTED_ARCH` in place
which would strip out non-x86 architectures. This unhelpful if we
want to use `FUZZER_SUPPORTED_ARCH` later.

To fix this we introduce `FUZZER_TEST_ARCH` which is similar to what we
have for for the other sanitizers. For non-Darwin host platforms
`FUZZER_TEST_ARCH` is the same as `FUZZER_SUPPORTED_ARCH` but for Darwin
host platforms we use `darwin_filter_host_archs(...)` as the previous
code did.

llvm-svn: 359394
2019-04-28 09:44:53 +00:00
Dan Liew 18bc872405 Revert "[CMake] Fix the value of `config.target_cflags` for non-macOS Apple"
This reverts commit 1bcdbd68616dc7f8debe126caafef7a7242a0e6b.

It's been reported that some bots are failing with this change with CMake
error like:

```
CMake Error at /b/s/w/ir/k/llvm-project/compiler-rt/cmake/config-ix.cmake:177 (message):
  Unsupported architecture: arm64
Call Stack (most recent call first):
  /b/s/w/ir/k/llvm-project/compiler-rt/cmake/config-ix.cmake:216 (get_target_flags_for_arch)
  /b/s/w/ir/k/llvm-project/compiler-rt/test/tsan/CMakeLists.txt:78 (get_test_cflags_for_apple_platform)
```

I'm reverting the patch now to unbreak builds. I will investigate properly when time permits.

rdar://problem/50124489

llvm-svn: 359327
2019-04-26 17:53:25 +00:00
Dan Liew 60f5df948b [asan_symbolize] Teach `asan_symbolize.py` to symbolicate using a module map
Summary:
The use case here is to be able get the UUIDs of the modules that need
to be symbolicated so that external plugins can see them. This
information can be extracted from ASan reports if the `print_module_map`
ASan option is enabled. Currently printing of the module map is only
implemented on Darwin and so this is effectively a Darwin only feature
right now.

The module map hooks into symbolization using the new plugin
infrastructure. A new hook in `AsanSymbolizerPlugInProxy` (and in
`AsanSymbolizerPlugIn`) is also provided to allow external plugins to hook
into the module look up process. This will allow external plugins to
look up modules with knowledge of their UUID.

The new plug-in is currently stored in the `asan_symbolize.py` script.
We could potentially move this into a separate file in the future (to
reduce clutter) if we can come up with a policy for where to search for
plugins that should always get loaded.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60531

llvm-svn: 359322
2019-04-26 16:54:09 +00:00
Dan Liew 9f59704a5d [CMake] Fix the value of `config.target_cflags` for non-macOS Apple
platforms.

The main problem here is that `-*-version_min=` was not being passed to
the compiler when building test cases. This can cause problems when
testing on devices running older OSs because Clang would previously
assume the minimum deployment target is the the latest OS in the SDK
which could be much newer than what the device is running.

Previously the generated value looked like this:

`-arch arm64 -isysroot
<path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`

With this change it now looks like:

`-arch arm64 -stdlib=libc++ -miphoneos-version-min=8.0 -isysroot
<path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`

This mirrors the setting of `config.target_cflags` on macOS.

This change is made for ASan, LibFuzzer, TSan, and UBSan.

To implement this a new `get_test_cflags_for_apple_platform()` function
has been added that when given an Apple platform name and architecture
returns a string containing the C compiler flags to use when building
tests. This also calls a new helper function `is_valid_apple_platform()`
that validates Apple platform names.

rdar://problem/50124489

Differential Revision: https://reviews.llvm.org/D58578

llvm-svn: 359305
2019-04-26 13:22:39 +00:00
Matt Morehouse 1b76063a5e [libFuzzer] Disable MSan interceptors in SIGINT handler.
Summary:
Avoids an MSan false positive if the SIGINT comes while the user
callback is running.  The false positive happens when the interrupt
handler calls opendir() to remove some temporary files, which is
intercepted by MSan.

Fixes https://github.com/google/oss-fuzz/issues/2332.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, Dor1s, metzman

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61163

llvm-svn: 359254
2019-04-26 00:17:41 +00:00
Matt Morehouse e9640afddd [libFuzzer] Require linux for libcxx.test
llvm-svn: 359223
2019-04-25 18:42:30 +00:00
Matt Morehouse 42ef2c6d06 [compiler-rt] Build custom libc++abi without exceptions.
Summary:
Since neither compiler-rt nor the libc++ we build use exceptions, we
don't need libc++abi to have them either.

This resolves an issue where libFuzzer's private libc++ contains
implementations for __cxa_throw and friends, causing fuzz targets built
with their own C++ library to segfault during exception unwinding.

See https://github.com/google/oss-fuzz/issues/2328.

Reviewers: phosek, EricWF, kcc

Reviewed By: phosek

Subscribers: kcc, dberris, mgorny, christof, llvm-commits, metzman

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61053

llvm-svn: 359218
2019-04-25 18:14:24 +00:00
Adhemerval Zanella 91cee68e1f [fuzzer] Fix reload.test on Linux/aarch64
The compiler generates a 'brk' instruction for __builtin_trap on aarch64
and Linux kernel issues a SIGTRAP. It is different from x86, where
compiler emits an 'ud2' and kernel issues a SIGILL.

A straightforward is to use abort instead.

llvm-svn: 359126
2019-04-24 19:02:54 +00:00
Julian Lettner 4d2b9426b9 [TSan] Support fiber API on macOS
Committing on behalf of Yuri Per (yuri).

Reviewers: dvyukov, kubamracek, yln

Reviewed By: kubamracek

Authored By: yuri

Differential Revision: https://reviews.llvm.org/D58110

llvm-svn: 358802
2019-04-20 00:18:44 +00:00
Mitch Phillips f4878d8905 Removed regiser dump checks from FP unit test.
llvm-svn: 358770
2019-04-19 17:36:58 +00:00
Mitch Phillips aa68f946dc [HWASan] Added no-FP unit test for register dump.
Summary: Unit test for D60798.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60806

llvm-svn: 358769
2019-04-19 17:36:56 +00:00
Mitch Phillips d1a18469a8 Updated test case.
llvm-svn: 358768
2019-04-19 17:36:55 +00:00
Mitch Phillips 2c3cecef3d Added no-FP unit test for register dump.
llvm-svn: 358767
2019-04-19 17:36:53 +00:00
Jonathan Metzman 139e216e66 Summary:
Add close_fd_mask functionality to AFL driver.

Summary:
Add support for env var AFL_DRIVER_CLOSE_FD_MASK which behaves
the same as libFuzzer's -close_fd_mask=1.

Also add tests.

Reviewers: kcc, vitalybuka, morehouse

Reviewed By: morehouse

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60334

llvm-svn: 358703
2019-04-18 18:49:11 +00:00
Dan Liew 403d3187a7 [asan_symbolize] Add a simple plugin architecture
Summary:
This change adds a simple plugin architecture to `asan_symbolize.py`.
The motivation here is that sometimes it's necessary to perform extra
work to figure out where binaries with debug symbols can actually be
found. For example it might be the case that a remote service needs
to be queried for binaries and then copied to the local system.

This "extra work" can be extremely site-specific such that adding the
code directly into the `asan_symbolize.py` would just clutter the code
for a very niche use case. To avoid this, the `asan_symbolize.py` can
now load external code via a new `--plugins` command line option.

These plugins are loaded before main command line argument parsing so
that they can add their own command line options.

Right now the only hook into the behaviour of symbolization is the
`filter_binary_path()` function which assumes a very similar role
to the `binary_name_filter` function that was previously in the code.
We can add more hooks as necessary.

Code in the `asan_symbolize.py` script does not call plugin code
directly. Instead it uses a `AsanSymbolizerPlugInProxy` object.
This object

* Loads plugins from files.
* Manages the lifetime of the plugins.
* Provides an interface for calling into plugin functions and handles
  calling into multiple plugins.

To unify the way binary paths are filtered the old `sysroot_path_filter`
function (and associated code) has been turned into a simple plugin
(`SysRootFilterPlugIn`) that is always loaded. The plugin unloads
itself if the `-s` option is not present on the command line. Users
should not see any functional change relating to this command line
option.

Some simple tests are provided to illustrate what plugin code looks
like and also to check the functionality continues to work.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: srhines, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60529

llvm-svn: 358657
2019-04-18 11:34:31 +00:00
Julian Lettner 2632643454 [TSan][libdispatch] Port gcd-sync-block-copy.mm to C++
Summary:
Apparently, it makes a difference on where a block lives depending on if
it's passed "inline" versus assigned and then passed via a variable.
Both tests in this commit now give a signal, if `Block_copy` is used in
`dispatch_sync`.

Since these tests use different mechanisms (Objective-C retain versus
C++ copy constructor) as proxies to observe if the block was copied, we
should keep both of them.

Commit, that first avoided the unnecessary copy:
faef7d034a

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60639

llvm-svn: 358469
2019-04-16 01:34:38 +00:00
Kostya Serebryany 4614cc3dfd [libFuzzer] add -features_dir= flag to dump unique input features on disk
llvm-svn: 358317
2019-04-13 00:20:31 +00:00
Julian Lettner c77bf89dcc [TSan][libdispatch] Move libdispatch tests out of Darwin folder
The libdispatch tests are now not tied to Darwin anymore. Move them to
their own platform-independent folder.

llvm-svn: 358316
2019-04-12 23:07:00 +00:00
Max Moroz b6e6d3c740 [libFuzzer] Fix DataFlow.cpp logic when tracing long inputs.
Summary:
1. Do not create DFSan labels for the bytes which we do not trace. This is where we run out of labels at the first place.
2. When dumping the traces on the disk, make sure to offset the label identifiers by the number of the first byte in the trace range.
3. For the last label, make sure to write it at the last position of the trace bit string, as that label represents the input size, not any particular byte.

Also fixed the bug with division in python which I've introduced when migrated the scripts to Python3 (`//` is required for integral division).

Otherwise, the scripts are wasting too much time unsuccessfully trying to
collect and process traces from the long inputs. For more context, see
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481761789

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60538

llvm-svn: 358311
2019-04-12 21:00:12 +00:00
Julian Lettner 6636978243 [TSan][libdispatch] Don't link against Foundation
Now that our tests don't depend on Foundation anymore,
don't link it in.

llvm-svn: 358309
2019-04-12 20:27:05 +00:00
Julian Lettner bfcbd677c8 [TSan][libdispatch] Replace NSTemporaryDirectory in tests
After this change, most tests don't have a dependency on Foundation.

Note: To hold the file name `tempnam` allocates a new buffer. We leak
      this buffer (omit the free), but I don't think we need to care.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60591

llvm-svn: 358308
2019-04-12 20:27:02 +00:00
Kostya Serebryany 65132e2118 [libFuzzer] support -runs=N in the fork mode. Make sure we see one-line reports from ubsan in the fork mode. Test both
llvm-svn: 358306
2019-04-12 20:20:57 +00:00
Julian Lettner 43e27eb87f [TSan][libdispatch] Fix failing test
The ThreadSanitizer-x86_64-iossim configuration (different SDK) seems to
require an additional #include.

rdar://49856637

llvm-svn: 358300
2019-04-12 18:31:09 +00:00
Julian Lettner 2edfcf9065 [TSan][libdispatch] Delete old tests
In a previous commit, I re-enabled the ported variants of these 2 tests:
tsan/Darwin/gcd-data.mm -> tsan/libdispatch/data.c
tsan/Darwin/gcd-source-serial.mm -> tsan/libdispatch/source-serial.c

So now we can delete the Darwin-only version.

llvm-svn: 358235
2019-04-11 22:27:57 +00:00
Julian Lettner 1740c6e0da [TSan][libdispatch] Re-enable disabled tests
I re-activated "broad strokes suppressions"
(ignore_noninstrumented_modules=1) in my last commit. Re-enable tests
that only fail on our bots to check if they work now.

llvm-svn: 358209
2019-04-11 18:34:47 +00:00
Julian Lettner 5b981638fb [TSan][libdispatch] Turn ignore_noninstrumented_modules=1 back on for Darwin
llvm-svn: 358208
2019-04-11 18:34:44 +00:00
Julian Lettner 3e58f94ced [TSan][libdispatch] Replace usage of NSMutableData with stack array
Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60477

llvm-svn: 358205
2019-04-11 18:14:19 +00:00
Julian Lettner edea731af2 [TSan][libdispatch] Change test to have two simultaneous timers
Change test semantics by waiting for both timer callbacks at the end
instead of serializing operations: start/wait timer 1 then 2.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60476

llvm-svn: 358204
2019-04-11 18:14:16 +00:00
Julian Lettner 0f7a0c71c6 [TSan][libdispatch] Replace CFRunLoop with dispatch_semaphore, pt. 2
Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60475

llvm-svn: 358203
2019-04-11 18:14:13 +00:00
John McCall 98da442b6d Prospective test fix in response to r358104.
llvm-svn: 358149
2019-04-11 03:04:04 +00:00
JF Bastien 2f46de8c0b [NFC] Use clearer naming for local variables
llvm-svn: 358145
2019-04-10 23:23:17 +00:00
Julian Lettner 19f52bde59 [TSan][libdispatch] Replace CFRunLoop with dispatch_semaphore, pt. 1
Remove the dependency on Foundation so we can start running those tests
on other platforms. Rename/move of tests will be done in a separate
commit.

Reviewed By: kubamracek, dvyukov

Differential Revision: https://reviews.llvm.org/D60347

llvm-svn: 358023
2019-04-09 17:51:55 +00:00
Peter Collingbourne df57979ba7 hwasan: Enable -hwasan-allow-ifunc by default.
It's been on in Android for a while without causing problems, so it's time
to make it the default and remove the flag.

Differential Revision: https://reviews.llvm.org/D60355

llvm-svn: 357960
2019-04-09 00:25:59 +00:00
Dan Liew 836da99af9 [asan_symbolize] Use proper logging infrastructure.
Summary:
The previous logging infrastructure had several problems:

* Debugging output was emitted to standard output which is also where
  the symbolized output would go. Interleaving these two separate
  bits of information makes inspecting the output difficult and could
  potentially break tests.

* Enabling debugging output requires modifying the script which is
  not very conveninent.

* When debugging it isn't immediately obvious where the output is
  coming from.

This patch uses the Python standard library logging infrastructure
which fixes all of the above problems. Logging is controlled using
two new options.

* `--log-level` - Sets the logging level, default is
`info`.
* `--log-dest` - Set the logging destination, default
is standard error.

Some simple test cases for the feature are included.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60343

llvm-svn: 357951
2019-04-08 22:28:27 +00:00
Pavel Labath 19cad6f089 [Sanitizer] Make wcrtomb test posix-only
wcrtomb is not intercepted on windows, so this test fails there. It's
not clear to me why we do not intercept this function there (I'll look
into that separately), but for now this should at least make the windows
sanitizer bot green again (broken by r357889, when I added this test).

I also add "UNSUPPORTED: android" as this function is also not
intercepted there.

llvm-svn: 357892
2019-04-08 09:13:13 +00:00
Pavel Labath 618dcfcaa3 [Sanitizer] Fix a possible write to freed memory in the wcrtomb interceptor
Summary:
r357240 added an interceptor for wctomb, which uses a temporary local
buffer to make sure we don't write to unallocated memory. This patch
applies the same technique to wcrtomb, and adds some additional tests
for this function.

Reviewers: vitalybuka, eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D59984

llvm-svn: 357889
2019-04-08 08:39:50 +00:00
Julian Lettner 18bc982c06 [TSan][libdispatch] Remove Darwin-only version of fully-ported tests
Remove 10 tests that already have a copy in tsan/libdispatch, without
dependencies on Darwin-specifis.

llvm-svn: 357832
2019-04-06 01:41:40 +00:00
Vitaly Buka 3db6ad2bcf Use binary write mode in WriteToFile function to avoid appended \r characters on Windows
Summary:
When using libfuzzer on Windows, in the contents of a crash sample, bytes that can be mistaken for a \n are replaced by a \r\n sequence. As a consequence, crashes are not reproducible. This patch will open files in binary mode to fix this issue. The patch does not affect POSIX systems.

Patch by tuktuk

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: dexonsmith, jdoerfert, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60008

llvm-svn: 357807
2019-04-05 20:17:03 +00:00
Julian Lettner 7d1ec7b07e [TSan][libdispatch] Make test work on Linux, pt. 2
llvm-svn: 357741
2019-04-05 02:05:57 +00:00
Julian Lettner be09db001a [TSan][libdispatch] Make test work on Linux
llvm-svn: 357729
2019-04-04 21:55:40 +00:00
Julian Lettner 13dace5d5d [TSan][libdispatch] Stricter checks via `--implicit-check-not`
`--implicit-check-not='ThreadSanitizer'` checks in the entire output
while `// CHECK-NOT: ThreadSanitizer` only checks after (before) the
previous (next) match.

llvm-svn: 357727
2019-04-04 21:25:56 +00:00
Julian Lettner f7461b8c5f [TSan][libdispatch] Specify libdispatch header dir for lit tests
Specify libdispatch header dir (include path) for lit tests. This is the
last missing piece in order to run the libdispatch tests on Linux even
when libdispatch is installed in a custom path instead of a default
(system) location.

llvm-svn: 357707
2019-04-04 17:25:43 +00:00
Julian Lettner b8c6effacf [NFC][TSan][libdispatch] Better CMake variable names
llvm-svn: 357664
2019-04-04 01:10:07 +00:00
Xing GUO 884c29e9ae Fix typos in tests. NFC.
Reviewers: Higuoxing

Reviewed By: Higuoxing

Subscribers: kubamracek, cfe-commits, #sanitizers, llvm-commits

Tags: #clang, #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60183

llvm-svn: 357577
2019-04-03 13:15:09 +00:00
Julian Lettner fc5a176f6a [NFC][TSan][libdispatch] Cleanup test
llvm-svn: 357530
2019-04-02 20:58:49 +00:00
Julian Lettner 226b992ebd [NFC] Test is C++, not C
llvm-svn: 357524
2019-04-02 20:39:10 +00:00
Julian Lettner 3a23fb1e04 [NFC][libdispatch] Improve a few tests
llvm-svn: 357510
2019-04-02 18:46:59 +00:00
Hans Wennborg 76f2259dde Fix tests after r357452
llvm-svn: 357462
2019-04-02 10:06:39 +00:00
Matt Davis 60a8816c2f [compiler-rt][test] Make instrprof-set-dir-mode test tolerant of group ID
Patch from 'troyj':
Hi, I ran into a problem with this test when the source was located in
certain directories. The mkdir(2) man page states that the set-group-ID
bit is inherited from the parent directory, but this test was written in
such a way that it assumes the bit is unset. Whether that assumption is
true depends on where the checkout lives, which leads to some people
being able to reproduce the problem whereas others cannot. I think the
correct fix is to exclude the bit from the check.

Making probinson a reviewer since they reviewed the original test.

Patch landed for troyj, thanks!

Differential Revision: D53832

llvm-svn: 357449
2019-04-02 03:24:12 +00:00
Pavel Labath 711bea9ded [Sanitizer] Add interceptor for wctomb
Summary:
This is required to avoid msan false positives for code using this
function (although generally one should avoid using this function in
favor of wcrtomb).

Reviewers: eugenis, EricWF, vitalybuka

Subscribers: srhines, kubamracek, fedor.sergeev, delcypher, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D59548

llvm-svn: 357240
2019-03-29 08:39:03 +00:00
Yi Kong e204d244ba Revert "[builtins] Rounding mode support for addxf3/subxf3"
This reverts commit 2cabea054e.

Test failure on buildbots.

llvm-svn: 357048
2019-03-27 04:18:37 +00:00
Yi Kong 2cabea054e [builtins] Rounding mode support for addxf3/subxf3
Implement rounding mode support for addxf3/subxf3.

On architectures that implemented the support, this will access the
corresponding floating point environment register to apply the
correct rounding. For other architectures, it will keep the current
behaviour and use IEEE-754 default rounding mode (to nearest, ties
to even).

ARM32/AArch64 support implemented in this change. i386 and AMD64
will be added in a follow up change.

Differential Revision: https://reviews.llvm.org/D57143

llvm-svn: 357035
2019-03-26 22:01:22 +00:00
Julian Lettner 1955c8f172 [NFC][TSan][libdispatch] Don't use ignore_noninstrumented_modules on Linux
llvm-svn: 356728
2019-03-22 01:43:43 +00:00
Dan Liew 5988d72243 Enable `asan/TestCases/Posix/start-deactivated.cc` test for iOS.
Summary:
To make this test pass it was necesary to change `iossim_run.py` to
propagate the `ASAN_ACTIVATION_OPTIONS` environment variable into the
testing environment.

rdar://problem/49114807

Reviewers: kubamracek, yln, serge-sans-paille

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D59660

llvm-svn: 356701
2019-03-21 19:20:44 +00:00
Eli Friedman d674d96bc5 [builtins] Divide shouldn't underflow if rounded result would be normal.
We were treating certain edge cases that are actually normal as denormal
results, and flushing them to zero; we shouldn't do that. Not sure this
is the cleanest way to implement this edge case, but I wanted to avoid
adding any code on the common path.

Differential Revision: https://reviews.llvm.org/D59070

llvm-svn: 356529
2019-03-19 21:55:58 +00:00
Reid Kleckner ed350f73c1 [asan] Disable -Wfortify-source in intentional OOB tests
Needed after r356397

llvm-svn: 356426
2019-03-18 23:03:46 +00:00
Julian Lettner 1814355655 [NFC][TSan][libdispatch] Fix test for dispatch_apply[_f]
* Array index out of bounds: 100 iterations, but size of array is 2.
* Unmatched barrier_init (2) with barrier_wait (200)
* Number of iterations must be smaller than the available parallelism
  for the queue, otherwise we deadlock (since every barrier_wait call
  blocks the thread).

Scary: All of this worked reliably in gcd-apply.mm (for Darwin)

Rievewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D59510

llvm-svn: 356418
2019-03-18 21:55:41 +00:00
Julian Lettner 769b73738d [TSan][libdispatch] Configure libdispatch lit tests
llvm-svn: 356311
2019-03-16 02:07:50 +00:00
Julian Lettner 764c2165e8 [TSan][libdispatch] Enable linking and running of tests on Linux
When COMPILER_RT_INTERCEPT_LIBDISPATCH is ON the TSan runtime library
now has a dependency on the blocks runtime and libdispatch. Make sure we
set all the required linking options.

Also add cmake options for specifying additional library paths to
instruct the linker where to search for libdispatch and the blocks
runtime. This allows us to build TSan runtime with libdispatch support
without installing those libraries into default linker library paths.

`CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY` is necessary to avoid
aborting the build due to failing the link step in CMake's
check_c_compiler test.

Reviewed By: dvyukov, kubamracek

Differential Revision: https://reviews.llvm.org/D59334

llvm-svn: 356281
2019-03-15 17:52:27 +00:00
Julian Lettner 4d4f64cddd [NFC][TSan] Move libdispatch tests into their own subfolder
Remove 'gcd' file prefix. GCD stands for Grand Central Dispatch, which
is another name for libdispatch.
https://apple.github.io/swift-corelibs-libdispatch/

Remove `REQUIRE: dispatch` from tests.

Also rename lit feature 'dispatch' -> 'libdispatch' to be more explicit
what this is about.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D59341

llvm-svn: 356202
2019-03-14 20:59:41 +00:00
Evgeniy Stepanov aedec3f684 Remove ASan asm instrumentation.
Summary: It is incomplete and has no users AFAIK.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, mgorny, krytarowski, eraman, hiraditya, jdoerfert, #sanitizers, llvm-commits, thakis

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D59154

llvm-svn: 355870
2019-03-11 21:50:10 +00:00
Nico Weber 885b790f89 Remove esan.
It hasn't seen active development in years, and it hasn't reached a
state where it was useful.

Remove the code until someone is interested in working on it again.

Differential Revision: https://reviews.llvm.org/D59133

llvm-svn: 355862
2019-03-11 20:23:40 +00:00
Mitch Phillips 790edbc16e [HWASan] Save + print registers when tag mismatch occurs in AArch64.
Summary:
This change change the instrumentation to allow users to view the registers at the point at which tag mismatch occured. Most of the heavy lifting is done in the runtime library, where we save the registers to the stack and emit unwind information. This allows us to reduce the overhead, as very little additional work needs to be done in each __hwasan_check instance.

In this implementation, the fast path of __hwasan_check is unmodified. There are an additional 4 instructions (16B) emitted in the slow path in every __hwasan_check instance. This may increase binary size somewhat, but as most of the work is done in the runtime library, it's manageable.

The failure trace now contains a list of registers at the point of which the failure occured, in a format similar to that of Android's tombstones. It currently has the following format:

Registers where the failure occurred (pc 0x0055555561b4):
    x0  0000000000000014  x1  0000007ffffff6c0  x2  1100007ffffff6d0  x3  12000056ffffe025
    x4  0000007fff800000  x5  0000000000000014  x6  0000007fff800000  x7  0000000000000001
    x8  12000056ffffe020  x9  0200007700000000  x10 0200007700000000  x11 0000000000000000
    x12 0000007fffffdde0  x13 0000000000000000  x14 02b65b01f7a97490  x15 0000000000000000
    x16 0000007fb77376b8  x17 0000000000000012  x18 0000007fb7ed6000  x19 0000005555556078
    x20 0000007ffffff768  x21 0000007ffffff778  x22 0000000000000001  x23 0000000000000000
    x24 0000000000000000  x25 0000000000000000  x26 0000000000000000  x27 0000000000000000
    x28 0000000000000000  x29 0000007ffffff6f0  x30 00000055555561b4

... and prints after the dump of memory tags around the buggy address.

Every register is saved exactly as it was at the point where the tag mismatch occurs, with the exception of x16/x17. These registers are used in the tag mismatch calculation as scratch registers during __hwasan_check, and cannot be saved without affecting the fast path. As these registers are designated as scratch registers for linking, there should be no important information in them that could aid in debugging.

Reviewers: pcc, eugenis

Reviewed By: pcc, eugenis

Subscribers: srhines, kubamracek, mgorny, javed.absar, krytarowski, kristof.beyls, hiraditya, jdoerfert, llvm-commits, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D58857

llvm-svn: 355738
2019-03-08 21:22:35 +00:00
Manman Ren e73ae9a142 Reland compiler-rt support for order file instrumentation.
r355343 was landed and was reverted in r355363 due to build breakage.
This patch adds Linux/Windows support on top of r355343.

In this patch, Darwin should be working with testing case. Linux should be working,
I will enable the testing case in a follwup diff. Windows/Other should be building.
Correct implementation for Other platforms will be added.

Thanks David for reviewing the original diff, helping me with issues on Linux, and
giving suggestions for adding support for Other platforms.

llvm-svn: 355701
2019-03-08 15:30:56 +00:00
Julian Lettner ed77926f99 [TSan] Temporarily disable test which fails on build bot
llvm-svn: 355642
2019-03-07 21:42:48 +00:00
Julian Lettner 9d782f998d [TSan] Temporarily disable test which fails on build bot
llvm-svn: 355633
2019-03-07 20:30:18 +00:00