Commit Graph

11763 Commits

Author SHA1 Message Date
Petr Hosek 156226bb70 [compiler-rt] Set the ZX_VMO_RESIZABLE option for zx_vmo_create
Currently VMO in Zircon create using the zx_vmo_create is resizable
by default, but we'll be changing this in the future, requiring an
explicit flag to make the VMO resizable.

Prepare for this change by passing ZX_VMO_RESIZABLE option to all
zx_vmo_create calls that need resizable VMO.

Differential Revision: https://reviews.llvm.org/D61450

llvm-svn: 359803
2019-05-02 17:24:53 +00:00
Jonathan Metzman 3d1d3ad50e [libFuzzer] Re-enable libFuzzer on i386 Linux and fix test
Summary:
Re-enable libFuzzer on i386 Linux after it was accidentally
disabled.

Also disable gc-sections.test on i386 since lld isn't
garbage collecting properly with ASAN on i386.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: srhines, mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61415

llvm-svn: 359802
2019-05-02 16:45:17 +00:00
Stephan Bergmann 2f0ef58e53 Fix for Windows
...after 5745eccef5 "Adapt -fsanitize=function to
SANITIZER_NON_UNIQUE_TYPEINFO"

llvm-svn: 359760
2019-05-02 07:05:29 +00:00
Stephan Bergmann 5745eccef5 Adapt -fsanitize=function to SANITIZER_NON_UNIQUE_TYPEINFO
This follows up after b7692bc3e9 "[UBSan] Fix
isDerivedFromAtOffset on iOS ARM64" fixed the RTTI comparison in
isDerivedFromAtOffset on just one platform and then
a25a2c7c9a "Always compare C++ typeinfo (based on
libstdc++ implementation)" extended that fix to more platforms.

But there is another RTTI comparison for -fsanitize=function generated in
clang's CodeGenFunction::EmitCall as just a pointer comparison.  For
SANITIZER_NON_UNIQUE_TYPEINFO platforms this needs to be extended to also do
string comparison.  For that, __ubsan_handle_function_type_mismatch[_abort]
takes the two std::type_info pointers as additional parameters now, checks them
internally for potential equivalence, and returns without reporting failure if
they turn out to be equivalent after all.  (NORETURN needed to be dropped from
the _abort variant for that.)  Also these functions depend on ABI-specific RTTI
now, so needed to be moved from plain UBSAN_SOURCES (ubsan_handlers.h/cc) to
UBSAN_CXXABI_SOURCES (ubsan_handlers_cxx.h/cc), but as -fsanitize=function is
only supported in C++ mode that's not a problem.

Differential Revision: https://reviews.llvm.org/D60760

llvm-svn: 359759
2019-05-02 06:40:33 +00:00
Hubert Tong 50721b27c4 [compiler-rt][tests] Propagate COMPILER_RT_UNITTEST_LINK_FLAGS
`COMPILER_RT_UNITTEST_LINK_FLAGS` is dropped in many places, unlike
`COMPILER_RT_UNITTEST_CFLAGS`. This patch attempts to remove that
inconsistency.

Previously reviewed as part of D58951.

Reviewers: sfertile, peter.smith, pzheng, phosek, Hahnfeld, nemanjai, jasonliu

Reviewed By: sfertile

Subscribers: jsji, kubamracek, dberris, mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60143

llvm-svn: 359733
2019-05-01 22:25:16 +00:00
Vitaly Buka fbcec6cad0 [tsan] Fix and re-enable user_malloc.cc test
Summary: no_sanitize_thread is not enough as it still puts some tsan instrumentation

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61393

llvm-svn: 359731
2019-05-01 21:53:39 +00:00
Vitaly Buka 9ef838761e [sanitizer][NFC] Remove unneeded SizeClassAllocatorLocalCache
llvm-svn: 359729
2019-05-01 21:23:07 +00:00
Vitaly Buka 59bcb788ec [sanitizer][NFC] Add const/static into a couple of methods
llvm-svn: 359728
2019-05-01 21:22:59 +00:00
Julian Lettner 91c166cbb0 [Sanitizer] Reland "Cleanup INTERCEPT_FUNCTION macro"
On Linux both version of the INTERCEPT_FUNCTION macro now return true
when interception was successful. Adapt and cleanup some usages.

Also note that `&(func) == &WRAP(func)` is a link-time property, but we
do a runtime check.

Tested on Linux and macOS.

Previous attempt reverted by: 5642c3feb0

This attempt to bring order to the interceptor macro goes the other
direction and aligns the Linux implementation with the way things are
done on Windows.

Reviewed By: vitalybuka, rnk

Differential Revision: https://reviews.llvm.org/D61358

llvm-svn: 359725
2019-05-01 20:57:59 +00:00
Vitaly Buka 20c5676416 [sanitizer][NFC] Set LargeMmapAllocator type from PrimaryAllocator
They need to have same AddressSpaceView and MapUnmapCallback.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61168

llvm-svn: 359719
2019-05-01 19:41:54 +00:00
Vitaly Buka 7780f51614 [scudo][NFC] Remove unneeded template from scudo::CombinedAllocator
Reviewers: cryptoad, eugenis

Reviewed By: cryptoad

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61162

llvm-svn: 359718
2019-05-01 19:36:26 +00:00
Vitaly Buka 76931df40f [sanitizer][NFC] Get type of AllocatorCache from CombinedAllocator
Reviewers: eugenis, cryptoad, kcc

Reviewed By: kcc

Subscribers: kcc, kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61155

llvm-svn: 359715
2019-05-01 19:30:49 +00:00
Evgeniy Stepanov d1a710047b [sanitizer] Implement reallocarray.
Summary:
It's a cross of calloc and realloc. Sanitizers implement calloc-like check for size
overflow.

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61108

llvm-svn: 359708
2019-05-01 17:33:01 +00:00
Peter Smith 101bf520d1 [libFuzzer] Add --dump-input-on-failure to help diagnose AArch64 failures
The fork-siguser.test and fork.test intermittently fail on the AArch64
buildbot. Unfortunately these failures are not reproducible on a similar
machine and seem to fail when the machines are under load. Before
suggesting the tests be marked unsupported for AArch64 we'd like to see
if we can get some more information about the failures to see if it helps
us reproduce. This patch adds --dump-input-on-failure to the FileCheck
commands to see if we can get some more information about the failures.

Differential Revision: https://reviews.llvm.org/D61315

llvm-svn: 359675
2019-05-01 12:30:04 +00:00
Petr Hosek 3e28075ecc [compiler-rt] Use correct spelling of the -Wno-pedantic variable
This addresses the issue introduced in r359646 which is causing
sanitizer bots to fail.

llvm-svn: 359659
2019-05-01 07:00:27 +00:00
Fangrui Song 99055b9c38 Use __extension__ to suppress __EH_FRAME_LIST__ warnings
gcc -pedantic => warning: ISO C forbids empty initializer braces
clang -pedantic => -Wzero-length-array + -Wgnu-empty-initializer

llvm-svn: 359655
2019-05-01 06:25:21 +00:00
Petr Hosek ad3faadc87 [compiler-rt] Cleanup the --target and --sysroot handling
This addresses issue introduced in r359646.

llvm-svn: 359650
2019-05-01 05:41:58 +00:00
Petr Hosek 738697ea4e [compiler-rt] Pass sysroot and disable pedantic for crtbegin.o/crtend.o
These are needed to make bots happy.

Differential Revision: https://reviews.llvm.org/D61363

llvm-svn: 359646
2019-05-01 03:30:51 +00:00
Petr Hosek a890919745 [compiler-rt] Rework the object build support
The initial implementation didn't properly support cross-compilation
via the runtime build, the updated implementation should address that
by expanding the CMAKE_C_COMPILE_OBJECT variable with correct values.

Differential Revision: https://reviews.llvm.org/D61356

llvm-svn: 359644
2019-05-01 02:49:45 +00:00
Jonathan Metzman a5147e0e15 [libFuzzer] Disable i386 on non-Linux platforms
Summary: Disable i386 on non-Linux platforms since it is unwanted and broken on Windows.

Reviewers: morehouse, rnk

Reviewed By: morehouse

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61354

llvm-svn: 359641
2019-05-01 01:40:31 +00:00
Jonathan Metzman c0806e0d24 [libFuzzer] Fix failing test: sigint.test
Summary:
Fix sigint.test by making it require msan rather
than enumerating unsupported platforms.

Reviewers: kcc

Reviewed By: kcc

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61353

llvm-svn: 359638
2019-04-30 23:46:52 +00:00
Mitch Phillips 40f98d4b78 Added test.
llvm-svn: 359632
2019-04-30 23:07:12 +00:00
Mitch Phillips 84ea3f2a48 [sanitizer_common] Added 64-bit signed flag parser.
Summary: Adds a 64-bit signed flag parser for GWP-ASan's use.

Reviewers: vlad.tsyrklevich, eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits, vitalybuka, morehouse, pcc, kcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61342

llvm-svn: 359631
2019-04-30 23:07:10 +00:00
Kostya Serebryany 905e60c422 [libFuzzer] temporarily disable a test on windows, where there is no memmem in the usual place
llvm-svn: 359624
2019-04-30 22:27:38 +00:00
Kostya Serebryany c239eda8a0 [libFuzzer] add MagicSeparatorTest
llvm-svn: 359620
2019-04-30 22:05:55 +00:00
Reid Kleckner 5642c3feb0 Revert r359325 "[NFC][Sanitizer] Change "return type" of INTERCEPT_FUNCTION to void"
Changing INTERCEPT_FUNCTION to return void is not functionally correct.
IMO the best way to communicate failure or success of interception is
with a return value, not some external address comparison.

This change was also creating link errors for _except_handler4_common,
which is exported from ucrtbase.dll in 32-bit Windows.

Also revert dependent changes r359362 and r359466.

llvm-svn: 359611
2019-04-30 20:59:56 +00:00
Jonathan Metzman f3ee97731e [libFuzzer] Replace -seed_corpus to better support fork mode on Win
Summary:
Pass seed corpus list in a file to get around argument length limits on Windows.
This limit was preventing many uses of fork mode on Windows.

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60980

llvm-svn: 359610
2019-04-30 20:56:18 +00:00
Amy Kwan 31dbbf1df9 [compiler-rt][builtins][sanitizers] Update compiler-rt test cases for
compatibility with system's toolchain

This patch aims to:
- Guard ompiler-rt/test/builtins/Unit/compiler_rt_logb_test.c with macros, so
the test runs on GLIBC versions >= 2.23. This is because the test relies on
comparing its computed values to libm. Oolder versions might not compute to the
same value as the compiler-rt value.
- Update compiler-rt/test/sanitizer_common/TestCases/Posix/getpw_getgr.cc
so that std::string is not used, since false positives may be detected.

Differential Revision: https://reviews.llvm.org/D60644

llvm-svn: 359606
2019-04-30 20:09:00 +00:00
Petr Hosek 999bb5ac27 Reland "[compiler-rt] Simple crtbegin.o and crtend.o implementation"
Clang relies on existence of certain symbols that are normally
provided by crtbegin.o/crtend.o. However, LLVM does not currently
provide implementation of these files, instead relying on either
libgcc or implementations provided as part of the system.

This change provides an initial implementation of crtbegin.o/crtend.o
that can be used on system that don't provide crtbegin.o/crtend.o as
part of their C library.

Differential Revision: https://reviews.llvm.org/D28791

llvm-svn: 359591
2019-04-30 18:13:22 +00:00
Jonathan Metzman 7a2ce3790b fix broken test
llvm-svn: 359590
2019-04-30 17:58:59 +00:00
Jonathan Metzman 17bd74d406 remove extra zeros
llvm-svn: 359589
2019-04-30 17:58:58 +00:00
Jonathan Metzman 5a271cd758 remove stale comment
llvm-svn: 359588
2019-04-30 17:58:58 +00:00
Jonathan Metzman d61084901c change order
llvm-svn: 359587
2019-04-30 17:58:57 +00:00
Jonathan Metzman 1fbc6116e1 fix comment
llvm-svn: 359586
2019-04-30 17:58:57 +00:00
Jonathan Metzman 2697664582 [libFuzzer] Enable for i386
Summary: Get libFuzzer to build on i386 and fix tests.

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61070

llvm-svn: 359585
2019-04-30 17:58:56 +00:00
Jonathan Metzman 6fa864b7a6 Fix issues with testing for i386
llvm-svn: 359584
2019-04-30 17:58:55 +00:00
Jonathan Metzman f721230831 Enable x86 builds
llvm-svn: 359583
2019-04-30 17:58:54 +00:00
Petr Hosek 9300f60c8c Revert "[compiler-rt] Simple crtbegin.o and crtend.o implementation"
This reverts commit r359576 since it's failing on Windows bots.

llvm-svn: 359579
2019-04-30 17:32:05 +00:00
Petr Hosek c8be6e670e [compiler-rt] Simple crtbegin.o and crtend.o implementation
Clang relies on existence of certain symbols that are normally
provided by crtbegin.o/crtend.o. However, LLVM does not currently
provide implementation of these files, instead relying on either
libgcc or implementations provided as part of the system.

This change provides an initial implementation of crtbegin.o/crtend.o
that can be used on system that don't provide crtbegin.o/crtend.o as
part of their C library.

Differential Revision: https://reviews.llvm.org/D28791

llvm-svn: 359576
2019-04-30 17:21:13 +00:00
Kostya Kortchinsky 21c31f5e7b [scudo][standalone] Add the memory reclaiming mechanism
Summary:
This CL implements the memory reclaiming function `releaseFreeMemoryToOS`
and its associated classes. Most of this code was originally written by
Aleksey for the Primary64 in sanitizer_common, and I made some changes to
be able to implement 32-bit reclaiming as well. The code has be restructured
a bit to accomodate for freelist of batches instead of the freearray used
in the current sanitizer_common code.

Reviewers: eugenis, vitalybuka, morehouse, hctim

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61214

llvm-svn: 359567
2019-04-30 14:56:18 +00:00
Reid Kleckner bb92282b29 Fix FIXME added in r359339
We have windows.h in asan_win.cc, so we can just use the correct
prototypes for these EH-related interceptors without worrying.

Also fix an unused variable warning while I'm here.

llvm-svn: 359500
2019-04-29 20:44:26 +00:00
Reid Kleckner c69cd30455 Pacify sanitizer lint script after r359498
llvm-svn: 359499
2019-04-29 20:40:08 +00:00
Matthew G McGovern d62416dfcd [AddressSanitizer] [Windows] Fix HeapReAlloc and _recalloc bugs in asan_malloc_win.cc
HeapReAlloc should allow for 0 sized reallocations without freeing the memory block provided by the user.

_recalloc previously did not zero new memory after reallocation.
https://reviews.llvm.org/D61268

llvm-svn: 359498
2019-04-29 20:26:19 +00:00
Julian Lettner 1f10f6ea51 [Sanitizer] Cleanup {ASAN, MSAN}_INTERCEPT_FUNC[_VER] macro
Note that this change is not strictly NFC since we add the
`(&(name) != &WRAP(name)` part to the conditional for the `_VER` variant
of the macro.

Reviewers: vitalybuka

Differential Revision: https://reviews.llvm.org/D61204

llvm-svn: 359466
2019-04-29 16:39:18 +00:00
Petr Hosek e62915bcc1 [builtins] Use __APPLE__ instead of __MACH__ in check
The latter doesn't seem to be working for all targets. This addresses
the issue introduced in r359413.

llvm-svn: 359423
2019-04-29 08:38:43 +00:00
Petr Hosek cb929dcebe [builtins] Fix the missing assembly on Darwin
This was introduced in r359413.

llvm-svn: 359421
2019-04-29 07:45:15 +00:00
Petr Hosek ba45daab14 [builtins] Fix the typo in the preprocessor check
This was introduced in r359413.

llvm-svn: 359419
2019-04-29 06:30:50 +00:00
Petr Hosek 84da0e1bb7 [builtins] Use aliases for function redirects
Symbol aliases are supported by all platforms that compiler-rt builtins
target, and we can use these instead of function redirects to avoid the
extra indirection.

This is part of the cleanup proposed in "[RFC] compiler-rt builtins
cleanup and refactoring".

Differential Revision: https://reviews.llvm.org/D60931

llvm-svn: 359413
2019-04-29 00:46:23 +00:00
Petr Hosek 0ba22f51d1 [builtins] Use single line C++/C99 comment style
Use the uniform single line C++/99 style for code comments.

This is part of the cleanup proposed in "[RFC] compiler-rt builtins
cleanup and refactoring".

Differential Revision: https://reviews.llvm.org/D60352

llvm-svn: 359411
2019-04-28 22:47:49 +00:00
Petr Hosek 082b89b25f [builtins] Reformat builtins with clang-format
Update formatting to use the LLVM style.

This is part of the cleanup proposed in "[RFC] compiler-rt builtins
cleanup and refactoring".

Differential Revision: https://reviews.llvm.org/D60351

llvm-svn: 359410
2019-04-28 21:53:32 +00:00
Dan Liew 8651edf898 [CMake] Don't modify `FUZZER_SUPPORTED_ARCH` is place.
On a Darwin host we were modifying the `FUZZER_SUPPORTED_ARCH` in place
which would strip out non-x86 architectures. This unhelpful if we
want to use `FUZZER_SUPPORTED_ARCH` later.

To fix this we introduce `FUZZER_TEST_ARCH` which is similar to what we
have for for the other sanitizers. For non-Darwin host platforms
`FUZZER_TEST_ARCH` is the same as `FUZZER_SUPPORTED_ARCH` but for Darwin
host platforms we use `darwin_filter_host_archs(...)` as the previous
code did.

llvm-svn: 359394
2019-04-28 09:44:53 +00:00
Vitaly Buka 0f21545a3c [sanitizer] Calculate SizeClassAllocator32::ByteMap type from Params::kSpaceSize and Params::kRegionSizeLog
Reviewers: eugenis

Subscribers: kubamracek, cryptoad, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61206

llvm-svn: 359374
2019-04-27 06:30:52 +00:00
Vitaly Buka 83c6d10b62 [sanitizer] NFC: add static_assert to confirm that we use reasonable ByteMap type
Summary: If bots work we can replace #ifs with template specialization by TwoLevelByteMapSize1.

There is known users of TwoLevelByteMap with TwoLevelByteMapSize1 equal 8,
and users of FlatByteMap with TwoLevelByteMapSize1 equal 2.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61200

llvm-svn: 359364
2019-04-27 02:13:26 +00:00
Julian Lettner c1008e4d3d [NFC][Sanitizer] Remove GetRealFunctionAddress and replace usages
Reviewers: vitalybuka

Differential Revision: https://reviews.llvm.org/D61205

llvm-svn: 359362
2019-04-27 00:49:14 +00:00
JF Bastien 8504b5f64f Revert "[sanitizer] NFC: add static_assert to confirm that we use optimal ByteMap type"
Fails on bots with:

/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-expensive/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary32.h:69:3: error: static_assert failed due to requirement 'TwoLevelByteMapSize1 > 128' "TwoLevelByteMap should be used"
  static_assert(TwoLevelByteMapSize1 > 128, "TwoLevelByteMap should be used");
  ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/buildslave/jenkins/workspace/clang-stage1-cmake-RA-expensive/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_allocator_combined.h:29:34: note: in instantiation of template class '__sanitizer::SizeClassAllocator32<__sanitizer::AP32>' requested here
                        typename PrimaryAllocator::AddressSpaceView>::value,
                                 ^

http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/13960/console

llvm-svn: 359352
2019-04-26 22:29:49 +00:00
Julian Lettner 93c05f097a [Sanitizer][Windows] Speculative fix for broken build on Windows
llvm-svn: 359339
2019-04-26 19:06:55 +00:00
Julian Lettner 74967cb4e0 [Sanitizer] Fix test
I broke the build, panicked and applied the wrong fix in my previous
commit. The ASSERT was obsolete, but not the call INTERCEPT_FUNCTION.

llvm-svn: 359336
2019-04-26 18:39:02 +00:00
Julian Lettner d0e5830017 [Sanitizer] Fix compliation error in test
Remove obsolete assert. I missed this in my previous patch:
https://reviews.llvm.org/D61145

llvm-svn: 359335
2019-04-26 18:27:35 +00:00
Vitaly Buka 077c502ddc Implement __sanitizer::conditional<B, T, F>
llvm-svn: 359334
2019-04-26 18:22:55 +00:00
Vitaly Buka 85dcdae5e3 [sanitizer] NFC: add static_assert to confirm that we use optimal ByteMap type
Summary: If bots work we can replace #ifs with template specialization by TwoLevelByteMapSize1.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61200

llvm-svn: 359333
2019-04-26 18:22:47 +00:00
Dan Liew 18bc872405 Revert "[CMake] Fix the value of `config.target_cflags` for non-macOS Apple"
This reverts commit 1bcdbd68616dc7f8debe126caafef7a7242a0e6b.

It's been reported that some bots are failing with this change with CMake
error like:

```
CMake Error at /b/s/w/ir/k/llvm-project/compiler-rt/cmake/config-ix.cmake:177 (message):
  Unsupported architecture: arm64
Call Stack (most recent call first):
  /b/s/w/ir/k/llvm-project/compiler-rt/cmake/config-ix.cmake:216 (get_target_flags_for_arch)
  /b/s/w/ir/k/llvm-project/compiler-rt/test/tsan/CMakeLists.txt:78 (get_test_cflags_for_apple_platform)
```

I'm reverting the patch now to unbreak builds. I will investigate properly when time permits.

rdar://problem/50124489

llvm-svn: 359327
2019-04-26 17:53:25 +00:00
Julian Lettner aebd3014a5 [NFC][Sanitizer] Change "return type" of INTERCEPT_FUNCTION to void
This temporary change tells us about all the places where the return
value of the INTERCEPT_FUNCTION macro is actually used. In the next
patch I will cleanup the macro and remove GetRealFuncAddress.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D61145

llvm-svn: 359325
2019-04-26 17:29:22 +00:00
Vitaly Buka efc94feef9 [NFC] Remove ::kForTest from AP64, it does not use it.
llvm-svn: 359323
2019-04-26 17:04:05 +00:00
Dan Liew 60f5df948b [asan_symbolize] Teach `asan_symbolize.py` to symbolicate using a module map
Summary:
The use case here is to be able get the UUIDs of the modules that need
to be symbolicated so that external plugins can see them. This
information can be extracted from ASan reports if the `print_module_map`
ASan option is enabled. Currently printing of the module map is only
implemented on Darwin and so this is effectively a Darwin only feature
right now.

The module map hooks into symbolization using the new plugin
infrastructure. A new hook in `AsanSymbolizerPlugInProxy` (and in
`AsanSymbolizerPlugIn`) is also provided to allow external plugins to hook
into the module look up process. This will allow external plugins to
look up modules with knowledge of their UUID.

The new plug-in is currently stored in the `asan_symbolize.py` script.
We could potentially move this into a separate file in the future (to
reduce clutter) if we can come up with a policy for where to search for
plugins that should always get loaded.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60531

llvm-svn: 359322
2019-04-26 16:54:09 +00:00
Dan Liew 9f59704a5d [CMake] Fix the value of `config.target_cflags` for non-macOS Apple
platforms.

The main problem here is that `-*-version_min=` was not being passed to
the compiler when building test cases. This can cause problems when
testing on devices running older OSs because Clang would previously
assume the minimum deployment target is the the latest OS in the SDK
which could be much newer than what the device is running.

Previously the generated value looked like this:

`-arch arm64 -isysroot
<path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`

With this change it now looks like:

`-arch arm64 -stdlib=libc++ -miphoneos-version-min=8.0 -isysroot
<path_to_xcode>/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk`

This mirrors the setting of `config.target_cflags` on macOS.

This change is made for ASan, LibFuzzer, TSan, and UBSan.

To implement this a new `get_test_cflags_for_apple_platform()` function
has been added that when given an Apple platform name and architecture
returns a string containing the C compiler flags to use when building
tests. This also calls a new helper function `is_valid_apple_platform()`
that validates Apple platform names.

rdar://problem/50124489

Differential Revision: https://reviews.llvm.org/D58578

llvm-svn: 359305
2019-04-26 13:22:39 +00:00
Vitaly Buka 1a607ff043 [lsan] Use SANITIZER_WORDSIZE when selecting ByteMap
Originally this code was added for 64-bit platform and it was never update.
Add static_assert to validate type of ByteMap.

llvm-svn: 359286
2019-04-26 08:24:38 +00:00
Vitaly Buka d8e9c3a999 Revert "[lsan] Use SANITIZER_WORDSIZE when selecting ByteMap"
New static_assert fails on a bot.

This reverts commit r359269.

llvm-svn: 359276
2019-04-26 05:19:32 +00:00
Vitaly Buka a9e7fd2188 NFC: Clang-format msan_allocator.cc
llvm-svn: 359270
2019-04-26 04:22:28 +00:00
Vitaly Buka 3db2a7a04f [lsan] Use SANITIZER_WORDSIZE when selecting ByteMap
Originally this code as added for 64-bit platform and was never changed.
Add static_assert to make sure that we have correct map on all platforms.

llvm-svn: 359269
2019-04-26 04:20:27 +00:00
Matt Morehouse 1b76063a5e [libFuzzer] Disable MSan interceptors in SIGINT handler.
Summary:
Avoids an MSan false positive if the SIGINT comes while the user
callback is running.  The false positive happens when the interrupt
handler calls opendir() to remove some temporary files, which is
intercepted by MSan.

Fixes https://github.com/google/oss-fuzz/issues/2332.

Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits, Dor1s, metzman

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61163

llvm-svn: 359254
2019-04-26 00:17:41 +00:00
Matt Morehouse e9640afddd [libFuzzer] Require linux for libcxx.test
llvm-svn: 359223
2019-04-25 18:42:30 +00:00
Matt Morehouse 42ef2c6d06 [compiler-rt] Build custom libc++abi without exceptions.
Summary:
Since neither compiler-rt nor the libc++ we build use exceptions, we
don't need libc++abi to have them either.

This resolves an issue where libFuzzer's private libc++ contains
implementations for __cxa_throw and friends, causing fuzz targets built
with their own C++ library to segfault during exception unwinding.

See https://github.com/google/oss-fuzz/issues/2328.

Reviewers: phosek, EricWF, kcc

Reviewed By: phosek

Subscribers: kcc, dberris, mgorny, christof, llvm-commits, metzman

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D61053

llvm-svn: 359218
2019-04-25 18:14:24 +00:00
Julian Lettner 8b36610bfa [NFC][Sanitizer] Extract GetFuncAddr from GetRealFunctionAddress
Summary:
Hopefully, this will enable cleanup/removal of GetRealFunctionAddress in
follow-up commits.

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D61110

llvm-svn: 359213
2019-04-25 17:46:29 +00:00
Kostya Kortchinsky 3fad6a206f [scudo][standalone] Introduce the SizeClassMap
Summary:
As with the sanitizer_common allocator, the SCM allows for efficient
mapping between sizes and size-classes, table-free.

It doesn't depart significantly from the original, except that we
allow the use of size-class 0 for other purposes (as opposed to
chunks of size 0). The Primary will use it to hold TransferBatches.

Reviewers: vitalybuka, eugenis, hctim, morehouse

Reviewed By: vitalybuka

Subscribers: srhines, mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D61088

llvm-svn: 359199
2019-04-25 15:49:34 +00:00
Mitch Phillips 1af5a85e48 [sanitizer_common] Remove some old commented-out printf statements, and fixed up the order of includes.
llvm-svn: 359150
2019-04-24 23:03:32 +00:00
Vitaly Buka 0ddd12e156 [compiler-rt] NFC fixed [whitespace/braces] LINT warning
llvm-svn: 359133
2019-04-24 20:01:50 +00:00
Adhemerval Zanella 91cee68e1f [fuzzer] Fix reload.test on Linux/aarch64
The compiler generates a 'brk' instruction for __builtin_trap on aarch64
and Linux kernel issues a SIGTRAP. It is different from x86, where
compiler emits an 'ud2' and kernel issues a SIGILL.

A straightforward is to use abort instead.

llvm-svn: 359126
2019-04-24 19:02:54 +00:00
Vitaly Buka 97bdb04a3b [compiler-rt] Fix warning about virtual destructor in sanitizer_flag_parser.h
llvm-svn: 359125
2019-04-24 19:01:04 +00:00
Mitch Phillips ca0a68705f [llvm-symbolizer] Quick fix for broken sanitizer bot
(sanitizer-x86_64-linux) until I can triage the issue properly. The
build has been broken due to the symbolizer build checks failing.

As the symbolizer build script relies on the old svn repo layout, it may
take a little while longer to find the responsible patch for the
breakage. This may be a completely valid fix, but I will need to confirm
it. For now, it unbreaks the build.

Tracking data:
Build where the break first occurred: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/21211
Changelist authors: @grimar, @maskray, @whitequark, @spatel, @dpreobra

For the authors, no action needed (yet). Will follow up when I can
identify the cause.

llvm-svn: 359123
2019-04-24 18:37:55 +00:00
Kostya Kortchinsky 475585655d [scudo][standalone] Introduce the Secondary allocator
Summary:
The Secondary allocator wraps the platform allocation primitives. It is
meant to be used for larger sizes that the Primary can't fullfill, as
it will be slower, and sizes are multiple of the system page size.

This also changes some of the existing code, notably the opaque
platform data being passed to the platform specific functions: we can
shave a couple of syscalls on Fuchsia by storing additional data (this
addresses a TODO).

Reviewers: eugenis, vitalybuka, hctim, morehouse

Reviewed By: morehouse

Subscribers: mgorny, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60787

llvm-svn: 359097
2019-04-24 14:20:49 +00:00
Dmitry Mikulin 312b5f86b7 The error message for mismatched value sites is very cryptic.
Make it more readable for an average user.

Differential Revision: https://reviews.llvm.org/D60896

llvm-svn: 359043
2019-04-23 22:26:55 +00:00
Julian Lettner 4d2b9426b9 [TSan] Support fiber API on macOS
Committing on behalf of Yuri Per (yuri).

Reviewers: dvyukov, kubamracek, yln

Reviewed By: kubamracek

Authored By: yuri

Differential Revision: https://reviews.llvm.org/D58110

llvm-svn: 358802
2019-04-20 00:18:44 +00:00
Chris Bieneman 2436237895 [CMake] Add fuzzer as a component for runtime builds
Calling `add_compiler_rt_component` sets up the component connection between runtime builds and the parent CMake configuration. Adding this call allows specifying `fuzzer` as a `LLVM_RUNTIME_DISTRIBUTION_COMPONENT`.

llvm-svn: 358780
2019-04-19 20:13:25 +00:00
Petr Hosek 45fc90326a [gn] Support dots in CMake paths in the sync script
Some file paths use dots to pick up sources from parent directories.

Differential Revision: https://reviews.llvm.org/D60734

llvm-svn: 358774
2019-04-19 18:29:17 +00:00
Mitch Phillips f4878d8905 Removed regiser dump checks from FP unit test.
llvm-svn: 358770
2019-04-19 17:36:58 +00:00
Mitch Phillips aa68f946dc [HWASan] Added no-FP unit test for register dump.
Summary: Unit test for D60798.

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60806

llvm-svn: 358769
2019-04-19 17:36:56 +00:00
Mitch Phillips d1a18469a8 Updated test case.
llvm-svn: 358768
2019-04-19 17:36:55 +00:00
Mitch Phillips 2c3cecef3d Added no-FP unit test for register dump.
llvm-svn: 358767
2019-04-19 17:36:53 +00:00
Kostya Serebryany 14ada6d1ad [libFuzzer] revert part of r358726 committed by mistake (oops)
llvm-svn: 358727
2019-04-19 01:40:16 +00:00
Kostya Serebryany bbb8522bde [libFuzzer] in fork mode, make sure the child stats are always collected. This should fix the flakiness in fork-ubsan.test
llvm-svn: 358726
2019-04-19 01:39:14 +00:00
Yi Kong 815a4c902d [builtins] Build x86_64 with GENERIC_TF_SOURCES
llvm-svn: 358706
2019-04-18 19:29:03 +00:00
Jonathan Metzman 139e216e66 Summary:
Add close_fd_mask functionality to AFL driver.

Summary:
Add support for env var AFL_DRIVER_CLOSE_FD_MASK which behaves
the same as libFuzzer's -close_fd_mask=1.

Also add tests.

Reviewers: kcc, vitalybuka, morehouse

Reviewed By: morehouse

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60334

llvm-svn: 358703
2019-04-18 18:49:11 +00:00
Dan Liew 16c3883b98 [asan_symbolize] Workaround bug in old Python 2 versions.
The change landed in r358657 broke some of the buildbots because they
use an older version of Python 2 that raises this error.

```
File "/Volumes/data/dev/llvm/upstream/master/src/projects/compiler-rt/lib/asan/scripts/asan_symbolize.py", line 509
  exec(f.read(), globals_space, None)
SyntaxError: unqualified exec is not allowed in function 'load_plugin_from_file' it contains a nested function with free variables
```

I can reproduce this problem when using Python 2.7.6.

To workaround this some indirection has been added to prevent the broken
(the line at fault would never be executed) SyntaxError error in old
Python versions from being raised.

rdar://problem/49476995

llvm-svn: 358682
2019-04-18 16:22:55 +00:00
Dan Liew 403d3187a7 [asan_symbolize] Add a simple plugin architecture
Summary:
This change adds a simple plugin architecture to `asan_symbolize.py`.
The motivation here is that sometimes it's necessary to perform extra
work to figure out where binaries with debug symbols can actually be
found. For example it might be the case that a remote service needs
to be queried for binaries and then copied to the local system.

This "extra work" can be extremely site-specific such that adding the
code directly into the `asan_symbolize.py` would just clutter the code
for a very niche use case. To avoid this, the `asan_symbolize.py` can
now load external code via a new `--plugins` command line option.

These plugins are loaded before main command line argument parsing so
that they can add their own command line options.

Right now the only hook into the behaviour of symbolization is the
`filter_binary_path()` function which assumes a very similar role
to the `binary_name_filter` function that was previously in the code.
We can add more hooks as necessary.

Code in the `asan_symbolize.py` script does not call plugin code
directly. Instead it uses a `AsanSymbolizerPlugInProxy` object.
This object

* Loads plugins from files.
* Manages the lifetime of the plugins.
* Provides an interface for calling into plugin functions and handles
  calling into multiple plugins.

To unify the way binary paths are filtered the old `sysroot_path_filter`
function (and associated code) has been turned into a simple plugin
(`SysRootFilterPlugIn`) that is always loaded. The plugin unloads
itself if the `-s` option is not present on the command line. Users
should not see any functional change relating to this command line
option.

Some simple tests are provided to illustrate what plugin code looks
like and also to check the functionality continues to work.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: srhines, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60529

llvm-svn: 358657
2019-04-18 11:34:31 +00:00
Yi Kong 64c32362f0 [builtins] Add __cmpsf2 for ARM version of comparesf2
The generic version of comparesf2 defines __cmpsf2 alias for libgcc
compatibility, but the ARM overlay is missing the alias.

Differential Revision: https://reviews.llvm.org/D60805

llvm-svn: 358542
2019-04-17 01:30:33 +00:00
Mitch Phillips 8f9d95d9d7 [HWASan] Fixed slow DWARF unwinding.
Summary: CFA was setup incorrectly, as there is an 8-byte gap at the top of the stack for SP 16-byte alignment purposes.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: kubamracek, javed.absar, #sanitizers, llvm-commits, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60798

llvm-svn: 358535
2019-04-16 22:16:01 +00:00
Matt Morehouse 57213d839a [libFuzzer] Remove tautological compare.
llvm-svn: 358510
2019-04-16 17:38:19 +00:00
Julian Lettner 2632643454 [TSan][libdispatch] Port gcd-sync-block-copy.mm to C++
Summary:
Apparently, it makes a difference on where a block lives depending on if
it's passed "inline" versus assigned and then passed via a variable.
Both tests in this commit now give a signal, if `Block_copy` is used in
`dispatch_sync`.

Since these tests use different mechanisms (Objective-C retain versus
C++ copy constructor) as proxies to observe if the block was copied, we
should keep both of them.

Commit, that first avoided the unnecessary copy:
faef7d034a

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60639

llvm-svn: 358469
2019-04-16 01:34:38 +00:00
Kostya Serebryany 5e67abd91f [libFuzzer] speedup the merge step in the fork mode by merging only the files that have unique features.
llvm-svn: 358320
2019-04-13 01:57:33 +00:00
Kostya Serebryany 4614cc3dfd [libFuzzer] add -features_dir= flag to dump unique input features on disk
llvm-svn: 358317
2019-04-13 00:20:31 +00:00
Julian Lettner c77bf89dcc [TSan][libdispatch] Move libdispatch tests out of Darwin folder
The libdispatch tests are now not tied to Darwin anymore. Move them to
their own platform-independent folder.

llvm-svn: 358316
2019-04-12 23:07:00 +00:00
Max Moroz b6e6d3c740 [libFuzzer] Fix DataFlow.cpp logic when tracing long inputs.
Summary:
1. Do not create DFSan labels for the bytes which we do not trace. This is where we run out of labels at the first place.
2. When dumping the traces on the disk, make sure to offset the label identifiers by the number of the first byte in the trace range.
3. For the last label, make sure to write it at the last position of the trace bit string, as that label represents the input size, not any particular byte.

Also fixed the bug with division in python which I've introduced when migrated the scripts to Python3 (`//` is required for integral division).

Otherwise, the scripts are wasting too much time unsuccessfully trying to
collect and process traces from the long inputs. For more context, see
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481761789

Reviewers: kcc

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60538

llvm-svn: 358311
2019-04-12 21:00:12 +00:00
Julian Lettner 6636978243 [TSan][libdispatch] Don't link against Foundation
Now that our tests don't depend on Foundation anymore,
don't link it in.

llvm-svn: 358309
2019-04-12 20:27:05 +00:00
Julian Lettner bfcbd677c8 [TSan][libdispatch] Replace NSTemporaryDirectory in tests
After this change, most tests don't have a dependency on Foundation.

Note: To hold the file name `tempnam` allocates a new buffer. We leak
      this buffer (omit the free), but I don't think we need to care.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60591

llvm-svn: 358308
2019-04-12 20:27:02 +00:00
Kostya Serebryany 65132e2118 [libFuzzer] support -runs=N in the fork mode. Make sure we see one-line reports from ubsan in the fork mode. Test both
llvm-svn: 358306
2019-04-12 20:20:57 +00:00
Julian Lettner 43e27eb87f [TSan][libdispatch] Fix failing test
The ThreadSanitizer-x86_64-iossim configuration (different SDK) seems to
require an additional #include.

rdar://49856637

llvm-svn: 358300
2019-04-12 18:31:09 +00:00
Julian Lettner 2edfcf9065 [TSan][libdispatch] Delete old tests
In a previous commit, I re-enabled the ported variants of these 2 tests:
tsan/Darwin/gcd-data.mm -> tsan/libdispatch/data.c
tsan/Darwin/gcd-source-serial.mm -> tsan/libdispatch/source-serial.c

So now we can delete the Darwin-only version.

llvm-svn: 358235
2019-04-11 22:27:57 +00:00
Julian Lettner 1740c6e0da [TSan][libdispatch] Re-enable disabled tests
I re-activated "broad strokes suppressions"
(ignore_noninstrumented_modules=1) in my last commit. Re-enable tests
that only fail on our bots to check if they work now.

llvm-svn: 358209
2019-04-11 18:34:47 +00:00
Julian Lettner 5b981638fb [TSan][libdispatch] Turn ignore_noninstrumented_modules=1 back on for Darwin
llvm-svn: 358208
2019-04-11 18:34:44 +00:00
Max Moroz bcb93a35c0 [libFuzzer] Make MutateWithMask work when the Mask is shorter than the input.
Summary:
Before this change, MutateWithMask used to assert that Mask should be
of sufficient length (>= Size of the input). However, in real cases we may have
inputs that are longer than the Mask they have inherited from the based inputs.

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60571

llvm-svn: 358207
2019-04-11 18:21:53 +00:00
Julian Lettner 3e58f94ced [TSan][libdispatch] Replace usage of NSMutableData with stack array
Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60477

llvm-svn: 358205
2019-04-11 18:14:19 +00:00
Julian Lettner edea731af2 [TSan][libdispatch] Change test to have two simultaneous timers
Change test semantics by waiting for both timer callbacks at the end
instead of serializing operations: start/wait timer 1 then 2.

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60476

llvm-svn: 358204
2019-04-11 18:14:16 +00:00
Julian Lettner 0f7a0c71c6 [TSan][libdispatch] Replace CFRunLoop with dispatch_semaphore, pt. 2
Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D60475

llvm-svn: 358203
2019-04-11 18:14:13 +00:00
Matt Morehouse abed4acc2c Revert "tsan: Update test expectations."
This reverts r357831 to go with r358113.

llvm-svn: 358202
2019-04-11 18:05:37 +00:00
Max Moroz 9d5e7ee296 [libFuzzer] Fallback to default Mutate when MutateWithMask fails.
Summary:
In case the current corpus input doesn't have bytes going into the
focus function, MutateWithMask is useless and may fail gently, allowing the
default mutation routine happen, rather than crashing on an assertion.

For more context and the initial fix suggestion, see:
https://github.com/google/oss-fuzz/issues/1632#issuecomment-481862879

Reviewers: kcc, morehouse

Reviewed By: kcc

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60567

llvm-svn: 358190
2019-04-11 16:24:53 +00:00
John McCall 98da442b6d Prospective test fix in response to r358104.
llvm-svn: 358149
2019-04-11 03:04:04 +00:00
JF Bastien 2f46de8c0b [NFC] Use clearer naming for local variables
llvm-svn: 358145
2019-04-10 23:23:17 +00:00
Julian Lettner 19f52bde59 [TSan][libdispatch] Replace CFRunLoop with dispatch_semaphore, pt. 1
Remove the dependency on Foundation so we can start running those tests
on other platforms. Rename/move of tests will be done in a separate
commit.

Reviewed By: kubamracek, dvyukov

Differential Revision: https://reviews.llvm.org/D60347

llvm-svn: 358023
2019-04-09 17:51:55 +00:00
Kostya Kortchinsky 7e2b15382c [scudo][standalone] Add flags & related parsers
Summary:
As with other Sanitizers, and the current version of Scudo, we can
provide flags in differents way: at compile time, through a weak
function, through an environment variable.

This change adds support for the configuration flags, and the string
parsers. Those are fairly similar to the sanitizer_common way of doing
things.

Reviewers: morehouse, hctim, vitalybuka

Reviewed By: morehouse, vitalybuka

Subscribers: mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D59597

llvm-svn: 358011
2019-04-09 14:57:25 +00:00
Dan Liew 2316429d04 [asan_symbolize] Move argument parser epilog text into script doc comment.
Summary:
This will make it easier to expand on the documentation in the future
that avoids cluttering the code.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60398

llvm-svn: 357978
2019-04-09 08:30:43 +00:00
Peter Collingbourne df57979ba7 hwasan: Enable -hwasan-allow-ifunc by default.
It's been on in Android for a while without causing problems, so it's time
to make it the default and remove the flag.

Differential Revision: https://reviews.llvm.org/D60355

llvm-svn: 357960
2019-04-09 00:25:59 +00:00
Dan Liew 836da99af9 [asan_symbolize] Use proper logging infrastructure.
Summary:
The previous logging infrastructure had several problems:

* Debugging output was emitted to standard output which is also where
  the symbolized output would go. Interleaving these two separate
  bits of information makes inspecting the output difficult and could
  potentially break tests.

* Enabling debugging output requires modifying the script which is
  not very conveninent.

* When debugging it isn't immediately obvious where the output is
  coming from.

This patch uses the Python standard library logging infrastructure
which fixes all of the above problems. Logging is controlled using
two new options.

* `--log-level` - Sets the logging level, default is
`info`.
* `--log-dest` - Set the logging destination, default
is standard error.

Some simple test cases for the feature are included.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60343

llvm-svn: 357951
2019-04-08 22:28:27 +00:00
Pavel Labath 19cad6f089 [Sanitizer] Make wcrtomb test posix-only
wcrtomb is not intercepted on windows, so this test fails there. It's
not clear to me why we do not intercept this function there (I'll look
into that separately), but for now this should at least make the windows
sanitizer bot green again (broken by r357889, when I added this test).

I also add "UNSUPPORTED: android" as this function is also not
intercepted there.

llvm-svn: 357892
2019-04-08 09:13:13 +00:00
Pavel Labath 618dcfcaa3 [Sanitizer] Fix a possible write to freed memory in the wcrtomb interceptor
Summary:
r357240 added an interceptor for wctomb, which uses a temporary local
buffer to make sure we don't write to unallocated memory. This patch
applies the same technique to wcrtomb, and adds some additional tests
for this function.

Reviewers: vitalybuka, eugenis

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D59984

llvm-svn: 357889
2019-04-08 08:39:50 +00:00
Julian Lettner 18bc982c06 [TSan][libdispatch] Remove Darwin-only version of fully-ported tests
Remove 10 tests that already have a copy in tsan/libdispatch, without
dependencies on Darwin-specifis.

llvm-svn: 357832
2019-04-06 01:41:40 +00:00
Peter Collingbourne c0da957bb0 tsan: Update test expectations.
llvm-svn: 357831
2019-04-06 00:43:17 +00:00
Petr Hosek 40442658db [gn] Support for building compiler-rt builtins
This is support for building compiler-rt builtins, The library build
should be complete for a subset of supported platforms, but not all
CMake options have been replicated in GN.

We always use the just built compiler to build all the runtimes, which
is equivalent to the CMake runtimes build. This simplifies the build
configuration because we don't need to support arbitrary host compiler
and can always assume the latest Clang. With GN's toolchain support,
this is significantly more efficient than the CMake runtimes build.

Differential Revision: https://reviews.llvm.org/D60331

llvm-svn: 357821
2019-04-05 21:30:40 +00:00
Vitaly Buka 3db6ad2bcf Use binary write mode in WriteToFile function to avoid appended \r characters on Windows
Summary:
When using libfuzzer on Windows, in the contents of a crash sample, bytes that can be mistaken for a \n are replaced by a \r\n sequence. As a consequence, crashes are not reproducible. This patch will open files in binary mode to fix this issue. The patch does not affect POSIX systems.

Patch by tuktuk

Reviewers: kcc, vitalybuka

Reviewed By: vitalybuka

Subscribers: dexonsmith, jdoerfert, llvm-commits, #sanitizers

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60008

llvm-svn: 357807
2019-04-05 20:17:03 +00:00
Rainer Orth 92db30fc00 [Sanitizers] Provide __internal_dup on Solaris
Both LLVM 8.0.0 and current trunk fail to link libclang_rt.ubsan.standalone-*.so
on Solaris 11/x86 with GCC 8.1.0:

  Undefined			first referenced
   symbol  			    in file
  Scanning dependencies of target clang_rt.asan_cxx-x86_64
  __sanitizer::internal_dup(int)      ../sanitizer_common/CMakeFiles/RTSanitizerCommon.i386.dir/sanitizer_posix.cc.o
  ld: fatal: symbol referencing errors
  collect2: error: ld returned 1 exit status
  make[2]: *** [projects/compiler-rt/lib/ubsan/CMakeFiles/clang_rt.ubsan_standalone-dynamic-i386.dir/build.make:233: lib/clang/9.0.0/lib/sunos/libclang_rt.ubsan_standalone-i386.so] Error 1

Fixed by providing an implementation, which allowed the i386-pc-solaris2.11 build
to finish.

Differential Revision: https://reviews.llvm.org/D60045

llvm-svn: 357753
2019-04-05 08:42:21 +00:00
Rainer Orth a0524521ea [Sanitizers] Fix sanitizer_linux_libcdep.cc compilation on Solaris
Both LLVM 8.0.0 and current trunk fail to compile on Solaris 11/x86 with
GCC 8.1.0:

  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc: In function ‘void __sanitizer::ReExec()’:
  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc:831:14: error: ‘getexecname’ was not declared in this scope
     pathname = getexecname();
                ^~~~~~~~~~~
  /vol/llvm/src/llvm/dist/projects/compiler-rt/lib/sanitizer_common/sanitizer_linux_libcdep.cc:831:14: note: suggested alternative: ‘gethostname’
     pathname = getexecname();
                ^~~~~~~~~~~
                gethostname

This is easily fixed by including <stdlib.h> which declares that function.
With that patch, compilation continues.

Differential Revision: https://reviews.llvm.org/D60044

llvm-svn: 357751
2019-04-05 08:40:08 +00:00
Julian Lettner 7d1ec7b07e [TSan][libdispatch] Make test work on Linux, pt. 2
llvm-svn: 357741
2019-04-05 02:05:57 +00:00
Max Moroz b250403590 [libFuzzer] Another follow up fix for Data Flow scripts in Py3.
Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60289

llvm-svn: 357732
2019-04-04 22:55:18 +00:00
Max Moroz 665b6b30dd [libFuzzer] Fix output format in data flow merge script after Py3 change.
Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60288

llvm-svn: 357730
2019-04-04 22:21:27 +00:00
Julian Lettner be09db001a [TSan][libdispatch] Make test work on Linux
llvm-svn: 357729
2019-04-04 21:55:40 +00:00
Reid Kleckner 6af8e1e64c Remove unneeded ymath.h include from int_math.h
This avoids a conflict between stdbool.h, which defines bool to _Bool in
xkeycheck.h. From what I can tell, ymath.h is an internal header, and
the intention is that users should include math.h directly instead. It
doesn't appear to provide declarations of anything required for our
builtins. This include was added back in r249513 from 2015, and it's
possible that ymath.h provided something this code needed at the time,
but today it does not.

llvm-svn: 357728
2019-04-04 21:47:15 +00:00
Julian Lettner 13dace5d5d [TSan][libdispatch] Stricter checks via `--implicit-check-not`
`--implicit-check-not='ThreadSanitizer'` checks in the entire output
while `// CHECK-NOT: ThreadSanitizer` only checks after (before) the
previous (next) match.

llvm-svn: 357727
2019-04-04 21:25:56 +00:00
Max Moroz c815210013 [libFuzzer] Make DataFlow scripts Python3 compatible.
Summary:
Python2 will hit end of life soon: https://pythonclock.org/
This change also makes the integration with OSS-Fuzz a bit simpler:
https://github.com/google/oss-fuzz/issues/1632

Reviewers: morehouse, kcc

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D60282

llvm-svn: 357726
2019-04-04 21:20:36 +00:00
Reid Kleckner 955b5a6331 [winasan] Fix strrchr interception with vs2019 CRT
strrchr in vs2019 CRT begins with unrecognized instructions.

VCRUNTIME140!strrchr:
4533c9         xor     r9d, r9d
4c8bc1         mov     r8, rcx

Patch by Christopher Reid!

Differential Revision: https://reviews.llvm.org/D60217

llvm-svn: 357725
2019-04-04 21:08:02 +00:00
Julian Lettner f7461b8c5f [TSan][libdispatch] Specify libdispatch header dir for lit tests
Specify libdispatch header dir (include path) for lit tests. This is the
last missing piece in order to run the libdispatch tests on Linux even
when libdispatch is installed in a custom path instead of a default
(system) location.

llvm-svn: 357707
2019-04-04 17:25:43 +00:00
Julian Lettner b8c6effacf [NFC][TSan][libdispatch] Better CMake variable names
llvm-svn: 357664
2019-04-04 01:10:07 +00:00
Julian Lettner 15f0057e66 [TSan][libdispatch] Add Cmake option for libdispatch install path
llvm-svn: 357657
2019-04-04 00:28:09 +00:00
Xing GUO 884c29e9ae Fix typos in tests. NFC.
Reviewers: Higuoxing

Reviewed By: Higuoxing

Subscribers: kubamracek, cfe-commits, #sanitizers, llvm-commits

Tags: #clang, #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D60183

llvm-svn: 357577
2019-04-03 13:15:09 +00:00
Julian Lettner fc5a176f6a [NFC][TSan][libdispatch] Cleanup test
llvm-svn: 357530
2019-04-02 20:58:49 +00:00
Julian Lettner 226b992ebd [NFC] Test is C++, not C
llvm-svn: 357524
2019-04-02 20:39:10 +00:00
Julian Lettner 9ca4ff2666 [TSan][libdispatch] We don't require any setup on Darwin
llvm-svn: 357519
2019-04-02 20:04:32 +00:00
Julian Lettner 3a23fb1e04 [NFC][libdispatch] Improve a few tests
llvm-svn: 357510
2019-04-02 18:46:59 +00:00
Peter Collingbourne a720381bb5 Fix problem with ar_to_bc.sh script introduced by r357450.
The new code wasn't matching absolute paths correctly. Should unbreak
sanitizer-x86_64-linux bot.

llvm-svn: 357502
2019-04-02 17:47:38 +00:00
Hans Wennborg 76f2259dde Fix tests after r357452
llvm-svn: 357462
2019-04-02 10:06:39 +00:00
Fangrui Song 9d1449246d ar_to_bc.sh: replace readlink -f with custom relative path resolution
llvm-ar is a crunchgen-style executable dispatching to dlltool,ranlib,lib,ar based on argv[0].
In our content-addressable storage, readlink -f resolves paths to some
digest and thus lost the original "llvm-ar" filename.

Replace it with a custom path resolution to fix the problem.

llvm-svn: 357450
2019-04-02 04:58:29 +00:00
Matt Davis 60a8816c2f [compiler-rt][test] Make instrprof-set-dir-mode test tolerant of group ID
Patch from 'troyj':
Hi, I ran into a problem with this test when the source was located in
certain directories. The mkdir(2) man page states that the set-group-ID
bit is inherited from the parent directory, but this test was written in
such a way that it assumes the bit is unset. Whether that assumption is
true depends on where the checkout lives, which leads to some people
being able to reproduce the problem whereas others cannot. I think the
correct fix is to exclude the bit from the check.

Making probinson a reviewer since they reviewed the original test.

Patch landed for troyj, thanks!

Differential Revision: D53832

llvm-svn: 357449
2019-04-02 03:24:12 +00:00