Commit Graph

12138 Commits

Author SHA1 Message Date
Kamil Rytarowski 8b83f50c03 Remove xfail i386 NetBSD mark in vptr-non-unique-typeinfo.cpp
This test passes now.

llvm-svn: 371575
2019-09-10 23:55:03 +00:00
Kamil Rytarowski 0910a03bc2 Remove xfail NetBSD mark from ignored-interceptors-mmap.cpp
This test now passes.

llvm-svn: 371574
2019-09-10 23:42:16 +00:00
Max Moroz ac3dce595c [UBSan] Follow up fix for r371442.
Reviewers: vitalybuka, hctim, Dor1s

Reviewed By: Dor1s

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D67371

llvm-svn: 371453
2019-09-09 21:00:25 +00:00
Julian Lettner 6d04ee0f86 [TSan] Add AnnotateIgnoreReadsBegin declaration to tsan/test.h
Declare the family of AnnotateIgnore[Read,Write][Begin,End] TSan
annotations in compiler-rt/test/tsan/test.h so that we don't have to
declare them separately in every test that needs them.  Replace usages.

Leave usages that explicitly test the annotation mechanism:
  thread_end_with_ignore.cpp
  thread_end_with_ignore3.cpp

llvm-svn: 371446
2019-09-09 20:07:03 +00:00
Max Moroz 9508738cd1 [UBSan] Do not overwrite the default print_summary sanitizer option.
Summary:
This option is true by default in sanitizer common. The default
false value was added a while ago without any reasoning in
524e934112

so, presumably it's safe to remove for consistency.

Reviewers: hctim, samsonov, morehouse, kcc, vitalybuka

Reviewed By: hctim, samsonov, vitalybuka

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D67193

llvm-svn: 371442
2019-09-09 19:30:48 +00:00
Julian Lettner fc910c507e [TSan] Add interceptors for mach_vm_[de]allocate
I verified that the test is red without the interceptors.

rdar://40334350

Reviewed By: kubamracek, vitalybuka

Differential Revision: https://reviews.llvm.org/D66616

llvm-svn: 371439
2019-09-09 18:57:32 +00:00
Ed Maste 1a3dd638c4 compiler-rt: use fp_t instead of long double, for consistency
Most builtins accepting or returning long double use the fp_t typedef.
Change the remaining few cases to do so.

Differential Revision:	https://reviews.llvm.org/D35034

llvm-svn: 371400
2019-09-09 13:50:20 +00:00
Kamil Rytarowski dd0c00b5f8 Enable LSan for NetBSD/i386 in test/asan/lit.cfg.py
llvm-svn: 371354
2019-09-08 23:53:36 +00:00
Kamil Rytarowski def6ca8b33 Enable LSan tests for NetBSD/i386
llvm-svn: 371338
2019-09-08 17:07:28 +00:00
Kamil Rytarowski 90d2be0163 Stop marking 5 ASan tests as failing on NetBSD/i386
Unexpected Passing Tests (4):
    AddressSanitizer-i386-netbsd :: TestCases/Posix/coverage-reset.cpp
    AddressSanitizer-i386-netbsd :: TestCases/Posix/coverage.cpp
    AddressSanitizer-i386-netbsd :: TestCases/Posix/interception-in-shared-lib-test.cpp
    AddressSanitizer-i386-netbsd :: TestCases/suppressions-library.cpp

llvm-svn: 371337
2019-09-08 16:15:18 +00:00
Jonas Hahnfeld 307daa71a8 [ASan] Only run dlopen-mixed-c-cxx.c with static runtime
This is what the original bug (http://llvm.org/PR39641) and the fix
in https://reviews.llvm.org/D63877 have been about.
With the dynamic runtime the test only passes when the asan library
is linked against libstdc++: In contrast to libc++abi, it does not
implement __cxa_rethrow_primary_exception so the regex matches the
line saying that asan cannot intercept this function. Indeed, there
is no message that the runtime failed to intercept  __cxa_throw.

Differential Revision: https://reviews.llvm.org/D67298

llvm-svn: 371336
2019-09-08 16:08:54 +00:00
Kamil Rytarowski 14f1990921 Enable leak-detection for NetBSD/amd64 in test/asan
llvm-svn: 371335
2019-09-08 15:54:48 +00:00
Kamil Rytarowski c8afbf3d95 Do not intercept malloc_usable_size on NetBSD
llvm-svn: 371334
2019-09-08 15:00:13 +00:00
Yi Kong 33b8a55329 Revert "Revert "[builtins] Rounding mode support for addxf3/subxf3""
Test failure fixed.

This reverts commit e204d244ba.

llvm-svn: 371003
2019-09-05 01:05:05 +00:00
Craig Topper 5465875e93 [X86] Add support for avx512bf16 for __builtin_cpu_supports and compiler-rt's cpu indicator.
llvm-svn: 370915
2019-09-04 16:01:43 +00:00
Ed Maste 1b6ee802c6 compiler-rt: use 64-bit time_t for all FreeBSD archs except i386
At present only i386 has 32-bit time_t on FreeBSD.

Reviewed by:	dim
Differential Revision:	https://reviews.llvm.org/D66758

llvm-svn: 370756
2019-09-03 13:44:47 +00:00
Ed Maste 14cf2b20ca compiler-rt: use more __sanitizer_time_t on FreeBSD
A few structs were using long for time_t members.  NFC.

Reviewed by:	devnexen
Differential Revision:	https://reviews.llvm.org/D66756

llvm-svn: 370755
2019-09-03 13:41:51 +00:00
Matthew G McGovern 11802ccc9f [cmake] enable x86 libfuzzer on Windows
- recent commit https://reviews.llvm.org/D66433 enabled libfuzzer
    to build on windows, this just enables the option to build as part
    of the the regular build.

llvm-svn: 370390
2019-08-29 17:47:43 +00:00
Vitaly Buka 342197e6fc [sanitizer] Support monorepo layout in symbolizer build script
llvm-svn: 370288
2019-08-29 02:02:03 +00:00
Vitaly Buka f6b9126890 [sanitizer] Simplify COMPILER_RT setup in lint script
llvm-svn: 370277
2019-08-28 23:52:21 +00:00
Alexander Richardson 83d2f0e799 Further relax checks in asan-symbolize-bad-path.cpp
It turns out that the DarwinSymbolizer does not print the "in" part for
invalid files but instead prints
 0xabcdabcd (.../asan-symbolize-bad-path.cpp.tmp/bad/path:i386+0x1234)
This tests is only checking that asan_symbolize.py doesn't hang or crash,
so further relax the checks to ensure that the test passes on macOS.

llvm-svn: 370243
2019-08-28 18:37:53 +00:00
David Carlier 72cb9db5c7 [XRay] Fixing one test case for FreeBSD
Reviewers: dberris

Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D66867

llvm-svn: 370209
2019-08-28 14:18:05 +00:00
Vitaly Buka acd8499028 [compiler-rt] Don't use libcxx if it's not enabled by LLVM_ENABLE_PROJECTS
This fixes sanitizer-x86_64-linux-fuzzer which switched to new layout and not it
can't link as libcxx is always there.
We should support and tests libcxx but still it was unexpected that libfuzzer
ignored LLVM_ENABLE_PROJECTS_USED.

llvm-svn: 370153
2019-08-28 02:41:14 +00:00
Julian Lettner d3136661ba [sanitizer_common] Close superfluous file descriptors in spawned process
Use attribute flag `POSIX_SPAWN_CLOEXEC_DEFAULT` in the call to
`posix_spawn`.

If this flag is set, then only file descriptors explicitly described by
the file_actions argument are available in the spawned process; all of
the other file descriptors are automatically closed in the spawned
process.

POSIX_SPAWN_CLOEXEC_DEFAULT is an Apple-specific extension.

llvm-svn: 370121
2019-08-27 22:12:26 +00:00
Alexander Richardson 79f3459deb Fix asan-symbolize-bad-path.cpp on Darwin
I accidentally made the CHECK line stricter when committing D65322.
While it happens to work for Linux and FreeBSD, it broke on Darwin.
This commit restores the previous behaviour.

llvm-svn: 370110
2019-08-27 21:10:47 +00:00
Mitch Phillips ae56e593b9 Add GWP-ASan fuzz target to compiler-rt/tools.
Summary:
@eugenis to approve addition of //compiler-rt/tools.
@pree-jackie please confirm that this WFY.

D66494 introduced the GWP-ASan stack_trace_compressor_fuzzer. Building fuzz
targets in compiler-rt is a new affair, and has some challenges:
- If the host compiler doesn't have compiler-rt, the -fsanitize=fuzzer may not
  be able to link against `libclang_rt.fuzzer*`.
- Things in compiler-rt generally aren't built when you want to build with
  sanitizers using `-DLLVM_USE_SANITIZER`. This tricky to work around, so
  we create the new tools directory so that we can build fuzz targets with
  sanitizers. This has the added bonus of fixing the problem above as well, as
  we can now just guard the fuzz target build to only be done with
  `-DLLVM_USE_SANITIZE_COVERAGE=On`.

Reviewers: eugenis, pree-jackie

Reviewed By: eugenis, pree-jackie

Subscribers: dberris, mgorny, #sanitizers, llvm-commits, eugenis, pree-jackie, lebedev.ri, vitalybuka, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66776

llvm-svn: 370094
2019-08-27 18:28:07 +00:00
Alexander Richardson 677c6ddd33 [asan_symbolize] Fix broken pipe handling for python 2.7
I D65322 I added a check for BrokenPipeError. However, python 2.7 doesn't
have BrokenPipeError. To be python 2.7 and 3 compatible we need to catch
IOError instead and check for errno == errno.EPIPE.

llvm-svn: 370025
2019-08-27 07:27:41 +00:00
Vitaly Buka 09fcec7c0a NFC: clang-format r370008 to suppress lint errors
llvm-svn: 370023
2019-08-27 06:59:57 +00:00
Alexander Richardson 23a12fc3ac Relax test introduced in D65322
It is possible that addr2line returns a valid function and file name for
the passed address on some build configuations.
The test is only checking that asan_symbolize doesn't assert any more when
passed a valid file with an invalid address so there is no need to check
that it can't find a valid function name.
This should fix http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux

llvm-svn: 370021
2019-08-27 06:50:36 +00:00
David Carlier 5058dd0f49 Fix buildbot
llvm-svn: 370011
2019-08-27 04:24:19 +00:00
David Carlier 4797f68b15 [Sanitizer] Using huge page on FreeBSD for shadow mapping
- Unless explicit configuration, using FreeBSD super pages feature for shadow mapping.
- asan only for now.

Reviewers: dim, emaste, vitalybuka

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D65851

llvm-svn: 370008
2019-08-27 04:02:19 +00:00
Vitaly Buka 4240c13274 [sanitizer] Add lld into dependency of sanitizer_common unittests
llvm-svn: 370007
2019-08-27 02:06:34 +00:00
Vitaly Buka aeca56964f msan, codegen, instcombine: Keep more lifetime markers used for msan
Reviewers: eugenis

Subscribers: hiraditya, cfe-commits, #sanitizers, llvm-commits

Tags: #clang, #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66695

llvm-svn: 369979
2019-08-26 22:15:50 +00:00
Evgeniy Stepanov ed4fefb0df [hwasan] Fix test failure in r369721.
Try harder to emulate "old runtime" in the test.
To get the old behavior with the new runtime library, we need both
disable personality function wrapping and enable landing pad
instrumentation.

llvm-svn: 369977
2019-08-26 21:44:55 +00:00
Alexander Richardson 2511b5a463 [asan_symbolize] Attempt to fix build-bot failure after latest change
llvm-svn: 369929
2019-08-26 17:11:25 +00:00
Alexander Richardson e320db434e [asan_symbolize] Avoid blocking when llvm-symbolizer is installed as addr2line
Summary:
Currently, llvm-symbolizer will print -1 when presented with -1 and not
print a second line. In that case we will block for ever trying to read
the file name. This also happens for non-existent files, in which case GNU
addr2line exits immediate, but llvm-symbolizer does not (see
https://llvm.org/PR42754). While touching these lines, I also added some
more debug logging to help diagnose this and potential future issues.

Reviewers: kcc, eugenis, glider, samsonov

Reviewed By: eugenis

Subscribers: kubamracek, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D65322

llvm-svn: 369924
2019-08-26 16:22:04 +00:00
Bjorn Pettersson 140f06f1b5 [GWP_ASAN] Avoid using VERSION_GREATER_EQUAL in cmake files
This is a fixup for r369823 which introduced the use of
VERSION_GREATER_EQUAL in the cmake config for gwp_asan.

Minimum supported version of cmake in LLVM is 3.4.3 and
VERSION_GREATER_EQUAL was not introduced until later
versions of cmake.

llvm-svn: 369891
2019-08-26 11:02:36 +00:00
Peter Collingbourne b2e7b85468 hwasan: Align n_namesz and n_descsz to 4 when reading notes.
There is no requirement for the producer of a note to include the note
alignment in these fields. As a result we can end up missing the HWASAN note
if one of the other notes in the binary has the alignment missing.

Differential Revision: https://reviews.llvm.org/D66692

llvm-svn: 369826
2019-08-23 23:33:26 +00:00
Mitch Phillips 27d69b2f4f [GWP-ASan] Split options_parser and backtrace_sanitizer_common.
Summary:
optional/options_parser and optional/backtrace_sanitizer_common are logically
separate components. They both use sanitizer-common to power their
functionality, but there was an unstated implicit dependency that in order for
backtrace_sanitizer_common to function correctly, one had to also use
options_parser.

This was because options_parser called __sanitizer::InitialiseCommonFlags. This
is a requirement for backtrace_sanitizer_common to work, as the sanitizer
unwinder uses the sanitizer_common flags and will SEGV on a null page if
they're not initialised correctly.

This patch removes this hidden dependency. You can now use
backtrace_sanitizer_common without the requirements of options_parser.

This patch also makes the GWP-ASan unit tests only have a soft dependency on
sanitizer-common. The unit tests previously explicitly used
__sanitizer::Printf, which is now provided under
tests/optional/printf_sanitizer_common. This allows Android to build the unit
tests using their own signal-safe printf().

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: srhines, mgorny, #sanitizers, llvm-commits, vlad.tsyrklevich, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66684

llvm-svn: 369825
2019-08-23 23:23:48 +00:00
Mitch Phillips 05bc1468c2 Fix stack_trace_compressor builds for Clang < 6.0
Summary:
Clang 4.* doesn't supply -fsanitize=fuzzer, and Clang 5.* doesn't supply
-fsanitize=fuzzer-no-link. Generally, in LLVM, fuzz targets are added through
the add_llvm_fuzzer build rule, which can't be used in compiler-rt (as it has
to be able to be standalone built).

Instead of adding tooling to add a dummy main (which kind of defeats the
purpose of these fuzz targets), we instead build the fuzz target only when the
Clang version is >= 6.*.

Reviewers: tejohnson

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66682

llvm-svn: 369823
2019-08-23 23:13:18 +00:00
Peter Collingbourne f7ca57468a Move a break into the correct place. NFCI.
Should silence new C fallthrough warning.

llvm-svn: 369813
2019-08-23 21:27:56 +00:00
Peter Collingbourne 21a1814417 hwasan: Untag unwound stack frames by wrapping personality functions.
One problem with untagging memory in landing pads is that it only works
correctly if the function that catches the exception is instrumented.
If the function is uninstrumented, we have no opportunity to untag the
memory.

To address this, replace landing pad instrumentation with personality function
wrapping. Each function with an instrumented stack has its personality function
replaced with a wrapper provided by the runtime. Functions that did not have
a personality function to begin with also get wrappers if they may be unwound
past. As the unwinder calls personality functions during stack unwinding,
the original personality function is called and the function's stack frame is
untagged by the wrapper if the personality function instructs the unwinder
to keep unwinding. If unwinding stops at a landing pad, the function is
still responsible for untagging its stack frame if it resumes unwinding.

The old landing pad mechanism is preserved for compatibility with old runtimes.

Differential Revision: https://reviews.llvm.org/D66377

llvm-svn: 369721
2019-08-23 01:28:44 +00:00
Taewook Oh e03f34dd09 [sanitizer] Resubmit D66620 from monorepo
Summary: https://reviews.llvm.org/D66620 is accepted but was based on the multi-repo setup, so I was not able to `arc patch` it. Resubmit the diff under monorepo

Committed on behalf of @sugak (Igor Sugak)

Reviewers: sugak

Subscribers: #sanitizers, llvm-commits, vitalybuka

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66624

llvm-svn: 369716
2019-08-22 23:34:08 +00:00
David Carlier f2003f4cb8 Fixing buildbot due to style.
llvm-svn: 369711
2019-08-22 21:53:21 +00:00
David Carlier e2ed800d62 [Sanitizer] checks ASLR on FreeBSD
- Especially MemorySanitizer fails if those sysctl configs are enabled.

Reviewers: vitalybuka, emaste, dim

Reviewed By: dim

Differential Revision: https://reviews.llvm.org/D66582

llvm-svn: 369708
2019-08-22 21:36:35 +00:00
Matthew G McGovern 2eaeba6f15 LibFuzzer support for 32bit MSVC
This fixes the two build errors when trying to compile LibFuzzer for
32bit with MSVC.

    - authored by Max Shavrick (mxms at microsoft)

llvm-svn: 369704
2019-08-22 20:44:34 +00:00
Petr Hosek 028b5499ff Revert "[GWP-ASan] Remove c++ standard lib dependency."
This reverts commit r369606: this doesn't addressed the underlying
problem and it's not the correct solution.

llvm-svn: 369623
2019-08-22 07:03:38 +00:00
Petr Hosek 3dd2258519 [GWP-ASan] Remove c++ standard lib dependency.
Remove c++ standard library dependency for now for @phosek. They have a
complicated build system that breaks with the fuzzer target here.

Also added a todo to remedy later.

Differential Revision: https://reviews.llvm.org/D66568

llvm-svn: 369606
2019-08-22 00:22:56 +00:00
Julian Lettner 894abb46f8 [TSan] #include header instead of forward declaring interceptees
llvm-svn: 369601
2019-08-21 23:42:06 +00:00
Mitch Phillips c776f3f3c2 [GWP-ASan] Add public-facing documentation [6].
Summary:
Note: Do not submit this documentation until Scudo support is reviewed and submitted (should be #[5]).

See D60593 for further information.

This patch introduces the public-facing documentation for GWP-ASan, as well as updating the definition of one of the options, which wasn't properly merged. The document describes the design and features of GWP-ASan, as well as how to use GWP-ASan from both a user's standpoint, and development documentation for supporting allocators.

Reviewers: jfb, morehouse, vlad.tsyrklevich

Reviewed By: morehouse, vlad.tsyrklevich

Subscribers: kcc, dexonsmith, kubamracek, cryptoad, jfb, #sanitizers, llvm-commits, vlad.tsyrklevich, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D62875

llvm-svn: 369552
2019-08-21 17:53:51 +00:00
Mitch Phillips f9269b5396 [GWP-ASan] Build stack_trace_compressor_fuzzer.
Summary:
Flips the switch to build stack_trace_compressor_fuzzer. This was recently
temporarily disabled in rL369079 as it was breaking the sanitizer buildbots.

My diagnosis of the problem is that on clang-only bootstrap builds, we build
gwp_asan before libfuzzer. This causes a discrepancy when the clang driver
attempts to link libclang_rt.fuzzer* as CMake doesn't see a dependency there.

I've (hopefully) fixed the issue by adding a direct dependency for the fuzz
target so CMake can resolve the build order properly. As part of this, the
libFuzzer 'fuzzer' target has to be discovered before the declaration of the
fuzz target.

pcc@ for mild review + notification as buildcop.

Reviewers: pcc

Reviewed By: pcc

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66494

llvm-svn: 369551
2019-08-21 17:52:51 +00:00
Ulrich Weigand f7489141be [Sanitizer] Disable -Wframe-larger-than on SystemZ
SystemZ builds show -Wframe-larger-than warnings in two functions:
 'sanitizer::SuspendedThreadsListLinux::GetRegistersAndSP'
 'sanitizer::SizeClassAllocator32<__sanitizer::AP32>::PopulateFreeList'

In both cases, the frame size looks correct; each of the functions has
a large local variable that brings the frame size close to the limit
even on x86, and the extra 160 bytes of the default register save areas
on SystemZ pushes it over the limit.

PowerPC and MIPS already disable this warning; do the same on SystemZ.

Differential Revision: https://reviews.llvm.org/D66021

llvm-svn: 369543
2019-08-21 15:53:08 +00:00
Nico Weber 98f800dabe compiler-rt: Fix warning if COMPILER_RT_HAS_FCNTL_LCK is 0
Fixes "warning: implicit declaration of function 'flock' is invalid in C99"
for flock().

llvm-svn: 369534
2019-08-21 15:24:25 +00:00
Sam McCall a451156bb6 reland [gtest] Fix printing of StringRef and SmallString in assert messages.
Renames GTEST_NO_LLVM_RAW_OSTREAM -> GTEST_NO_LLVM_SUPPORT and guards
the new features behind it.

This reverts commit a063bcf3ef5a879adbe9639a3c187d876eee0e66.

llvm-svn: 369527
2019-08-21 13:56:29 +00:00
Vitaly Buka 93a3cbc746 Revert r369472 and r369441
check-sanitizer does not work on Linux

llvm-svn: 369495
2019-08-21 05:06:21 +00:00
Sebastian Pop 5a7bba09ac [AArch64][asan] fix typo in AsanStats::Print
This created an infinite loop that timed out several build bots while
executing the test in compiler-rt/test/asan/TestCases/atexit_stats.cpp

Differential Revision: https://reviews.llvm.org/D60243

llvm-svn: 369472
2019-08-20 23:28:05 +00:00
Sebastian Pop 63487bfec9 [AArch64] Speed-up leak and address sanitizers on AArch64 for 48-bit VMA
This patch fixes https://github.com/google/sanitizers/issues/703
On a Graviton-A1 aarch64 machine with 48-bit VMA,
the time spent in LSan and ASan reduced from 2.5s to 0.01s when running

clang -fsanitize=leak compiler-rt/test/lsan/TestCases/sanity_check_pure_c.c && time ./a.out
clang -fsanitize=address compiler-rt/test/lsan/TestCases/sanity_check_pure_c.c && time ./a.out

With this patch, LSan and ASan create both the 32 and 64 allocators and select
at run time between the two allocators following a global variable that is
initialized at init time to whether the allocator64 can be used in the virtual
address space.

Differential Revision: https://reviews.llvm.org/D60243

llvm-svn: 369441
2019-08-20 20:54:05 +00:00
Mitch Phillips 8f5e1755ca [GWP-ASan] Fix typos.
Summary:
Fix two spelling typos and de-indent a guarded #define so that it's
consistent with clang-format.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66311

llvm-svn: 369433
2019-08-20 20:16:11 +00:00
Kostya Kortchinsky 3e5360f194 [scudo][standalone] Fix malloc_iterate
Summary:
cferris's Bionic tests found an issue in Scudo's `malloc_iterate`.

We were inclusive of both boundaries, which resulted in a `Block` that
was located on said boundary to be possibly accounted for twice, or
just being accounted for while iterating on regions that are not ours
(usually the unmapped ones in between Primary regions).

The fix is to exclude the upper boundary in `iterateOverChunks`, and
add a regression test.

This additionally corrects a typo in a comment, and change the 64-bit
Primary iteration function to not assume that `BatchClassId` is 0.

Reviewers: cferris, morehouse, hctim, vitalybuka, eugenis

Reviewed By: hctim

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66231

llvm-svn: 369400
2019-08-20 16:17:08 +00:00
Hans Wennborg 9e8b011195 [compiler-rt][crt] Pass -fno-lto in check_cxx_section_exists
Otherwise it doesn't work when building with -DLLVM_ENABLE_LTO=thin

(We hit this in Chromium in
https://bugs.chromium.org/p/chromium/issues/detail?id=966403)

llvm-svn: 369336
2019-08-20 07:41:14 +00:00
Julian Lettner def061e6f0 [TSan] Rename file with libdispatch interceptors
llvm-svn: 369314
2019-08-20 00:04:19 +00:00
Julian Lettner 9f985dd380 [sanitizer_common] Extend test after switch to posix_spawn
llvm-svn: 369311
2019-08-19 23:47:35 +00:00
Peter Collingbourne 4340c2cf8f Build symbolizer runtime with C++14.
Should hopefully fix sanitizer-x86_64-linux bot.

llvm-svn: 369290
2019-08-19 18:43:52 +00:00
Julian Lettner e644d5e230 [TSan] Rename file to make it clear that it defines interceptors
Rename file `tsan_libdispatch.cpp -> tsan_libdispatch_interceptors.cpp`
to make it clear that it's main purpose is defining interceptors.

llvm-svn: 369289
2019-08-19 18:41:20 +00:00
David Carlier 949f190810 [Sanitizer] arc4random interception on Mac
Reviewers: yln,vitalybuka

Reviewed By: yln

Differential Revision: https://reviews.llvm.org/D66391

llvm-svn: 369285
2019-08-19 18:12:15 +00:00
Petr Hosek edf504f503 [Fuchsia] Create the VMO during initialization, not during exit
We want to avoid doing expensive work during atexit since the process
might be terminated before we can publish the VMO and write out the
symbolizer markup, so move the VMO creation to the initialization
phase and only write data during the atexit phase.

Differential Revision: https://reviews.llvm.org/D66323

llvm-svn: 369180
2019-08-17 00:54:22 +00:00
Julian Lettner 0c2f26d664 Revert "[TSan] Don't guard #include <xpc/xpc.h>"
This reverts commit 8191585b36.

llvm-svn: 369165
2019-08-16 22:58:13 +00:00
Julian Lettner 4a9b747bfb [TSan] Add interceptors for os_unfair_lock
llvm-svn: 369164
2019-08-16 22:41:25 +00:00
Julian Lettner 8191585b36 [TSan] Don't guard #include <xpc/xpc.h>
The xpc_connection_* APIs that we are intercepting are available
starting at macOS 10.7. This is old enough so that we don't need to
guard them.

llvm-svn: 369150
2019-08-16 20:43:09 +00:00
Mitch Phillips effc28c139 Disable stack_trace_compressor_fuzzer.
Should hopefully fix the remainder of the buildbot issues. Just disabling this
for now with a comment that I'm working on it. Can actually fix the real problem
when I'm at a real computer.

llvm-svn: 369079
2019-08-16 02:03:33 +00:00
Mitch Phillips 4660ea953e Moved binary off add_llvm_executable.
Used add_executable instead, as this allows a standalone compiler-rt to build,
as the add_llvm_executable build target isn't accessible in a standalone CRT
preparation.

llvm-svn: 369071
2019-08-15 23:50:05 +00:00
Mitch Phillips 4be5d53a33 Re-instate 369051.
Looks like I accidentally reverted r369051 to the old
CMake-version-specific flag when committing 369055.

llvm-svn: 369067
2019-08-15 23:28:33 +00:00
Mitch Phillips 2fb4e754c6 Guard fuzzer build behind Clang-only flags.
Should fix sanitizer buildbots and any one else who's building
compiler-rt using gcc.

llvm-svn: 369055
2019-08-15 22:15:46 +00:00
Mitch Phillips fdee340bea Remove CMake >= v3.13 target_link_options.
Instead, use set_target_properties.

llvm-svn: 369051
2019-08-15 21:34:13 +00:00
Mitch Phillips be8a2f7565 [GWP-ASan] Implement stack frame compression.
Summary:
This patch introduces stack frame compression to GWP-ASan. Each stack frame is
variable-length integer encoded as the difference between frame[i] and
frame[i - 1]. Furthermore, we use zig-zag encoding on the difference to ensure
that negative differences are also encoded into a relatively small number of
bytes.

Examples of what the compression looks like can be seen in
`gwp_asan/tests/compression.cpp`.

This compression can reduce the memory consumption cost of stack traces by
~50%.

Reviewers: vlad.tsyrklevich

Reviewed By: vlad.tsyrklevich

Subscribers: mgorny, #sanitizers, llvm-commits, eugenis, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66189

llvm-svn: 369048
2019-08-15 21:09:09 +00:00
Julian Lettner bb99437f6d [sanitizer_common] Always use posix_spawn on Darwin
On Darwin we have two external symbolizers: atos and llvm-symbolizer.
atos was changed to use posix_spawn (instead of fork+execv) in a
previous commit [1]. Let's use posix_spawn for llvm-symbolizer as well.

Our hope is that eventually we can transition to posix_spawn on other
platforms too.

[1] 399408a92f

llvm-svn: 369021
2019-08-15 17:25:10 +00:00
Jonas Devlieghere 8903433bb2 Re-land "[compiler-rt] Migrate llvm::make_unique to std::make_unique"
With the compiler-rt check for C++14 updated in r368960, this should now
be fine to land.

llvm-svn: 369009
2019-08-15 14:57:44 +00:00
Jonas Devlieghere 4d474e078a [CMake] Check for C++14 instead of C++11
Now that LLVM moved to C++14, `COMPILER_RT_HAS_STD_CXX11_FLAG` should
become `COMPILER_RT_HAS_STD_CXX14_FLAG`.

I ran into this issue when replacing llvm::make_unique with
std::make_unique in an X-ray unit test. We are correctly passing
`-std=c++14`, but this got overwritten further down the invocation by
the compiler-rt flags. Given that this unit test is using LLVM headers,
this is bound to break sooner than later, regardless of my change.

Differential revision: https://reviews.llvm.org/D66271

llvm-svn: 368960
2019-08-15 04:42:15 +00:00
Jonas Devlieghere d65f37dbcc Revert "[compiler-rt] Migrate llvm::make_unique to std::make_unique"
The X-ray unit tests in compiler-rt are overriding the C++ version by
explicitly passing -std=c++11 in the compiler invocation. This poses a
problem as these tests are including LLVM headers that can now use C++14
features. I'm temporarily reverting this as I investigate the correct
solution.

llvm-svn: 368952
2019-08-15 03:02:57 +00:00
Julian Lettner 399408a92f [sanitizer_common] Replace forkpty with posix_spawn on Darwin
On Darwin, we currently use forkpty to communicate with the "atos"
symbolizer. There are several problems that fork[pty] has, e.g. that
after fork, interceptors are still active and this sometimes causes
crashes or hangs. This is especially problematic for TSan, which uses
interceptors for OS-provided locks and mutexes, and even Libc functions
use those.

This patch replaces forkpty with posix_spawn on Darwin. Since
posix_spawn doesn't fork (at least on Darwin), the interceptors are not
a problem. Another benefit is that we'll handle post-fork failures (e.g.
sandbox disallows "exec") gracefully now.

Related revisions and previous attempts that were blocked by or had to
be revered due to test failures:
https://reviews.llvm.org/D48451
https://reviews.llvm.org/D40032

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D65253

llvm-svn: 368947
2019-08-15 00:18:55 +00:00
Jonas Devlieghere 5edd6843fa [compiler-rt] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368946
2019-08-15 00:06:49 +00:00
Kostya Kortchinsky 2be59170d4 [scudo][standalone] Add more stats to mallinfo
Summary:
Android requires additional stats in mallinfo. While we can provide
right away the number of bytes mapped (Primary+Secondary), there was
no way to get the number of free bytes (only makes sense for the
Primary since the Secondary unmaps everything on deallocation).

An approximation could be `StatMapped - StatAllocated`, but since we
are mapping in `1<<17` increments for the 64-bit Primary, it's fairly
inaccurate.

So we introduce `StatFree` (note it's `Free`, not `Freed`!), which
keeps track of the amount of Primary blocks currently unallocated.

Reviewers: cferris, eugenis, vitalybuka, hctim, morehouse

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66112

llvm-svn: 368866
2019-08-14 16:04:01 +00:00
Julian Lettner d8c47d52da [TSan] Fix test failing on Linux
llvm-svn: 368641
2019-08-13 00:37:48 +00:00
Mitch Phillips 352d1b59c0 [GWP-ASan] Update backtrace function signature.
Summary:
Updates the function signature and comments for backtracing (and printing
backtraces). This update brings GWP-ASan in line with future requirements for
stack frame compression, wherein the length of the trace is provided
explicitly, rather than relying on nullptr-termination.

Reviewers: vlad.tsyrklevich

Reviewed By: vlad.tsyrklevich

Subscribers: #sanitizers, llvm-commits, morehouse

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D66099

llvm-svn: 368619
2019-08-12 21:36:44 +00:00
Max Moroz 74cec618f3 [libFuzzer] Merge: print feature coverage number as well.
Summary:
feature coverage is a useful signal that is available during the merge
process, but was not printed previously.

Output example:

```
$ ./fuzzer -use_value_profile=1 -merge=1 new_corpus/ seed_corpus/
INFO: Seed: 1676551929
INFO: Loaded 1 modules   (2380 inline 8-bit counters): 2380 [0x90d180, 0x90dacc), 
INFO: Loaded 1 PC tables (2380 PCs): 2380 [0x684018,0x68d4d8), 
MERGE-OUTER: 180 files, 78 in the initial corpus
MERGE-OUTER: attempt 1
INFO: Seed: 1676574577
INFO: Loaded 1 modules   (2380 inline 8-bit counters): 2380 [0x90d180, 0x90dacc), 
INFO: Loaded 1 PC tables (2380 PCs): 2380 [0x684018,0x68d4d8), 
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1048576 bytes
MERGE-INNER: using the control file '/tmp/libFuzzerTemp.111754.txt'
MERGE-INNER: 180 total files; 0 processed earlier; will process 180 files now
	pulse  cov: 134 ft: 330 exec/s: 0 rss: 37Mb
	pulse  cov: 142 ft: 462 exec/s: 0 rss: 38Mb
	pulse  cov: 152 ft: 651 exec/s: 0 rss: 38Mb
	pulse  cov: 152 ft: 943 exec/s: 0 rss: 38Mb
	pulse  cov: 520 ft: 2783 exec/s: 0 rss: 39Mb
	pulse  cov: 552 ft: 3280 exec/s: 0 rss: 41Mb
	pulse  cov: 576 ft: 3641 exec/s: 0 rss: 50Mb
	LOADED cov: 602 ft: 3936 exec/s: 0 rss: 88Mb
	pulse  cov: 611 ft: 3996 exec/s: 0 rss: 93Mb
	DONE   cov: 611 ft: 4016 exec/s: 0 rss: 155Mb
MERGE-OUTER: succesfull in 1 attempt(s)
MERGE-OUTER: the control file has 39741 bytes
MERGE-OUTER: consumed 0Mb (37Mb rss) to parse the control file
MERGE-OUTER: 9 new files with 80 new features added; 9 new coverage edges
```

Reviewers: hctim, morehouse

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66030

llvm-svn: 368617
2019-08-12 20:21:27 +00:00
Nico Weber d2e493c337 Fix Wnewline-eof after r368598
llvm-svn: 368613
2019-08-12 19:57:17 +00:00
Dan Liew c3b93bed29 [asan_symbolize] Fix bug where the frame counter was not incremented.
Summary:
This bug occurred when a plug-in requested that a binary not be
symbolized while the script is trying to symbolize a stack frame. In
this case `self.frame_no` would not be incremented. This would cause
subsequent stack frames that are symbolized to be incorrectly numbered.

To fix this `get_symbolized_lines()` has been modified to take an
argument that indicates whether the stack frame counter should
incremented. In `process_line_posix()` `get_symbolized_lines(None, ...)`
is now used in in the case where we don't want to symbolize a line so
that we can keep the frame counter increment in a single function.

A test case is included. The test uses a dummy plugin that always asks
`asan_symbolize.py` script to not symbolize the first binary that the
script asks about. Prior to the patch this would cause the output to
script to look something like

```
   0x0
   0x0 in do_access
   0x0 in main
```

This is the second attempt at landing this patch. The first (r368373)
failed due to failing some android bots and so was reverted in r368472.
The new test is now disabled for Android. It turns out that the patch
also fails for iOS too so it is also disabled for that family of
platforms too.

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D65495

llvm-svn: 368603
2019-08-12 18:51:25 +00:00
Matthew G McGovern 38a1aa117f [builtins] MSVC warning disable for clean build
- https://reviews.llvm.org/D66023
    - amended for ifdef/if gcc errors in previous verison

llvm-svn: 368598
2019-08-12 18:08:44 +00:00
Kostya Kortchinsky 8095449e68 [scudo][standalone] Minor corrections
Summary:
Few corrections with no functional change:
- replacing `%zd` with `%zu` all around: the values are unsigned
- prefer `MAP_ANONYMOUS` to `MAP_ANON` (it's deprecated)
- remove the unused `enum LinkerInitialized`
- mark a parameter as `UNUSED` in Fuchsia's `getRandom`
- correct the casing of a variable and use `nullptr` instead of 0 for
  pointers in `list.h`
- reorder some `typedef` to be consistent between `signed` and
  `unsigned`

Reviewers: eugenis, vitalybuka, morehouse, hctim

Reviewed By: vitalybuka, morehouse

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D65660

llvm-svn: 368585
2019-08-12 15:35:09 +00:00
Dimitry Andric c09d888632 Add support for FreeBSD's LD_32_LIBRARY_PATH
Summary:
Because the dynamic linker for 32-bit executables on 64-bit FreeBSD uses
the environment variable `LD_32_LIBRARY_PATH` instead of
`LD_LIBRARY_PATH` to find needed dynamic libraries, running the 32-bit
parts of the dynamic ASan tests will fail with errors similar to:

```
ld-elf32.so.1: Shared object "libclang_rt.asan-i386.so" not found, required by "Asan-i386-inline-Dynamic-Test"
```

This adds support for setting up `LD_32_LIBRARY_PATH` for the unit and
regression tests.  It will likely also require a minor change to the
`TestingConfig` class in `llvm/utils/lit/lit`.

Reviewers: emaste, kcc, rnk, arichardson

Reviewed By: arichardson

Subscribers: kubamracek, krytarowski, fedor.sergeev, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D65772

llvm-svn: 368516
2019-08-10 19:07:38 +00:00
David Carlier 8d1646bf66 [Sanitizer] Reenable getusershell interception
and disabling it forAndroid.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D66027

llvm-svn: 368504
2019-08-10 03:35:54 +00:00
Julian Lettner 5ef4b190d9 [Sanitizer][Darwin] Add interceptor for malloc_zone_from_ptr
Ensure that malloc_default_zone and malloc_zone_from_ptr return the
sanitizer-installed malloc zone even when MallocStackLogging (MSL) is
requested. This prevents crashes in certain situations. Note that the
sanitizers and MSL cannot be used together. If both are enabled, MSL
functionality is essentially deactivated since it only hooks the default
allocator which is replaced by a custom sanitizer allocator.

rdar://53686175

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D65990

llvm-svn: 368492
2019-08-09 21:46:32 +00:00
Eric Christopher 11c1847237 Revert "[sanitizers] MSVC warning disable for clean build" and follow-up that tried to fix the build as it's still broken.
This reverts commit 368476 and 368480.

llvm-svn: 368481
2019-08-09 20:43:36 +00:00
Martin Storsjo 96a2b25bcb Fix compilation after SVN r368476
That revision broke compilation with this error:

lib/builtins/fixunsxfdi.c:13:2: error: unterminated conditional directive
 #if !_ARCH_PPC

llvm-svn: 368480
2019-08-09 20:36:00 +00:00
Matthew G McGovern 8e2842cc85 [sanitizers] MSVC warning disable for clean build
- https://reviews.llvm.org/D66023

llvm-svn: 368476
2019-08-09 20:09:46 +00:00
Mitch Phillips cace571c91 Revert "[asan_symbolize] Fix bug where the frame counter was not incremented."
This reverts commit 52a36fae2a.

This commit broke the sanitizer_android buildbot. See comments at
https://reviews.llvm.org/rL368373 for more details.

llvm-svn: 368472
2019-08-09 19:36:41 +00:00
Max Moroz 3653aeeffa [libFuzzer] Merge: print stats after reading the output corpus dir.
Summary:
The purpose is to be able to extract the number of new edges added to
the original (i.e. output) corpus directory after doing the merge. Use case
example: in ClusterFuzz, we do merge after every fuzzing session, to avoid
uploading too many corpus files, and we also record coverage stats at that
point. Having a separate line indicating stats after reading the initial output
corpus directory would make the stats extraction easier for both humans and
parsing scripts.

Context: https://github.com/google/clusterfuzz/issues/802.

Reviewers: morehouse, hctim

Reviewed By: hctim

Subscribers: delcypher, #sanitizers, llvm-commits, kcc

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66020

llvm-svn: 368461
2019-08-09 18:20:53 +00:00
Max Moroz f8744ab57a [compiler-rt] FuzzedDataProvider: use C++ headers only instead of a C/C++ mix.
Reviewers: Dor1s

Reviewed By: Dor1s

Subscribers: dberris, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D66017

llvm-svn: 368448
2019-08-09 16:00:53 +00:00
Benjamin Kramer 8819a734ae [sanitizer] Update symbolizer/scripts/global_symbols.txt
llvm-svn: 368440
2019-08-09 14:03:30 +00:00
Dan Liew 52a36fae2a [asan_symbolize] Fix bug where the frame counter was not incremented.
Summary:
This bug occurred when a plug-in requested that a binary not be
symbolized while the script is trying to symbolize a stack frame. In
this case `self.frame_no` would not be incremented. This would cause
subsequent stack frames that are symbolized to be incorrectly numbered.

To fix this `get_symbolized_lines()` has been modified to take an
argument that indicates whether the stack frame counter should
incremented. In `process_line_posix()` `get_symbolized_lines(None, ...)`
is now used in in the case where we don't want to symbolize a line so
that we can keep the frame counter increment in a single function.

A test case is included. The test uses a dummy plugin that always asks
`asan_symbolize.py` script to not symbolize the first binary that the
script asks about. Prior to the patch this would cause the output to
script to look something like

```
   0x0
   0x0 in do_access
   0x0 in main
```

rdar://problem/49476995

Reviewers: kubamracek, yln, samsonov, dvyukov, vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D65495

llvm-svn: 368373
2019-08-09 00:52:07 +00:00