Commit Graph

9434 Commits

Author SHA1 Message Date
Fangrui Song c5b94ea265 [profile] Support merge pool size >= 10
The executable acquires an advisory record lock (`fcntl(fd, F_SETLKW, *)`) on a profile file.
Merge pool size >= 10 may be beneficial when the concurrency is large.

Also fix a small problem about snprintf. It can cause the filename to be truncated after %m.

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D71970
2020-01-12 00:27:18 -08:00
Lirong Yuan a5a6fd3f95 Summary: update macro for OFF_T so that sanitizer works on AARCH64.
Reviewers: vitalybuka, eugenis, MaskRay

Reviewed By: eugenis, MaskRay

Subscribers: MaskRay, kristof.beyls, #sanitizers, llvm-commits, jkz, scw

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D72367
2020-01-10 13:34:49 -08:00
Evgenii Stepanov 7ba4595c86 [msan] Fix underflow in qsort interceptor. 2020-01-06 16:12:52 -08:00
Evgenii Stepanov b5e7f95cfb [msan] Check qsort input.
Summary:
Qsort interceptor suppresses all checks by unpoisoning the data in the
wrapper of a comparator function, and then unpoisoning the output array
as well.

This change adds an explicit run of the comparator on all elements of
the input array to catch any sanitizer bugs.

Reviewers: vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71780
2020-01-06 15:07:02 -08:00
Kamil Rytarowski 0da15ff318 [compiler-rt] [netbsd] Switch to syscall for ThreadSelfTlsTcb()
This change avoids using internal, namespaced per-CPU calls that
are not a stable interface to read the TSL base value.
2020-01-03 02:40:52 +01:00
Sterling Augustine 108daf7611 Check for aarch64 when not using .init_arrays
Reviewers: vvereschaka

Subscribers: kristof.beyls, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D72005
2020-01-02 10:10:49 -08:00
Fangrui Song 33a1b3d8fc [sanitizer] Link Sanitizer-x86_64-Test-Nolibc with -static
Pass -static so that clang will not pass -Wl,--dynamic-linker,... to the
linker. The test is not expected to run under a ld.so. (Technically it
works under a ld.so but glibc expects to see a PT_DYNAMIC. lld
intentionally does not follow GNU ld's complex rules regarding
PT_DYNAMIC.)

This allows commit 1417558e4a to be
relanded.
2019-12-27 15:25:08 -08:00
Sterling Augustine 1d891a32cf Support powerpc and sparc when building without init_array.
Summary: Support powerpc and sparc when building without init_array.

Reviewers: rdhindsa, gribozavr

Subscribers: jyknight, nemanjai, fedor.sergeev, jsji, shchenz, steven.zhang, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71944
2019-12-27 13:02:11 -08:00
Reid Kleckner ef7a659c21 Reland "[msan] Intercept qsort, qsort_r."
This reverts commit 8fcce5ac73.

I spoke too soon, the revert does not actually cause the startup crash
to go away.
2019-12-27 11:29:00 -08:00
Reid Kleckner 8fcce5ac73 Revert "[msan] Intercept qsort, qsort_r."
This reverts commit 7a9ebe9512, and
dependent commit 54c5224203, which
disables qsort interception for some iOS platforms.

After this change, the -Nolibc sanitizer common test binary crashes on
startup on my regular Linux workstation, as well as on our bots:
https://ci.chromium.org/p/chromium/builders/try/linux_upload_clang/740

 ********************
  Failing Tests (1):
       SanitizerCommon-Unit ::
       ./Sanitizer-x86_64-Test/SanitizerCommon.NolibcMain

Loading it up in gdb shows that it crashes during relocation processing,
which suggests that some glibc loader versions do not support the
THREADLOCAL data added in this interceptor.
2019-12-27 11:24:07 -08:00
Kamil Rytarowski 84afd9c536 [compiler-rt] [netbsd] Add support for versioned statvfs interceptors
Summary:
Add support for NetBSD 9.0 and newer versions of interceptors
operating on struct statvfs: fstatvfs, fstatvfs1, getmntinfo,
getvfsstat, statvfs, statvfs1.

The default promoted interceptors are for NetBSD 9.99.26. Older
ones (currently 9.0) are kept in a new NetBSD specific file:
/sanitizer_common_interceptors_netbsd_compat.inc. This file
defines compat interceptors and mangles `INIT_*` macros,
concatenating the current interceptors and the compat ones.
This redefinition is not elegant, but it avoids preprocessor madness.

Define struct_statvfs90_sz for the compat purposes.

Reviewers: mgorny, kcc, vitalybuka, joerg

Reviewed By: mgorny

Subscribers: dberris, llvm-commits, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71700
2019-12-27 18:50:14 +01:00
Eric Christopher 8f9304f559 Temporarily Revert "[compiler-rt] [netbsd] Add support for versioned statvfs interceptors"
as it's failing the netbsd specific linter parts of the sanitizer linter:

llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors_netbsd_compat.inc:23:  Lines should be <= 80 characters long  [whitespace/line_length]
llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_netbsd.cpp:2450:  Do not use variable-length arrays.  Use an appropriately named ('k' followed by CamelCase) compile-time constant for the size.

This reverts commit 78f714f824.
2019-12-26 18:40:20 -08:00
Kamil Rytarowski 78f714f824 [compiler-rt] [netbsd] Add support for versioned statvfs interceptors
Summary:
Add support for NetBSD 9.0 and newer versions of interceptors
operating on struct statvfs: fstatvfs, fstatvfs1, getmntinfo,
getvfsstat, statvfs, statvfs1.

The default promoted interceptors are for NetBSD 9.99.26. Older
ones (currently 9.0) are kept in a new NetBSD specific file:
/sanitizer_common_interceptors_netbsd_compat.inc. This file
defines compat interceptors and mangles `INIT_*` macros,
concatenating the current interceptors and the compat ones.
This redefinition is not elegant, but it avoids preprocessor madness.

Define struct_statvfs90_sz for the compat purposes.

Reviewers: mgorny, kcc, vitalybuka, joerg

Reviewed By: mgorny

Subscribers: dberris, llvm-commits, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71700
2019-12-27 01:15:39 +01:00
Evgenii Stepanov 04926e67fd Revert "[msan] Check qsort input."
This change breaks LLVM bootstrap with ASan and MSan.

FAILED: lib/ToolDrivers/llvm-lib/Options.inc
OptParser.td:137:1: error: Option is equivalent to
def INPUT : Option<[], "<input>", KIND_INPUT>;
^
OptParser.td:137:1: error: Other defined here
def INPUT : Option<[], "<input>", KIND_INPUT>;

This reverts commit caa48a6b88.
2019-12-26 12:29:48 -08:00
Florian Hahn 54c5224203 [compiler-rt] Disable QSORT interception on watchOS and tvOS.
Building the sanitizers for watchOS currently fails with
    sanitizer_common_interceptors.inc:9656:8: error: thread-local storage is not supported for the current target
    static THREADLOCAL SIZE_T qsort_size;

I've also speculatively disabled QSORT interception for tvOS to unblock
failing builds. I'll ask someone with more sanitizer knowledge to check
after the holidays.
2019-12-26 10:39:29 +01:00
Kamil Rytarowski aff6c9db9a [compiler-rt] [netbsd] Correct the fallback definition of PT_LWPNEXT
Fixes build on NetBSD 9.0.
2019-12-25 18:21:48 +01:00
Kamil Rytarowski 52b5fe5f45 [compiler-rt] [netbsd] Define _RTLD_SOURCE to fix build
The TLS base (LWP private pointer) functions are namespaced and
hidden i.e. inside the _RTLD_SOURCE namespace.
2019-12-25 04:09:50 +01:00
Kamil Rytarowski 4b8232d4f0 [compiler-rt] Adapt for ptrace(2) changes in NetBSD-9.99.30
Enable compat support for now legacy PT_LWPINFO.
Support PT_LWPSTATUS and PT_LWPNEXT.
2019-12-24 20:34:58 +01:00
Kamil Rytarowski fc356dcc11 [compiler-rt] Adapt stop-the-world for ptrace changes in NetBSD-9.99.30
Handle PT_LWPNEXT for newer kernels and keep PT_LWPINFO for older ones.
2019-12-24 20:33:54 +01:00
Evgenii Stepanov caa48a6b88 [msan] Check qsort input.
Summary:
Qsort interceptor suppresses all checks by unpoisoning the data in the
wrapper of a comparator function, and then unpoisoning the output array
as well.

This change adds an explicit run of the comparator on all elements of
the input array to catch any sanitizer bugs.

Reviewers: vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71780
2019-12-23 11:34:49 -08:00
Evgenii Stepanov 7a9ebe9512 [msan] Intercept qsort, qsort_r.
Summary:
This fixes qsort-related false positives with glibc-2.27.
I'm not entirely sure why they did not show up with the earlier
versions; the code seems similar enough.

Reviewers: vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71740
2019-12-23 11:34:49 -08:00
Jonas Devlieghere 60236fedc9 Revert "[msan] Check qsort input." and "[msan] Intercept qsort, qsort_r."
Temporarily revert the qsort changes because they fail to build on bots
that build with modules:

> error: thread-local storage is not supported for the current
> target (iossim)

http://green.lab.llvm.org/green/job/clang-stage2-Rthinlto/1820/console
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/4983/console

This reverts commit ddf897fc80.
This reverts commit 07861e955d.
2019-12-20 21:34:35 -08:00
Kostya Kortchinsky 0fd6f19025 [scudo][standalone] Support __BIONIC__
Summary:
Some Android builds that we are interested in define `__BIONIC__`
but not `__ANDROID__`, so expand `SCUDO_ANDROID` to encompass those.

Reviewers: cferris, hctim, pcc, eugenis, morehouse

Subscribers: krytarowski, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71772
2019-12-20 12:59:33 -08:00
Evgenii Stepanov ddf897fc80 [msan] Check qsort input.
Summary:
Qsort interceptor suppresses all checks by unpoisoning the data in the
wrapper of a comparator function, and then unpoisoning the output array
as well.

This change adds an explicit run of the comparator on all elements of
the input array to catch any sanitizer bugs.

Reviewers: vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71780
2019-12-20 12:41:57 -08:00
Evgenii Stepanov 07861e955d [msan] Intercept qsort, qsort_r.
Summary:
This fixes qsort-related false positives with glibc-2.27.
I'm not entirely sure why they did not show up with the earlier
versions; the code seems similar enough.

Reviewers: vitalybuka

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71740
2019-12-20 12:27:09 -08:00
Kostya Kortchinsky 77e906ac78 [scudo][standalone] Implement TSD registry disabling
Summary:
In order to implement `malloc_{enable|disable}` we were just disabling
(or really locking) the Primary and the Secondary. That meant that
allocations could still be serviced from the TSD as long as the cache
wouldn't have to be filled from the Primary.

This wasn't working out for Android tests, so this change implements
registry disabling (eg: locking) so that `getTSDAndLock` doesn't
return a TSD if the allocator is disabled. This also means that the
Primary doesn't have to be disabled in this situation.

For the Shared Registry, we loop through all the TSDs and lock them.
For the Exclusive Registry, we add a `Disabled` boolean to the Registry
that forces `getTSDAndLock` to use the Fallback TSD instead of the
thread local one. Disabling the Registry is then done by locking the
Fallback TSD and setting the boolean in question (I don't think this
needed an atomic variable but I might be wrong).

I clang-formatted the whole thing as usual hence the couple of extra
whiteline changes in this CL.

Reviewers: cferris, pcc, hctim, morehouse, eugenis

Subscribers: jfb, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71719
2019-12-20 06:52:13 -08:00
Dmitri Gribenko 0109efe751 Revert "Support powerpc when builing without init_array."
This reverts commit 5789e83ded. It broke
the build on aarch64.
2019-12-19 11:25:14 +01:00
Kamil Rytarowski 3a189bac9b [compiler-rt] Enable SANITIZER_CAN_USE_PREINIT_ARRAY on NetBSD
.preinit_array is supported since 9.0.
2019-12-19 03:23:51 +01:00
Kamil Rytarowski 7e8541f3df [compiler-rt] [fuzzer] Include stdarg.h for va_list
Fixes build on NetBSD after 139e216e66.
2019-12-19 01:11:39 +01:00
Sterling Augustine 5789e83ded Support powerpc when builing without init_array.
Summary: Also add an error case when targetting an unimplement architecture.

Subscribers: nemanjai, jsji, shchenz, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71670
2019-12-18 13:37:08 -08:00
Kamil Rytarowski 3f96686700 Correct version check 9.9.26 -> 9.99.26 2019-12-18 21:10:33 +01:00
Mitch Phillips 2f4c3c565a [NFC] Fix sanitizer lint problem that's breaking Android bot.
Fixes the lint issue introduced by
e14ff22694
2019-12-18 08:47:31 -08:00
Kamil Rytarowski e14ff22694 [compiler-rt] Sync NetBSD ioctl definitions with 9.99.26
Document the minimal version supported to 9.0.
2019-12-18 15:44:01 +01:00
Igor Kudrin a57adc7a0b [sanitizer] Construct InternalMmapVector without memory allocation.
Construction of InternalMmapVector is often followed by a call to
reserve(), which may result in immediate reallocation of the memory
for the internal storage. This patch delays that allocation until
it is really needed.

Differential Revision: https://reviews.llvm.org/D71342
2019-12-17 15:03:23 +07:00
Julian Lettner f9a706a36a [TSan][Darwin] Avoid calling pthread_self() before libpthread is initialized
This skips calling `pthread_self` when `main_thread_identity` hasn't
been initialized yet.  `main_thread_identity` is only ever assigned in
`__tsan::InitializePlatform`.  This change should be relatively safe; we
are not changing behavior other than skipping the call to `pthread_self`
when `main_thread_identity == 0`.

rdar://57822138

Reviewed By: kubamracek

Differential Revision: https://reviews.llvm.org/D71559
2019-12-16 13:10:51 -08:00
Mitch Phillips d6c445ea69 [NFC] Guard scudo_standalone's optional dependency on GWP-ASan behind flags. 2019-12-13 10:55:53 -08:00
Mitch Phillips a00cd6dfea [GWP-ASan] [Scudo] ifdef entire GWP-ASan tests.
Turns out that gtest in LLVM is only 1.8.0 (the newest version 1.10.0)
supports the GTEST_SKIP() macro, and apparently I didn't build w/o
GWP-ASan.

Should fix the GN bot, as well as any bots that may spuriously break on
platforms where the code wasn't correctly ifdef'd out as well.
2019-12-13 09:45:28 -08:00
Mitch Phillips ed4618edb3 [Scudo] [GWP-ASan] Add GWP-ASan to Scudo Standalone.
Summary:
Adds GWP-ASan to Scudo standalone. Default parameters are pulled across from the
GWP-ASan build. No backtrace support as of yet.

Reviewers: cryptoad, eugenis, pcc

Reviewed By: cryptoad

Subscribers: merge_guards_bot, mgorny, #sanitizers, llvm-commits, cferris, vlad.tsyrklevich, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71229
2019-12-13 09:09:41 -08:00
Eric Christopher 3ecfe64fe3 [sanitizers] Redirect stdout and stderr to TASK_LOG
At some point cpplint.py became very noisy during a build spewing
a few hundred lines of "Done processing..." even with SILENT=1 in
cmake. This attempts to redirect the stdout of "Done processing" to
the task log along with any errors.

Tested by this with and without SILENT=1 to check things.

Differential Revision: https://reviews.llvm.org/D71402
Reviewed By: eugenis
2019-12-12 13:11:16 -08:00
Max Moroz 926fa4088c [compiler-rt] libFuzzer: update -merge_control_file= help message.
Summary:
The motivation for this change is to have a distinguisher in libFuzzer
that would let the runner know whether multistep merge is supported or not by
a particular fuzz target binary. Otherwise, multistep merge fails to execute
with older version of libFuzzer, and there is no way to verify that easily.

Reviewers: kcc

Subscribers: dberris, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71423
2019-12-12 11:09:40 -08:00
Jonathan Metzman 23bee0b0cf [fuzzer] Add basic support for emscripten.
Summary:
Add basic support for emscripten.

This enables libFuzzer to build (using build.sh) for emscripten and fuzz
a target compiled with
-fsanitize-coverage=inline-8bit-counters.

Basic fuzzing and bug finding work with this commit.
RSS limit and timeouts will not work because they depend on system
functions that are not implemented/widely supported in emscripten.

Reviewers: kcc, vitalybuka, hctim

Reviewed By: hctim

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71285
2019-12-12 08:56:47 -08:00
Eric Christopher 6c79095576 Revert unintentional change to compiler-rt as part of the __bit_reference revert. 2019-12-12 00:31:23 -08:00
Calixte Denizet 02ce9d8ef5 [compiler-rt] Add a critical section when flushing gcov counters
Summary:
Counters can be flushed in a multi-threaded context for example when the process is forked in different threads (https://github.com/llvm/llvm-project/blob/master/llvm/lib/Transforms/Instrumentation/GCOVProfiling.cpp#L632-L663).
In order to avoid pretty bad things, a critical section is needed around the flush.
We had a lot of crashes in this code in Firefox CI when we switched to clang for linux ccov builds and those crashes disappeared with this patch.

Reviewers: marco-c, froydnj, dmajor, davidxl, vsk

Reviewed By: marco-c, dmajor

Subscribers: ahatanak, froydnj, dmajor, dberris, jfb, #sanitizers, llvm-commits, sylvestre.ledru

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70910
2019-12-12 09:23:32 +01:00
Eric Christopher fa0fc04a4f Temporarily Revert "[libc++] Fix -Wdeprecated-copy warnings in __bit_reference"
as it's causing test failures due to mismatched visibility.

This reverts commit 02bb20223b.
2019-12-12 00:22:37 -08:00
Vedant Kumar 5a486e0f15 [profile] Avoid allocating a page on the stack, NFC
When writing out a profile, avoid allocating a page on the stack for the
purpose of writing out zeroes, as some embedded environments do not have
enough stack space to accomodate this.

Instead, use a small, fixed-size zero buffer that can be written
repeatedly.

For a synthetic file with >100,000 functions, I did not measure a
significant difference in profile write times. We are removing a
page-length zero-fill `memset()` in favor of several smaller buffered
`fwrite()` calls: in practice, I am not sure there is much of a
difference. The performance impact is only expected to affect the
continuous sync mode (%c) -- zero padding is less than 8 bytes in all
other cases.

rdar://57810014

Differential Revision: https://reviews.llvm.org/D71323
2019-12-11 10:04:00 -08:00
Igor Kudrin 3a713ae5fa [SanitizerCommon] Reduce wasting memory in LowLevelAllocator.
MmapOrDie allocates memory multiple to page size. LowLevelAllocator
should use all that memory for the internal buffer because there are
chances that subsequent requests may be small enough to fit in that
space.

Differential Revision: https://reviews.llvm.org/D71275
2019-12-11 12:44:55 +07:00
Peter Collingbourne 9fbfdd2bfe scudo: Tweak how we align UserPtr. NFCI.
Instead of testing whether the pointer is aligned, just align it
unconditionally and compare it to the original pointer.

This moves the computation of UserPtr up to before we start preparing the
header, so that the memory tagging code will be able to read the original
header containing the bounds of the previous allocation before it gets
potentially clobbered by the pointer realignment code.

Differential Revision: https://reviews.llvm.org/D71292
2019-12-10 12:09:47 -08:00
Peter Collingbourne e966416ff1 scudo: Move getChunkFromBlock() allocated check into caller. NFCI.
With tag-on-free we will need to get the chunk of a deallocated block. Change
getChunkFromBlock() so that it doesn't check that the chunk is allocated,
and move the check into the caller, so that it can be reused for this purpose.

Differential Revision: https://reviews.llvm.org/D71291
2019-12-10 12:08:52 -08:00
Kostya Kortchinsky b36b16372d [scudo][standalone] Define hasHardwareCRC32 for other archs
Summary:
The function was only defined for x86 and arm families, which ends
up being an issue for PPC in g3.

Define the function, simply returning `false` for "other"
architectures.

Reviewers: hctim, pcc, cferris, eugenis, vitalybuka

Subscribers: kristof.beyls, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71223
2019-12-10 07:41:46 -08:00
Akira Hatanaka 78a7af456d Revert "[compiler-rt] Add a critical section when flushing gcov counters"
This reverts commit 88f5bf77f9 as it broke
green dragon bots.

http://lab.llvm.org:8080/green/job/clang-stage1-RA/4401/
2019-12-09 19:58:26 -08:00