Commit Graph

13855 Commits

Author SHA1 Message Date
Daniel Kiss 201550852b [AArch64][compiler-rt] Add Pointer Authentication support for VFORK.
The LR is stored to off-stack spill area where it is vulnerable.
"paciasp" add an auth code to the LR while the "autiasp" verifies that so
LR can't be modiifed on the spill area.

Test: build with -DCMAKE_C_FLAGS="-mbranch-protection=standard",
run on Armv8.3 capable hardware with PAuth.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D98009
2021-03-10 16:39:39 +01:00
Alex Richardson cbcab14b19 [compiler-rt] Silence a deprecation warning on FreeBSD
On FreeBSD the sys/timeb.h header has a #warning that it's deprecated.
However, we need to include this header here, so silence this warning that
is printed multiple times otherwise.

Reviewed By: dim

Differential Revision: https://reviews.llvm.org/D94963
2021-03-10 11:17:33 +00:00
Alex Richardson 6840fc4e7f [asan][tests] Handle FreeBSD in large_func_test and use-after-delete
This is the same fix as ab8a460241 for
Solaris and 23cd8d51ad (Windows+Darwin).
2021-03-10 11:17:33 +00:00
Alex Richardson 5853e1ca7f [sanitizers] Fix typo in ca9815fc24
I accidentally committed the wrong version of this patch which didn't
actually enable the hooks for FreeBSD. Fixing the typo allows the tests
to actually pass.
2021-03-10 11:17:33 +00:00
Martin Storsjö 8bd2722f65 [compiler-rt] Normalize i?86 to i386 and armv* to arm for COMPILER_RT_DEFAULT_TARGET_ARCH
This corresponds to getArchNameForCompilerRTLib in clang; any
32 bit x86 architecture triple (except on android, but those
exceptions are already handled in compiler-rt on a different level)
get the compiler rt library names with i386; arm targets get either
"arm" or "armhf". (Mapping to "armhf" is handled in the toplevel
CMakeLists.txt.)

Differential Revision: https://reviews.llvm.org/D98173
2021-03-10 08:43:54 +02:00
Jianzhou Zhao 8506fe5b41 [dfsan] Tracking origins at memory transfer
This is a part of https://reviews.llvm.org/D95835.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D98192
2021-03-09 22:15:07 +00:00
Fangrui Song 27ddcd51d3 [profile] Fix InstrProfGetRangeRepValue to use UINT64_C
This is a minor issue because the TargetValue parameter of `__llvm_profile_instrument_memop`
is usually small and cannot exceed 2**31 at all.

Differential Revision: https://reviews.llvm.org/D97640
2021-03-09 12:46:13 -08:00
Peter Collingbourne 1f55fa0b99 scudo: Add support for tracking stack traces of secondary allocations.
There is no centralized store of information related to secondary
allocations. Moreover the allocations themselves become inaccessible
when the allocation is freed in order to implement UAF detection,
so we can't store information there to be used in case of UAF
anyway.

Therefore our storage location for tracking stack traces of secondary
allocations is a ring buffer. The ring buffer is copied to the process
creating the crash dump when a fault occurs.

The ring buffer is also used to store stack traces for primary
deallocations. Stack traces for primary allocations continue to be
stored inline.

In order to support the scenario where an access to the ring buffer
is interrupted by a concurrently occurring crash, the ring buffer is
accessed in a lock-free manner.

Differential Revision: https://reviews.llvm.org/D94212
2021-03-09 11:43:03 -08:00
MarcoFalke 0cccccf0d2
[compiler-rt] FuzzedDataProvider: Add missing include
Differential Revision: https://reviews.llvm.org/D98057
2021-03-09 12:26:34 +01:00
Dmitry Vyukov 373e1681c9 [sanitizers] Enable runtime vma for mips64 in buildgo.sh
Go requires 47 bits VA for tsan.
Go will run race_detector testcases unless tsan warns about "unsupported VMA range"

Author: mzh (Meng Zhuo)
Reviewed-in: https://reviews.llvm.org/D98238
2021-03-09 11:45:55 +01:00
Dmitry Vyukov 85801b4c68 [sanitizers] fix wrong enum of memory_order for mips
Author: mzh (Meng Zhuo)
Reviewed-in: https://reviews.llvm.org/D98231
2021-03-09 11:28:45 +01:00
Peter Collingbourne 3f71ce8589 scudo: Support memory tagging in the secondary allocator.
This patch enhances the secondary allocator to be able to detect buffer
overflow, and (on hardware supporting memory tagging) use-after-free
and buffer underflow.

Use-after-free detection is implemented by setting memory page
protection to PROT_NONE on free. Because this must be done immediately
rather than after the memory has been quarantined, we no longer use the
combined allocator quarantine for secondary allocations. Instead, a
quarantine has been added to the secondary allocator cache.

Buffer overflow detection is implemented by aligning the allocation
to the right of the writable pages, so that any overflows will
spill into the guard page to the right of the allocation, which
will have PROT_NONE page protection. Because this would require the
secondary allocator to produce a header at the correct position,
the responsibility for ensuring chunk alignment has been moved to
the secondary allocator.

Buffer underflow detection has been implemented on hardware supporting
memory tagging by tagging the memory region between the start of the
mapping and the start of the allocation with a non-zero tag. Due to
the cost of pre-tagging secondary allocations and the memory bandwidth
cost of tagged accesses, the allocation itself uses a tag of 0 and
only the first four pages have memory tagging enabled.

This is a reland of commit 7a0da88943 which was reverted in commit
9678b07e42. This reland includes the following changes:

- Fix the calculation of BlockSize which led to incorrect statistics
  returned by mallinfo().
- Add -Wno-pedantic to silence GCC warning.
- Optionally add some slack at the end of secondary allocations to help
  work around buggy applications that read off the end of their
  allocation.

Differential Revision: https://reviews.llvm.org/D93731
2021-03-08 14:39:33 -08:00
Luís Marques c5a6ad86b0 [Sanitizer][RISCV] Fix internal_clone
A RISC-V implementation of `internal_clone` was introduced in D87573, as
part of the RISC-V ASan patch set by @EccoTheDolphin. That function was
never used/tested until I ported LSan for RISC-V, as part of D92403. That
port revealed problems in the original implementation, so I provided a fix
in D92403. Unfortunately, my choice of replacing the assembly with regular
C++ code wasn't correct. The clone syscall arguments specify a separate
stack, so non-inlined calls, spills, etc. aren't going to work. This wasn't
a problem in practice for optimized builds of Compiler-RT, but it breaks
for debug builds. This patch fixes the original problem while keeping the
assembly.

Differential Revision: https://reviews.llvm.org/D96954
2021-03-08 22:29:59 +00:00
Kuba Mracek 8a8fa09cb7 [tsan] Use large address space mapping on Apple Silicon Macs
Differential Revision: https://reviews.llvm.org/D86377
2021-03-08 14:09:42 -08:00
Alex Richardson ad294e572b [sanitizers] Fix interception of GLibc regexec
Previously, on GLibc systems, the interceptor was calling __compat_regexec
(regexec@GLIBC_2.2.5) insead of the newer __regexec (regexec@GLIBC_2.3.4).
The __compat_regexec strips the REG_STARTEND flag but does not report an
error if other flags are present. This can result in infinite loops for
programs that use REG_STARTEND to find all matches inside a buffer (since
ignoring REG_STARTEND means that the search always starts from the first
character).

The underlying issue is that GLibc's dlsym(RTLD_NEXT, ...) appears to
always return the oldest versioned symbol instead of the default. This
means it does not match the behaviour of dlsym(RTLD_DEFAULT, ...) or the
behaviour documented in the manpage.

It appears a similar issue was encountered with realpath and worked around
in 77ef78a0a5.

See also https://sourceware.org/bugzilla/show_bug.cgi?id=14932 and
https://sourceware.org/bugzilla/show_bug.cgi?id=1319.

Fixes https://github.com/google/sanitizers/issues/1371

Reviewed By: #sanitizers, vitalybuka, marxin

Differential Revision: https://reviews.llvm.org/D96348
2021-03-08 10:53:55 +00:00
Jeremy Morse 66e3a4abe9 Revert "[tsan] Use large address space mapping on Apple Silicon Macs"
This reverts commit bde2e56071.

This patch produces a compile failure on linux amd64 environments, when
running:

  ninja GotsanRuntimeCheck

I get various build errors:

  ../rtl/tsan_platform.h:608: error: use of undeclared identifier 'Mapping'
    return MappingImpl<Mapping, Type>();

Here's a buildbot with the same failure during stage "check-tsan in gcc
build", there are other unrelated failures in there.

  http://lab.llvm.org:8011/#/builders/37/builds/2831
2021-03-08 09:51:51 +00:00
Raul Tambre ba860963b1 [CMake][compiler-rt] Use copying instead of symlinking for LSE builtins on non-Unix-likes
As reported in D93278 post-review symlinking requires privilege escalation on Windows.
Copying is functionally same, so fallback to it for systems that aren't Unix-like.
This is similar to the solution in AddLLVM.cmake.

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D98111
2021-03-08 09:10:36 +02:00
Kuba Mracek bde2e56071 [tsan] Use large address space mapping on Apple Silicon Macs
Differential Revision: https://reviews.llvm.org/D86377
2021-03-07 12:47:08 -08:00
Elia Geretto b46c89892f [XRay][compiler-rt][x86_64] Fix CFI directives in assembly trampolines
This patch modifies the x86_64 XRay trampolines to fix the CFI information
generated by the assembler. One of the main issues in correcting the CFI
directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on
the alignment of the stack. However, this macro is not really necessary because
some additional assumptions can be made on the alignment of the stack when the
trampolines are called. The code has been written as if the stack is guaranteed
to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8
bytes with respect to a 16-bytes alignment. For this reason, always moving the
stack pointer by 8 bytes is sufficient to restore the appropriate alignment.

Trampolines that are called from within a function as a result of the builtins
`__xray_typedevent` and `__xray_customevent` are necessarely called with the
stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be
eliminated.

Fixes https://bugs.llvm.org/show_bug.cgi?id=49060

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D96785
2021-03-06 10:38:27 -08:00
Fangrui Song ca747e48af [sanitizer] Restrict clock_gettime workaround to glibc
The hackery is due to glibc clock_gettime crashing from preinit_array (D40679).
32-bit musl architectures do not define `__NR_clock_gettime` so the code causes a compile error.

Tested on Alpine Linux x86-64 (musl) and FreeBSD x86-64.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D96925
2021-03-06 10:32:27 -08:00
Vitaly Buka 56ed64dfa9 [sanitizer] Don't expect ABORTING in print-module-map
ABORTING message is inconsistent across sanitizers.

Another followup for D98089
2021-03-05 19:22:34 -08:00
Jianzhou Zhao 469d5462fa [dfsan] Re-enable origin tracking test cases 2021-03-06 02:41:56 +00:00
Vitaly Buka 2fcd872d8a [dfsan] Remove dfsan_get_origin from done_abilist.txt
Followup for D95835
2021-03-05 17:59:39 -08:00
Vitaly Buka 1c5f083128 [NFC] Fix module map test
Followup for D98089
2021-03-05 17:23:19 -08:00
Jianzhou Zhao d02e0ba070 [dfsan] Disable origin test cases temporarily 2021-03-06 01:12:54 +00:00
Emily Shi c0503df15d [sanitizers] fix print-module-map test on linux
Looks like the default options for halt_on_error are different between linux and mac. set it to 0 in the test so the behavior is the same on both platforms.

rdar://75110847

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D98089
2021-03-05 16:41:37 -08:00
Rainer Orth ab8a460241 [asan][test] Handle Solaris in large_func_test.cpp etc.
Two ASan tests currently `FAIL' on Solaris

  AddressSanitizer-i386-sunos :: TestCases/large_func_test.cpp
  AddressSanitizer-i386-sunos :: TestCases/use-after-delete.cpp

both for the same reason:

  error: no check strings found with prefix 'CHECK-SunOS:'

Fixed by adding the appropriate check strings.

Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D97931
2021-03-06 01:28:17 +01:00
Jianzhou Zhao c20db7ea6a [dfsan] Add utils to get and print origin paths and some test cases
This is a part of https://reviews.llvm.org/D95835.

Reviewed By: morehouse, gbalats

Differential Revision: https://reviews.llvm.org/D97962
2021-03-06 00:11:35 +00:00
Emily Shi 51d8f598ad [sanitizers] run print module map test run on posix
Previously, this test only ran for mac because platforms have different messaging. This diff enables the test for all posix

rdar://75110847

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D98079
2021-03-05 15:41:14 -08:00
Petr Hosek b654b22197 Partially revert "[runtimes] Use add_lit_testsuite to register lit testsuites"
This partially reverts commit e1173c8794
until we find out why compiler-rt tests are failing under runtimes build.
2021-03-05 15:24:38 -08:00
Emily Shi 9059903f2d [ubsan] support print_module_map flag in standalone mode
Currently, `print_module_map` is only respected for ubsan if it is ran in tandem with asan. This patch adds support for this flag in standalone mode. I copied the pattern used to implement this for asan.

Also added a common `print_module_map` lit test for Darwin only. Since the print messages are different per platform, we need to write a regex test to cover them. This test is coming in a separate patch

rdar://56135732

Reviewed By: vitalybuka, vsk, delcypher

Differential Revision: https://reviews.llvm.org/D97746
2021-03-05 13:59:56 -08:00
Petr Hosek e1173c8794 [runtimes] Use add_lit_testsuite to register lit testsuites
The runtimes build uses variables set by add_lit_testsuite to collect
testsuites from all the runtimes.

Differential Revision: https://reviews.llvm.org/D97913
2021-03-05 10:37:21 -08:00
Fangrui Song 0c5bfe6bb8 [profile][test] Pin Linux/instrprof-value-prof-warn.test to -fuse-ld=bfd
To work around https://sourceware.org/bugzilla/show_bug.cgi?id=27490
2021-03-05 00:52:25 -08:00
Rainer Orth 579fd02597 [asan][test] Don't XFAIL Posix/unpoison-alternate-stack.cpp on Solaris
One ASan test currently `XPASS`es on Solaris:

  AddressSanitizer-i386-sunos :: TestCases/Posix/unpoison-alternate-stack.cpp

It was originally `XFAIL`ed in D88501 <https://reviews.llvm.org/D88501>
because `longjmp` from a signal handled is highly unportable, warned
against in XPG7, and was not supported by Solaris `libc` at the time.

However, since then support has been added for some cases including the
current one, so the `XFAIL` can go.

Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D97933
2021-03-05 09:43:47 +01:00
Rainer Orth 1d0dee51da [asan][test] Don't XFAIL Posix/no_asan_gen_globals.c on Solaris
One ASan test currently `XPASS`es on Solaris:

  AddressSanitizer-i386-sunos :: TestCases/Posix/no_asan_gen_globals.c

It was originally `XFAIL`ed in D88218 <https://reviews.llvm.org/D88218>
because Solaris `ld`, unlike GNU `ld`, doesn't strip local labels.  Since
then, the integrated assembler has stopped emitting those local labels, so
the difference becomes moot and the `XFAIL` can go.

Tested on `amd64-pc-solaris2.11` and `x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D97932
2021-03-05 09:42:29 +01:00
Vitaly Buka 8a07c4a10b [sanitizer,NFC] Fix long comment formating 2021-03-04 20:58:42 -08:00
Vitaly Buka 657a58a571 [dfsan,NFC] Suppress cpplint warning 2021-03-04 20:42:18 -08:00
Mitch Phillips 1be97975cc Change instrprof LLVM_VP_MAX_NUM_VALS_PER_SITE threshold.
We're having flaky failures on this test on the sanitizer slow
buildbot. Not per-run flaky, but it'll be green for a while, then red
for a while. I suspect that changes in codegen are causing the
LLVM_VP_MAX_NUM_VALS_PER_SITE=150 to be above and below the limit
sporadically. The limit on my machine using lld and a non-bootstrapped
compiler is 175, but the bot uses GNU ld and ld.gold at different
points, which could be affecting behaviour.

Change this threshold to LLVM_VP_MAX_NUM_VALS_PER_SITE=130 in order to
try and get it below the failure point, at least for the foreseeable
future.

http://lab.llvm.org:8011/#/builders/37/builds/2744
2021-03-04 12:13:47 -08:00
Zequan Wu 9783e20988 Revert "Revert "[Coverage] Emit gap region between statements if first statements contains terminate statements.""
Reland with update on test case ContinuousSyncmode/basic.c.

This reverts commit fe5c2c3ca6.
2021-03-04 11:52:43 -08:00
Nico Weber fe5c2c3ca6 Revert "[Coverage] Emit gap region between statements if first statements contains terminate statements."
This reverts commit 2d7374a0c6.
Breaks ContinuousSyncMode/basic.c in check-profile on macOS.
2021-03-04 08:53:30 -05:00
Hans Wennborg 840a16d3c4 Revert "[darwin] switch blocking mutex from osspinlock to os_unfair_lock"
> OSSpinLock is deprecated, so we are switching to `os_unfair_lock`. However, `os_unfair_lock` isn't available on older OSs, so we keep `OSSpinLock` as fallback.
>
> Also change runtime assumption check to static since they only ever check constant values.
>
> rdar://69588111
>
> Reviewed By: delcypher, yln
>
> Differential Revision: https://reviews.llvm.org/D97509

This reverts commit 71ef54337d.
2021-03-04 12:22:39 +01:00
Petr Hosek 46a3f4ae27 Revert "[XRay][x86_64] Fix CFI directives in assembly trampolines"
This reverts commit 9ee61cf3f6 since
it's failing to compile on Darwin.
2021-03-04 01:03:04 -08:00
Siva Chandra Reddy 0106370bee [compiler-rt | interceptors] Provide an intercept override knob.
This knob is useful for downstream users who want that some of their
libc functions to not be intercepted.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D97740
2021-03-03 21:03:46 -08:00
Fangrui Song a84f4fc0df [InstrProfiling] Place __llvm_prf_vnodes and __llvm_prf_names in llvm.used on ELF
`__llvm_prf_vnodes` and `__llvm_prf_names` are used by runtime but not
referenced via relocation in the translation unit.

With `-z start-stop-gc` (LLD 13 (D96914); GNU ld 2.37 https://sourceware.org/bugzilla/show_bug.cgi?id=27451),
the linker does not let `__start_/__stop_` references retain their sections.

Place `__llvm_prf_vnodes` and `__llvm_prf_names` in `llvm.used` to make
them retained by the linker.

This patch changes most existing `UsedVars` cases to `CompilerUsedVars`
to reflect the ideal state - if the binary format properly supports
section based GC (dead stripping), `llvm.compiler.used` should be sufficient.

`__llvm_prf_vnodes` and `__llvm_prf_names` are switched to `UsedVars`
since we want them to be unconditionally retained by both compiler and linker.

Behaviors on COFF/Mach-O are not affected.

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D97649
2021-03-03 11:32:24 -08:00
Fangrui Song 75df61e93d [test] Improve PGO tests 2021-03-03 11:32:24 -08:00
Zequan Wu 2d7374a0c6 [Coverage] Emit gap region between statements if first statements contains terminate statements.
Differential Revision: https://reviews.llvm.org/D97101
2021-03-03 11:25:49 -08:00
Petr Hosek 61a792b39b [CMake] Rename RUNTIMES_BUILD to LLVM_RUNTIMES_BUILD
This avoid potential conflict with other internal variables.

Differential Revision: https://reviews.llvm.org/D97838
2021-03-03 10:58:51 -08:00
Elia Geretto 9ee61cf3f6 [XRay][x86_64] Fix CFI directives in assembly trampolines
This patch modifies the x86_64 XRay trampolines to fix the CFI information
generated by the assembler. One of the main issues in correcting the CFI
directives is the `ALIGNED_CALL_RAX` macro, which makes the CFA dependent on
the alignment of the stack. However, this macro is not really necessary because
some additional assumptions can be made on the alignment of the stack when the
trampolines are called. The code has been written as if the stack is guaranteed
to be 8-bytes aligned; however, it is instead guaranteed to be misaligned by 8
bytes with respect to a 16-bytes alignment. For this reason, always moving the
stack pointer by 8 bytes is sufficient to restore the appropriate alignment.

Trampolines that are called from within a function as a result of the builtins
`__xray_typedevent` and `__xray_customevent` are necessarely called with the
stack properly aligned so, in this case too, `ALIGNED_CALL_RAX` can be
eliminated.

Fixes: https://bugs.llvm.org/show_bug.cgi?id=49060

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D96785
2021-03-03 10:01:28 -08:00
Nico Weber 64f5d7e972 Revert "[InstrProfiling] Place __llvm_prf_vnodes and __llvm_prf_names in llvm.used on ELF"
This reverts commit 04c3040f41.
Breaks instrprof-value-merge.c in bootstrap builds.
2021-03-03 10:21:17 -05:00
Petr Hosek 6e3946c9f5 [runtimes] Use standalone build only for compiler-rt
compiler-rt needs to use standalone build because of the assumptions
made by its build, but other runtimes can use non-standalone build.

Differential Revision: https://reviews.llvm.org/D97575
2021-03-03 00:06:20 -08:00
Petr Hosek b3ac90da1d Revert "[runtimes] Use standalone build only for compiler-rt"
This reverts commit 4e421b2323 as this
seemed to have broke Python 3 executable detection on some builders.
2021-03-02 16:59:32 -08:00
Petr Hosek 4e421b2323 [runtimes] Use standalone build only for compiler-rt
compiler-rt needs to use standalone build because of the assumptions
made by its build, but other runtimes can use non-standalone build.

Differential Revision: https://reviews.llvm.org/D97575
2021-03-02 16:21:35 -08:00
Luís Marques 2b5f3f446f [Sanitizer][NFC] Fix typo 2021-03-01 23:47:03 +00:00
Fangrui Song 04c3040f41 [InstrProfiling] Place __llvm_prf_vnodes and __llvm_prf_names in llvm.used on ELF
`__llvm_prf_vnodes` and `__llvm_prf_names` are used by runtime but not
referenced via relocation in the translation unit.

With `-z start-stop-gc` (D96914 https://sourceware.org/bugzilla/show_bug.cgi?id=27451),
the linker no longer lets `__start_/__stop_` references retain them.

Place `__llvm_prf_vnodes` and `__llvm_prf_names` in `llvm.used` to make
them retained by the linker.

This patch changes most existing `UsedVars` cases to `CompilerUsedVars`
to reflect the ideal state - if the binary format properly supports
section based GC (dead stripping), `llvm.compiler.used` should be sufficient.

`__llvm_prf_vnodes` and `__llvm_prf_names` are switched to `UsedVars`
since we want them to be unconditionally retained by both compiler and linker.

Behaviors on other COFF/Mach-O are not affected.

Differential Revision: https://reviews.llvm.org/D97649
2021-03-01 13:43:23 -08:00
Vedant Kumar 2fcc3f4b18 [test/profile] Disable instrprof-write-buffer-internal.c on Windows
This is failing due to:

"instrprof-write-buffer-internal.c.tmp.buf.profraw: Invalid
instrumentation profile data (file header is corrupt)"

https://lab.llvm.org/buildbot/#/builders/127/builds/6830
2021-03-01 11:06:38 -08:00
Vedant Kumar 8f7dc99647 [test/profile] Pass -w to suppress suggestion to use fopen_s 2021-03-01 11:06:38 -08:00
Emily Shi 71ef54337d [darwin] switch blocking mutex from osspinlock to os_unfair_lock
OSSpinLock is deprecated, so we are switching to `os_unfair_lock`. However, `os_unfair_lock` isn't available on older OSs, so we keep `OSSpinLock` as fallback.

Also change runtime assumption check to static since they only ever check constant values.

rdar://69588111

Reviewed By: delcypher, yln

Differential Revision: https://reviews.llvm.org/D97509
2021-03-01 10:52:47 -08:00
Vedant Kumar 18adbb86f9 [test/profile] Add test coverage for __llvm_profile_write_buffer_internal
Reviewed By: davidxl, MaskRay

Differential Revision: https://reviews.llvm.org/D97697
2021-03-01 10:46:14 -08:00
Fangrui Song dc93b1127c [profile] Delete zero-size dummy sections
They were added so that if no metadata section is present,
`__start_llvm_prf_*` references would not cause "undefined symbol"
errors.  By switching to undefined weak symbols in D96936, the dummy
sections are not needed.

This patch is also needed to work around
https://sourceware.org/bugzilla/show_bug.cgi?id=27490

Differential Revision: https://reviews.llvm.org/D97648
2021-02-28 21:07:30 -08:00
Vitaly Buka c88c46080a [NFC] Remove tab from the source 2021-02-26 19:40:24 -08:00
Jianzhou Zhao c0dc885d29 [msan] Use non-transparent-huge-page at SetShadow
This prevents from getting THP ranges more and more.

Did not see any issues in practice, just found this by code review.

Reviewed By: eugenis, vitalybuka

Differential Revision: https://reviews.llvm.org/D97593
2021-02-27 00:28:57 +00:00
Ryan Prichard d202201410 Reland "[builtins] Define fmax and scalbn inline"
This reverts commit 680f836c2f.

Disable the non-default-rounding-mode scalbn[f] tests when we're using
the MSVC libraries.

Differential Revision: https://reviews.llvm.org/D91841
2021-02-26 16:20:14 -08:00
Vitaly Buka 812a906133 [sanitizers][NFC] Change typesto avoid warnings
Warning was enabled by D94640
2021-02-26 14:32:54 -08:00
Vitaly Buka e29063b16e [NFC] Suppress "warning: ignoring return value" 2021-02-26 14:32:54 -08:00
Jianzhou Zhao c5c316f6d9 [dfsan] Do not test origin-tracking in atomic.cpp
This would cause linking errors after https://reviews.llvm.org/D97483
that introduced new prefixes for ABI wrappers with origin tracking mode.
We will renable this after the full origin tracking is checked in.
2021-02-26 19:44:18 +00:00
Leonard Chan bed8882426 [scudo][test] Disable -Wfree-nonheap-object
As of 4f395db86b which contains updates to
-Wfree-nonheap-object, a line in this test will trigger the warning. This
particular line is ok though since it's meant to test a free on a bad pointer.

Differential Revision: https://reviews.llvm.org/D97516
2021-02-26 11:14:50 -08:00
Jianzhou Zhao a47d435bc4 [dfsan] Propagate origins for callsites
This is a part of https://reviews.llvm.org/D95835.

Each customized function has two wrappers. The
first one dfsw is for the normal shadow propagation. The second one dfso is used
when origin tracking is on. It calls the first one, and does additional
origin propagation. Which one to use can be decided at instrumentation
time. This is to ensure minimal additional overhead when origin tracking
is off.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D97483
2021-02-26 19:12:03 +00:00
Peter Collingbourne 9678b07e42 Revert 7a0da88943, "scudo: Support memory tagging in the secondary allocator."
We measured a 2.5 seconds (17.5%) regression in Android boot time
performance with this change.
2021-02-25 16:50:02 -08:00
Jianzhou Zhao c88fedef2a [dfsan] Conservative solution to atomic load/store
DFSan at store does store shadow data; store app data; and at load does
load shadow data; load app data.

When an application data is atomic, one overtainting case is

thread A: load shadow
thread B: store shadow
thread B: store app
thread A: load app

If the application address had been used by other flows, thread A reads
previous shadow, causing overtainting.

The change is similar to MSan's solution.
1) enforce ordering of app load/store
2) load shadow after load app; store shadow before shadow app
3) do not track atomic store by reseting its shadow to be 0.
The last one is to address a case like this.

Thread A: load app
Thread B: store shadow
Thread A: load shadow
Thread B: store app

This approach eliminates overtainting as a trade-off between undertainting
flows via shadow data race.

Note that this change addresses only native atomic instructions, but
does not support builtin libcalls yet.
   https://llvm.org/docs/Atomics.html#libcalls-atomic

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D97310
2021-02-25 23:34:58 +00:00
Kostya Kortchinsky 2c56776a31 [scudo][standalone] Compact pointers for Caches/Batches
This CL introduces configuration options to allow pointers to be
compacted in the thread-specific caches and transfer batches. This
offers the possibility to have them use 32-bit of space instead of
64-bit for the 64-bit Primary, thus cutting the size of the caches
and batches by nearly half (and as such the memory used in size
class 0). The cost is an additional read from the region information
in the fast path.

This is not a new idea, as it's being used in the sanitizer_common
64-bit primary. The difference here is that it is configurable via
the allocator config, with the possibility of not compacting at all.

This CL enables compacting pointers in the Android and Fuchsia default
configurations.

Differential Revision: https://reviews.llvm.org/D96435
2021-02-25 12:14:38 -08:00
Vedant Kumar a7d4826101 [profile] Fix buffer overrun when parsing %c in filename string
Fix a buffer overrun that can occur when parsing '%c' at the end of a
filename pattern string.

rdar://74571261

Reviewed By: kastiglione

Differential Revision: https://reviews.llvm.org/D97239
2021-02-24 14:49:45 -08:00
Ryan Prichard 680f836c2f Revert "[builtins] Define fmax and scalbn inline"
This reverts commit 341889ee9e.

The new unit tests fail on sanitizer-windows.
2021-02-24 14:47:48 -08:00
Ryan Prichard 341889ee9e [builtins] Define fmax and scalbn inline
Define inline versions of __compiler_rt_fmax* and __compiler_rt_scalbn*
rather than depend on the versions in libm. As with
__compiler_rt_logbn*, these functions are only defined for single,
double, and quad precision (binary128).

Fixes PR32279 for targets using only these FP formats (e.g. Android
on arm/arm64/x86/x86_64).

For single and double precision, on AArch64, use __builtin_fmax[f]
instead of the new inline function, because the builtin expands to the
AArch64 fmaxnm instruction.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D91841
2021-02-24 14:27:37 -08:00
Vitaly Buka bc897bad66 [asan] Increase CHECK limit in __sanitizer_annotate_contiguous_container
Asan allocator already support up to (1 << 40) bytes allocations.
2021-02-23 22:14:42 -08:00
Jianzhou Zhao a05aa0dd5e [dfsan] Update memset and dfsan_(set|add)_label with origin tracking
This is a part of https://reviews.llvm.org/D95835.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D97302
2021-02-23 23:16:33 +00:00
Emily Shi 956c90d347 [darwin] use new crash reporter api
Add support for the new crash reporter api if the headers are available. Falls back to the old API if they are not available. This change was based on [[ 0164d546d2/llvm/lib/Support/PrettyStackTrace.cpp (L111) | /llvm/lib/Support/PrettyStackTrace.cpp ]]

There is a lit for this behavior here: https://reviews.llvm.org/D96737 but is not included in this diff because it is potentially flaky.

rdar://69767688

Reviewed By: delcypher, yln

Commited by Dan Liew on behalf of Emily Shi.

Differential Revision: https://reviews.llvm.org/D96830
2021-02-23 09:23:23 -08:00
Emily Shi b6099fa515 [darwin][asan] add test for application specific information in crash logs
Added a lit test that finds its corresponding crash log and checks to make sure it has asn output under `Application Specific Information`.

This required adding two python commands:
- `get_pid_from_output`: takes the output from the asan instrumentation and parses out the process ID
- `print_crashreport_for_pid`: takes in the pid of the process and the file name of the binary that was run and prints the contents of the corresponding crash log.

This test was added in preparation for changing the integration with crash reporter from the old api to the new api, which is implemented in a subsequent commit.

rdar://69767688

Reviewed By: delcypher

Commited by Dan Liew on behalf of Emily Shi.

Differential Revision: https://reviews.llvm.org/D96737
2021-02-23 09:22:11 -08:00
Martin Liska 16ede0956c Fix UBSAN in __ubsan::Value::getSIntValue
/home/marxin/Programming/gcc2/libsanitizer/ubsan/ubsan_value.cpp:77:25: runtime error: left shift of 0x0000000000000000fffffffffffffffb by 96 places cannot be represented in type '__int128'
    #0 0x7ffff754edfe in __ubsan::Value::getSIntValue() const /home/marxin/Programming/gcc2/libsanitizer/ubsan/ubsan_value.cpp:77
    #1 0x7ffff7548719 in __ubsan::Value::isNegative() const /home/marxin/Programming/gcc2/libsanitizer/ubsan/ubsan_value.h:190
    #2 0x7ffff7542a34 in handleShiftOutOfBoundsImpl /home/marxin/Programming/gcc2/libsanitizer/ubsan/ubsan_handlers.cpp:338
    #3 0x7ffff75431b7 in __ubsan_handle_shift_out_of_bounds /home/marxin/Programming/gcc2/libsanitizer/ubsan/ubsan_handlers.cpp:370
    #4 0x40067f in main (/home/marxin/Programming/testcases/a.out+0x40067f)
    #5 0x7ffff72c8b24 in __libc_start_main (/lib64/libc.so.6+0x27b24)
    #6 0x4005bd in _start (/home/marxin/Programming/testcases/a.out+0x4005bd)

Differential Revision: https://reviews.llvm.org/D97263
2021-02-23 11:10:21 +01:00
Luís Marques ebca13c665 [Sanitizer][NFC] Fix typo 2021-02-23 09:24:35 +00:00
Petr Hosek b6876ddc82 [CMake][profile] Don't use `TARGET lld` to avoid ordering issues
Depending on the order in which lld and compiler-rt projects are
processed by CMake, `TARGET lld` might evaluate to `TRUE` or `FALSE`
even though `lld-available` lit stanza is always set because lld is
being built. We check whether lld project is enabled instead which
is used by other compiler-rt tests.

The ideal solution here would be to use CMake generator expressions,
but those cannot be used for dependencies yet, see:
https://gitlab.kitware.com/cmake/cmake/-/issues/19467

Differential Revision: https://reviews.llvm.org/D97256
2021-02-22 23:33:21 -08:00
Kamlesh Kumar 18035991cd [builtins] Replace __SOFT_FP__ with __SOFTFP__
Fix PR46294

Differential Revision: https://reviews.llvm.org/D82014
2021-02-22 22:57:23 -08:00
Peter Collingbourne 7a0da88943 scudo: Support memory tagging in the secondary allocator.
This patch enhances the secondary allocator to be able to detect buffer
overflow, and (on hardware supporting memory tagging) use-after-free
and buffer underflow.

Use-after-free detection is implemented by setting memory page
protection to PROT_NONE on free. Because this must be done immediately
rather than after the memory has been quarantined, we no longer use the
combined allocator quarantine for secondary allocations. Instead, a
quarantine has been added to the secondary allocator cache.

Buffer overflow detection is implemented by aligning the allocation
to the right of the writable pages, so that any overflows will
spill into the guard page to the right of the allocation, which
will have PROT_NONE page protection. Because this would require the
secondary allocator to produce a header at the correct position,
the responsibility for ensuring chunk alignment has been moved to
the secondary allocator.

Buffer underflow detection has been implemented on hardware supporting
memory tagging by tagging the memory region between the start of the
mapping and the start of the allocation with a non-zero tag. Due to
the cost of pre-tagging secondary allocations and the memory bandwidth
cost of tagged accesses, the allocation itself uses a tag of 0 and
only the first four pages have memory tagging enabled.

Differential Revision: https://reviews.llvm.org/D93731
2021-02-22 14:35:39 -08:00
Petr Hosek c24b7a16b1 [InstrProfiling] Use ELF section groups for counters, data and values
__start_/__stop_ references retain C identifier name sections such as
__llvm_prf_*. Putting these into a section group disables this logic.

The ELF section group semantics ensures that group members are retained
or discarded as a unit. When a function symbol is discarded, this allows
allows linker to discard counters, data and values associated with that
function symbol as well.

Note that `noduplicates` COMDAT is lowered to zero-flag section group in
ELF. We only set this for functions that aren't already in a COMDAT and
for those that don't have available_externally linkage since we already
use regular COMDAT groups for those.

Differential Revision: https://reviews.llvm.org/D96757
2021-02-22 14:00:02 -08:00
Petr Hosek 4827492d9f Revert "[InstrProfiling] Use ELF section groups for counters, data and values"
This reverts commits:
5ca21175e0
97184ab99c

The instrprof-gc-sections.c is failing on AArch64 LLD bot.
2021-02-22 11:13:55 -08:00
Joachim Protze ed4230732a [sanitizers] Pass CMAKE_C_FLAGS into TSan buildgo script
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.

If CMAKE_C_FLAGS contains -Wno-unused-command-line-argument or
-Wno-error=unused-command-line-argument, the latter flag is passed into the
build script.

This is a re-commit. The previous version was reverted because of failing
tests.

Differential Revision: https://reviews.llvm.org/D96762
2021-02-22 18:49:02 +01:00
Petr Hosek 97184ab99c [InstrProfiling] Fix instrprof-gc-sections.c test
After D97110 __llvm_prof_cnts has the nobits type so it's empty.
2021-02-21 23:47:18 -08:00
Nico Weber 4b34e0c797 Revert "[sanitizers] Pass CMAKE_C_FLAGS into TSan buildgo script"
This reverts commit ac6c13bfc4.
Breaks building with PGO, see https://reviews.llvm.org/D96762#2574009
2021-02-21 22:13:59 -05:00
Petr Hosek 5ca21175e0 [InstrProfiling] Use ELF section groups for counters, data and values
__start_/__stop_ references retain C identifier name sections such as
__llvm_prf_*. Putting these into a section group disables this logic.

The ELF section group semantics ensures that group members are retained
or discarded as a unit. When a function symbol is discarded, this allows
allows linker to discard counters, data and values associated with that
function symbol as well.

Note that `noduplicates` COMDAT is lowered to zero-flag section group in
ELF. We only set this for functions that aren't already in a COMDAT and
for those that don't have available_externally linkage since we already
use regular COMDAT groups for those.

Differential Revision: https://reviews.llvm.org/D96757
2021-02-21 16:13:06 -08:00
Luís Marques 7c31661b22 [Sanitizers][NFC] Fix typo 2021-02-20 10:54:00 +00:00
Luís Marques 43fa23a01f [Sanitizer][NFC] Fix typo 2021-02-19 17:46:02 +00:00
Matthew Malcomson c1653b8cc7 Hwasan InitPrctl check for error using internal_iserror
When adding this function in https://reviews.llvm.org/D68794 I did not
notice that internal_prctl has the API of the syscall to prctl rather
than the API of the glibc (posix) wrapper.

This means that the error return value is not necessarily -1 and that
errno is not set by the call.

For InitPrctl this means that the checks do not catch running on a
kernel *without* the required ABI (not caught since I only tested this
function correctly enables the ABI when it exists).
This commit updates the two calls which check for an error condition to
use internal_iserror. That function sets a provided integer to an
equivalent errno value and returns a boolean to indicate success or not.

Tested by running on a kernel that has this ABI and on one that does
not. Verified that running on the kernel without this ABI the current
code prints the provided error message and does not attempt to run the
program. Verified that running on the kernel with this ABI the current
code does not print an error message and turns on the ABI.
This done on an x86 kernel (where the ABI does not exist), an AArch64
kernel without this ABI, and an AArch64 kernel with this ABI.

In order to keep running the testsuite on kernels that do not provide
this new ABI we add another option to the HWASAN_OPTIONS environment
variable, this option determines whether the library kills the process
if it fails to enable the relaxed syscall ABI or not.
This new flag is `fail_without_syscall_abi`.
The check-hwasan testsuite results do not change with this patch on
either x86, AArch64 without a kernel supporting this ABI, and AArch64
with a kernel supporting this ABI.

Differential Revision: https://reviews.llvm.org/D96964
2021-02-19 16:30:56 +00:00
Vitaly Buka df2940aa85 [android] Fix testing adb call
Looks like some debug leftover which force another retry.
2021-02-18 17:24:07 -08:00
Petr Hosek 5fbd1a333a [Coverage] Store compilation dir separately in coverage mapping
We currently always store absolute filenames in coverage mapping.  This
is problematic for several reasons. It poses a problem for distributed
compilation as source location might vary across machines.  We are also
duplicating the path prefix potentially wasting space.

This change modifies how we store filenames in coverage mapping. Rather
than absolute paths, it stores the compilation directory and file paths
as given to the compiler, either relative or absolute. Later when
reading the coverage mapping information, we recombine relative paths
with the working directory. This approach is similar to handling
ofDW_AT_comp_dir in DWARF.

Finally, we also provide a new option, -fprofile-compilation-dir akin
to -fdebug-compilation-dir which can be used to manually override the
compilation directory which is useful in distributed compilation cases.

Differential Revision: https://reviews.llvm.org/D95753
2021-02-18 14:34:39 -08:00
Petr Hosek fbf8b957fd Revert "[Coverage] Store compilation dir separately in coverage mapping"
This reverts commit 97ec8fa5bb since
the test is failing on some bots.
2021-02-18 12:50:24 -08:00
Petr Hosek 97ec8fa5bb [Coverage] Store compilation dir separately in coverage mapping
We currently always store absolute filenames in coverage mapping.  This
is problematic for several reasons. It poses a problem for distributed
compilation as source location might vary across machines.  We are also
duplicating the path prefix potentially wasting space.

This change modifies how we store filenames in coverage mapping. Rather
than absolute paths, it stores the compilation directory and file paths
as given to the compiler, either relative or absolute. Later when
reading the coverage mapping information, we recombine relative paths
with the working directory. This approach is similar to handling
ofDW_AT_comp_dir in DWARF.

Finally, we also provide a new option, -fprofile-compilation-dir akin
to -fdebug-compilation-dir which can be used to manually override the
compilation directory which is useful in distributed compilation cases.

Differential Revision: https://reviews.llvm.org/D95753
2021-02-18 12:27:42 -08:00
Jianzhou Zhao 063a6fa87e [dfsan] Add origin tls/move/read APIs
This is a part of https://reviews.llvm.org/D95835.

Added
1) TLS storage
2) a weak global used to set by instrumented code
3) move origins

These APIs are similar to MSan's APIs
  https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/msan/msan_poisoning.cpp
We first improved MSan's by https://reviews.llvm.org/D94572 and https://reviews.llvm.org/D94552.
So the correctness has been verified by MSan.
After the DFSan instrument code is ready, we wil be adding more test
cases

4) read

To reduce origin tracking cost, some of the read APIs return only
the origin from the first taint data.

Note that we did not add origin set APIs here because they are related
to code instrumentation, will be added later with IR transformation
code.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96564
2021-02-18 17:48:20 +00:00
Fangrui Song 833d4d8e89 [profile] Make {__start_,__stop_}__llvm_prf_* symbols undefined weak
To make a kind of metadata section usage work, we want to drop the
`__start_/__stop_ references retain C identifier name sections` rule from LLD (see D96914).

If an application has no `__llvm_prf_data` input section surviving --gc-sections,
LLD will error for undefined hidden `{__start_,__stop_}__llvm_prf_*` from `libclang_rt.profile-*`.
Other `__llvm_prf_*` sections have similar issues.

Making the references weak can address the problem.
This probably enables the opportunity to drop zero size dummy sections in `InstrProfilingPlatformLinux.c`.

Reviewed By: davidxl

Differential Revision: https://reviews.llvm.org/D96936
2021-02-17 23:33:13 -08:00
Fangrui Song da59c2e4dc [GWP-ASan] Change sys/cdefs.h to features.h
sys/cdefs.h is a glibc internal header which is not supposed to be included by applications.
(Some libc implementations provide this file for compatibility.)
Android features.h includes sys/cdefs.h, so we can include features.h instead.

This change makes `ninja gwp_asan` build on musl.
2021-02-17 20:03:16 -08:00
Fangrui Song 58ecfccd0d [profile] Add __attribute__((used)) to zero size dummy sections
D14468 added these dummy sections. This patch adds `__attribute__((used))` so
that when compiled by GCC>=11 or (expected, D96838) Clang>=13 on some ELF platforms,
these sections will get SHF_GNU_RETAIN to make sure they will not be discarded
by ld --gc-sections.

We are trying to get rid of LLD's "__start_/__stop_ references retain C identifier name sections" rule.
If LLD drops the rule in the future (we will retain compatibility for `__llvm_prf_*` for a while),
`__llvm_prf_*` will need to have the SHF_GNU_RETAIN flag, otherwise:

```
// __llvm_prf_cnts/__llvm_prf_data usually exist, but {names,vnds} may not exist.
// Such diagnostics will happen with {cnts,data} as well if no input object file is instrumented.
% clang++ -fprofile-generate a.cc -fuse-ld=lld -Wl,--gc-sections
ld.lld: error: undefined hidden symbol: __start___llvm_prf_names
>>> referenced by InstrProfilingPlatformLinux.c
>>>               InstrProfilingPlatformLinux.c.o:(__llvm_profile_begin_names) in archive /tmp/RelA/lib/clang/13.0.0/lib/linux/libclang_rt.profile-x86_64.a
...
```

Differential Revision: https://reviews.llvm.org/D96902
2021-02-17 19:22:25 -08:00
Aaron Green 10993bf072 Bugfix for collecting features from very small DSOs.
During unit tests, it was observed that crafting an artificially small DSO could cause OOB memory to be accessed. This change fixes that (but again, the affected DSOs are unlikely to ever occur outside unit tests).

Reviewed By: morehouse, charco

Differential Revision: https://reviews.llvm.org/D94507
2021-02-17 13:04:49 -08:00
Dmitry Vyukov fb19400d4e tsan: fix mmap_lots test
If tsan runtime will try to allocate something during exit handling,
the allocation will fail because there is no VA whatsoever.
It's observed to fail with the following error in some cases:
failed to allocate 0x1000 (4096) bytes of DTLS_NextBlock.
So terminate the process immediately.

Reviewed-in: https://reviews.llvm.org/D96874
2021-02-17 19:03:17 +01:00
Jonny Farley f8ed31cd99 [Fuzzer][Test] Use %python substitution for trace-malloc-unbalanced.test
This test was found to fail for some of our downstream builds, on
computers where python was not on the default $PATH. Therefore
add a %python substitution to use sys.executable, based on similar
solutions for python calls in tests elsewhere in LLVM.

Differential Revision: https://reviews.llvm.org/D96799
2021-02-17 15:00:46 +00:00
Alex Richardson ca9815fc24 [sanitizers] Define SANITIZER_INTERCEPTOR_HOOKS on FreeBSD
This fixes the weak_hooks.cpp test on FreeBSD. Since this feature appears
to be supported on almost all platforms, it might also make sense to turn
it into an opt-out list instead of being opt-in.

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D96255
2021-02-17 10:36:47 +00:00
Joachim Protze ac6c13bfc4 [sanitizers] Pass CMAKE_C_FLAGS into TSan buildgo script
When compiling with ccache, compiler commands get split into smaller steps
and clang's default -Wunused-command-line-argument complains about unused
include directory arguments. In combination -Werror, compilation aborts.

This patch passes the CMAKE_C_FLAGS into the build script. Configuring with
-DCMAKE_C_FLAGS=-Wno-unused-command-line-argument allows successful testing.

Differential Revision: https://reviews.llvm.org/D96762
2021-02-17 09:14:23 +01:00
Dmitry Vyukov 0984b8de0b tsan: don't leave unmapped hole in non-app memory
If an app mmaps lots of memory, a user mmap may end up
in the tsan region for traces. Shadow for this range
overlaps with shadow for other user regions.
This causes havok: from false positives to crashes.
Don't leave unmapped holes in the traces region.

Reviewed-in: https://reviews.llvm.org/D96697
2021-02-17 08:37:04 +01:00
Adhemerval Zanella b9d3234c70 [sanitizer] [arm] Disable some LSAN tests for arm-linux-gnueabihf
Reinstate D90628 since the fix done by D96337 does not change the outcome
of the https://bugs.llvm.org/show_bug.cgi?id=48052
2021-02-15 09:35:02 -03:00
Amy Huang 60a55337e0 Fix test in external_symbolizer_path.cpp, by adding a REQUIRES: static-lib.
Follow-up to https://reviews.llvm.org/D94563.
2021-02-12 14:04:43 -08:00
Amy Huang 1e92b1730f Fix test failure for external_symbolizer_path.cpp 2021-02-12 12:49:41 -08:00
Amy Huang 5815b71eac Disable test in external_symbolizer_path.cpp temporarily to debug test failures. 2021-02-12 11:51:13 -08:00
Marco Vanotti 0fe4701e51 Expand unit tests for fuzzer::Merger
This change adds additional unit tests for fuzzer::Merger::Parse and fuzzer::Merger::Merge in anticipation of additional changes to the merge control file format to support cross-process fuzzing.

It modifies the parameter handling of Merge slightly in order to make NewFeatures and NewCov consistent with NewFiles; namely, Merge *replaces* the contents of these output parameters rather than accumulating them (thereby fixing a buggy return value).

This is change 1 of (at least) 18 for cross-process fuzzing support.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D94506
2021-02-12 11:18:34 -08:00
Amy Huang 394913fdb9 Try to fix external_symbolizer_path.cpp test to stop breaking on buildbots.
Not sure what the issue is, but it might be because the test copies
llvm-symbolizer to a different directory, and it can't find libc++.
Try to add some REQUIRES that we use in other tests where we copy
llvm tools out of their original directories.
2021-02-12 10:16:49 -08:00
Jianzhou Zhao a7538fee3a [dfsan] Comment out ChainOrigin temporarily
It was added by D96160, will be used by D96564.
Some OS got errors if it is not used.
Comment it out for the time being.
2021-02-12 18:13:24 +00:00
Matthew G McGovern 81b1d3da09 [sanitizers][Windows] Implement __sanitizer_purge_allocator for Win64
Windows' memory unmapping has to be explicit, there is no madvise.
Similarly, re-mapping memory has to be explicit as well. This patch
implements a basic method for remapping memory which was previously
returned to the OS on Windows.

Patch by Matthew G. McGovern and Jordyn Puryear
2021-02-12 09:49:04 -08:00
Jianzhou Zhao 7590c0078d [dfsan] Turn off THP at dfsan_flush
https://reviews.llvm.org/D89662 turned this off at dfsan_init.
dfsan_flush also needs to turn it off.
W/o this a program may get more and more memory usage after hours.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96569
2021-02-12 17:10:09 +00:00
Vitaly Buka 0b3d31222d [ASAN][NFC] Improve language 2021-02-12 02:55:58 -08:00
Vitaly Buka fc05b2d9e5 [NFC][ProfileData] Improve language 2021-02-12 02:55:58 -08:00
Jianzhou Zhao 083d45b21c [dfsan] Fix building OriginAddr at non-linux OS
Fix the broken build by D96545
2021-02-12 05:02:14 +00:00
Vitaly Buka f2133f2e31 [NFC,memprof] Update test after D96319 2021-02-11 16:36:16 -08:00
Julian Lettner 9360f1a191 [Sanitizer] Fix sanitizer tests without reducing optimization levels
As discussed, these tests are compiled with optimization to mimic real
sanitizer usage [1].

Let's mark relevant functions with `noinline` so we can continue to
check against the stack traces in the report.

[1] https://reviews.llvm.org/D96198

This reverts commit 04af72c542.

Differential Revision: https://reviews.llvm.org/D96357
2021-02-11 15:22:20 -08:00
Jianzhou Zhao 5ebbc5802f [dfsan] Introduce memory mapping for origin tracking
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96545
2021-02-11 22:33:16 +00:00
Jianzhou Zhao 2d9c6e10e9 [dfsan] Add origin chain utils
This is a part of https://reviews.llvm.org/D95835.

The design is based on MSan origin chains.

An 4-byte origin is a hash of an origin chain. An origin chain is a
pair of a stack hash id and a hash to its previous origin chain. 0 means
no previous origin chains exist. We limit the length of a chain to be
16. With origin_history_size = 0, the limit is removed.

The change does not have any test cases yet. The following change
will be adding test cases when the APIs are used.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D96160
2021-02-11 19:10:11 +00:00
Fangrui Song 0dc6122dd3 [asan][test] Fix Linux/odr-violation.cpp on gcc 2021-02-11 09:53:48 -08:00
Adhemerval Zanella 88d1724d9b [sanitizer] Fix suffix-log-path_test.c on arm-linux-gnu
The recent suffix-log-path_test.c checks for a full stacktrace and
since on some arm-linux-gnu configuration the slow unwinder is used
on default (when the compiler emits thumb code as default), it
requires -funwind-tables on tests.

It also seems to fix the issues disable by d025df3c1d.

Reviewed By: ostannard

Differential Revision: https://reviews.llvm.org/D96337
2021-02-11 14:26:04 -03:00
Jianzhou Zhao b4993cf54d [sanitizer] Move MSan's chained_origin_depot to sanitizer_common
https://reviews.llvm.org/D95835 implements origin tracking for DFSan.
It reuses the chained origin depot of MSan.

This change moves the utility to sanitizer_common to share between
MSan and DFSan.

Reviewed-by: eugenis, morehouse

Differential Revision: https://reviews.llvm.org/D96319
2021-02-11 01:25:56 +00:00
Vedant Kumar 897bec937e [test/ubsan] Allow unused FileCheck prefixes in TypeCheck/vptr.cpp
This test started failing after https://reviews.llvm.org/D95849
defaulted --allow-unused-prefixes to false.

Taking a look at the test, I didn't see an obvious need to add
OS-specific check lines for each supported value of %os.

rdar://74207657
2021-02-10 16:06:51 -08:00
Fangrui Song 096f547208 [asan][test] Fix Linux/odr-violation.cpp after symbolizer change D95927 2021-02-10 14:27:12 -08:00
Mitch Phillips b93786907c [GWP-ASan] Add back some headers removed by IWYU.
These headers are required for Android.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D96374
2021-02-10 09:13:45 -08:00
Todd Lipcon 5dd29d9922
Fix xray fdr mode to allow multiple flushes
Reviewed By: dberris

Differential Revision: https://reviews.llvm.org/D96382
2021-02-10 12:57:24 +11:00
Nico Weber 23cd8d51ad Try to fix compiler-rt tests after 87dbdd2e3b on mac and win
These tests use `--check-prefix=CHECK-%os` but then didn't have
a CHECK line for every os.

In most tests, the linux expectations were sufficient (they match
the "wrap_" prefix with .*), so just remove the check-prefix there.
In the places where this didn't easily work, make sure there are
at least CHECK-Windows and CHECK-Darwin lines.
2021-02-09 09:28:52 -05:00
Roland McGrath 4c9adbb287 [scudo/standalone] Use .arch_extension memtag, not mte
GNU binutils accepts only `.arch_extension memtag` while Clang
accepts either that or `.arch_extension mte` to mean the same thing.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D95996
2021-02-08 12:24:47 -08:00
Mitch Phillips 3d8823b8e4 [GWP-ASan] Add aligned allocations.
Adds a new allocation API to GWP-ASan that handles size+alignment
restrictions.

Reviewed By: cryptoad, eugenis

Differential Revision: https://reviews.llvm.org/D94830
2021-02-08 11:22:29 -08:00
Julian Lettner 04af72c542 [Sanitizer] Fix failing sanitizer tests
The new pass manager was enabled by default [1].

The commit message states the following relevant differences:
  * The inliner works slightly differently
  * -O1 does some amount of inlining

These tests are affected because they specify `-O1` and then check the
reported stack trace.

[1] https://reviews.llvm.org/D95380

Differential Revision: https://reviews.llvm.org/D96198
2021-02-08 09:56:32 -08:00
Jianzhou Zhao 0f3fd3b281 [dfsan] Add thread registration
This is a part of https://reviews.llvm.org/D95835.

This change is to address two problems
1) When recording stacks in origin tracking, libunwind is not async signal safe. Inside signal callbacks, we need
to use fast unwind. Fast unwind needs threads
2) StackDepot used by origin tracking is not async signal safe, we set a flag per thread inside
a signal callback to prevent from using it.

The thread registration is similar to ASan and MSan.

Related MSan changes are
* 98f5ea0dba
* f653cda269
* 5a7c364343

Some changes in the diff are used in the next diffs
1) The test case pthread.c is not very interesting for now. It will be
  extended to test origin tracking later.
2) DFsanThread::InSignalHandler will be used by origin tracking later.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95963
2021-02-05 17:38:59 +00:00
Emily Shi 039567b664 [Darwin] Switch to new logging api for sanitizers
Switch to new logging api added in [[ https://developer.apple.com/documentation/os/os_log_error | macOS 10.12 ]] that is more memory safe and enables us to label the log messages in the future. Falls back to old API if ran on older OS versions.

Commited by Dan Liew on behalf of Emily Shi.

rdar://25181524

Reviewed By: delcypher, yln

Differential Revision: https://reviews.llvm.org/D95977
2021-02-04 21:04:51 -08:00
Vitaly Buka 758928e3b8 [sanitizer] Fix the test on android 2021-02-04 18:02:02 -08:00
Amy Huang 8d7d2deb37 Fix test case from D94563.
The added test case failed on ppc, android, and other buildbots,
so require x86 targets.
2021-02-04 16:34:39 -08:00
Amy Huang 4737aab53d Fix a test case after committing D94563.
D94563 implemented `ReadBinaryName` on Windows, which causes a test case
to now pass, so remove the `XFAIL: windows-msvc` line.
2021-02-04 16:06:51 -08:00
Amy Huang 9ba623c655 [asan] Add %d variable to external_symbolizer_path option, so that user can specify paths relative to the location of the binary.
We want way to set a path to llvm-symbolizer that isn't relative
to the current working directory; this change adds a variable that
expands to the path relative to the current binary.
This approach came from comments in https://reviews.llvm.org/D93070

Differential Revision: https://reviews.llvm.org/D94563
2021-02-04 15:43:02 -08:00
Bill Torpey dd5c2b8de9 [sanitizer] Add suffix to report file name
For those using a GUI, it can be very helpful to have a
particular suffix appended to the report file name, so
it can be opened with a double-click.

(see also: https://github.com/google/sanitizers/issues/951)

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D46546
2021-02-04 13:50:11 -08:00
Nico Weber b995314143 Revert "[InstrProfiling] Use !associated metadata for counters, data and values"
This reverts commit 97ba5cde52.
Still breaks tests: https://reviews.llvm.org/D76802#2540647
2021-02-03 19:14:34 -05:00
Vitaly Buka 9da05cf6ed [asan] Fix pthread_create interceptor
AsanThread::Destroy implementation expected to be called on
child thread.

I missed authors concern regarding this reviewing D95184.

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D95731
2021-02-03 12:57:56 -08:00
Jianzhou Zhao 15f26c5f51 [dfsan] Wrap strcat
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95923
2021-02-03 18:50:29 +00:00
Jianzhou Zhao eb5c0a90e7 [dfsan] Test IGN and DFL for sigaction
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95957
2021-02-03 18:46:49 +00:00
Roland McGrath 15aa78abb6 [sanitizer_common] Use zx_system_get_page_size() on Fuchsia
Fuchsia is migrating to a variable page size.

Reviewed By: phosek, charco

Differential Revision: https://reviews.llvm.org/D95919
2021-02-03 10:45:46 -08:00
Roland McGrath 09fe23a61c [gwp_asan] Use zx_system_get_page_size() on Fuchsia
Fuchsia is migrating to a variable page size.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D95920
2021-02-03 10:35:07 -08:00
Roland McGrath d81069e796 [scudo/standalone] Use zx_system_get_page_size() on Fuchsia
Fuchsia is migrating to a variable page size.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D95921
2021-02-03 10:28:05 -08:00
Jianzhou Zhao 93afc3452c [dfsan] Clean TLS after signal callbacks
Similar to https://reviews.llvm.org/D95642, this diff fixes signal.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95896
2021-02-03 17:21:28 +00:00
Petr Hosek 97ba5cde52 [InstrProfiling] Use !associated metadata for counters, data and values
C identifier name input sections such as __llvm_prf_* are GC roots so
they cannot be discarded. In LLD, the SHF_LINK_ORDER flag overrides the
C identifier name semantics.

The !associated metadata may be attached to a global object declaration
with a single argument that references another global object, and it
gets lowered to SHF_LINK_ORDER flag. When a function symbol is discarded
by the linker, setting up !associated metadata allows linker to discard
counters, data and values associated with that function symbol.

Note that !associated metadata is only supported by ELF, it does not have
any effect on non-ELF targets.

Differential Revision: https://reviews.llvm.org/D76802
2021-02-02 23:19:51 -08:00
Jianzhou Zhao 3f568e1fbb [dfsan] Wrap memmove
Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95883
2021-02-03 05:15:56 +00:00
Mitch Phillips 0dcf3324cf Revert "[GWP-ASan] Add aligned allocations."
This reverts commit 9dc0676247.

Should not have been committed, was a bad rebase. Review still
continuing on D94830.
2021-02-02 14:46:25 -08:00
Mitch Phillips 30973f6fe0 [GWP-ASan] Add locking around unwinder for atfork protection.
Unwinders (like libc's backtrace()) can call their own locks (like the
libdl lock). We need to let the unwinder release the locks before
forking. Wrap a new lock around the unwinder for atfork protection.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D95889
2021-02-02 14:44:35 -08:00
Mitch Phillips 9dc0676247 [GWP-ASan] Add aligned allocations.
Adds a new allocation API to GWP-ASan that handles size+alignment
restrictions.

Differential Revision: https://reviews.llvm.org/D94830
2021-02-02 14:44:35 -08:00
Jianzhou Zhao e1a4322f81 [dfsan] Clean TLS after sigaction callbacks
DFSan uses TLS to pass metadata of arguments and return values. When an
instrumented function accesses the TLS, if a signal callback happens, and
the callback calls other instrumented functions with updating the same TLS,
the TLS is in an inconsistent state after the callback ends. This may cause
either under-tainting or over-tainting.

This fix follows MSan's workaround.
  cb22c67a21
It simply resets TLS at restore. This prevents from over-tainting. Although
under-tainting may still happen, a taint flow can be found eventually if we
run a DFSan-instrumented program multiple times. The alternative option is
saving the entire TLS. However the TLS storage takes 2k bytes, and signal calls
could be nested. So it does not seem worth.

This diff fixes sigaction. A following diff will be fixing signal.

Reviewed-by: morehouse

Differential Revision: https://reviews.llvm.org/D95642
2021-02-02 22:07:17 +00:00
Tom Weaver 4f1320b77d Revert "[InstrProfiling] Use !associated metadata for counters, data and values"
This reverts commit df3e39f60b.

introduced failing test instrprof-gc-sections.c
causing build bot to fail:
http://lab.llvm.org:8011/#/builders/53/builds/1184
2021-02-02 14:19:31 +00:00
Fangrui Song a59283a745 [test] Fix unused FileCheck prefixes in compiler-rt 2021-02-01 22:32:13 -08:00
Fangrui Song 327196d688 [test] Fix unused FileCheck prefixes in compiler-rt/test 2021-02-01 21:24:58 -08:00
Petr Hosek df3e39f60b [InstrProfiling] Use !associated metadata for counters, data and values
C identifier name input sections such as __llvm_prf_* are GC roots so
they cannot be discarded. In LLD, the SHF_LINK_ORDER flag overrides the
C identifier name semantics.

The !associated metadata may be attached to a global object declaration
with a single argument that references another global object, and it
gets lowered to SHF_LINK_ORDER flag. When a function symbol is discarded
by the linker, setting up !associated metadata allows linker to discard
counters, data and values associated with that function symbol.

Note that !associated metadata is only supported by ELF, it does not have
any effect on non-ELF targets.

Differential Revision: https://reviews.llvm.org/D76802
2021-02-01 15:01:43 -08:00
Erik Pilkington 5f3c4923e4 [compiler-rt] Fix AArch64 out-of-line atomics visibility on mach-o
This fixes an apparent oversight in D91156, where the symbol was defined
without the leading underscore, then the visibility was later declared with it.
rdar://73364185

Differential revision: https://reviews.llvm.org/D95639
2021-02-01 11:29:20 -05:00
xgupta 94fac81fcc [Branch-Rename] Fix some links
According to the [[ https://foundation.llvm.org/docs/branch-rename/ | status of branch rename ]], the master branch of the LLVM repository is removed on 28 Jan 2021.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D95766
2021-02-01 16:43:21 +05:30
Luís Marques 68e5614b42 [Sanitizer][RISCV] Fix FastUnwindTest
Fixes the `FastUnwindTest` unit test for RISC-V.
These changes reflect the different stack organization commonly used for
that architecture.

Differential Revision: https://reviews.llvm.org/D90574
2021-02-01 00:00:15 +00:00
Luís Marques 3203c96842 [CMake][compiler-rt][RISCV] Support RISC-V cross-compilation
This seems to be a safe way to ensure that the Compiler-RT test compiler
flags are properly set in all cross-compilation scenarios. Without this
when `BUILTINS_TEST_TARGET_CFLAGS` is set in
`compiler-rt/test/builtins/CMakeLists.txt` the other flags are cleared.

Differential Revision: https://reviews.llvm.org/D92124
2021-01-31 23:52:57 +00:00
Luís Marques 2de4f19ecd [LSan][RISCV] Enable LSan for RISCV64
Fixes the broken RISCV64 implementation of `internal_clone` and
adds RISCV64 support for LSan.

Differential Revision: https://reviews.llvm.org/D92403
2021-01-31 21:53:25 +00:00
Harald van Dijk b01b964d37
[compiler-rt][tests] Define TARGET_FLAGS
D36116 refactored the logic of tests and removed the definition of TARGET_FLAGS, but left one use of it. Restore its definition for that one use, so that an x86_64 test is compiled with -m64.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93634
2021-01-30 13:06:08 +00:00
Dimitry Andric e056fc6cb6 [sanitizer] Fix msan test build on FreeBSD after 7afdc89c20
This commit accidentally enabled fgetgrent_r() in the msan tests under
FreeBSD, but this function is not supported. Also remove FreeBSD from
the SANITIZER_INTERCEPT_FGETGRENT_R macro.
2021-01-28 23:54:04 +01:00
Vyacheslav Chigrin f79937140d [sanitizer] Fix crash on empty strings.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D95431
2021-01-27 13:22:43 -08:00
Evgenii Stepanov e53247ca42 Revert "[sanitizer] Fix crash on empty strings."
Wrong author name, will recommit shortly.

This reverts commit 262c50856a.
2021-01-27 13:22:43 -08:00
Evgenii Stepanov 262c50856a [sanitizer] Fix crash on empty strings.
Patch by Vyacheslav Chigrin.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D95431
2021-01-27 13:08:10 -08:00
Kostya Kortchinsky ba379fe527 [scudo][standalone] Restore GWP-ASan flag parsing
With D92696, the Scudo Standalone GWP-ASan flag parsing was changed to
the new GWP-ASan optional one. We do not necessarily want this, as this
duplicates flag parsing code in Scudo Standalone when using the
GWP-ASan integration.

This CL reverts the changes within Scudo Standalone, and increases
`MaxFlags` to 20 as an addionnal option got us to the current max.

Differential Revision: https://reviews.llvm.org/D95542
2021-01-27 12:35:34 -08:00
Jessica Clarke e47a388bd6 [builtins] Fix build after D95537 due to missing size_t
Strangely int_lib.h only includes stdint.h and not stddef.h, so we need
to include the latter to be able to use size_t here.
2021-01-27 19:44:01 +00:00
Jessica Clarke 5748a71ba3 [builtins] Fix integer/pointer confusion in gcc_personality_v0.c
This fixes the implementation for architectures like CHERI with strong
pointer provenance (pointers, and thus uintptr_t, are represented as
hardware capabilities). Specifically, adding two uintptr_t's together
(as is done for `start + length` and `funcStart + landingPad`) has
ambiguous provenance, whereas using a plain integer (such as size_t) for
the offset operand does not. Also, readULEB128 is creating a plain
integer, not a pointer.

On all currently-supported architectures this should be an NFC, as
size_t and uintptr_t end up being the same underlying plain integer
type.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D95537
2021-01-27 19:28:20 +00:00
Marco Vanotti 98c21ae7a8 [sanitizer][fuchsia] Implement ReleaseMemoryPagesToOS
The `zx_vmar_op_range` allows us to decommit memory pages without
needing a handle to the underlying vmo, as long as we have a handle to
a vmar that contains this mapping. This allows us to implement the
`ReleaseMemoryPagesToOS` function by decommitting the memory using a
handle to the root vmar.

Reviewed By: mcgrathr

Differential Revision: https://reviews.llvm.org/D95384
2021-01-27 11:13:24 -08:00
Jessica Clarke add98b2462 [compiler-rt] Support FreeBSD standalone (boot) environment
FreeBSD uses -Ddouble=jagged-little-pill -Dfloat=floaty-mcfloatface to
poison uses of floating point in its standalone environment. It also
deprecates machine/limits.h in favour of sys/limits.h and does not even
provide the former on newer architectures.

This is a cleaner reimplementation of equivalent patches in FreeBSD's
vendored copy of compiler-rt.

Reviewed By: dim

Differential Revision: https://reviews.llvm.org/D95264
2021-01-27 15:29:39 +00:00
Kostya Kortchinsky e9cc5fef64 [scudo][standalone] Enable death tests on Fuchsia
zxtest doesn't have `EXPECT_DEATH` and the Scudo unit-tests were
defining it as a no-op.

This enables death tests on Fuchsia by using `ASSERT_DEATH` instead.
I used a lambda to wrap the expressions as this appears to not be
working the same way as `EXPECT_DEATH`.

Additionnally, a death test using `alarm` was failing with the change,
as it's currently not implemented in Fuchsia, so move that test within
a `!SCUDO_FUCHSIA` block.

Differential Revision: https://reviews.llvm.org/D94362
2021-01-25 09:19:10 -08:00
Dan Liew 757b93bb7b [ASan] Fix broken Windows build due to 596d534ac3.
In that change I forgot to update the call to
`AsanThread::ThreadStart()` in `asan_win.cpp`.
2021-01-23 09:09:06 -08:00
Dan Liew 596d534ac3 [ASan] Stop blocking child thread progress from parent thread in `pthread_create` interceptor.
Previously in ASan's `pthread_create` interceptor we would block in the
`pthread_create` interceptor waiting for the child thread to start.

Unfortunately this has bad performance characteristics because the OS
scheduler doesn't know the relationship between the parent and child
thread (i.e. the parent thread cannot make progress until the child
thread makes progress) and may make the wrong scheduling decision which
stalls progress.

It turns out that ASan didn't use to block in this interceptor but was
changed to do so to try to address
http://llvm.org/bugs/show_bug.cgi?id=21621/.

In that bug the problem being addressed was a LeakSanitizer false
positive. That bug concerns a heap object being passed
as `arg` to `pthread_create`. If:

* The calling thread loses a live reference to the object (e.g.
  `pthread_create` finishes and the thread no longer has a live
  reference to the object).
* Leak checking is triggered.
* The child thread has not yet started (once it starts it will have a
  live reference).

then the heap object will incorrectly appear to be leaked.

This bug is covered by the `lsan/TestCases/leak_check_before_thread_started.cpp` test case.

In b029c5101f ASan was changed to block
in `pthread_create()` until the child thread starts so that `arg` is
kept alive for the purposes of leaking check.

While this change "works" its problematic due to the performance
problems it causes. The change is also completely unnecessary if leak
checking is disabled (via detect_leaks runtime option or
CAN_SANITIZE_LEAKS compile time config).

This patch does two things:

1. Takes a different approach to solving the leak false positive by
   making LSan's leak checking mechanism treat the `arg` pointer of
   created but not started threads as reachable.  This is done by
   implementing the `ForEachRegisteredThreadContextCb` callback for
   ASan.

2. Removes the blocking behaviour in the ASan `pthread_create`
   interceptor.

rdar://problem/63537240

Differential Revision: https://reviews.llvm.org/D95184
2021-01-22 23:34:43 -08:00
Dan Liew dd922bc2a6 [LSan] Introduce a callback mechanism to allow adding data reachable from ThreadContexts to the frontier.
This mechanism is intended to provide a way to treat the `arg` pointer
of a created (but not yet started) thread as reachable. In future
patches this will be implemented in `GetAdditionalThreadContextPtrs`.

A separate implementation of `GetAdditionalThreadContextPtrs` exists
for ASan and LSan runtimes because they need to be implemented
differently in future patches.

rdar://problem/63537240

Differential Revision: https://reviews.llvm.org/D95183
2021-01-22 19:26:02 -08:00
Jianzhou Zhao f86db34def [MSan] Move origins for overlapped memory transfer
Reviewed-by: eugenis

Differential Revision: https://reviews.llvm.org/D94572
2021-01-21 02:11:26 +00:00
Raul Tambre 480643a95c [CMake] Remove dead code setting policies to NEW
cmake_minimum_required(VERSION) calls cmake_policy(VERSION),
which sets all policies up to VERSION to NEW.
LLVM started requiring CMake 3.13 last year, so we can remove
a bunch of code setting policies prior to 3.13 to NEW as it
no longer has any effect.

Reviewed By: phosek, #libunwind, #libc, #libc_abi, ldionne

Differential Revision: https://reviews.llvm.org/D94374
2021-01-19 17:19:36 +02:00
Peter Collingbourne d302398ff0 hwasan: Update register-dump-read.c test to reserve x23 instead of x20.
D90422 changed this test to write a fixed value into register x23
instead of x20, but it did not update the list of reserved registers.
This meant that x23 may have been live across the register write,
although this happens to not be the case with the current compiler.
Fix the problem by updating the reserved register list.
2021-01-15 16:14:36 -08:00
Mitch Phillips 6a42cbf6d2 [GWP-ASan] Add inbuilt options parser.
Adds a modified options parser (shamefully pulled from Scudo, which
shamefully pulled it from sanitizer-common) to GWP-ASan. This allows
customers (Android) to parse options strings in a common way.

Depends on D94117.

AOSP side of these patches is staged at:

 - sepolicy (sysprops should only be settable by the shell, in both root and
 unrooted conditions):
 https://android-review.googlesource.com/c/platform/system/sepolicy/+/1517238

 - zygote updates:
 https://android-review.googlesource.com/c/platform/frameworks/base/+/1515009

 - bionic changes to add `gwp_asan.<process_name>` system property, and
 GWP_ASAN_OPTIONS environment variable:
 https://android-review.googlesource.com/c/platform/bionic/+/1514989

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D92696
2021-01-15 12:57:05 -08:00
Julian Lettner 84de4faf4c GetMacosAlignedVersion() fails if sysctl is not setup
`GetMacosAlignedVersion()` fails for ASan-ified launchd because the
sanitizer initialization code runs before `sysctl` has been setup by
launchd.  In this situation, `sysctl kern.osproductversion` returns a
non-empty string that does not match our expectations of a
well-formatted version string.

Retrieving the kernel version (via `sysctl kern.osrelease`) still works,
so we can use it to add a fallback for this corner case.

Differential Revision: https://reviews.llvm.org/D94190
2021-01-15 11:42:25 -08:00
Mitch Phillips a8520f6970 [GWP-ASan] Minor refactor of optional components.
In preparation for the inbuilt options parser, this is a minor refactor
of optional components including:
 - Putting certain optional elements in the right header files,
 according to their function and their dependencies.
 - Cleaning up some old and mostly-dead code.
 - Moving some functions into anonymous namespaces to prevent symbol
 export.

Reviewed By: cryptoad, eugenis

Differential Revision: https://reviews.llvm.org/D94117
2021-01-14 11:14:11 -08:00
Nico Weber 704831fe1f Revert "Hwasan InitPrctl check for error using internal_iserror"
This reverts commit 1854594b80.
See https://reviews.llvm.org/D94425#2495621
2021-01-13 08:30:11 -05:00
Matthew Malcomson 1854594b80 Hwasan InitPrctl check for error using internal_iserror
When adding this function in https://reviews.llvm.org/D68794 I did not
notice that internal_prctl has the API of the syscall to prctl rather
than the API of the glibc (posix) wrapper.

This means that the error return value is not necessarily -1 and that
errno is not set by the call.

For InitPrctl this means that the checks do not catch running on a
kernel *without* the required ABI (not caught since I only tested this
function correctly enables the ABI when it exists).
This commit updates the two calls which check for an error condition to
use `internal_iserror`.  That function sets a provided integer to an
equivalent errno value and returns a boolean to indicate success or not.

Tested by running on a kernel that has this ABI and on one that does
not.  Verified that running on the kernel without this ABI the current
code prints the provided error message and does not attempt to run the
program.  Verified that running on the kernel with this ABI the current
code does not print an error message and turns on the ABI.
All tests done on an AArch64 Linux machine.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D94425
2021-01-13 11:35:09 +00:00
Jianzhou Zhao 0b99385e15 [MSan] Partially revert some changes from D94552
Because of line 55, actually aligned_beg always equals to beg.
2021-01-13 07:03:17 +00:00
Jianzhou Zhao 82655c1514 [MSan] Tweak CopyOrigin
There could be some mis-alignments when copying origins not aligned.

I believe inaligned memcpy is rare so the cases do not matter too much
in practice.

1) About the change at line 50

Let dst be (void*)5,
then d=5, beg=4
so we need to write 3 (4+4-5) bytes from 5 to 7.

2) About the change around line 77.

Let dst be (void*)5,
because of lines 50-55, the bytes from 5-7 were already writen.
So the aligned copy is from 8.

Reviewed-by: eugenis
Differential Revision: https://reviews.llvm.org/D94552
2021-01-13 01:22:05 +00:00
Julian Lettner 8f5ec45937 [Sanitizer][Darwin] Fix test for macOS 11+ point releases
This test wrongly asserted that the minor version is always 0 when
running on macOS 11 and above.
2021-01-12 15:23:43 -08:00
Kazushi (Jam) Marukawa 70474dfebc [VE][compiler-rt] Add VE as a target of crt
SX Aurora VE is an experimental target.  We upstreamed many part of
ported llvm and clang.  In order to continue this move, we need to
support libraries next, then we need to show the ability of llvm for
VE through test cases.  As a first step for that, we need to use
crt in compiler-rt.  VE has it's own crt but they are a part of
proprietary compiler.  So, we want to use crt in compiler-rt as an
alternative.

This patch enables VE as a candidate of crt in compiler-rt.

Reviewed By: phosek, compnerd

Differential Revision: https://reviews.llvm.org/D92748
2021-01-12 10:20:21 +09:00
Martin Storsjö ffac9001d9 [compiler-rt] [sanitizer] Silence -Wframe-larger-than= for a few windows functions with large stack buffers
Also update a documentation url while touching code nearby, as
requested in review.

Differential Revision: https://reviews.llvm.org/D91853
2021-01-08 22:54:03 +02:00
Alex Richardson 00530dee5d [compiler-rt] Implement __atomic_is_lock_free
This function is called by the __atomic_is_lock_free() builtin if the value
cannot be resolved to true at compile time. Lack of this function is
causing the non-lockfree atomics tests in libc++ to not be run (see D91911)

This function is also added in D85044, but that review also adds support
for using lock-free atomics in more cases, whereas this is a minimal change
that just adds __atomic_is_lock_free() for the implementation of atomic.c.

Reviewed By: ldionne

Differential Revision: https://reviews.llvm.org/D92302
2021-01-08 12:48:22 +00:00
Ryan Prichard 658a1be76b [builtins] Add COMPILER_RT_BUILTINS_HIDE_SYMBOLS
On Android, when the builtins are linked into a binary, they are
typically linked using -Wl,--exclude-libs so that the symbols aren't
reexported. For the NDK, compiler-rt's default behavior (build the
builtins archive with -fvisibility=hidden) is better so that builtins
are hidden even without -Wl,--exclude-libs.

Android needs the builtins with non-hidden symbols only for a special
case: for backwards compatibility with old binaries, the libc.so and
libm.so DSOs in the platform need to export some builtins for arm32 and
32-bit x86. See D56977.

Control the behavior with a new flag,
`COMPILER_RT_BUILTINS_HIDE_SYMBOLS`, that behaves similarly to the
`*_HERMETIC_STATIC_LIBRARY` in libunwind/libcxx/libcxxabi, so that
Android can build a special builtins variant for libc.so/libm.so.

Unlike the hermetic flags for other projects, this new flag is enabled
by default.

Reviewed By: compnerd, MaskRay

Differential Revision: https://reviews.llvm.org/D93431
2021-01-07 17:53:44 -08:00
Fangrui Song 7afdc89c20 [sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi lsan msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
  Unsupported:  7
  Passed     : 65
  Failed     : 22

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Note: we need to place `_FILE_OFFSET_BITS` above `#include "sanitizer_platform.h"` to avoid `#define __USE_FILE_OFFSET64 1` in 32-bit ARM `features.h`

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2021-01-06 10:55:40 -08:00
Oliver Stannard 4839378ca0 Revert "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl"
This reverts commit b7718b6175, because it
is causing build failures on all 32-bit ARM bots which build
compiler-rt.
2021-01-06 10:31:59 +00:00
Peter Collingbourne 6dc3c117a3 scudo: Update a comment to match the Linux kernel behavior. NFCI. 2021-01-05 19:51:47 -08:00
Julian Lettner 8b0bd54d0e [sanitizer][Darwin] Suppress -Wno-non-virtual-dtor warning
Suppress the warning:
```
'fake_shared_weak_count' has virtual functions but non-virtual destructor [-Wnon-virtual-dtor]
```

The warning has been recently enabled [1], but the associated cleanup
missed this instance in Darwin code [2].

[1] 9c31e12609
[2] d48f2d7c02

Differential Revision: https://reviews.llvm.org/D94139
2021-01-05 17:09:18 -08:00
Fangrui Song b7718b6175 Reland D93848 "[sanitizer] Define SANITIZER_GLIBC to refine SANITIZER_LINUX feature detection and support musl""
Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are replaced
with the more appropriate `#if SANITIZER_GLIBC` (the headers are glibc
extensions, not specific to Linux (i.e. if we ever support GNU/kFreeBSD
or Hurd, the guards may automatically work)).

Several `#if SANITIZER_LINUX && !SANITIZER_ANDROID` guards are refined
with `#if SANITIZER_GLIBC` (the definitions are available on Linux glibc,
but may not be available on other libc (e.g. musl) implementations).

This patch makes `ninja asan cfi msan stats tsan ubsan xray` build on a musl based Linux distribution (apk install musl-libintl)
Notes about disabled interceptors for musl:

* `SANITIZER_INTERCEPT_GLOB`: musl does not implement `GLOB_ALTDIRFUNC` (GNU extension)
* Some ioctl structs and functions operating on them.
* `SANITIZER_INTERCEPT___PRINTF_CHK`: `_FORTIFY_SOURCE` functions are GNU extension
* `SANITIZER_INTERCEPT___STRNDUP`: `dlsym(RTLD_NEXT, "__strndup")` errors so a diagnostic is formed. The diagnostic uses `write` which hasn't been intercepted => SIGSEGV
* `SANITIZER_INTERCEPT_*64`: the `_LARGEFILE64_SOURCE` functions are glibc specific. musl does something like `#define pread64 pread`
* Disabled `msg_iovlen msg_controllen cmsg_len` checks: musl is conforming while many implementations (Linux/FreeBSD/NetBSD/Solaris) are non-conforming. Since we pick the glibc definition, exclude the checks for musl (incompatible sizes but compatible offsets)

Pass through LIBCXX_HAS_MUSL_LIBC to make check-msan/check-tsan able to build libc++ (https://bugs.llvm.org/show_bug.cgi?id=48618).

Many sanitizer features are available now.

```
% ninja check-asan
(known issues:
* ASAN_OPTIONS=fast_unwind_on_malloc=0 odr-violations hangs
)
...
Testing Time: 53.69s
  Unsupported      : 185
  Passed           : 512
  Expectedly Failed:   1
  Failed           :  12

% ninja check-ubsan check-ubsan-minimal check-memprof # all passed

% ninja check-cfi
( all cross-dso/)
...
Testing Time: 8.68s
  Unsupported      : 264
  Passed           :  80
  Expectedly Failed:   8
  Failed           :  32

% ninja check-lsan
(With GetTls (D93972), 10 failures)
Testing Time: 4.09s
  Unsupported:  7
  Passed     : 65
  Failed     : 22

% ninja check-msan
(Many are due to functions not marked unsupported.)
Testing Time: 23.09s
  Unsupported      :   6
  Passed           : 764
  Expectedly Failed:   2
  Failed           :  58

% ninja check-tsan
Testing Time: 23.21s
  Unsupported      :  86
  Passed           : 295
  Expectedly Failed:   1
  Failed           :  25
```

Used `ASAN_OPTIONS=verbosity=2` to verify there is no unneeded interceptor.

Partly based on Jari Ronkainen's https://reviews.llvm.org/D63785#1921014

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D93848
2021-01-05 16:18:43 -08:00
Roland McGrath 90b8fd6136 scudo: Fix compilation for non-Linux aarch64
Linux-specific aarch64 code was deconditionalized in commit
dfa40840e0.  This broke builds for
non-Linux aarch64 platforms.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D94108
2021-01-05 13:21:52 -08:00
Mitch Phillips 1f8031cd74 [android] Fix some tests for AOSP-master devices.
Some tests are broken at API level 30 on AOSP-master devices. When we
change the buildbuit to API level 30, the following tests get enabled.
They're currently broken due to various issues, and so fix up those
issues.

Reviewed By: oontvoo, eugenis

Differential Revision: https://reviews.llvm.org/D94100
2021-01-05 12:54:09 -08:00
Alan Phipps 9f2967bcfe [Coverage] Add support for Branch Coverage in LLVM Source-Based Code Coverage
This is an enhancement to LLVM Source-Based Code Coverage in clang to track how
many times individual branch-generating conditions are taken (evaluate to TRUE)
and not taken (evaluate to FALSE).  Individual conditions may comprise larger
boolean expressions using boolean logical operators.  This functionality is
very similar to what is supported by GCOV except that it is very closely
anchored to the ASTs.

Differential Revision: https://reviews.llvm.org/D84467
2021-01-05 09:51:51 -06:00
Martin Storsjö 979c38cc74 [compiler-rt] [windows] Add UNUSED attributes on variables/functions only used for 64 bit targets
This fixes warnings when building for 32 bit targets.

Differential Revision: https://reviews.llvm.org/D91852
2021-01-05 08:59:08 +02:00