Commit Graph

12320 Commits

Author SHA1 Message Date
Akira Hatanaka 78a7af456d Revert "[compiler-rt] Add a critical section when flushing gcov counters"
This reverts commit 88f5bf77f9 as it broke
green dragon bots.

http://lab.llvm.org:8080/green/job/clang-stage1-RA/4401/
2019-12-09 19:58:26 -08:00
Eric Christopher 22d516261a Remove unused Unit test directory for xray. 2019-12-09 18:40:44 -08:00
Evgenii Stepanov 9ef451d1fd [hwasan] Offline symbolization script.
Summary:
A script to symbolize hwasan reports after the fact using unstripped
binaries. Supports stack-based reports. Requires llvm-symbolizer
(addr2line is not an option).

Reviewers: pcc, hctim

Subscribers: mgorny, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71148
2019-12-09 13:46:11 -08:00
Peter Collingbourne 29f0a65671 scudo: Add a basic malloc/free benchmark.
Differential Revision: https://reviews.llvm.org/D71104
2019-12-09 10:10:19 -08:00
Mitch Phillips 26fd95680b [GWP-ASan] 32-bit test pointers, allow multi-init for test.
Summary:
GWP-ASan test currently fail on 32-bit platforms, as some of the pointers are
larger than `uintptr_t` on 32-bit platforms. Fix up all those instances.

Also add an uncompress varint test where the result is an underflow.

Furthermore, allow multi-init for testing. Each gtest when running
`check-gwp_asan` apparently runs in its own instance, but when integrating
these tests into Android, this behaviour isn't the same. We remove the
global multi-init check here, to allow for testing to work elsewhere, and we're
not really worried about multi-init anyway as it's part of our contract with
the allocator.

Reviewers: eugenis, vlad.tsyrklevich

Reviewed By: eugenis

Subscribers: #sanitizers, llvm-commits, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71121
2019-12-09 08:19:54 -08:00
Calixte Denizet 88f5bf77f9 [compiler-rt] Add a critical section when flushing gcov counters
Summary:
Counters can be flushed in a multi-threaded context for example when the process is forked in different threads (https://github.com/llvm/llvm-project/blob/master/llvm/lib/Transforms/Instrumentation/GCOVProfiling.cpp#L632-L663).
In order to avoid pretty bad things, a critical section is needed around the flush.
We had a lot of crashes in this code in Firefox CI when we switched to clang for linux ccov builds and those crashes disappeared with this patch.

Reviewers: marco-c, froydnj, dmajor, davidxl

Reviewed By: marco-c, dmajor

Subscribers: froydnj, dmajor, dberris, jfb, #sanitizers, llvm-commits, sylvestre.ledru

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70910
2019-12-09 10:39:55 +01:00
Michał Górny 6c2b2b9e20 [compiler-rt] [test] Disable ASLR on ASAN/MSAN/TSAN tests on NetBSD
Use a new %run wrapper for ASAN/MSAN/TSAN tests that calls paxctl
in order to disable ASLR on the test executables.  This makes it
possible to test sanitizers on systems where ASLR is enabled by default.

Differential Revision: https://reviews.llvm.org/D70958
2019-12-06 08:09:01 +01:00
Peter Collingbourne c8a2882a97 scudo: Fix one of the C wrapper tests on Android.
The test ScudoWrappersCTest.Realloc expects realloc of memalign to work on
Android, but this relies on dealloc_type_mismatch being set to false. Commit
0d3d4d3b0 caused us to start setting it to true in the C wrapper tests,
which broke the test. Set it to the correct value on Android.

Differential Revision: https://reviews.llvm.org/D71078
2019-12-05 11:15:31 -08:00
Peter Collingbourne 0cfe0a7bb0 scudo: Fix the build of wrappers_c_test.cpp on Android.
The Android headers don't provide a declaration of valloc or pvalloc, so we
need to declare them ourselves.

Differential Revision: https://reviews.llvm.org/D71077
2019-12-05 11:15:00 -08:00
Max Moroz a44ef027eb [compiler-rt] FuzzedDataProvider: do not call memcpy on empty vector.
Summary:
Some versions of memcpy mark pointer arguments as __nonnull, that triggers UBSan
errors even when the length passed is 0.

Reviewers: manojgupta, metzman

Subscribers: dberris, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D71031

[compiler-rt] FDP: assert that num_bytes_to_consume == 0 when size == 0.
2019-12-04 14:18:52 -08:00
Vedant Kumar f208b70fbc Revert "[Coverage] Revise format to reduce binary size"
This reverts commit e18531595b.

On Windows, there is an error:

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/54963/steps/stage%201%20check/logs/stdio

error: C:\b\slave\sanitizer-windows\build\stage1\projects\compiler-rt\test\profile\Profile-x86_64\Output\instrprof-merging.cpp.tmp.v1.o: Failed to load coverage: Malformed coverage data
2019-12-04 10:35:14 -08:00
Vedant Kumar e18531595b [Coverage] Revise format to reduce binary size
Revise the coverage mapping format to reduce binary size by:

1. Naming function records and marking them `linkonce_odr`, and
2. Compressing filenames.

This shrinks the size of llc's coverage segment by 82% (334MB -> 62MB)
and speeds up end-to-end single-threaded report generation by 10%. For
reference the compressed name data in llc is 81MB (__llvm_prf_names).

Rationale for changes to the format:

- With the current format, most coverage function records are discarded.
  E.g., more than 97% of the records in llc are *duplicate* placeholders
  for functions visible-but-not-used in TUs. Placeholders *are* used to
  show under-covered functions, but duplicate placeholders waste space.

- We reached general consensus about giving (1) a try at the 2017 code
  coverage BoF [1]. The thinking was that using `linkonce_odr` to merge
  duplicates is simpler than alternatives like teaching build systems
  about a coverage-aware database/module/etc on the side.

- Revising the format is expensive due to the backwards compatibility
  requirement, so we might as well compress filenames while we're at it.
  This shrinks the encoded filenames in llc by 86% (12MB -> 1.6MB).

See CoverageMappingFormat.rst for the details on what exactly has
changed.

Fixes PR34533 [2], hopefully.

[1] http://lists.llvm.org/pipermail/llvm-dev/2017-October/118428.html
[2] https://bugs.llvm.org/show_bug.cgi?id=34533

Differential Revision: https://reviews.llvm.org/D69471
2019-12-04 10:10:55 -08:00
Alex Lorenz ad871e4295 [compiler-rt] Disable fuzzer large.test when LLVM_ENABLE_EXPENSIVE_CHECKS=ON
This test is timing out on Green Dragon http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/
and looks like it's not executed on other bots with expensive checks
enabled
http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-ubuntu
http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win

The test times out at the C++ source file takes too long to build (2+ hours on my machine), as
clang spends a lot of time in IR/MIR verifiers.

Differential Revision: https://reviews.llvm.org/D70024
2019-12-03 14:37:37 -08:00
Dan Liew 96c8024e2e Rename `tsan/race_range_pc.cc` to `test/tsan/race_range_pc.cpp`.
The old suffix was preventing it from being executed by default.
2019-12-03 09:49:25 -08:00
Kostya Kortchinsky 5595249e48 [scudo][standalone] Add chunk ownership function
Summary:
In order to be compliant with tcmalloc's extension ownership
determination function, we have to expose a function that will
say if a chunk was allocated by us.

As to whether or not this has security consequences: someone
able to call this function repeatedly could use it to determine
secrets (cookie) or craft a valid header. So this should not be
exposed directly to untrusted user input.

Add related tests.

Additionally clang-format caught a few things to change.

Reviewers: hctim, pcc, cferris, eugenis, vitalybuka

Subscribers: JDevlieghere, jfb, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70908
2019-12-03 08:32:26 -08:00
Yi Kong acc79aa0e7 Revert "Revert 1689ad27af "[builtins] Implement rounding mode support for i386/x86_64""
Don't build specilised fp_mode.c on MSVC since it does not support
inline ASM on x86_64.

This reverts commit a19f0eec94.
2019-11-27 17:29:20 -08:00
Roman Lebedev b98a0c7f6c
[clang][CodeGen] Implicit Conversion Sanitizer: handle increment/decrement (PR44054)(take 2)
Summary:
Implicit Conversion Sanitizer is *almost* feature complete.
There aren't *that* much unsanitized things left,
two major ones are increment/decrement (this patch) and bit fields.

As it was discussed in
[[ https://bugs.llvm.org/show_bug.cgi?id=39519 | PR39519 ]],
unlike `CompoundAssignOperator` (which is promoted internally),
or `BinaryOperator` (for which we always have promotion/demotion in AST)
or parts of `UnaryOperator` (we have promotion/demotion but only for
certain operations), for inc/dec, clang omits promotion/demotion
altogether, under as-if rule.

This is technically correct: https://rise4fun.com/Alive/zPgD
As it can be seen in `InstCombineCasts.cpp` `canEvaluateTruncated()`,
`add`/`sub`/`mul`/`and`/`or`/`xor` operators can all arbitrarily
be extended or truncated:
901cd3b3f6/llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp (L1320-L1334)

But that has serious implications:
1. Since we no longer model implicit casts, do we pessimise
   their AST representation and everything that uses it?
2. There is no demotion, so lossy demotion sanitizer does not trigger :]

Now, i'm not going to argue about the first problem here,
but the second one **needs** to be addressed. As it was stated
in the report, this is done intentionally, so changing
this in all modes would be considered a penalization/regression.
Which means, the sanitization-less codegen must not be altered.

It was also suggested to not change the sanitized codegen
to the one with demotion, but i quite strongly believe
that will not be the wise choice here:
1. One will need to re-engineer the check that the inc/dec was lossy
   in terms of `@llvm.{u,s}{add,sub}.with.overflow` builtins
2. We will still need to compute the result we would lossily demote.
   (i.e. the result of wide `add`ition/`sub`traction)
3. I suspect it would need to be done right here, in sanitization.
   Which kinda defeats the point of
   using `@llvm.{u,s}{add,sub}.with.overflow` builtins:
   we'd have two `add`s with basically the same arguments,
   one of which is used for check+error-less codepath and other one
   for the error reporting. That seems worse than a single wide op+check.
4. OR, we would need to do that in the compiler-rt handler.
   Which means we'll need a whole new handler.
   But then what about the `CompoundAssignOperator`,
   it would also be applicable for it.
   So this also doesn't really seem like the right path to me.
5. At least X86 (but likely others) pessimizes all sub-`i32` operations
   (due to partial register stalls), so even if we avoid promotion+demotion,
   the computations will //likely// be performed in `i32` anyways.

So i'm not really seeing much benefit of
not doing the straight-forward thing.

While looking into this, i have noticed a few more LLVM middle-end
missed canonicalizations, and filed
[[ https://bugs.llvm.org/show_bug.cgi?id=44100 | PR44100 ]],
[[ https://bugs.llvm.org/show_bug.cgi?id=44102 | PR44102 ]].

Those are not specific to inc/dec, we also have them for
`CompoundAssignOperator`, and it can happen for normal arithmetics, too.
But if we take some other path in the patch, it will not be applicable
here, and we will have most likely played ourselves.

TLDR: front-end should emit canonical, easy-to-optimize yet
un-optimized code. It is middle-end's job to make it optimal.

I'm really hoping reviewers agree with my personal assessment
of the path this patch should take..

This originally landed in 9872ea4ed1
but got immediately reverted in cbfa237892
because the assertion was faulty. That fault ended up being caused
by the enum - while there will be promotion, both types are unsigned,
with same width. So we still don't need to sanitize non-signed cases.
So far. Maybe the assert will tell us this isn't so.

Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=44054 | PR44054 ]].
Refs. https://github.com/google/sanitizers/issues/940

Reviewers: rjmccall, erichkeane, rsmith, vsk

Reviewed By: erichkeane

Subscribers: mehdi_amini, dexonsmith, cfe-commits, #sanitizers, llvm-commits, aaron.ballman, t.p.northover, efriedma, regehr

Tags: #llvm, #clang, #sanitizers

Differential Revision: https://reviews.llvm.org/D70539
2019-11-27 21:52:41 +03:00
Peter Collingbourne b208088a21 scudo: Limit the number of bytes tested in a realloc test.
This test was previously effectively doing:
P = malloc(X); write X bytes to P; P = realloc(P, X - Y); P = realloc(P, X)
and expecting that all X bytes stored to P would still be identical after
the final realloc.

This happens to be true for the current scudo implementation of realloc,
but is not guaranteed to be true by the C standard ("Any bytes in the new
object beyond the size of the old object have indeterminate values.").
This implementation detail will change with the new memory tagging support,
which unconditionally zeros newly allocated granules when memory tagging
is enabled. Fix this by limiting the number of bytes that we test to the
minimum size that we realloc the allocation to.

Differential Revision: https://reviews.llvm.org/D70761
2019-11-27 10:32:34 -08:00
Peter Collingbourne 6fd6cfdf72 scudo: Replace a couple of macros with their expansions.
The macros INLINE and COMPILER_CHECK always expand to the same thing (inline
and static_assert respectively). Both expansions are standards compliant C++
and are used consistently in the rest of LLVM, so let's improve consistency
with the rest of LLVM by replacing them with the expansions.

Differential Revision: https://reviews.llvm.org/D70793
2019-11-27 10:12:27 -08:00
Peter Collingbourne f30fe16d49 scudo: Call setCurrentTSD(nullptr) when bringing down the TSD registry in tests.
Otherwise, we will hit a use-after-free when testing multiple instances of
the same allocator on the same thread. This only recently became a problem
with D70552 which caused us to run both ScudoCombinedTest.BasicCombined and
ScudoCombinedTest.ReleaseToOS on the unit tests' main thread.

Differential Revision: https://reviews.llvm.org/D70760
2019-11-27 09:55:14 -08:00
Martin Liska 2045d2c90e
Make memory dump same as the one in asan.
Shadow memory (and short granules) are not prepended with memory
address and arrow at the end of line is removed.

Differential Revision: https://reviews.llvm.org/D70707
2019-11-27 18:46:38 +01:00
Kostya Kortchinsky 0d3d4d3b0f [scudo][standalone] Make tests work on Fuchsia
Summary:
This CL makes unit tests compatible with Fuchsia's zxtest. This
required a few changes here and there, but also unearthed some
incompatibilities that had to be addressed.

A header is introduced to allow to account for the zxtest/gtest
differences, some `#if SCUDO_FUCHSIA` are used to disable incompatible
code (the 32-bit primary, or the exclusive TSD).

It also brought to my attention that I was using
`__scudo_default_options` in different tests, which ended up in a
single binary, and I am not sure how that ever worked. So move
this to the main cpp.

Additionally fully disable the secondary freelist on Fuchsia as we do
not track VMOs for secondary allocations, so no release possible.

With some modifications to Scudo's BUILD.gn in Fuchsia:
```
[==========] 79 tests from 23 test cases ran (10280 ms total).
[  PASSED  ] 79 tests
```

Reviewers: mcgrathr, phosek, hctim, pcc, eugenis, cferris

Subscribers: srhines, jfb, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70682
2019-11-27 09:17:40 -08:00
Hans Wennborg 900d8a9a3b [profile] Fix file contention causing dropped counts on Windows under -fprofile-generate
See PR43425:
https://bugs.llvm.org/show_bug.cgi?id=43425

When writing profile data on Windows we were opening profile file with
exclusive read/write access.

In case we are trying to write to the file from multiple processes
simultaneously, subsequent calls to CreateFileA would return
INVALID_HANDLE_VALUE.

To fix this, I changed to open without exclusive access and then take a
lock.

Patch by Michael Holman!

Differential revision: https://reviews.llvm.org/D70330
2019-11-27 15:55:13 +01:00
Roman Lebedev cbfa237892
Revert "[clang][CodeGen] Implicit Conversion Sanitizer: handle increment/decrement (PR44054)"
The asssertion that was added does not hold,
breaks on test-suite/MultiSource/Applications/SPASS/analyze.c
Will reduce the testcase and revisit.

This reverts commit 9872ea4ed1, 870f3542d3.
2019-11-27 17:05:21 +03:00
Roman Lebedev 9872ea4ed1
[clang][CodeGen] Implicit Conversion Sanitizer: handle increment/decrement (PR44054)
Summary:
Implicit Conversion Sanitizer is *almost* feature complete.
There aren't *that* much unsanitized things left,
two major ones are increment/decrement (this patch) and bit fields.

As it was discussed in
[[ https://bugs.llvm.org/show_bug.cgi?id=39519 | PR39519 ]],
unlike `CompoundAssignOperator` (which is promoted internally),
or `BinaryOperator` (for which we always have promotion/demotion in AST)
or parts of `UnaryOperator` (we have promotion/demotion but only for
certain operations), for inc/dec, clang omits promotion/demotion
altogether, under as-if rule.

This is technically correct: https://rise4fun.com/Alive/zPgD
As it can be seen in `InstCombineCasts.cpp` `canEvaluateTruncated()`,
`add`/`sub`/`mul`/`and`/`or`/`xor` operators can all arbitrarily
be extended or truncated:
901cd3b3f6/llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp (L1320-L1334)

But that has serious implications:
1. Since we no longer model implicit casts, do we pessimise
   their AST representation and everything that uses it?
2. There is no demotion, so lossy demotion sanitizer does not trigger :]

Now, i'm not going to argue about the first problem here,
but the second one **needs** to be addressed. As it was stated
in the report, this is done intentionally, so changing
this in all modes would be considered a penalization/regression.
Which means, the sanitization-less codegen must not be altered.

It was also suggested to not change the sanitized codegen
to the one with demotion, but i quite strongly believe
that will not be the wise choice here:
1. One will need to re-engineer the check that the inc/dec was lossy
   in terms of `@llvm.{u,s}{add,sub}.with.overflow` builtins
2. We will still need to compute the result we would lossily demote.
   (i.e. the result of wide `add`ition/`sub`traction)
3. I suspect it would need to be done right here, in sanitization.
   Which kinda defeats the point of
   using `@llvm.{u,s}{add,sub}.with.overflow` builtins:
   we'd have two `add`s with basically the same arguments,
   one of which is used for check+error-less codepath and other one
   for the error reporting. That seems worse than a single wide op+check.
4. OR, we would need to do that in the compiler-rt handler.
   Which means we'll need a whole new handler.
   But then what about the `CompoundAssignOperator`,
   it would also be applicable for it.
   So this also doesn't really seem like the right path to me.
5. At least X86 (but likely others) pessimizes all sub-`i32` operations
   (due to partial register stalls), so even if we avoid promotion+demotion,
   the computations will //likely// be performed in `i32` anyways.

So i'm not really seeing much benefit of
not doing the straight-forward thing.

While looking into this, i have noticed a few more LLVM middle-end
missed canonicalizations, and filed
[[ https://bugs.llvm.org/show_bug.cgi?id=44100 | PR44100 ]],
[[ https://bugs.llvm.org/show_bug.cgi?id=44102 | PR44102 ]].

Those are not specific to inc/dec, we also have them for
`CompoundAssignOperator`, and it can happen for normal arithmetics, too.
But if we take some other path in the patch, it will not be applicable
here, and we will have most likely played ourselves.

TLDR: front-end should emit canonical, easy-to-optimize yet
un-optimized code. It is middle-end's job to make it optimal.

I'm really hoping reviewers agree with my personal assessment
of the path this patch should take..

Fixes [[ https://bugs.llvm.org/show_bug.cgi?id=44054 | PR44054 ]].

Reviewers: rjmccall, erichkeane, rsmith, vsk

Reviewed By: erichkeane

Subscribers: mehdi_amini, dexonsmith, cfe-commits, #sanitizers, llvm-commits, aaron.ballman, t.p.northover, efriedma, regehr

Tags: #llvm, #clang, #sanitizers

Differential Revision: https://reviews.llvm.org/D70539
2019-11-27 15:39:55 +03:00
Evgenii Stepanov 947f969244 Fix sanitizer-common build with glibc 2.31
Summary:
As mentioned in D69104, glibc changed ABI recently with the [[ https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=2f959dfe849e0646e27403f2e4091536496ac0f0| 2f959dfe ]] change.
D69104 dealt with just 32-bit ARM, but that is just one of the many affected architectures.
E.g. x86_64, i?86, riscv64, sparc 32-bit, s390 31-bit are affected too (and various others).

This patch instead of adding a long list of further architectures that wouldn't be checked ever next to arm 32-bit changes the structures to match the 2.31 layout and performs the checking on Linux for ipc_perm mode position/size only on non-Linux or on Linux with glibc 2.31 or later.  I think this matches what is done for aarch64 already.
If needed, we could list architectures that haven't changed ABI (e.g. powerpc), so that they would be checked even with older glibcs.  AFAIK sanitizers don't actually use ipc_perm.mode and
so all they care about is the size and alignment of the whole structure.

Note, s390 31-bit and arm 32-bit big-endian changed ABI even further, there will now be shmctl with old symbol version and shmctl@@GLIBC_2.31 which will be incompatible.  I'm afraid this isn't really solvable unless the sanitizer libraries are symbol versioned and use matching symbol versions to glibc symbols for stuff they intercept, plus use dlvsym.
This patch doesn't try to address that.

Patch by Jakub Jelinek.

Reviewers: kcc, eugenis, dvyukov

Reviewed By: eugenis

Subscribers: jyknight, kristof.beyls, fedor.sergeev, simoncook, PkmX, s.egerton, steven.zhang, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70662
2019-11-25 14:38:10 -08:00
Lei Huang 9e676d9c7e [PowerPC][compiler-rt][builtins]Add __fixtfti builtin on PowerPC
Implements __fixtfti builtin for PowerPC. This builtin converts a
long double (IBM double-double) to a signed int128. The conversion relies on
the unsigned conversion of the absolute value of the long double.

Tests included for both positive and negative long doubles.

Patch By: Baptiste Saleil

Differential Revision: https://reviews.llvm.org/D69730
2019-11-25 14:54:03 -06:00
Mitch Phillips 19edfb3728 [GWP-ASan] Add GWP_ASAN_ prefix to macros.
Summary:
When platforms use their own `LIKELY()` definitions, it can be quite
troublesome to ensure they don't conflict with the GWP-ASan internal
definitions. Just force the GWP_ASAN_ prefix to help this issue.

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: #sanitizers, llvm-commits, cferris, pcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70683
2019-11-25 12:27:00 -08:00
Kostya Kortchinsky 15664fe2c4 [scudo][standalone] Fix for releaseToOS prior to init
Summary:
cferris@ found an issue where calling `releaseToOS` prior to any other
heap operation would lead to a crash, due to the allocator not being
properly initialized (it was discovered via `mallopt`).

The fix is to call `initThreadMaybe` prior to calling `releaseToOS` for
the Primary.

Add a test that crashes prior to fix.

Reviewers: hctim, cferris, pcc, eugenis

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70552
2019-11-25 08:38:45 -08:00
Petr Hosek e8a4e5892b [CMake] Use correct include path for InstrProfData.inc on Darwin
On Darwin, part of the profile runtime is included in the builtin
library. f35032e changed the location of InstrProfData.inc but the
builtin build for Darwin hasn't been updated to include the new
path which causes a breakage when building for Darwin. This change
addresses this breakage.
2019-11-24 13:26:01 -08:00
Petr Hosek f35032e03d Reland "[CMake] Support installation of InstrProfData.inc"
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.

Differential Revision: https://reviews.llvm.org/D64532
2019-11-22 14:09:46 -08:00
Petr Hosek 262b10ba1f Revert "[CMake] Support installation of InstrProfData.inc"
This reverts commit f11bc1776f since it's
failing to build on some bots.
2019-11-22 12:00:23 -08:00
Petr Hosek f11bc1776f [CMake] Support installation of InstrProfData.inc
This header fragment is useful on its own for any consumer that wants
to use custom instruction profile runtime with the LLVM instrumentation.
The concrete use case is in Fuchsia's kernel where we want to use
instruction profile instrumentation, but we cannot use the compiler-rt
runtime because it's not designed for use in the kernel environment.
This change allows installing this header as part of compiler-rt.

Differential Revision: https://reviews.llvm.org/D64532
2019-11-22 11:52:50 -08:00
Vitaly Buka 97e0fd27eb Don't forward __pthread_mutex_* interceptors to pthread_mutex_* version
Summary:
Allows to use rr with asan

Fixes PR41095

Reviewers: eugenis

Subscribers: jfb, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70581
2019-11-22 11:19:39 -08:00
Marco Vanotti e5b603a4c3 [libFuzzer] don't use /dev/null for DiscardOuput in Fuchsia.
Summary:

This commit moves the `DiscardOutput` function in FuzzerIO to
FuzzerUtil, so fuchsia can have its own specialized version.

In fuchsia, accessing `/dev/null` is not supported, and there's nothing
similar to a file that discards everything that is written to it. The
way of doing something similar in fuchsia is by using `fdio_null_create`
and binding that to a file descriptor with `fdio_bind_to_fd`.

This change should fix one of the issues with the `-close_fd_mask` flag
in libfuzzer, in which closing stdout was not working due to
`fopen("/dev/null", "w")` returning `NULL`.

Reviewers: kcc, aarongreen

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D69593
2019-11-21 16:56:05 -08:00
Marco Vanotti 46c7fc22cf [libFuzzer] Fix unwinding for Fuchsia
Summary:
This commit fixes part of the issues with stack unwinding in fuchsia for
arm64 and x86_64. It consists of multiple fixes:

(1) The cfa_offset calculation was wrong, instead of pointing to the
previous stack pointer, it was pointing to the current  one. It worked in
most of the cases because the crashing functions already had a
prologue and had their cfa information relative to another register. The
fix consists on adding a constant that can be used to calculate the
crashing function's stack pointer, and base all the cfi information
relative to that offset.

(2) (arm64) Due to errors with the syntax for the dwarf information, most
of the `OP_NUM` macros were not working. The problem was that they were
referred to as `r##NUM` (like `r14`), when it should have been `x##num`
(like `x14`), or even without the x.

(3) (arm64) The link register was being considered a part of the main
registers (`r30`), when in the real struct it has its own field. Given
that the link register is in the same spot in the struct as r[30] would be,
and that C++ doesn't care about anything, the calculation was still correct.

(4) (x86_64) The stack doesn't need to be aligned to 16 bytes when we
jump to the trampoline function, but it needs to be before performing
call instructions. Encoding that logic in cfi information was tricky, so
we decided to make the cfa information relative to `rbp` and align `rsp`.
Note that this could have been done using another register directly,
but it seems cleaner to make a new fake stack frame.

There are some other minor changes like adding a `brk 1` instruction in
arm64 to make sure that we never return to the crash trampoline (similar to
what we do in x86_64).

Sadly this commit does not fix unwinding for all use cases for arm64.
Crashing functions that do not add information related to the return column in
their cfi information will fail to unwind due to a bug in libunwinder.

Reviewers: mcgrathr, jakehehrlich, phosek, kcc, aarongreen

Subscribers: aprantl, kristof.beyls, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D69579
2019-11-21 15:47:07 -08:00
Marco Vanotti 16d9f44fd1 [libFuzzer] Fix fd check in DupAndCloseStderr.
Summary:
This commit fixes the check in the return value from the `DuplicateFile`
function, which returns a new file descriptor. `DuplicateFile` can
return 0 if that file descriptor is available (for example, if stdin has
already been closed).

In particular, this could cause a bug with the `-close_fd_mask` flag in
some platforms: just call the fuzzer with stdin closed and the
`-close_fd_mask=2` flag, and stderr will not be muted.

Example fuzzer:

```

extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) {
  fprintf(stderr, "STDERR\n");
  fprintf(stdout, "STDOUT\n");
  return 0;
}
```

Invocation (muting both stderr and stdout):
```
./test -close_fd_mask=3 -runs=1 0<&-
INFO: Seed: 1155116940
INFO: Loaded 1 modules   (1 inline 8-bit counters): 1 [0x48b020, 0x48b021),
INFO: Loaded 1 PC tables (1 PCs): 1 [0x478dc8,0x478dd8),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
STDERR
INFO: A corpus is not provided, starting from an empty corpus
STDERR
Done 2 runs in 0 second(s)
```

Reviewers: mcgrathr, jakehehrlich, phosek, kcc, aarongreen

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D68775
2019-11-21 15:25:10 -08:00
Julian Lettner 0163329dbd [TSan] Make `mach_vm_allocate.c` test less flaky
rdar://57365733
2019-11-21 15:19:31 -08:00
Kostya Kortchinsky 46240c3872 [scudo][standalone] Minor optimization & improvements
Summary:
A few small improvements and optimizations:
- when refilling the free list, push back the last batch and return
  the front one: this allows to keep the allocations towards the front
  of the region;
- instead of using 48 entries in the shuffle array, use a multiple of
  `MaxNumCached`;
- make the maximum number of batches to create on refil a constant;
  ultimately it should be configurable, but that's for later;
- `initCache` doesn't need to zero out the cache, it's already done.
- it turns out that when using `||` or `&&`, the compiler is adamant
  on adding a short circuit for every part of the expression. Which
  ends up making somewhat annoying asm with lots of test and
  conditional jump. I am changing that to bitwise `|` or `&` in two
  place so that the generated code looks better. Added comments since
  it might feel weird to people.

This yields to some small performance gains overall, nothing drastic
though.

Reviewers: hctim, morehouse, cferris, eugenis

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70452
2019-11-21 10:05:39 -08:00
Florian Hahn a70c3f9f45 [compiler-rt] Don't check XCR0 when detecting avx512 on Darwin.
Darwin lazily saves the AVX512 context on first use [1]: instead of checking
that it already does to figure out if the OS supports AVX512, trust that
the kernel will do the right thing and always assume the context save
support is available.

[1] https://github.com/apple/darwin-xnu/blob/xnu-4903.221.2/osfmk/i386/fpu.c#L174

Reviewers: ab, RKSimon, craig.topper

Reviewed By: craig.topper

Subscribers: dberris, JDevlieghere, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D70454
2019-11-21 09:19:17 +00:00
Peter Collingbourne f751a79173 scudo: Only use the Android reserved TLS slot when building libc's copy of the allocator.
When we're not building libc's allocator, just use a regular TLS variable. This
lets the unit tests pass on Android devices whose libc uses Scudo. Otherwise
libc's copy of Scudo and the unit tests' copy will both try to use the same
TLS slot, in likely incompatible ways.

This requires using ELF TLS, so start passing -fno-emulated-tls when building
the library and the unit tests on Android.

Differential Revision: https://reviews.llvm.org/D70472
2019-11-20 11:30:58 -08:00
Peter Collingbourne 10089ce202 scudo: Switch from std::random_shuffle to std::shuffle in a test.
This lets the test build with C++17.

Differential Revision: https://reviews.llvm.org/D70471
2019-11-19 16:13:12 -08:00
Vedant Kumar 0d4211f4e7 [profile] Address unused function warnings on Windows after D69586
This '#ifdef's out two functions which are unused on Windows, to prevent
-Wunused-function warnings.
2019-11-19 12:51:12 -08:00
Vedant Kumar 1aacf58819 [profile] Unbreak Fuchsia/Windows after D68351
Continuous mode is not yet supported on Fuchsia/Windows, however an
error should not be reported unless the user attempted to actually
enable continuous mode.
2019-11-19 12:29:57 -08:00
Dan Liew b9a92206b6 [Symbolizers] On Darwin compute function offset when possible.
Summary:
The sanitizer symbolizers support printing the function offset
(difference between pc and function start) of a stackframe using the
`%q` format specifier.

Unfortunately this didn't actually work because neither the atos
or dladdr symbolizer set the `AddressInfo::function_offset` field.

This patch teaches both symbolizers to try to compute the function
offset. In the case of the atos symbolizer, atos might not report the
function offset (e.g. it reports a source location instead) so in this
case it fallsback to using `dladdr()` to compute the function offset.

Two test cases are included.

rdar://problem/56695185

Reviewers: kubamracek, yln

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D69549
2019-11-19 10:04:44 -08:00
Hans Wennborg a19f0eec94 Revert 1689ad27af "[builtins] Implement rounding mode support for i386/x86_64"
It broke the build with MSVC:

fp_mode.c(20): error C2065: '__asm__': undeclared identifier

> Differential Revision: https://reviews.llvm.org/D69870
2019-11-19 09:37:31 +01:00
Vedant Kumar 2492b5a125 [profile] Support online merging with continuous sync mode
Make it possible to use online profile merging ("%m" mode) with
continuous sync ("%c" mode).

To implement this, the merged profile is locked in the runtime
initialization step and either a) filled out for the first time or b)
checked for compatibility. Then, the profile can simply be mmap()'d with
MAP_SHARED set. With the mmap() in place, counter updates from every
process which uses an image are mapped onto the same set of physical
pages assigned by the filesystem cache. After the mmap() is set up, the
profile is unlocked.

Differential Revision: https://reviews.llvm.org/D69586
2019-11-18 12:56:58 -08:00
Craig Topper ff75bf6ac9 [X86] Add AMD Matisse (znver2) model number to getHostCPUName and compiler-rt's getAMDProcessorTypeAndSubtype.
This is the CPUID model used on Ryzen 3000 series (Zen 2/Matisse) CPUs.

Patch by Alex James

Differential Revision: https://reviews.llvm.org/D70279
2019-11-18 11:57:04 -08:00
Yi Kong 1689ad27af [builtins] Implement rounding mode support for i386/x86_64
Differential Revision: https://reviews.llvm.org/D69870
2019-11-18 10:32:40 -08:00
Julian Lettner 7e4d386b77 [cmake] Factor out DEFAULT_SANITIZER_MIN_OSX_VERSION 2019-11-18 09:41:19 -08:00