The MemProf compiler-rt support relies on some of the support only built
when COMPILER_RT_BUILD_SANITIZERS was enabled. This showed up in some
initial bot failures, and I addressed those by making the memprof
runtime build also conditional on COMPILER_RT_BUILD_SANITIZERS
(3ed77ecd0a). However, this resulted in
another inconsistency with how the tests were set up that was hit by
Chromium:
https://bugs.chromium.org/p/chromium/issues/detail?id=1142191
Undo the original bot fix and address this with a more comprehensive fix
that enables memprof to be built even when COMPILER_RT_BUILD_SANITIZERS
is disabled, by also building the necessary pieces under
COMPILER_RT_BUILD_MEMPROF.
Tested by configuring with a similar command as to what was used in the
failing Chromium configure. I reproduced the Chromium failure, as well
as the original bot failure I tried to fix in
3ed77ecd0a, with that fix reverted.
Confirmed it now works.
Differential Revision: https://reviews.llvm.org/D90190
In preparation for Fuchsia support, this CL refactors the memory
mapping functions.
The new functions are as follows:
- for Freeslots and Metadata:
`void *map(size_t Size, const char *Name) const;`
`void unmap(void *Ptr, size_t Size) const;`
- for the Pool:
`void *reservePool(size_t Size);`
`void commitPool(void *Ptr, size_t Size) const;`
`void decommitPool(void *Ptr, size_t Size) const;`
`void unreservePool();`
Note that those don't need a `Name` parameter as those are fixed per
function. `{reserve,unreserve}Pool` are not `const` because they will
modify platform specific class member on Fuchsia.
I added a plethora of `assert()` as the initial code was not enforcing
page alignment for sizes and addresses, which caused problem in the
initial Fuchsia draft. All sizes should now be properly rounded up to
a page.
Differential Revision: https://reviews.llvm.org/D89993
Disable the part of this test that started failing only on the
llvm-avr-linux bot after 5c20d7db9f.
Unfortunately, "XFAIL: avr" does not work. Still in the process of
trying to figure out how to debug.
While some platforms call `AsanThread::Init()` from the context of the
thread being started, others (like Fuchsia) call `AsanThread::Init()`
from the context of the thread spawning a child. Since
`AsyncSignalSafeLazyInitFakeStack` writes to a thread-local, we need to
avoid calling it from the spawning thread on Fuchsia. Skipping the call
here on Fuchsia is fine; it'll get called from the new thread lazily on first
attempted access.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D89607
When enabling stack use-after-free detection, we discovered that we read
the thread ID on the main thread while it is still set to 2^24-1.
This patch moves our call to AsanThread::Init() out of CreateAsanThread,
so that we can call SetCurrentThread first on the main thread.
Reviewed By: mcgrathr
Differential Revision: https://reviews.llvm.org/D89606
-print_full_coverage=1 produces a detailed branch coverage dump when run on a single file.
Uses same infrastructure as -print_coverage flag, but prints all branches (regardless of coverage status) in an easy-to-parse format.
Usage: For internal use with machine learning fuzzing models which require detailed coverage information on seed files to generate mutations.
Differential Revision: https://reviews.llvm.org/D85928
Reverts the XFAIL added in b67a2aef8a,
which had no effect.
Adjust the test to make sure all output is dumped to stderr, so that
hopefully I can get a better idea of where/why this is failing.
Remove some redundant checking while here.
While implementing inline stack traces on Windows I noticed that the stack
traces in many asan tests included an inlined frame that shouldn't be there.
Currently we get the PC and then do a stack unwind and use the PC to
find the beginning of the stack trace.
In the failing tests the first thing in the stack trace is inside an inline
call site that shouldn't be in the stack trace, so replace it with the PC.
Differential Revision: https://reviews.llvm.org/D89996
For unknown reasons, this test started failing only on the
llvm-avr-linux bot after 5c20d7db9f2791367b9311130eb44afecb16829c:
http://lab.llvm.org:8011/#/builders/112/builds/365
The error message is not helpful, and I have an email out to the bot
owner to help with debugging. XFAIL it on avr for now.
These compiler-rt tests should be UNSUPPORTED instead of XFAIL, which seems to be the real intent of the authors.
Reviewed By: vvereschaka
Differential Revision: https://reviews.llvm.org/D89840
This is a redo of D89908, which triggered some `-Werror=conversion`
errors with GCC due to assignments to the 31-bit variable.
This CL adds to the original one a 31-bit mask variable that is used
at every assignment to silence the warning.
Differential Revision: https://reviews.llvm.org/D89984
This reverts commit 9903b0586c.
Causes build failures (on GCC 10.2) with the following error:
In file included from /home/nikic/llvm-project/compiler-rt/lib/scudo/standalone/combined.h:29,
from /home/nikic/llvm-project/compiler-rt/lib/scudo/standalone/allocator_config.h:12,
from /home/nikic/llvm-project/compiler-rt/lib/scudo/standalone/wrappers_cpp.cpp:14:
/home/nikic/llvm-project/compiler-rt/lib/scudo/standalone/../../gwp_asan/guarded_pool_allocator.h: In member function ‘bool gwp_asan::GuardedPoolAllocator::shouldSample()’:
/home/nikic/llvm-project/compiler-rt/lib/scudo/standalone/../../gwp_asan/guarded_pool_allocator.h:82:69: error: conversion from ‘uint32_t’ {aka ‘unsigned int’} to ‘unsigned int:31’ may change value [-Werror=conversion]
82 | (getRandomUnsigned32() % (AdjustedSampleRatePlusOne - 1)) + 1;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
We need to have all thread specific data packed into a single `uintptr_t`
for the upcoming Fuchsia support. We can move the `RandomState` into the
`ThreadLocalPackedVariables`, reducing the size of `NextSampleCounter`
to 31 bits (or we could reduce `RandomState` to 31 bits).
We move `getRandomUnsigned32` into the platform agnostic part of the
class, and `initPRNG` in the platform specific part.
`ScopedBoolean` is replaced by actual assignments since non-const
references to bitfields are prohibited.
`random.{h,cpp}` are removed.
Differential Revision: https://reviews.llvm.org/D89908
This will allow the output directory to be specified by a build time
option, similar to the directory specified for regular PGO profiles via
-fprofile-generate=. The memory profiling instrumentation pass will
set up the variable. This is the same mechanism used by the PGO
instrumentation and runtime.
Depends on D87120 and D89629.
Differential Revision: https://reviews.llvm.org/D89086
Split out of D89086 as suggested.
Change the default of the log_path flag to nullptr, and the code
consuming that flag (ReportFile::SetReportPath), to treat nullptr as
stderr (so no change to the behavior of existing users). This allows
code to distinguish between the log_path being specified explicitly as
stderr vs the default.
This is so the flag can be used to override the new report path variable
that will be encoded in the binary for memprof for runtime testing.
Differential Revision: https://reviews.llvm.org/D89629
As discussed in the review for D87120 (specifically at
https://reviews.llvm.org/D87120#inline-831939), clean up PrintModuleMap
and DumpProcessMap usage differences. The former is only implemented for
Mac OSX, whereas the latter is implemented for all OSes. The former is
called by asan and tsan, and the latter by hwasan and now memprof, under
the same option. Simply rename the PrintModuleMap implementation for Mac
to DumpProcessMap, remove other empty PrintModuleMap implementations,
and convert asan/tsan to new name. The existing posix DumpProcessMap is
disabled for SANITIZER_MAC.
Differential Revision: https://reviews.llvm.org/D89630
The RISC-V implementations of the `__mulsi3`, `__muldi3` builtins were
conditionally compiling the actual function definitions depending on whether
the M extension was present or not. This caused Compiler-RT testing failures
for RISC-V targets with the M extension, as when these sources were included
the `librt_has_mul*i3` features were still being defined. These `librt_has_*`
definitions are used to conditionally run the respective tests. Since the
actual functions were not being compiled-in, the generic test for `__muldi3`
would fail. This patch makes these implementations follow the normal
Compiler-RT convention of always including the definition, and conditionally
running the respective tests by using the lit conditional
`REQUIRES: librt_has_*`.
Since the `mulsi3_test.c` wasn't actually RISC-V-specific, this patch also
moves it out of the `riscv` directory. It now only depends on
`librt_has_mulsi3` to run.
Differential Revision: https://reviews.llvm.org/D86457
Few changes wrt utilities:
- split `Check` into a platform agnostic condition test and a platform
specific termination, for which we introduce the function `die`.
- add a platform agnostic `utilities.cpp` that gets the allocation
alignment functions original in the platform specific file, as they
are reusable by all platforms.
Differential Revision: https://reviews.llvm.org/D89811
Do not crash when AsanThread::GetStackVariableShadowStart does not find
a variable for a pointer on a shadow stack.
Differential Revision: https://reviews.llvm.org/D89552
It turned out that at dynamic shared library mode, the memory access
pattern can increase memory footprint significantly on OS when transparent
hugepages (THP) are enabled. This could cause >70x memory overhead than
running a static linked binary. For example, a static binary with RSS
overhead 300M can use > 23G RSS if it is built dynamically.
/proc/../smaps shows in 6204552 kB RSS 6141952 kB relates to
AnonHugePages.
Also such a high RSS happens in some rate: around 25% runs may use > 23G RSS, the
rest uses in between 6-23G. I guess this may relate to how user memory
is allocated and distributted across huge pages.
THP is a trade-off between time and space. We have a flag
no_huge_pages_for_shadow for sanitizer. It is true by default but DFSan
did not follow this. Depending on if a target is built statically or
dynamically, maybe Clang can set no_huge_pages_for_shadow accordingly
after this change. But it still seems fine to follow the default setting of
no_huge_pages_for_shadow. If time is an issue, and users are fine with
high RSS, this flag can be set to false selectively.
This is a follow up patch of https://reviews.llvm.org/D88755.
When set 0 label for an address range, we can release pages within the
corresponding shadow address range to OS, and set only addresses outside
the pages to be 0.
Reviewed-by: morehouse, eugenis
Differential Revision: https://reviews.llvm.org/D89199
Adds some simple sanity checks that the support functions for the atomic
builtins do the right thing. This doesn't test concurrency and memory model
issues.
Differential Revision: https://reviews.llvm.org/D86278
- Fixing VS compiler and other cases settings this time.
Reviewers: dmajor, hans
Reviewed By: hans
Differential Revision: https://reviews.llvm.org/D89759
Cleaning up some of the GWP-ASan code base:
- lots of headers didn't have the correct file name
- adding `#ifdef` guard to `utilities.h`
- correcting an `#ifdef` guard based on actual file name
- removing an extra `;`
- clang-format'ing the code (`-style=llvm`)
Differential Revision: https://reviews.llvm.org/D89721
Revert "Fix compiler-rt build on Windows after D89640"
This reverts commit a7acee89d6.
This reverts commit d09b08919c.
Reason: breaks Linux / x86_64 build.
See RFC for background:
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142744.html
Follow on companion to the clang/llvm instrumentation support in D85948
and committed earlier.
This patch adds the compiler-rt runtime support for the memory
profiling.
Note that much of this support was cloned from asan (and then greatly
simplified and renamed). For example the interactions with the
sanitizer_common allocators, error handling, interception, etc.
The bulk of the memory profiling specific code can be found in the
MemInfoBlock, MemInfoBlockCache, and related classes defined and used
in memprof_allocator.cpp.
For now, the memory profile is dumped to text (stderr by default, but
honors the sanitizer_common log_path flag). It is dumped in either a
default verbose format, or an optional terse format.
This patch also adds a set of tests for the core functionality.
Differential Revision: https://reviews.llvm.org/D87120
Following up D81682 and D83903, remove the code for the old value profiling
buckets, which have been replaced with the new, extended buckets and disabled by
default.
Also syncing InstrProfData.inc between compiler-rt and llvm.
Differential Revision: https://reviews.llvm.org/D88838
While sanitizers don't use C++ standard library, we could still end
up accidentally including or linking it just by the virtue of using
the C++ compiler. Pass -nostdinc++ and -nostdlib++ to avoid these
accidental dependencies.
Differential Revision: https://reviews.llvm.org/D88922
Summary:
According the mmap man page (https://man7.org/linux/man-pages/man2/mmap.2.html) is only required to precisely control updates, so we can safely remove it.
Since gcda files are dumped just before to call exec** functions, dump need to be fast.
On my computer, Firefox built with --coverage needs ~1min40 to display something and in removing msync it needs ~8s.
Reviewers: void
Subscribers: #sanitizers, marco-c, sylvestre.ledru
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D81060
Teresa Johnson