Data involving struct accesses accounting work (plan to support only efficiency-cache-frag flag in the frontend side).
Reviewers: krytarowski, vitalybuka, jfb
Reviewed By : vitalybuka
Differential Revision: https://reviews.llvm.org/D52608
llvm-svn: 343812
r337531 changed return type of MmapFixedNoReserve, but esan wasn't updated.
As the result esan shadow setup always fails.
We probably need to make MmapFixedNoAccess signature consistent
with MmapFixedNoReserve. But this is just to unbreak tests.
llvm-svn: 337550
when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.
To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.
The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.
The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).
The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:
* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
avoid because it would include a bunch of stuff we don't need.
Differential Revision: https://reviews.llvm.org/D48422
llvm-svn: 336663
Summary:
Add fgets, fputs and puts to sanitizer_common. This adds ASAN coverage
for these functions, extends MSAN support from fgets to fputs/puts and
extends TSAN support from puts to fputs.
Fixes: https://github.com/google/sanitizers/issues/952
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D46545
llvm-svn: 334450
Summary:
Host symbolizer & stacktraces related code in their own RT:
`RTSanitizerCommonSymbolizer`, which is "libcdep" by nature. Symbolizer &
stacktraces specific code that used to live in common files is moved to a new
file `sanitizer_symbolizer_report.cc` as is.
The purpose of this is the enforce a separation between code that relies on
symbolization and code that doesn't. This saves the inclusion of spurious code
due to the interface functions with default visibility, and the extra data
associated.
The following sanitizers makefiles were modified & tested locally:
- dfsan: doesn't require the new symbolizer RT
- esan: requires it
- hwasan: requires it
- lsan: requires it
- msan: requires it
- safestack: doesn't require it
- xray: doesn't require it
- tsan: requires it
- ubsan: requires it
- ubsan_minimal: doesn't require it
- scudo: requires it (but not for Fuchsia that has a minimal runtime)
This was tested locally on Linux, Android, Fuchsia.
Reviewers: alekseyshl, eugenis, dberris, kubamracek, vitalybuka, dvyukov, mcgrathr
Reviewed By: alekseyshl, vitalybuka
Subscribers: srhines, kubamracek, mgorny, krytarowski, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D45457
llvm-svn: 330131
Summary:
Implement the skeleton of NetBSD syscall hooks for use with sanitizers.
Add a script that generates the rules to handle syscalls
on NetBSD: generate_netbsd_syscalls.awk. It has been written
in NetBSD awk(1) (patched nawk) and is compatible with gawk.
Generate lib/sanitizer_common/sanitizer_platform_limits_netbsd.h
that is a public header for applications, and included as:
<sanitizer_common/sanitizer_platform_limits_netbsd.h>.
Generate sanitizer_syscalls_netbsd.inc that defines all the
syscall rules for NetBSD. This file is modeled after the Linux
specific file: sanitizer_common_syscalls.inc.
Start recognizing NetBSD syscalls with existing sanitizers:
ASan, ESan, HWASan, TSan, MSan.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, vitalybuka, kcc, dvyukov, eugenis
Reviewed By: vitalybuka
Subscribers: hintonda, kubamracek, mgorny, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D42048
llvm-svn: 325206
The struct tag is going away in soon-to-be-released glibc 2.26 and the
stack_t typedef seems to have been there forever.
Patch by Bernhard Rosenkraenzer!
llvm-svn: 311495
Summary:
This is the first in a series of patches to refactor sanitizer_procmaps
to allow MachO section information to be exposed on darwin.
In addition, grouping all segment information in a single struct is
cleaner than passing it through a large set of output parameters, and
avoids the need for annotations of NULL parameters for unneeded
information.
The filename string is optional and must be managed and supplied by the
calling function. This is to allow the MemoryMappedSegment struct to be
stored on the stack without causing overly large stack sizes.
Reviewers: alekseyshl, kubamracek, glider
Subscribers: emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D35135
llvm-svn: 307688
{M, T, E}San have fread and fwrite interceptors, let's move them to sanitizer_common to enable ASan checks as well.
Differential Revision: https://reviews.llvm.org/D31456
llvm-svn: 299061
Summary:
Handles early allocation from dlsym by allocating memory from a local
static buffer.
Reviewers: bruening
Subscribers: kubabrecka
Differential Revision: https://reviews.llvm.org/D25193
llvm-svn: 283139
For mips assember '#' is the start of comment. We get assembler error messages if # is used in the struct names. Therefore using '$' which works for all architectures.
Differential: D24335
Reviewed by: zhaoqin
llvm-svn: 282142
The definitions in sanitizer_common may conflict with definitions from system headers because:
The runtime includes the system headers after the project headers (as per LLVM coding guidelines).
lib/sanitizer_common/sanitizer_internal_defs.h pollutes the namespace of everything defined after it, which is all/most of the sanitizer .h and .cc files and the included system headers with: using namespace __sanitizer; // NOLINT
This patch solves the problem by introducing the namespace only within the sanitizer namespaces as proposed by Dmitry.
Differential Revision: https://reviews.llvm.org/D21947
llvm-svn: 281657
With this patch 10 out of 13 tests are passing.
Following is the list of failing tests:
struct-simple.cpp
workingset-signal-posix.cpp
mmap-shadow-conflict.c
Reviewed by bruening
Differential: D23799
llvm-svn: 280795
This patch builds on LLVM r279776.
In this patch I've done some cleanup and abstracted three common steps runtime components have in their CMakeLists files, and added a fourth.
The three steps I abstract are:
(1) Add a top-level target (i.e asan, msan, ...)
(2) Set the target properties for sorting files in IDE generators
(3) Make the compiler-rt target depend on the top-level target
The new step is to check if a command named "runtime_register_component" is defined, and to call it with the component name.
The runtime_register_component command is defined in llvm/runtimes/CMakeLists.txt, and presently just adds the component to a list of sub-components, which later gets used to generate target mappings.
With this patch a new workflow for runtimes builds is supported. The new workflow when building runtimes from the LLVM runtimes directory is:
> cmake [...]
> ninja runtimes-configure
> ninja asan
The "runtimes-configure" target builds all the dependencies for configuring the runtimes projects, and runs CMake on the runtimes projects. Running the runtimes CMake generates a list of targets to bind into the top-level CMake so subsequent build invocations will have access to some of Compiler-RT's targets through the top-level build.
Note: This patch does exclude some top-level targets from compiler-rt libraries because they either don't install files (sanitizer_common), or don't have a cooresponding `check` target (stats).
llvm-svn: 279863
Summary:
Adds a new, generic, resizing hashtable data structure for use by esan
tools. No existing sanitizer hashtable is suitable for the use case for
most esan tools: we need non-fixed-size tables, parameterized keys and
payloads, and write access to payloads. The new hashtable uses either
simple internal or external mutex locking and supports custom hash and
comparision operators. The focus is on functionality, not performance, to
catalyze creation of a variety of tools. We can optimize the more
successful tools later.
Adds tests of the data structure.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: https://reviews.llvm.org/D22681
llvm-svn: 278024
Adds a new esan public interface routine __esan_get_sample_count() and uses
it to ensure that tests of sampling receive the minimum number of samples.
llvm-svn: 275948
Fixes an esan sideline thread CHECK that failed to account for the sideline
thread reaching its code before the internal_clone() return value was
assigned in the parent.
llvm-svn: 275946
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.
Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.
This patch doesn't contain behavior changes.
Reviewers: kubabrecka, rnk
Subscribers: wang0109, llvm-commits, kubabrecka, chrisha
Differential Revision: http://reviews.llvm.org/D21952
llvm-svn: 275111
Summary:
Adds a new public interface routine __esan_report() which can be used to
request profiling results prior to abnormal termination (e.g., for a server
process killed by its parent where the normal exit does not allow for
normal result reporting).
Implements this for the working-set tool. The cache frag tool is left
unimplemented as it requires missing iteration capabilities.
Adds a new test.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D22098
llvm-svn: 274964
Summary:
Adds interception of sigprocmask and pthread_sigmask to esan so that the
working set tool can prevent SIGSEGV from being blocked. A blocked SIGSEGV
results in crashes due to our lazy shadow page allocation scheme.
Adds new sanitizer helper functions internal_sigemptyset and
internal_sigismember.
Adds a test to workingset-signal-posix.cpp.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D22063
llvm-svn: 274672
Summary:
Adds the struct field size array in the struct StructInfo.
Prints struct field size info in the report.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, bruening, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D21342
llvm-svn: 272988
Fixes another interceptor issue where an app with a static tcmalloc
library that prevents our early-calloc handling from triggering yet
does not have a static mmap crashes in our mmap interceptor. The
solution is to call internal_mmap when REAL(mmap) is not yet set up.
llvm-svn: 272870
Summary:
Reports the struct field access info only if it has been used at least
once.
Adds type printing size limit.
Formats the cache_frag.cpp with clang-format.
Reviewers: bruening
Subscribers: llvm-commits, eugenis, kcc, zhaoqin, vitalybuka, aizatsky, kubabrecka
Differential Revision: http://reviews.llvm.org/D21351
llvm-svn: 272810
Summary:
Adds a version of sigaction that uses a raw system call, to avoid circular
dependencies and support calling sigaction prior to setting up
interceptors. The new sigaction relies on an assembly sigreturn routine
for its restorer, which is Linux x86_64-only for now.
Uses the new sigaction to initialize the working set tool's shadow fault
handler prior to libc interceptor being set up. This is required to
support instrumentation invoked during interceptor setup, which happens
with an instrumented tcmalloc or other allocator compiled with esan.
Adds a test that emulates an instrumented allocator.
Reviewers: aizatsky
Subscribers: vitalybuka, tberghammer, zhaoqin, danalbert, kcc, srhines, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D21083
llvm-svn: 272676
Summary:
Adds a version of sigaction that uses a raw system call, to avoid circular
dependencies and support calling sigaction prior to setting up
interceptors. The new sigaction relies on an assembly sigreturn routine
for its restorer, which is Linux x86_64-only for now.
Uses the new sigaction to initialize the working set tool's shadow fault
handler prior to libc interceptor being set up. This is required to
support instrumentation invoked during interceptor setup, which happens
with an instrumented tcmalloc or other allocator compiled with esan.
Adds a test that emulates an instrumented allocator.
Reviewers: aizatsky
Subscribers: vitalybuka, tberghammer, zhaoqin, danalbert, kcc, srhines, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D21083
llvm-svn: 272591
Summary:
Adds a version of sigaction that uses a raw system call, to avoid circular
dependencies and support calling sigaction prior to setting up
interceptors. The new sigaction relies on an assembly sigreturn routine
for its restorer, which is Linux x86_64-only for now.
Uses the new sigaction to initialize the working set tool's shadow fault
handler prior to libc interceptor being set up. This is required to
support instrumentation invoked during interceptor setup, which happens
with an instrumented tcmalloc or other allocator compiled with esan.
Adds a test that emulates an instrumented allocator.
Reviewers: aizatsky
Subscribers: vitalybuka, tberghammer, zhaoqin, danalbert, kcc, srhines, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D21083
llvm-svn: 272553
Summary:
Adds the struct field offset array in the struct StructInfo.
Prints struct size and field offset info in the report.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, bruening, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D21191
llvm-svn: 272363
Summary:
When tcmalloc initializes before esan, esan's initialization ends up
calling back into tcmalloc due to the calloc done by dlsym. This results
in a deadlock. We avoid this by special-casing this single allocation.
Intercepting calloc also gives us the opportunity to act on its zeroing as
stores by the application.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D21086
llvm-svn: 272076
Summary:
Adds initialization of esan's runtime library during any early interceptors
that are sometimes called prior to the official __esan_init() invocation
(we see this with apps using tcmalloc).
Adds handling of interceptors called during interceptor initialization.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D20976
llvm-svn: 271744
Summary:
Computes the struct field access variation based on each field access
count.
Adds a flag to control the report thresholds.
Updates struct-simple.cpp with variance report output.
Reviewers: aizatsky
Subscribers: kubabrecka, zhaoqin, llvm-commits, eugenis, vitalybuka, kcc, bruening
Differential Revision: http://reviews.llvm.org/D20914
llvm-svn: 271734
Summary:
Adds a global variable to specify the tool, to support handling early
interceptors that invoke instrumented code, thus requiring shadow memory to
be initialized prior to __esan_init() being invoked.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D20974
llvm-svn: 271714
Summary:
Adds a new option -snapshot_step controlling the frequency distribution for
an 8-level series of samples using each bit of each shadow byte.
Implements accumulation from each level to the next higher level at the
specified frequency.
Adds storage of the 8 series of samples using CircularBuffer instances.
Fixes an error in the circular buffer data structure where a static
object's destructor will be called too early.
Prints the results out at the end in a simple manner to give us something
to start with.
Updates the workingset-samples test to test the new feature.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D20833
llvm-svn: 271683
Summary:
Adds support for creating a separate thread for performing "sideline"
actions on a periodic basis via an itimer. A new class SidelineThread
implements this feature, exposing a sampling callback to the caller.
Adds initial usage of sideline sampling to the working set tool. For now
it simply prints the usage at each snapshot at verbosity level 1. Adds a
test of this behavior. Adds a new option -record_snapshots to control
whether we sample and a new option -sample_freq to control the periodicity
of the sampling.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D20751
llvm-svn: 271682
Summary:
Implements real_sigaction() which it turns out is required for
internal_sigaction() to bypass the libc interceptors.
Without real_sigaction(), our internal_sigaction() calls during init happen
to work due to the EsanDuringInit check in COMMON_INTERCEPTOR_ENTER (though
even here it does not feel right for an "internal_" call to go through the
interceptor). The real problem is when we call internal_sigaction() after
we're initialized, which only happens on an unhandled SIGSEGV for which the
app has no handler: then we'll spin in an infinite loop as our attempts to
remove our own handler repeatedly fail. It's not easy to add a test for
that, unfortunately.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D20832
llvm-svn: 271626
David Carlier