Summary:
This is the first mostly working version of the Sanitizer port to 32-bit Solaris/x86.
It is currently based on Solaris 11.4 Beta.
This part was initially developed inside libsanitizer in the GCC tree and should apply to
both. Subsequent parts will address changes to clang, the compiler-rt build system
and testsuite.
I'm not yet sure what the right patch granularity is: if it's profitable to split the patch
up, I'd like to get guidance on how to do so.
Most of the changes are probably straightforward with a few exceptions:
* The Solaris syscall interface isn't stable, undocumented and can change within an
OS release. The stable interface is the libc interface, which I'm using here, if possible
using the internal _-prefixed names.
* While the patch primarily target 32-bit x86, I've left a few sparc changes in. They
cannot currently be used with clang due to a backend limitation, but have worked
fine inside the gcc tree.
* Some functions (e.g. largefile versions of functions like open64) only exist in 32-bit
Solaris, so I've introduced a separate SANITIZER_SOLARIS32 to check for that.
The patch (with the subsequent ones to be submitted shortly) was tested
on i386-pc-solaris2.11. Only a few failures remain, some of them analyzed, some
still TBD:
AddressSanitizer-i386-sunos :: TestCases/Posix/concurrent_overflow.cc
AddressSanitizer-i386-sunos :: TestCases/init-order-atexit.cc
AddressSanitizer-i386-sunos :: TestCases/log-path_test.cc
AddressSanitizer-i386-sunos :: TestCases/malloc-no-intercept.c
AddressSanitizer-i386-sunos-dynamic :: TestCases/Posix/concurrent_overflow.cc
AddressSanitizer-i386-sunos-dynamic :: TestCases/Posix/start-deactivated.cc
AddressSanitizer-i386-sunos-dynamic :: TestCases/default_options.cc
AddressSanitizer-i386-sunos-dynamic :: TestCases/init-order-atexit.cc
AddressSanitizer-i386-sunos-dynamic :: TestCases/log-path_test.cc
AddressSanitizer-i386-sunos-dynamic :: TestCases/malloc-no-intercept.c
SanitizerCommon-Unit :: ./Sanitizer-i386-Test/MemoryMappingLayout.DumpListOfModules
SanitizerCommon-Unit :: ./Sanitizer-i386-Test/SanitizerCommon.PthreadDestructorIterations
Maybe this is good enough the get the ball rolling.
Reviewers: kcc, alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, jyknight, kubamracek, krytarowski, fedor.sergeev, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D40898
llvm-svn: 320740
Summary:
See D40657 & D40679 for previous versions of this patch & description.
A couple of things were fixed here to have it not break some bots.
Weak symbols can't be used with `SANITIZER_GO` so the previous version was
breakin TsanGo. I set up some additional local tests and those pass now.
I changed the workaround for the glibc vDSO issue: `__progname` is initialized
after the vDSO and is actually public and of known type, unlike
`__vdso_clock_gettime`. This works better, and with all compilers.
The rest is the same.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D41121
llvm-svn: 320594
Summary:
Redo of D40657, which had the initial discussion. The initial code had to move
into a libcdep file, and things had to be shuffled accordingly.
`NanoTime` is a time sink when checking whether or not to release memory to
the OS. While reducing the amount of calls to said function is in the works,
another solution that was found to be beneficial was to use a timing function
that can leverage the vDSO.
We hit a couple of snags along the way, like the fact that the glibc crashes
when clock_gettime is called from a preinit_array, or the fact that
`__vdso_clock_gettime` is mangled (for security purposes) and can't be used
directly, and also that clock_gettime can be intercepted.
The proposed solution takes care of all this as far as I can tell, and
significantly improve performances and some Scudo load tests with memory
reclaiming enabled.
@mcgrathr: please feel free to follow up on
https://reviews.llvm.org/D40657#940857 here. I posted a reply at
https://reviews.llvm.org/D40657#940974.
Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek
Reviewed By: alekseyshl, krytarowski
Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D40679
llvm-svn: 320409
Summary:
This change reverts r318575 and changes FindDynamicShadowStart() to
keep the memory range it found mapped PROT_NONE to make sure it is
not reused. We also skip MemoryRangeIsAvailable() check, because it
is (a) unnecessary, and (b) would fail anyway.
Reviewers: pcc, vitalybuka, kcc
Subscribers: srhines, kubamracek, mgorny, llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D40203
llvm-svn: 318666
Revert the following commits:
r318369 [asan] Fallback to non-ifunc dynamic shadow on android<22.
r318235 [asan] Prevent rematerialization of &__asan_shadow.
r317948 [sanitizer] Remove unnecessary attribute hidden.
r317943 [asan] Use dynamic shadow on 32-bit Android.
MemoryRangeIsAvailable() reads /proc/$PID/maps into an mmap-ed buffer
that may overlap with the address range that we plan to use for the
dynamic shadow mapping. This is causing random startup crashes.
llvm-svn: 318575
Summary:
The following kernel change has moved ET_DYN base to 0x4000000 on arm32:
https://marc.info/?l=linux-kernel&m=149825162606848&w=2
Switch to dynamic shadow base to avoid such conflicts in the future.
Reserve shadow memory in an ifunc resolver, but don't use it in the instrumentation
until PR35221 is fixed. This will eventually let use save one load per function.
Reviewers: kcc
Subscribers: aemerson, srhines, kubamracek, kristof.beyls, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D39393
llvm-svn: 317943
Summary:
These will be used in an ifunc resolver, when the binary may not be
completely relocated, and syscall() function from libc could not be
used.
Reviewers: dvyukov, vitalybuka
Subscribers: aemerson, kubamracek, javed.absar, llvm-commits, kristof.beyls
Differential Revision: https://reviews.llvm.org/D39701
llvm-svn: 317640
Summary:
Relanding D38600, which was reverted due to various PPC bot failures.
If it breaks something again, please provide some pointers to broken
bots, not just revert it, otherwise it's very hard to reason what's
wrong with this commit.
Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D39275
llvm-svn: 317512
Summary:
Changes:
* Add initial msan stub support.
* Handle NetBSD specific pthread_setname_np(3).
* NetBSD supports __attribute__((tls_model("initial-exec"))),
define it in SANITIZER_TLS_INITIAL_EXEC_ATTRIBUTE.
* Add ReExec() specific bits for NetBSD.
* Simplify code and add syscall64 and syscall_ptr for !NetBSD.
* Correct bunch of syscall wrappers for NetBSD.
* Disable test/tsan/map32bit on NetBSD as not applicable.
* Port test/tsan/strerror_r to a POSIX-compliant OSes.
* Disable __libc_stack_end on NetBSD.
* Disable ReadNullSepFileToArray() on NetBSD.
* Define struct_ElfW_Phdr_sz, detected missing symbol by msan.
* Change type of __sanitizer_FILE from void to char. This helps
to reuse this type as an array. Long term it will be properly
implemented along with SANITIZER_HAS_STRUCT_FILE setting to 1.
* Add initial NetBSD support in lib/tsan/go/buildgo.sh.
* Correct referencing stdout and stderr in tsan_interceptors.cc
on NetBSD.
* Document NetBSD x86_64 specific virtual memory layout in
tsan_platform.h.
* Port tests/rtl/tsan_test_util_posix.cc to NetBSD.
* Enable NetBSD tests in test/msan/lit.cfg.
* Enable NetBSD tests in test/tsan/lit.cfg.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, vitalybuka, eugenis, kcc, dvyukov
Reviewed By: dvyukov
Subscribers: #sanitizers, llvm-commits, kubamracek
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D39124
llvm-svn: 316591
Summary:
Relanding D33859, which was reverted because it has "broken LOTS of
ARM/AArch64 bots for two days".
If it breaks something again, please provide some pointers to broken
bots, not just revert it, otherwise it's very hard to reason what's
wrong with this commit.
Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.
Reviewers: eugenis
Subscribers: aemerson, kubamracek, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D38600
llvm-svn: 315024
This patch tackles with two issues:
Output stat st_[a|m|c]time fields were holding wrong values.
st_[a|m|c]time fields should have contained value of seconds and instead
these are filled with st_[a|m|c]time_nsec fields which hold nanoseconds.
Build fails for MIPS64 if SANITIZER_ANDROID. Recently <sys/stat.h> from
bionic introduced st_[a|m|c]time_nsec macros for compatibility with old NDKs
and those clashed with the field names of the <asm/stat.h> kernel_stat
structure.
To fix both issues and make sure sanitizer builds on all platforms, we must
un-define all compatibility macros and access the fields directly when
copying the 'time' fields.
Patch by Miodrag Dinic <miodrag.dinic@imgtec.com>
Differential Revision: https://reviews.llvm.org/D35671
llvm-svn: 313360
Summary:
On platforms with `getrandom`, the system call defaults to blocking. This
becomes an issue in the very early stage of the boot for Scudo, when the RNG
source is not set-up yet: the syscall will block and we'll stall.
Introduce a parameter to specify that the function should not block, defaulting
to blocking as the underlying syscall does.
Update Scudo to use the non-blocking version.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D36399
llvm-svn: 310839
Summary:
Follow FreeBSD and reuse sanitizer_linux for NetBSD.
Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.
Sponsored by <The NetBSD Foundation>
Reviewers: joerg, kcc, filcab, vitalybuka, fjricci, dvyukov
Reviewed By: fjricci
Subscribers: dvyukov, emaste, kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D36325
llvm-svn: 310411
Summary:
libsanitizer doesn't build against latest glibc anymore, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81066 for details.
One of the changes is that stack_t changed from typedef struct sigaltstack { ... } stack_t; to typedef struct { ... } stack_t; for conformance reasons.
And the other change is that the glibc internal __need_res_state macro is now ignored, so when doing
```
#define __need_res_state
#include <resolv.h>
```
the effect is now the same as just
```
#include <resolv.h>
```
and thus one doesn't get just the
```
struct __res_state { ... };
```
definition, but newly also the
```
extern struct __res_state *__res_state(void) __attribute__ ((__const__));
```
prototype. So __res_state is no longer a type, but a function.
Reviewers: kcc, ygribov
Reviewed By: kcc
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D35246
llvm-svn: 307969
On iOS/AArch64, the address space is very limited and has a dynamic maximum address based on the configuration of the device. We're already using a dynamic shadow, and we find a large-enough "gap" in the VM where we place the shadow memory. In some cases and some device configuration, we might not be able to find a large-enough gap: E.g. if the main executable is linked against a large number of libraries that are not part of the system, these libraries can fragment the address space, and this happens before ASan starts initializing.
This patch has a solution, where we have a "backup plan" when we cannot find a large-enough gap: We will restrict the address space (via MmapFixedNoAccess) to a limit, for which the shadow limit will fit.
Differential Revision: https://reviews.llvm.org/D35098
llvm-svn: 307865
Summary:
This is the first in a series of patches to refactor sanitizer_procmaps
to allow MachO section information to be exposed on darwin.
In addition, grouping all segment information in a single struct is
cleaner than passing it through a large set of output parameters, and
avoids the need for annotations of NULL parameters for unneeded
information.
The filename string is optional and must be managed and supplied by the
calling function. This is to allow the MemoryMappedSegment struct to be
stored on the stack without causing overly large stack sizes.
Reviewers: alekseyshl, kubamracek, glider
Subscribers: emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D35135
llvm-svn: 307688
The logic in GetMaxVirtualAddress is already pretty complex, and I want to get rid of the hardcoded value for iOS/AArch64, which would need adding more Darwin-specific code, so let's split the implementation into sanitizer_linux.cc and sanitizer_mac.cc files. NFC.
Differential Revision: https://reviews.llvm.org/D35031
llvm-svn: 307281
Summary:
AFAICT compiler-rt doesn't have a function that would return 'good' random
bytes to seed a PRNG. Currently, the `SizeClassAllocator64` uses addresses
returned by `mmap` to seed its PRNG, which is not ideal, and
`SizeClassAllocator32` doesn't benefit from the entropy offered by its 64-bit
counterpart address space, so right now it has nothing. This function aims at
solving this, allowing to implement good 32-bit chunk randomization. Scudo also
has a function that does this for Cookie purposes, which would go away in a
later CL once this lands.
This function will try the `getrandom` syscall if available, and fallback to
`/dev/urandom` if not.
Unfortunately, I do not have a way to implement and test a Mac and Windows
version, so those are unimplemented as of now. Note that `kRandomShuffleChunks`
is only used on Linux for now.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: zturner, rnk, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34412
llvm-svn: 305922
Summary:
After r303941 it was not possible to setup ASAN_OPTIONS to have the same
behavior for pre r303941 and post r303941 builds.
Pre r303941 Asan does not accept handle_sigbus=2.
Post r303941 Asan does not accept allow_user_segv_handler.
This fix ignores allow_user_segv_handler=1, but for allow_user_segv_handler=0
it will upgrade flags like handle_sigbus=1 to handle_sigbus=2. So user can set
ASAN_OPTIONS=allow_user_segv_handler=0 and have same behavior on old and new
clang builds (except range from r303941 to this revision).
In future users which need to prevent third party handlers should switch to
handle_sigbus=2 and remove allow_user_segv_handler as soon as suport of older
builds is not needed.
Related bugs:
https://github.com/google/oss-fuzz/issues/675https://bugs.chromium.org/p/chromium/issues/detail?id=731130
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34227
llvm-svn: 305433
Revert "Mark sancov test as unsupported on Darwin"
Revert "[LSan] Detect dynamic loader by its base address."
This reverts commit r304633.
This reverts commit r304673.
This reverts commit r304632.
Those commit have broken LOTS of ARM/AArch64 bots for two days.
llvm-svn: 304699
Summary:
Very recently, FreeBSD 12 has been updated to use 64-bit inode numbers:
<https://svnweb.freebsd.org/changeset/base/318737>. This entails many
user-visible changes, but for the sanitizers the modifications are
limited in scope:
* The `stat` and `lstat` syscalls were removed, and should be replaced
with calls to `fstatat`.
* The `getdents` syscall was removed, and should be replaced with calls
to `getdirentries`.
* The layout of `struct dirent` was changed to accomodate 64-bit inode
numbers, and a new `d_off` field was added.
* The system header <sys/_types.h> now contains a macro `__INO64` to
determine whether the system uses 64-bit inode numbers.
I tested these changes on both FreeBSD 12.0-CURRENT (after r318959,
which adds the `__INO64` macro), and FreeBSD 11.0-STABLE (which still
uses 32-bit inode numbers).
Reviewers: emaste, kcc, vitalybuka, kubamracek
Reviewed By: vitalybuka
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D33600
llvm-svn: 304658
Summary:
Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D33859
llvm-svn: 304633
Summary:
allow_user_segv_handler had confusing name did not allow to control behavior for
signals separately.
Reviewers: eugenis, alekseyshl, kcc
Subscribers: llvm-commits, dberris, kubamracek
Differential Revision: https://reviews.llvm.org/D33371
llvm-svn: 303941
Summary:
Dmitry, seeking your expertise. I believe, the proper way to implement
Lock/Unlock here would be to use acquire/release semantics. Am I missing
something?
Reviewers: dvyukov
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33521
llvm-svn: 303869
We seem to assume that OS-provided thread IDs are either uptr or int, neither of which is true on Darwin. This introduces a tid_t type, which holds a OS-provided thread ID (gettid on Linux, pthread_threadid_np on Darwin, pthread_self on FreeBSD).
Differential Revision: https://reviews.llvm.org/D31774
llvm-svn: 300473
Summary:
Set up the proper stack frame for the thread spawned in internal_clone,
the current code does not follow ABI (and causes SEGV trying to use this
malformed frame).
Reviewers: wschmidt
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D31555
llvm-svn: 299896
Summary:
sysconf(_SC_PAGESIZE) is called very early during sanitizer init and
any instrumented code (sysconf() wrapper/interceptor will likely be
instrumented) calling back to sanitizer before init is done will
most surely crash.
2nd attempt, now with glibc version checks (D31092 was reverted).
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D31221
llvm-svn: 298613
Summary:
sysconf(_SC_PAGESIZE) is called very early, during sanitizer init and
any instrumented code (a wrapper/interceptor will likely be instrumented)
calling back to sanitizer before init is done will most surely crash.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D31092
llvm-svn: 298305
check for the existence of RTLD_DEEPBIND, since this constant is only
supported for glibc >= 2.3.4. This fixes builds for FreeBSD and other
platforms that do not have RTLD_DEEPBIND.
llvm-svn: 297763
Summary: This is useful in some platforms where one of these signals is special.
Reviewers: kubamracek, kcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D30783
llvm-svn: 297665
People keep hitting on spurious failures in malloc/free routines when using sanitizers
with shared libraries dlopened with RTLD_DEEPBIND (see https://github.com/google/sanitizers/issues/611 for details).
Let's check for this flag and bail out with warning message instead of failing in random places.
Differential Revision: https://reviews.llvm.org/D30504
llvm-svn: 297370
Breaks tests on i686/Linux due to missing clang driver support:
error: unsupported option '-fsanitize=leak' for target 'i386-unknown-linux-gnu'
llvm-svn: 292844
People keep asking LSan to be available on 32 bit targets (e.g. https://github.com/google/sanitizers/issues/403)
despite the fact that false negative ratio might be huge (up to 85%). This happens for big real world applications
that may contain random binary data (e.g. browser), but for smaller apps situation is not so terrible and LSan still might be useful.
This patch adds initial support for x86 Linux (disabled by default), ARM32 is in TODO list.
We used this patch (well, ported to GCC) on our 32 bit mobile emulators and it worked pretty fine
thus I'm posting it here to initiate further discussion.
Differential Revision: https://reviews.llvm.org/D28609
llvm-svn: 292775
This patch add a new sanitizer flag, print_module_map, which enables printing a module map when the process exits, or after each report (for TSan). The output format is very similar to what Crash Reporter produces on Darwin (e.g. the format of module UUIDs). This enables users to use the existing symbol servers to offline symbolicate and aggregate reports.
Differential Revision: https://reviews.llvm.org/D27400
llvm-svn: 291277
This patch prints out all CPU registers after a SIGSEGV. These are available in the signal handler context. Only implemented for Darwin. Can be turned off with the dump_registers flag.
Differential Revision: https://reviews.llvm.org/D11365
llvm-svn: 287957
Currently we either define SANITIZER_GO for Go or don't define it at all for C++.
This works fine with preprocessor (ifdef/ifndef/defined), but does not work
for C++ if statements (e.g. if (SANITIZER_GO) {...}). Also this is different
from majority of SANITIZER_FOO macros which are always defined to either 0 or 1.
Always define SANITIZER_GO to either 0 or 1.
This allows to use SANITIZER_GO in expressions and in flag default values.
Also remove kGoMode and kCppMode, which were meant to be used in expressions,
but they are not defined in sanitizer_common code, so SANITIZER_GO become prevalent.
Also convert some preprocessor checks to C++ if's or ternary expressions.
Majority of this change is done mechanically with:
sed "s#ifdef SANITIZER_GO#if SANITIZER_GO#g"
sed "s#ifndef SANITIZER_GO#if \!SANITIZER_GO#g"
sed "s#defined(SANITIZER_GO)#SANITIZER_GO#g"
llvm-svn: 285443
Kamil Rytarowski