Commit Graph

2616 Commits

Author SHA1 Message Date
Vitaly Buka 7af8fa41d3 [compiler-rt] Replace VPrintf with VReport in sanitizer_tls_get_addr.cc
Summary:
Thread id will be added to VRerort. Having thread here is useful.
This is also common place for logging for all sanitizers, so I can use this in
common test.

Reviewers: kcc, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D35655

llvm-svn: 308578
2017-07-20 01:36:16 +00:00
Francis Ricci bf60f68c2c Revert "Add MemoryMappedSection struct for two-level memory map iteration"
This reverts commit c8095ce74118dee8544b0f1ffaba8f46aa10215c.

Reverted due to some buildbot timeouts, perhaps due to 10.11 issues.

llvm-svn: 308395
2017-07-18 23:51:47 +00:00
Francis Ricci 1b5f773859 Revert "Only scan global sections containing data in LSan on darwin"
This reverts commit 7e46d78d47832f03ce42adcf56417fbfd47cbaad.

llvm-svn: 308394
2017-07-18 23:51:44 +00:00
Alex Shlyapnikov 42bea018af [Sanitizers] ASan/MSan/LSan allocators set errno on failure.
Summary:
ASan/MSan/LSan allocators set errno on allocation failures according to
malloc/calloc/etc. expected behavior.

MSan allocator was refactored a bit to make its structure more similar
with other allocators.

Also switch Scudo allocator to the internal errno definitions.

TSan allocator changes will follow.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D35275

llvm-svn: 308344
2017-07-18 19:11:04 +00:00
Francis Ricci 7096b08cd7 Only scan global sections containing data in LSan on darwin
Summary:
__DATA segments on Darwin contain a large number of separate sections,
most of which cannot actually contain pointers, and contain const values or
objc metadata. Only scanning sections which can contain pointers greatly improves
performance.

On a medium-sized (~4000 files) internal project, I saw a speedup of about 50%
in standalone LSan's execution time (50% improvement in the time spent running
LSan, not the total program time).

Reviewers: kcc, kubamracek, alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35432

llvm-svn: 308231
2017-07-17 23:03:03 +00:00
Vitaly Buka 4a8f16ec9d [sanitizers] Make sure that all complex macros uses parenthesis
Summary:
Without them expressions like this may have different values.
(SANITIZER_INTERCEPT_MEMRCHR && SANITIZER_INTERCEPT_PREADV)

Reviewers: alekseyshl

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D35512

llvm-svn: 308228
2017-07-17 22:49:46 +00:00
Kamil Rytarowski e9140e5b95 Add missing && to fix syntax.
llvm-svn: 308221
2017-07-17 21:09:34 +00:00
Kamil Rytarowski 20f5a5c64c Add NetBSD support in platform_interceptors.h
Summary:
Introduce SI_NETBSD for NetBSD.

Add NetBSD support for appropriate `SANITIZER_INTERCEPT_*`.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dim, kcc, alekseyshl, filcab, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35468

llvm-svn: 308217
2017-07-17 20:49:13 +00:00
Kamil Rytarowski 27169a7709 Introduce SANITIZER_NETBSD in sanitizer_platform.h
Summary:
Add defines for new NetBSD: SANITIZER_NETBSD,
it will be used across the codebase for sanitizers.

NetBSD is a POSIX-like platform, add it to SANITIZER_POSIX.

Part of the code inspired by the original work on libsanitizer in GCC 5.4 by Christos Zoulas.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, dim, alekseyshl, filcab, eugenis, vitalybuka

Reviewed By: kcc

Subscribers: kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35467

llvm-svn: 308216
2017-07-17 20:47:53 +00:00
Francis Ricci 455ba3fd79 Add MemoryMappedSection struct for two-level memory map iteration
Summary: This will allow sanitizer_procmaps on mac to expose section information.

Reviewers: kubamracek, alekseyshl, kcc

Subscribers: llvm-commits, emaste

Differential Revision: https://reviews.llvm.org/D35422

llvm-svn: 308210
2017-07-17 20:09:20 +00:00
Vitaly Buka 3f25b53485 [asan] Attempt to fix test in Windows after r308064
llvm-svn: 308120
2017-07-16 00:40:40 +00:00
Vitaly Buka 50648af623 [compiler-rt] Fix fix format specifies type in test
llvm-svn: 308117
2017-07-16 00:17:11 +00:00
Kostya Serebryany c56d444b75 Fix sanitizer build against latest glibc
Summary:
libsanitizer doesn't build against latest glibc anymore, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81066 for details.
One of the changes is that stack_t changed from typedef struct sigaltstack { ... } stack_t; to typedef struct { ... } stack_t; for conformance reasons.
And the other change is that the glibc internal __need_res_state macro is now ignored, so when doing
```
#define __need_res_state
#include <resolv.h>
```
the effect is now the same as just
```
#include <resolv.h>
```
and thus one doesn't get just the
```
struct __res_state { ... };
```
definition, but newly also the
```
extern struct __res_state *__res_state(void) __attribute__ ((__const__));
```
prototype.  So __res_state is no longer a type, but a function.

Reviewers: kcc, ygribov

Reviewed By: kcc

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D35246

llvm-svn: 307969
2017-07-13 21:59:01 +00:00
Kuba Mracek f0c652a52e For Darwin's GetTaskInfoMaxAddress, define the real structure with real fields to avoid confusion. NFC.
llvm-svn: 307945
2017-07-13 20:02:45 +00:00
Kuba Mracek c1e903be19 [asan] For iOS/AArch64, if the dynamic shadow doesn't fit, restrict the VM space
On iOS/AArch64, the address space is very limited and has a dynamic maximum address based on the configuration of the device. We're already using a dynamic shadow, and we find a large-enough "gap" in the VM where we place the shadow memory. In some cases and some device configuration, we might not be able to find a large-enough gap: E.g. if the main executable is linked against a large number of libraries that are not part of the system, these libraries can fragment the address space, and this happens before ASan starts initializing.

This patch has a solution, where we have a "backup plan" when we cannot find a large-enough gap: We will restrict the address space (via MmapFixedNoAccess) to a limit, for which the shadow limit will fit.

Differential Revision: https://reviews.llvm.org/D35098

llvm-svn: 307865
2017-07-12 23:29:21 +00:00
Francis Ricci 9fd8a6381c Use internal_strncpy to copy filename in linux procmaps
Cleaner than using a while loop to copy the string character by character.

Reviewers: alekseyshl, glider

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35136

llvm-svn: 307696
2017-07-11 19:40:54 +00:00
Francis Ricci edd53cb652 Inline function to get mac segment address range
Summary:
This function is only called once and is fairly simple. Inline to
keep API simple.

Reviewers: alekseyshl, kubamracek

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35270

llvm-svn: 307695
2017-07-11 19:40:53 +00:00
Francis Ricci f6a4329b7d Refactor MemoryMappingLayout::Next to use a single struct instead of output parameters. NFC.
Summary:
This is the first in a series of patches to refactor sanitizer_procmaps
to allow MachO section information to be exposed on darwin.

In addition, grouping all segment information in a single struct is
cleaner than passing it through a large set of output parameters, and
avoids the need for annotations of NULL parameters for unneeded
information.

The filename string is optional and must be managed and supplied by the
calling function. This is to allow the MemoryMappedSegment struct to be
stored on the stack without causing overly large stack sizes.

Reviewers: alekseyshl, kubamracek, glider

Subscribers: emaste, llvm-commits

Differential Revision: https://reviews.llvm.org/D35135

llvm-svn: 307688
2017-07-11 18:54:00 +00:00
Petar Jovanovic adf4ce3a2c Remove duplicate assignments in stat64/kernel_stat_to_stat functions
Remove duplicate assignments in stat64_to_stat() and kernel_stat_to_stat().

llvm-svn: 307657
2017-07-11 13:03:48 +00:00
George Karpenkov 0b4ebb1d0b Do not crash with missing symbolication when running in DEDUP mode
Printing stacktrace from ASAN crashes with a segfault in DEDUP mode when
symbolication is missing.

Differential Revision: https://reviews.llvm.org/D34914

llvm-svn: 307577
2017-07-10 20:06:06 +00:00
Kuba Mracek 0fa9d68740 Fix-up for r307307: vm_info.max_address is the first non-addressable pointer, so we need to subtract one.
llvm-svn: 307408
2017-07-07 15:32:44 +00:00
Kuba Mracek 560440975a Fix whitespace lint issue (introduced in r307307).
llvm-svn: 307309
2017-07-06 20:38:33 +00:00
Kuba Mracek e9b5857db3 [sanitizer] Use TASK_VM_INFO to get the maximum VM address on iOS/AArch64
We currently hardcode the maximum VM address on iOS/AArch64, which is not really correct and this value changes between device configurations. Let's use TASK_VM_INFO to retrieve the maximum VM address dynamically.

Differential Revision: https://reviews.llvm.org/D35032

llvm-svn: 307307
2017-07-06 20:30:09 +00:00
Kuba Mracek c7f895351e More fixup for r307281: Move the #includes to sanitizer_linux.cc
llvm-svn: 307284
2017-07-06 17:45:01 +00:00
Kuba Mracek 47e994c4f1 Fixup for r307281: Also move GetKernelAreaSize into sanitizer_linux.cc
llvm-svn: 307282
2017-07-06 17:17:50 +00:00
Kuba Mracek 72665d6c15 [sanitizer] Split GetMaxVirtualAddress into separate Linux version and Mac version [NFC]
The logic in GetMaxVirtualAddress is already pretty complex, and I want to get rid of the hardcoded value for iOS/AArch64, which would need adding more Darwin-specific code, so let's split the implementation into sanitizer_linux.cc and sanitizer_mac.cc files. NFC.

Differential Revision: https://reviews.llvm.org/D35031

llvm-svn: 307281
2017-07-06 17:13:40 +00:00
Alex Shlyapnikov 35adb43950 [Sanitizers] Consolidate internal errno definitions.
Move internal errno definitions to common to be shared by all sanitizers
and to be used by allocators.

llvm-svn: 307233
2017-07-06 00:50:57 +00:00
Kuba Mracek 7149ca04ad [tsan] Use pthread_sigmask instead of sigprocmask to block signals in a thread on Darwin
On Darwin, sigprocmask changes the signal mask for the entire process. This has some unwanted consequences, because e.g. internal_start_thread wants to disable signals only in the current thread (to make the new thread inherit the signal mask), which is currently broken on Darwin. This patch switches to pthread_sigmask.

Differential Revision: https://reviews.llvm.org/D35016

llvm-svn: 307212
2017-07-05 22:17:44 +00:00
Hiroshi Inoue 03656aa3dc fix trivial typos in comments; NFC
llvm-svn: 307005
2017-07-03 06:44:05 +00:00
Kostya Kortchinsky b138ab810f [sanitizer] Small tweaks and fixes to allocator related functions
Summary:
In `sanitizer_allocator_primary32.h`:
- rounding up in `MapWithCallback` is not needed as `MmapOrDie` does it. Note
  that the 64-bit counterpart doesn't round up, this keeps the behavior
  consistent;
- since `IsAligned` exists, use it in `AllocateRegion`;
- in `PopulateFreeList`:
  - checking `b->Count` to be greater than 0 when `b->Count() == max_count` is
    redundant when done more than once. Just check that `max_count` is greater
    than 0 out of the loop; the compiler (at least on ARM) didn't optimize it;
  - mark the batch creation failure as `UNLIKELY`;

In `sanitizer_allocator_primary64.h`:
- in `MapWithCallback`, mark the failure condition as `UNLIKELY`;

In `sanitizer_posix.h`:
- mark a bunch of Mmap related failure conditions as `UNLIKELY`;
- in `MmapAlignedOrDieOnFatalError`, we have `IsAligned`, so use it; rearrange
  the conditions as one test was redudant;
- in `MmapFixedImpl`, 30 chars was not large enough to hold the message and a
  full 64-bit address (or at least a 48-bit usermode address), increase to 40.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: aemerson, kubamracek, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D34840

llvm-svn: 306834
2017-06-30 16:05:40 +00:00
Alex Shlyapnikov 93c1a8c7c7 Merge
llvm-svn: 306746
2017-06-29 21:54:36 +00:00
Reid Kleckner 4acf36beb6 Fix WinASan after moving wcslen interceptor to sanitizer_common
Do this by removing SANITIZER_INTERCEPT_WCSLEN and intercept wcslen
everywhere. Before this change, we were already intercepting wcslen on
Windows, but the interceptor was in asan, not sanitizer_common. After
this change, we stopped intercepting wcslen on Windows, which broke
asan_dll_thunk.c, which attempts to thunk to __asan_wcslen in the ASan
runtime.

llvm-svn: 306706
2017-06-29 17:15:53 +00:00
Alex Shlyapnikov 4b450685d3 [Sanitizers] Operator new() interceptors always die on allocation error
Summary:
Operator new interceptors behavior is now controlled by their nothrow
property as well as by allocator_may_return_null flag value:

- allocator_may_return_null=* + new()        - die on allocation error
- allocator_may_return_null=0 + new(nothrow) - die on allocation error
- allocator_may_return_null=1 + new(nothrow) - return null

Ideally new() should throw std::bad_alloc exception, but that is not
trivial to achieve, hence TODO.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34731

llvm-svn: 306604
2017-06-28 21:58:57 +00:00
Evgeniy Stepanov 6f75e2dd48 [msan] Intercept wcscat, wcsncat.
Also move wcslen, wscnlen to common interceptors.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34656

llvm-svn: 306482
2017-06-27 22:52:38 +00:00
Alex Shlyapnikov 01676883cd [Sanitizers] 64 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator64 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow
allocator_may_return_null=1 policy
(LargeMmapAllocator: D34243, SizeClassAllocator64: D34433).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34540

llvm-svn: 306342
2017-06-26 22:54:10 +00:00
Evgeniy Stepanov 90e5c30836 [asan] Add support for Android debug message.
Add ASan report to the "debug message" field in Android tombstones.

llvm-svn: 306184
2017-06-23 23:38:20 +00:00
Alex Shlyapnikov f3cc7cc3d8 [Sanitizers] 32 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).

Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34433

llvm-svn: 305972
2017-06-22 00:02:37 +00:00
Kostya Kortchinsky 2b053b1c28 [sanitizer] Add a function to gather random bytes
Summary:
AFAICT compiler-rt doesn't have a function that would return 'good' random
bytes to seed a PRNG. Currently, the `SizeClassAllocator64` uses addresses
returned by `mmap` to seed its PRNG, which is not ideal, and  
`SizeClassAllocator32` doesn't benefit from the entropy offered by its 64-bit
counterpart address space, so right now it has nothing. This function aims at
solving this, allowing to implement good 32-bit chunk randomization. Scudo also
has a function that does this for Cookie purposes, which would go away in a
later CL once this lands.

This function will try the `getrandom` syscall if available, and fallback to
`/dev/urandom` if not.

Unfortunately, I do not have a way to implement and test a Mac and Windows
version, so those are unimplemented as of now. Note that `kRandomShuffleChunks`
is only used on Linux for now.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: zturner, rnk, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34412

llvm-svn: 305922
2017-06-21 15:56:03 +00:00
Simon Dardis f62399945c [mips][compiler-rt] Fix build breakage.
Change some reinterpret_casts to c-style casts due to template instantiation
restrictions and build breakage due to missing paranthesises.

llvm-svn: 305899
2017-06-21 11:29:15 +00:00
Vitaly Buka ff7b8ea2a0 Revert "[compiler-rt] Don't reset non-default user handler if allow_user_segv_handler is true."
Summary:
On Android we still need to reset preinstalled handlers and allow use handlers later.

This reverts commit r304039.

Reviewers: eugenis

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D34434

llvm-svn: 305871
2017-06-21 01:10:23 +00:00
Alex Shlyapnikov ccab11b0e8 [Sanitizers] Move cached allocator_may_return_null flag to sanitizer_allocator
Summary:
Move cached allocator_may_return_null flag to sanitizer_allocator.cc and
provide API to consolidate and unify the behavior of all specific allocators.

Make all sanitizers using CombinedAllocator to follow
AllocatorReturnNullOrDieOnOOM() rules to behave the same way when OOM
happens.

When OOM happens, turn allocator_out_of_memory flag on regardless of
allocator_may_return_null flag value (it used to not to be set when
allocator_may_return_null == true).

release_to_os_interval_ms and rss_limit_exceeded will likely be moved to
sanitizer_allocator.cc too (later).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34310

llvm-svn: 305858
2017-06-20 21:23:02 +00:00
Sagar Thakur 6478d14a0d [scudo] Enabling MIPS support for Scudo
Adding MIPS 32-bit and 64-bit support for Scudo.

Reviewed by cryptoad, sdardis.
Differential: D31803

llvm-svn: 305682
2017-06-19 11:28:59 +00:00
Alex Shlyapnikov 5a308f24c3 [Sanitizer] Remove CombinedAllocator::Allocate's 'cleared' parameter
Summary:
CombinedAllocator::Allocate cleared parameter is not used anywhere and
seem to be obsolete.

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34289

llvm-svn: 305590
2017-06-16 21:00:03 +00:00
Alex Shlyapnikov 9092fe6f4b [Sanitizers] Secondary allocator respects allocator_may_return_null=1.
Summary:
Context: https://github.com/google/sanitizers/issues/740.

Making secondary allocator to respect allocator_may_return_null=1 flag
and return nullptr when "out of memory" happens.

More changes in primary allocator and operator new will follow.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34243

llvm-svn: 305569
2017-06-16 18:48:08 +00:00
Vitaly Buka 4a2cc96f1f [asan] Fix typo in doc string.
llvm-svn: 305436
2017-06-15 00:31:59 +00:00
Vitaly Buka 7901b47094 [asan] Return allow_user_segv_handler=0 to fix compatibility issues.
Summary:
After r303941 it was not possible to setup ASAN_OPTIONS to have the same
behavior for pre r303941 and post r303941 builds.
Pre r303941 Asan does not accept handle_sigbus=2.
Post r303941 Asan does not accept allow_user_segv_handler.

This fix ignores allow_user_segv_handler=1, but for allow_user_segv_handler=0
it will upgrade flags like handle_sigbus=1 to handle_sigbus=2. So user can set
ASAN_OPTIONS=allow_user_segv_handler=0 and have same behavior on old and new
clang builds (except range from r303941 to this revision).

In future users which need to prevent third party handlers should switch to
handle_sigbus=2 and remove allow_user_segv_handler as soon as suport of older
builds is not needed.

Related bugs:
  https://github.com/google/oss-fuzz/issues/675
  https://bugs.chromium.org/p/chromium/issues/detail?id=731130

Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34227

llvm-svn: 305433
2017-06-15 00:19:13 +00:00
Kostya Kortchinsky 77f30c9c31 [sanitizer] Reverting D34152
Summary:
This broke thread_local_quarantine_pthread_join.cc on some architectures, due
to the overhead of the stashed regions. Reverting while figuring out the best
way to deal with it.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34213

llvm-svn: 305404
2017-06-14 17:32:26 +00:00
Kostya Kortchinsky eca926ab3a [sanitizer] MmapAlignedOrDie changes to reduce fragmentation
Summary:
The reasoning behind this change is explained in D33454, which unfortunately
broke the Windows version (due to the platform not supporting partial unmapping
of a memory region).

This new approach changes `MmapAlignedOrDie` to allow for the specification of
a `padding_chunk`. If non-null, and the initial allocation is aligned, this
padding chunk will hold the address of the extra memory (of `alignment` bytes).
This allows `AllocateRegion` to get 2 regions if the memory is aligned
properly, and thus help reduce fragmentation (and saves on unmapping
operations). As with the initial D33454, we use a stash in the 32-bit Primary
to hold those extra regions and return them on the fast-path.

The Windows version of `MmapAlignedOrDie` will always return a 0
`padding_chunk` if one was requested.

Reviewers: alekseyshl, dvyukov, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34152

llvm-svn: 305391
2017-06-14 15:32:17 +00:00
Alex Shlyapnikov 8531fa3028 [ASan] Move rss_limit_is_exceeded_ flag to ASan.
Summary:
Move the OOM decision based on RSS limits out of generic allocator to
ASan allocator, where it makes more sense at the moment.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34180

llvm-svn: 305342
2017-06-13 23:57:24 +00:00
Evgeniy Stepanov cde90a66b5 [sanitize] Remove stack size limits from secondary threads.
If pthread_attr_getstack tell us the stack is 2G, why would we doubt that?

Differential Revision: https://reviews.llvm.org/D34169

llvm-svn: 305330
2017-06-13 22:29:15 +00:00
Evgeniy Stepanov 7922ac23f1 [sanitizer] Fix symbolizer build after r304864.
r304864 moved code into a new BinaryFormat library. Update the
symbolizer build script.

llvm-svn: 304930
2017-06-07 17:24:58 +00:00
Vitaly Buka d9bc851fb3 [tsan]: Fix GNU version of strerror_r interceptor
GNU version of strerror_r returns a result pointer that doesn't match the input
buffer. The result pointer is in fact a pointer to some internal storage.
TSAN was recording a write to this location, which was incorrect.

Fixed https://github.com/google/sanitizers/issues/696

llvm-svn: 304858
2017-06-07 01:53:38 +00:00
Renato Golin 77a12b3972 Revert "[sanitizer-coverage] test for -fsanitize-coverage=inline-8bit-counters"
Revert "Mark sancov test as unsupported on Darwin"
Revert "[LSan] Detect dynamic loader by its base address."

This reverts commit r304633.
This reverts commit r304673.
This reverts commit r304632.

Those commit have broken LOTS of ARM/AArch64 bots for two days.

llvm-svn: 304699
2017-06-05 07:36:02 +00:00
Dimitry Andric 6c00c9950c Adjust sanitizers for FreeBSD 64-bit inode update
Summary:
Very recently, FreeBSD 12 has been updated to use 64-bit inode numbers:
<https://svnweb.freebsd.org/changeset/base/318737>.  This entails many
user-visible changes, but for the sanitizers the modifications are
limited in scope:
* The `stat` and `lstat` syscalls were removed, and should be replaced
  with calls to `fstatat`.
* The `getdents` syscall was removed, and should be replaced with calls
  to `getdirentries`.
* The layout of `struct dirent` was changed to accomodate 64-bit inode
  numbers, and a new `d_off` field was added.
* The system header <sys/_types.h> now contains a macro `__INO64` to
  determine whether the system uses 64-bit inode numbers.

I tested these changes on both FreeBSD 12.0-CURRENT (after r318959,
which adds the `__INO64` macro), and FreeBSD 11.0-STABLE (which still
uses 32-bit inode numbers).

Reviewers: emaste, kcc, vitalybuka, kubamracek

Reviewed By: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33600

llvm-svn: 304658
2017-06-03 11:11:36 +00:00
Alex Shlyapnikov 2090504453 [LSan] Detect dynamic loader by its base address.
Summary:
Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D33859

llvm-svn: 304633
2017-06-03 01:43:44 +00:00
Kostya Serebryany ad272b0861 [asan] fix one more case where stack-use-after-return is not async-signal-safe (during thread startup). beef-up the test to give it a chance to catch regressions. Also relax the lint to make C++11 more usable.
llvm-svn: 304598
2017-06-02 21:32:04 +00:00
Kostya Serebryany 124c2ae4fa [sanitizer-coverage] nuke more stale code
llvm-svn: 304508
2017-06-02 01:17:04 +00:00
Kostya Serebryany 1800814b41 [sanitizer-coverage] nuke more stale code
llvm-svn: 304504
2017-06-02 00:52:35 +00:00
Kostya Serebryany c1a56baa3e [sanitizer-coverage] nuke more stale code
llvm-svn: 304503
2017-06-02 00:17:54 +00:00
Kostya Serebryany 70c64869b5 [sanitizer-coverage] nuke more stale code
llvm-svn: 304500
2017-06-01 23:56:49 +00:00
Pierre Gousseau 183d1368f3 [asan] Add strndup/__strndup interceptors.
Recommit of r302781 with Vitaly Buka's fix for non zero terminated strings.

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 304399
2017-06-01 09:37:22 +00:00
Kostya Serebryany 9c50876120 [sanitizer-coverage] remove stale code (old coverage); compiler-rt part
llvm-svn: 304318
2017-05-31 18:26:32 +00:00
Alex Shlyapnikov 0145dee366 [sanitizer] Add "isapla" to symbolizer's global symbols whitelist.
Summary: D33637 introduced isalpha, whitelist need to reflect that.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D33687

llvm-svn: 304234
2017-05-30 19:52:34 +00:00
Vitaly Buka e8b09195fa [compiler-rt] Don't reset non-default user handler if allow_user_segv_handler is true.
Reviewers: eugenis, kcc

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32457

llvm-svn: 304039
2017-05-26 21:51:26 +00:00
Vitaly Buka a05da1fca9 [compiler-rt] Replace allow_user_segv_handler=0 with kHandleSignalExclusive
Summary:
allow_user_segv_handler had confusing name did not allow to control behavior for
signals separately.

Reviewers: eugenis, alekseyshl, kcc

Subscribers: llvm-commits, dberris, kubamracek

Differential Revision: https://reviews.llvm.org/D33371

llvm-svn: 303941
2017-05-25 23:42:33 +00:00
Vitaly Buka 40d54d408b [compiler-rt] Make print_module_map description consistent with the rest.
Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33160

llvm-svn: 303892
2017-05-25 18:07:48 +00:00
Francis Ricci 86e070f7e9 Implement tls scanning for darwin LSan
Summary:
This required for any users who call exit() after creating
thread-specific data, as tls destructors are only called when
pthread_exit() or pthread_cancel() are used. This should also
match tls behavior on linux.

Getting the base address of the tls section is straightforward,
as it's stored as a section offset in %gs. The size is a bit trickier
to work out, as there doesn't appear to be any official documentation
or source code referring to it. The size used in this patch was determined
by taking the difference between the base address and the address of the
subsequent memory region returned by vm_region_recurse_64, which was
1024 * sizeof(uptr) on all threads except the main thread, where it was
larger. Since the section must be the same size on all of the threads,
1024 * sizeof(uptr) seemed to be a reasonable size to use, barring
a more programtic way to get the size.

1024 seems like a reasonable number, given that PTHREAD_KEYS_MAX
is 512 on darwin, so pthread keys will fit inside the region while
leaving space for other tls data. A larger size would overflow the
memory region returned by vm_region_recurse_64, and a smaller size
wouldn't leave room for all the pthread keys. In addition, the
stress test added here passes, which means that we are scanning at
least the full set of possible pthread keys, and probably
the full tls section.

Reviewers: alekseyshl, kubamracek

Subscribers: krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D33215

llvm-svn: 303887
2017-05-25 17:41:13 +00:00
Kostya Kortchinsky 5d0ecbc8d9 [sanitizer] Revert rL303879 as it breaks Windows
Summary:
Apparently Windows's `UnmapOrDie` doesn't support partial unmapping. Which
makes the new region allocation technique not Windows compliant.

Reviewers: alekseyshl, dvyukov

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33554

llvm-svn: 303883
2017-05-25 16:54:44 +00:00
Kostya Kortchinsky 0dd40cf28d [sanitizer] Change the 32-bit Primary AllocateRegion to reduce fragmentation
Summary:
Currently, AllocateRegion has a tendency to fragment memory: it allocates
`2*kRegionSize`, and if the memory is aligned, will unmap `kRegionSize` bytes,
thus creating a hole, which can't itself be reused for another region. This
is exacerbated by the fact that if 2 regions get allocated one after another
without any `mmap` in between, the second will be aligned due to mappings 
generally being contiguous.

An idea, suggested by @alekseyshl, to prevent such a behavior is to have a
stash of regions: if the `2*kRegionSize` allocation is properly aligned, split
it in two, and stash the second part to be returned next time a region is
requested.

At this point, I thought about a couple of ways to implement this:
 - either an `IntrusiveList` of regions candidates, storing `next` at the
   begining of the region;
 - a small array of regions candidates existing in the Primary.

While the second option is more constrained in terms of size, it offers several
advantages:
 - security wise, a pointer in a region candidate could be overflowed into, and
   abused when popping an element;
 - we do not dirty the first page of the region by storing something in it;
 - unless several threads request regions simultaneously from different size
   classes, the stash rarely goes above 1 entry.

I am not certain about the Windows impact of this change, as `sanitizer_win.cc`
has its own version of MmapAlignedOrDie, maybe someone could chime in on this.

MmapAlignedOrDie is effectively unused after this change and could be removed
at a later point. I didn't notice any sizeable performance gain, even though we
are saving a few `mmap`/`munmap` syscalls.

Reviewers: alekseyshl, kcc, dvyukov

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33454

llvm-svn: 303879
2017-05-25 16:19:57 +00:00
Alex Shlyapnikov 3ea9499e75 [sanitizer] Pair atomic acquire with release in BlockingMutex::Unlock
Summary:
Dmitry, seeking your expertise. I believe, the proper way to implement
Lock/Unlock here would be to use acquire/release semantics. Am I missing
something?

Reviewers: dvyukov

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33521

llvm-svn: 303869
2017-05-25 15:07:07 +00:00
Vitaly Buka 4974f108ac [compiler-rt] Change default of allow_user_segv_handler to true
Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32443

llvm-svn: 303842
2017-05-25 06:29:30 +00:00
Vitaly Buka 62882c93c8 Revert "[compiler-rt] Change default of allow_user_segv_handler to true"
Breaks sanitizer-x86_64-linux-fuzzer bot.

This reverts commit r303729.

llvm-svn: 303795
2017-05-24 19:09:24 +00:00
Ulrich Weigand 66f2260837 [sanitizer] [SystemZ] Update CVE-2016-2143 check for Ubuntu 16.04
The Ubuntu 16.04 kernel contains a backport of the CVE check
starting with version 4.4.0-13.  Update FixedCVE_2016_2143.

llvm-svn: 303757
2017-05-24 15:06:33 +00:00
Vitaly Buka 354439a5a1 [compiler-rt] Change default of allow_user_segv_handler to true
Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32443

llvm-svn: 303729
2017-05-24 07:21:39 +00:00
Vitaly Buka 4eff87c36d Revert "[compiler-rt] Change default of allow_user_segv_handler to true"
Failed libFuzzer tests on Windows.

This reverts commit r303476.

llvm-svn: 303481
2017-05-20 02:09:25 +00:00
Vitaly Buka 99534e8040 [compiler-rt] Change default of allow_user_segv_handler to true
Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D32443

llvm-svn: 303476
2017-05-20 01:04:59 +00:00
Vitaly Buka 8018780d39 [compiler-rt] Switch handle_<signal> flags from bool to enum.
Summary: We are going to make it tri-state and remove allow_user_segv_handler.

Reviewers: eugenis, alekseys, kcc

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D33159

llvm-svn: 303464
2017-05-19 22:37:16 +00:00
Vitaly Buka 5934882576 [compiler-rt] Add negative test for boolean flags.
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D33344

llvm-svn: 303462
2017-05-19 22:37:13 +00:00
Francis Ricci dd592ff467 Use write instead of read permissions to check for global sections on mac
Summary:
The LINKEDIT section is very large and is read-only. Scanning this
section caused LSan on darwin to be very slow. When only writable sections
are scanned for global pointers, performance improved by a factor of about 25x.

Reviewers: alekseyshl, kubamracek

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D33322

llvm-svn: 303422
2017-05-19 13:34:02 +00:00
Vitaly Buka 1cfc81f1de [compiler-rt] Replace ifs with switch statement in IsHandledDeadlySignal
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D33158

llvm-svn: 303392
2017-05-18 23:13:22 +00:00
Simon Dardis 657899bad1 [compiler-rt][cmake] Build unit tests conditionally with _FILE_OFFSET_BITS=64 and _LARGEFILE_SOURCE
The sanitizer library unit tests for libc can get a different definition
of 'struct stat' to what the sanitizer library is built with for certain
targets.

For MIPS the size element of 'struct stat' is after a macro guarded
explicit padding element.

This patch resolves any possible inconsistency by adding the same
_FILE_OFFSET_BITS=64 and _LARGE_SOURCE with the same
conditions as the sanitizer library to the build flags for the unit tests.

This resolves a recurring build failure on the MIPS buildbots due to
'struct stat' defintion differences.

Reviewers: slthakur

Differential Revision: https://reviews.llvm.org/D33131

llvm-svn: 303350
2017-05-18 13:19:35 +00:00
Daniel Jasper f97310fb7a Revert r302781 and subsequent attempts to disable part of it.
The Msan unit tests are still broken and by this point, I think we
should start over.

llvm-svn: 303339
2017-05-18 09:31:37 +00:00
Kostya Serebryany 9aa8ef8504 temporary disable stndup interceptor, due to r302781 being buggy
llvm-svn: 303324
2017-05-18 03:00:07 +00:00
Francis Ricci 3b57da0c73 Revert "Implement tls scanning for darwin LSan"
This reverts r303262, due to TSan buildbot breakages.

llvm-svn: 303266
2017-05-17 15:25:41 +00:00
Francis Ricci eab89eb850 Implement tls scanning for darwin LSan
Summary:
This required for any users who call exit() after creating
thread-specific data, as tls destructors are only called when
pthread_exit() or pthread_cancel() are used. This should also
match tls behavior on linux.

Getting the base address of the tls section is straightforward,
as it's stored as a section offset in %gs. The size is a bit trickier
to work out, as there doesn't appear to be any official documentation
or source code referring to it. The size used in this patch was determined
by taking the difference between the base address and the address of the
subsequent memory region returned by vm_region_recurse_64, which was
1024 * sizeof(uptr) on all threads except the main thread, where it was
larger. Since the section must be the same size on all of the threads,
1024 * sizeof(uptr) seemed to be a reasonable size to use, barring
a more programtic way to get the size.

1024 seems like a reasonable number, given that PTHREAD_KEYS_MAX
is 512 on darwin, so pthread keys will fit inside the region while
leaving space for other tls data. A larger size would overflow the
memory region returned by vm_region_recurse_64, and a smaller size
wouldn't leave room for all the pthread keys. In addition, the
stress test added here passes, which means that we are scanning at
least the full set of possible pthread keys, and probably
the full tls section.

Reviewers: alekseyshl, kubamracek

Subscribers: krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D33215

llvm-svn: 303262
2017-05-17 14:35:17 +00:00
Kostya Serebryany 130fca1882 [asan] make asan under sandboxes more robust
llvm-svn: 303132
2017-05-15 23:37:54 +00:00
Kostya Kortchinsky dc646a0889 [sanitizer] Change SizeClassAllocator32 to accept just one template
Summary:
With rL279771, SizeClassAllocator64 was changed to accept only one template
instead of 5, for the following reasons: "First, this will make the mangled
names shorter. Second, this will make adding more parameters simpler". This
patch mirrors that work for SizeClassAllocator32.

This is in preparation for introducing the randomization of chunks in the
32-bit SizeClassAllocator in a later patch.

Reviewers: kcc, alekseyshl, dvyukov

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33141

llvm-svn: 303071
2017-05-15 14:47:19 +00:00
Francis Ricci 61ed9345c9 Add dyld to sanitizer procmaps on darwin
Summary:
Sanitizer procmaps uses dyld apis to iterate over the list of images
in the process. This is much more performan than manually recursing
over all of the memory regions in the process, however, dyld does
not report itself in the list of images. In order to prevent reporting
leaks from dyld globals and to symbolize dyld functions in stack traces,
this patch special-cases dyld and ensures that it is added to the
list of modules.

This is accomplished by recursing through the memory map of the process
until a dyld Mach header is found. While this recursion is expensive,
it is run before the full set of images has been loaded in the process,
so only a few calls are required. The result is cached so that it never
needs to be searched for when the full process memory map exists, as this
would be incredibly slow, on the order of minutes for leak sanitizer with
only 25 or so libraries loaded.

Reviewers: alekseyshl, kubamracek

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32968

llvm-svn: 302899
2017-05-12 14:10:53 +00:00
Francis Ricci 5d4ad5c7d2 Account for stack redzone when computing sp on darwin
thread_get_register_pointer_values handles the redzone computation
automatically, but is marked as an unavailable API function. This
patch replicates its logic accounting for the stack redzone on
x86_64.

Should fix flakiness in the use_stack_threaded test for lsan on darwin.

llvm-svn: 302898
2017-05-12 14:10:51 +00:00
Pierre Gousseau 0550581070 [asan] Recommit of r301904: Add strndup/__strndup interceptors
Fix undeclared __interceptor_malloc in esan_interceptors.cc
Fix undeclared strnlen on OSX

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 302781
2017-05-11 08:53:24 +00:00
Francis Ricci 4cd7a15cea Revert "Add dyld to sanitizer procmaps on darwin"
This breaks several tests because we don't always have
access to __cxa_guard functions

This reverts commit 45eb470c3e9e8f6993a204e247c33d4092237efe.

llvm-svn: 302693
2017-05-10 16:33:46 +00:00
Francis Ricci 48eab42101 Revert "Disable static caching of dyld header on Go sanitizers"
This is a problem on more than just the go sanitizers, so it's
not a good enough fix for the issue.

llvm-svn: 302692
2017-05-10 16:33:43 +00:00
Francis Ricci 15dc8c93bc Disable static caching of dyld header on Go sanitizers
This causes buildbot failures due to undefined __cxa_guard_acquire

llvm-svn: 302681
2017-05-10 15:40:29 +00:00
Francis Ricci 1cdcbcdb92 Add dyld to sanitizer procmaps on darwin
Summary:
Sanitizer procmaps uses dyld apis to iterate over the list of images
in the process. This is much more performan than manually recursing
over all of the memory regions in the process, however, dyld does
not report itself in the list of images. In order to prevent reporting
leaks from dyld globals and to symbolize dyld functions in stack traces,
this patch special-cases dyld and ensures that it is added to the
list of modules.

This is accomplished by recursing through the memory map of the process
until a dyld Mach header is found. While this recursion is expensive,
it is run before the full set of images has been loaded in the process,
so only a few calls are required. The result is cached so that it never
needs to be searched for when the full process memory map exists, as this
would be incredibly slow, on the order of minutes for leak sanitizer with
only 25 or so libraries loaded.

Reviewers: alekseyshl, kubamracek

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32968

llvm-svn: 302673
2017-05-10 14:38:04 +00:00
Kostya Serebryany 8c34243a13 [asan] print the 'unexpected format specifier in printf interceptor' warning just once (came up in https://github.com/google/oss-fuzz/pull/562). Not touching a similar scanf warning -- for some reason it does not fire for me.
llvm-svn: 302064
2017-05-03 18:38:34 +00:00
Maxim Ostapenko 726701b0ed [sanitizer] Intercept mcheck and mprobe on Linux
This patch addresses https://github.com/google/sanitizers/issues/804.
Users can use mcheck and mprobe functions to verify heap state so we should intercept them to avoid breakage of valid code.

Differential Revision: https://reviews.llvm.org/D32589

llvm-svn: 302001
2017-05-03 07:09:10 +00:00
Kostya Kortchinsky 7fc481e561 [compiler-rt] move tsan's Android __get_tls() to sanitizer_common
Summary:
TSan's Android `__get_tls()` and `TLS_SLOT_TSAN` can be used by other sanitizers as well (see D32649), this change moves them to sanitizer_common.
I picked sanitizer_linux.h as their new home.
In the process, add the 32-bit versions for ARM, i386 & MIPS.

Can the address of `__get_tls()[TLS_SLOT_TSAN]` change in between the calls?
I am not sure if there is a need to repeat the construct as opposed to using a variable. So I left things as they were.

Testing on my side was restricted to a successful cross-compilation.

Reviewers: dvyukov, kubamracek

Reviewed By: dvyukov

Subscribers: aemerson, rengolin, srhines, dberris, arichardson, llvm-commits

Differential Revision: https://reviews.llvm.org/D32705

llvm-svn: 301926
2017-05-02 15:13:36 +00:00
Pierre Gousseau 1c5550671d Revert r301904 causing tsan test failure in x86_64-linux-autoconf
llvm-svn: 301909
2017-05-02 10:22:05 +00:00
Pierre Gousseau b7101479a8 [asan] Add strndup/__strndup interceptors if targeting linux.
Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 301904
2017-05-02 09:01:02 +00:00
Kostya Serebryany 5508ffaef2 [sanitizer-coverage] add a deprecation note for the old sanitizer-coverage; remove a TODO printf
llvm-svn: 301889
2017-05-02 00:44:24 +00:00