Commit Graph

416 Commits

Author SHA1 Message Date
Martin Storsjo 05394af3ff [sanitizers] Don't use Windows Trace Logging on MinGW
mingw-w64 currently is lacking the headers for this feature.

Make the include lowercase at the same time. We consistently
use lowercase for windows header includes, as windows itself is
case insensitive, the SDK headers (in general, not necessarily
considering this particular header) aren't consistent among themselves
about what the proper canonical capitalization for headers are,
and MinGW uses all lowercase names for the headers (as it is often
used on case sensitive filesystems).

In case mingw-w64 later gets this header, we can revert this
(but keep the include lowercased).

Differential Revision: https://reviews.llvm.org/D58765

llvm-svn: 355236
2019-03-01 22:30:14 +00:00
Matthew G McGovern 8edffdb4f0 [compiler-rt] Windows Trace Logging for error reports.
Adds option for collecting sanitixer dumps via trace logging.
    - Set log_to_syslog=1 to enable this output.
    - Consult https://aka.ms/windowstracelogging for details on use.

llvm-svn: 355045
2019-02-27 23:43:50 +00:00
Evgeniy Stepanov 443c034391 [sanitizer] Decorate /proc/self/maps better.
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.

Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.

Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.

The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D57720

llvm-svn: 353255
2019-02-06 01:14:50 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Evgeniy Stepanov 0df20e893f [sanitizer] Move android's GetPageSize to a header (NFC)
No need to pay function call overhead for a function that returns a
constant.

llvm-svn: 350983
2019-01-12 00:09:24 +00:00
Michal Gorny a939b40eae [xray] Detect MPROTECT and error out when it's enabled (on NetBSD)
Add a CheckMPROTECT() routine to detect when pax MPROTECT is enabled
on NetBSD, and error xray out when it is.  The solution is adapted
from existing CheckASLR().

Differential Revision: https://reviews.llvm.org/D56049

llvm-svn: 350030
2018-12-23 15:09:28 +00:00
Kuba Mracek c4b8eb53c4 [sanitizer] Use "fast mmap" kernel flag for shadow memory on macOS 10.13.4+
This speeds up process startup and teardown and also reduces lock contention when running multiple ASanified/TSanified processes simultaneously. Should greatly improve lit testing time.

Differential Revision: https://reviews.llvm.org/D48445

llvm-svn: 346262
2018-11-06 19:55:19 +00:00
Vitaly Buka ecb2eb46bc [sanitizer] Add char **GetEnviron() on all other platforms
Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D54165

llvm-svn: 346257
2018-11-06 19:23:12 +00:00
Vlad Tsyrklevich 05ee22e31d Reland "SafeStack: Delay thread stack clean-up""
This relands commit r339405 (reverted in commit r339408.) The original
revert was due to tests failing on a darwin buildbot; however, after
looking at the affected code more I realized that the Darwin safestack
support has always been broken and disabled it in r339719. This relands
the original commit.

llvm-svn: 339723
2018-08-14 20:28:58 +00:00
Vlad Tsyrklevich bd85115c6e Revert "SafeStack: Delay thread stack clean-up"
This reverts commit r339405, it's failing on Darwin buildbots because
it doesn't seem to have a tgkill/thr_kill2 interface. It has a
__pthread_kill() syscall, but that relies on having a handle to the
thread's port which is not equivalent to it's tid.

llvm-svn: 339408
2018-08-10 00:36:04 +00:00
Vlad Tsyrklevich a24ecc337f SafeStack: Delay thread stack clean-up
Summary:
glibc can call SafeStack instrumented code even after the last pthread
data destructor has run. Delay cleaning-up unsafe stacks for threads
until the thread is dead by having future threads clean-up prior threads
stacks.

Reviewers: pcc, eugenis

Reviewed By: eugenis

Subscribers: cryptoad, eugenis, kubamracek, delcypher, llvm-commits, #sanitizers, kcc

Differential Revision: https://reviews.llvm.org/D50406

llvm-svn: 339405
2018-08-09 22:56:41 +00:00
Dmitry Vyukov f52726aae9 sanitizers: consistently check result of MmapFixedNoReserve
MmapFixedNoReserve does not terminate process on failure.
Failure to check its result and die will always lead to harder
to debug crashes later in execution. This was observed in Go
processes due to some address space conflicts.

Consistently check result of MmapFixedNoReserve.
While we are here also add warn_unused_result attribute
to prevent such bugs in future and change return type to bool
as that's what all callers want.

Reviewed in https://reviews.llvm.org/D49367

llvm-svn: 337531
2018-07-20 08:33:41 +00:00
Alex Shlyapnikov 4bf9b5612a [TSan] Fix madvise(MADV_NOHUGEPAGE) for meta shadow memory
Summary:
Move madvise(MADV_NOHUGEPAGE) for the meta shadow memory after the meta
shadow memory is mapped (currently it silently fails with ENOMEM).

Add a diagnostic message to detect similar problems in the future.

Reviewers: dvyukov

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D48097

llvm-svn: 334624
2018-06-13 17:18:41 +00:00
Vitaly Buka 4ef9117b49 [sanitizer] Cleanup ReadFileToVector and ReadFileToBuffer
Summary:
Added unit-test.
Fixed behavior of max_len argument.
Call read syscall with all available buffer, not just a page.

Reviewers: eugenis

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D46618

llvm-svn: 334130
2018-06-06 20:53:43 +00:00
Kamil Rytarowski 7d260775f3 Introduce CheckASLR() in sanitizers
Summary:
At least the ASan, MSan, TSan sanitizers require disabled ASLR on a NetBSD.

Introduce a generic CheckASLR() routine, that implements a check for the
current process. This flag depends on the global or per-process settings.

There is no simple way to disable ASLR in the build process from the
level of a sanitizer or during the runtime execution.

With ASLR enabled sanitizers that operate over the process virtual address
space can misbehave usually breaking with cryptic messages.

This check is dummy for !NetBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: cryptoad, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D47442

llvm-svn: 333985
2018-06-05 07:29:23 +00:00
Vitaly Buka c779388c7b [sanitizer] Use all available rounded up capacity
Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D46617

llvm-svn: 331940
2018-05-09 23:31:05 +00:00
Vitaly Buka d3e55bf7fc [sanitizer] Cleanup sorting functions
llvm-svn: 331915
2018-05-09 20:42:11 +00:00
Vitaly Buka e21fbe5af2 [sanitizer] Read file to InternalMmapVectorNoCtor
llvm-svn: 331791
2018-05-08 18:35:11 +00:00
Vitaly Buka 7381f26b45 [sanitizer] Fix InternalMmapVectorNoCtor reserve and resize
Remap on reserve of more than the current size.
Don't remap on downsize.

llvm-svn: 331784
2018-05-08 17:59:44 +00:00
Vitaly Buka 4e76d7fcb6 [sanitizer] Add InternalMmapVector::swap
llvm-svn: 331736
2018-05-08 04:57:08 +00:00
Vitaly Buka fac095caba [sanitizer] Add operator== and operator!= for InternalMmapVectorNoCtor
llvm-svn: 331619
2018-05-07 06:14:12 +00:00
Vitaly Buka 2a20955169 [sanitizer] Replace InternalScopedBuffer with InternalMmapVector
llvm-svn: 331618
2018-05-07 05:56:36 +00:00
Vitaly Buka 44f55509d7 [sanitizer] Remove reserving constructor from InternalMmapVector
llvm-svn: 331617
2018-05-07 05:56:24 +00:00
Vitaly Buka db4241a046 [sanitizer] Implement InternalScopedBuffer with InternalMmapVector
llvm-svn: 331616
2018-05-07 05:56:12 +00:00
Vitaly Buka 20f49662f6 [sanitizer] Make InternalScopedBuffer::size() behavior similar to vector.
llvm-svn: 331612
2018-05-07 01:08:13 +00:00
Kostya Kortchinsky efe5afbc3d [sanitizer] More dead code removal
Summary:
The following functions are only used in tests: `SetEnv`,
`SanitizerSetThreadName`, `SanitizerGetThreadName`. I don't think they are
going to be used in the future, and I propose to get rid of them, and associated
tests and include.

Reviewers: alekseyshl, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: dvyukov, vitalybuka, kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D45838

llvm-svn: 330724
2018-04-24 14:58:10 +00:00
Kostya Kortchinsky 2c5f944015 [sanitizer] Remove empty Symbolizer PrepareForSandboxing
Summary:
`Symbolizer::PrepareForSandboxing` is empty for all platforms and apparently
has been for a while (D10213). Remove it, and shuffle things around so that the
platform specific code is now in `PlatformPrepareForSandboxing`.

This allows to have one less symbolizer dependency in a common file, which
helps for the upcoming split.

Also remove `SymbolizerPrepareForSandboxing` in tsan_go which appears to not
be used anywhere.

Reviewers: alekseyshl, eugenis, dvyukov, mcgrathr

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44953

llvm-svn: 329094
2018-04-03 18:07:22 +00:00
Vitaly Buka 8616363017 Mmap interceptor providing mprotect support
Summary:
- Intercepting mprotect calls.
- Fixing forgotten flag check.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Subscribers: delcypher, srhines, kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44777

llvm-svn: 328415
2018-03-24 07:45:24 +00:00
Vitaly Buka bb096c27bf Revert "Mmap interceptor providing mprotect support"
Breaks Darwin.

This reverts commit r328369.

llvm-svn: 328375
2018-03-23 21:44:59 +00:00
Vitaly Buka e4d34c0d56 Mmap interceptor providing mprotect support
Summary:
- Intercepting mprotect calls.
- Fixing forgotten flag check.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D44777

llvm-svn: 328369
2018-03-23 20:59:51 +00:00
Vitaly Buka 8980449bb6 Mmap interceptor new option, Write Exec runtime detector
Summary:
Following-up the refactoring of mmap interceptors, adding a new common
option to detect PROT_WRITE|PROT_EXEC pages request.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: krytarowski, #sanitizers

Differential Revision: https://reviews.llvm.org/D44194

llvm-svn: 328151
2018-03-21 21:25:07 +00:00
Dmitry Vyukov f5675e7e2a tsan: support inlined frames in external symbolization
New API passes a callback function to the external symbolizer,
allowing it to add multiple frames to the traceback. Note that
the old interface API will be still supported until the clients
migrate to the new one.

Author: asmundak (Alexander Smundak)
Reviewed in: https://reviews.llvm.org/D44714

llvm-svn: 328079
2018-03-21 08:44:14 +00:00
Vitaly Buka ec6327bab8 Revert "Mmap interceptor new option, Write Exec runtime detector"
Breaks Android bot.

This reverts commit r327747.

llvm-svn: 327762
2018-03-17 00:31:41 +00:00
Vitaly Buka d2aca7bf89 Mmap interceptor new option, Write Exec runtime detector
Summary: Following-up the refactoring of mmap interceptors, adding a new common option to detect PROT_WRITE|PROT_EXEC pages request.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: krytarowski, #sanitizers

Differential Revision: https://reviews.llvm.org/D44194

llvm-svn: 327747
2018-03-16 21:25:09 +00:00
Jonas Devlieghere bf0ed8e9b8 Revert "Mmap interceptor new option, Write Exec runtime detector"
This reverts r327696 because it is failing on GreenDragon.

  http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/43605/
  http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/10957/

llvm-svn: 327719
2018-03-16 15:34:09 +00:00
Vitaly Buka 21f1b69862 Mmap interceptor new option, Write Exec runtime detector
Summary: Following-up the refactoring of mmap interceptors, adding a new common option to detect PROT_WRITE|PROT_EXEC pages request.

Patch by David CARLIER

Reviewers: vitalybuka, vsk

Reviewed By: vitalybuka

Subscribers: krytarowski, #sanitizers

Differential Revision: https://reviews.llvm.org/D44194

llvm-svn: 327696
2018-03-16 08:22:18 +00:00
Kostya Kortchinsky 85ecac5f3f [sanitizer] Align & pad the allocator structures to the cacheline size v2
Summary:
This is a new version of D44261, which broke some builds with older gcc, as
they can't align on a constexpr, but rather require an integer (see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=56859) among others.

We introduce `SANITIZER_CACHE_LINE_SIZE` in `sanitizer_platform.h` to be
used in `ALIGNED` attributes instead of using directly `kCacheLineSize`.

Reviewers: alekseyshl, thakis

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44326

llvm-svn: 327297
2018-03-12 17:18:26 +00:00
Kostya Kortchinsky 112d7a435b [sanitizer] Revert rCRT327145
Summary:
It breaks the Chromium toolchain due to:
```
lib/sanitizer_common/sanitizer_allocator_primary32.h:269:34: error: requested alignment is not an integer constant
   struct ALIGNED(kCacheLineSize) SizeClassInfo {
```

Reviewers: alekseyshl, thakis

Reviewed By: thakis

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44320

llvm-svn: 327167
2018-03-09 20:17:12 +00:00
Kostya Kortchinsky 69df838b52 [sanitizer] Align & pad the allocator structures to the cacheline size
Summary:
Both `SizeClassInfo` structures for the 32-bit primary & `RegionInfo`
structures for the 64-bit primary can be used by different threads, and as such
they should be aligned & padded to the cacheline size to avoid false sharing.
The former was padded but the array was not aligned, the latter was not padded
but we lucked up as the size of the structure was 192 bytes, and aligned by
the properties of `mmap`.

I plan on adding a couple of fields to the `RegionInfo`, and some highly
threaded tests pointed out that without proper padding & alignment, performance
was getting a hit - and it is going away with proper padding.

This patch makes sure that we are properly padded & aligned for both. I used
a template to avoid padding if the size is already a multiple of the cacheline
size. There might be a better way to do this, I am open to suggestions.

Reviewers: alekseyshl, dvyukov

Reviewed By: alekseyshl

Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D44261

llvm-svn: 327145
2018-03-09 16:18:38 +00:00
Kuba Mracek 061f3589cc [asan] Be more careful and verbose when allocating dynamic shadow memory
FindAvailableMemoryRange can currently overwrite existing memory (by restricting the VM below addresses that are already used). This patch adds a check to make sure we don't restrict the VM space too much. We are also now more explicit about why the lookup failed and print out verbose values.

Differential Revision: https://reviews.llvm.org/D43318

llvm-svn: 326106
2018-02-26 18:33:21 +00:00
Kostya Kortchinsky f50246da65 [sanitizer] Introduce a vDSO aware timing function
Summary:
See D40657 & D40679 for previous versions of this patch & description.

A couple of things were fixed here to have it not break some bots.
Weak symbols can't be used with `SANITIZER_GO` so the previous version was
breakin TsanGo. I set up some additional local tests and those pass now.

I changed the workaround for the glibc vDSO issue: `__progname` is initialized
after the vDSO and is actually public and of known type, unlike
`__vdso_clock_gettime`. This works better, and with all compilers.

The rest is the same.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41121

llvm-svn: 320594
2017-12-13 16:23:54 +00:00
Kostya Kortchinsky ab5f6aaa75 [sanitizer] Revert rL320409
Summary: D40679 broke a couple of builds, reverting while investigating.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41088

llvm-svn: 320417
2017-12-11 21:03:12 +00:00
Kostya Kortchinsky d276d72441 [sanitizer] Introduce a vDSO aware time function, and use it in the allocator [redo]
Summary:
Redo of D40657, which had the initial discussion. The initial code had to move
into a libcdep file, and things had to be shuffled accordingly.

`NanoTime` is a time sink when checking whether or not to release memory to
the OS. While reducing the amount of calls to said function is in the works,
another solution that was found to be beneficial was to use a timing function
that can leverage the vDSO.

We hit a couple of snags along the way, like the fact that the glibc crashes
when clock_gettime is called from a preinit_array, or the fact that
`__vdso_clock_gettime` is mangled (for security purposes) and can't be used
directly, and also that clock_gettime can be intercepted.

The proposed solution takes care of all this as far as I can tell, and
significantly improve performances and some Scudo load tests with memory
reclaiming enabled.

@mcgrathr: please feel free to follow up on
https://reviews.llvm.org/D40657#940857 here. I posted a reply at
https://reviews.llvm.org/D40657#940974.

Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek

Reviewed By: alekseyshl, krytarowski

Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D40679

llvm-svn: 320409
2017-12-11 19:23:12 +00:00
Kostya Kortchinsky fbff7fcdf4 [fuchsia] Update Fuchsia with a new mmap implementation.
Summary:
    Now that the sanitizer_common interface for MmapNoAccess / MmapFixed
    have been refactored to allow a more OO-esque access pattern, update the
    Fuchsia mmap implementation to take advantage of this.
    
    Previously MmapNoAccess / MmapFixed relied on a global allocator_vmar,
    since the sanitizer_allocator only called MmapNoAccess once.  Now, we
    create a new VMAR per ReservedAddressRange object.
    
    This allows the sanitizer allocator to work in tandem with the Scudo
    secondary allocator.
    
    This is part 4 of a 4 part changeset:
    * part 1 https://reviews.llvm.org/D38593
    * part 2 https://reviews.llvm.org/D38592
    * part 3 https://reviews.llvm.org/D38593

Reviewers: mcgrathr, cryptoad

Reviewed By: cryptoad

Subscribers: alekseyshl, mcgrathr, kubamracek, mehdi_amini

Differential Revision: https://reviews.llvm.org/D38595

llvm-svn: 319083
2017-11-27 19:53:53 +00:00
Kostya Kortchinsky 5a3fdbd829 [scudo] Make getNumberOfCPUs Fuchsia compliant v2
Summary:
This change allows Fuchsia to boot properly using the Scudo allocator.

A first version of this commit was reverted by rL317834 because it broke Android
builds for toolchains generated with older NDKs. This commit introduces a
fall back to solve that issue.

Reviewers: cryptoad, krytarowski, rnk, alekseyshl

Reviewed By: cryptoad, krytarowski, alekseyshl

Subscribers: llvm-commits, srhines, kubamracek, krytarowski

Differential Revision: https://reviews.llvm.org/D40121

llvm-svn: 318802
2017-11-21 21:14:00 +00:00
Walter Lee 6c529f7e96 [sanitizers] Add init function to set alignment of low level allocator
ASan requires that the min alignment be at least the shadow
granularity, so add an init function to do that.

Differential Revision: https://reviews.llvm.org/D39473

llvm-svn: 318717
2017-11-21 01:01:32 +00:00
Evgeniy Stepanov 8e7018d92f [asan] Use dynamic shadow on 32-bit Android, try 2.
Summary:
This change reverts r318575 and changes FindDynamicShadowStart() to
keep the memory range it found mapped PROT_NONE to make sure it is
not reused. We also skip MemoryRangeIsAvailable() check, because it
is (a) unnecessary, and (b) would fail anyway.

Reviewers: pcc, vitalybuka, kcc

Subscribers: srhines, kubamracek, mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D40203

llvm-svn: 318666
2017-11-20 17:41:57 +00:00
Evgeniy Stepanov 9d564cdcb0 Revert "[asan] Use dynamic shadow on 32-bit Android" and 3 more.
Revert the following commits:
  r318369 [asan] Fallback to non-ifunc dynamic shadow on android<22.
  r318235 [asan] Prevent rematerialization of &__asan_shadow.
  r317948 [sanitizer] Remove unnecessary attribute hidden.
  r317943 [asan] Use dynamic shadow on 32-bit Android.

MemoryRangeIsAvailable() reads /proc/$PID/maps into an mmap-ed buffer
that may overlap with the address range that we plan to use for the
dynamic shadow mapping. This is causing random startup crashes.

llvm-svn: 318575
2017-11-18 00:22:34 +00:00
Evgeniy Stepanov 989299c42b [asan] Use dynamic shadow on 32-bit Android.
Summary:
The following kernel change has moved ET_DYN base to 0x4000000 on arm32:
https://marc.info/?l=linux-kernel&m=149825162606848&w=2

Switch to dynamic shadow base to avoid such conflicts in the future.

Reserve shadow memory in an ifunc resolver, but don't use it in the instrumentation
until PR35221 is fixed. This will eventually let use save one load per function.

Reviewers: kcc

Subscribers: aemerson, srhines, kubamracek, kristof.beyls, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D39393

llvm-svn: 317943
2017-11-10 22:27:48 +00:00
Vitaly Buka f2827e133b [sanitizer] Remove references and hide CommonSanitizerReportMutex
llvm-svn: 317865
2017-11-10 02:07:11 +00:00
Kostya Kortchinsky 5604ad1c9b [sanitizer] Revert rL317822
Summary:
This reverts D39490.

For toolchains generated with older NDKs (<=r13b as far as we tested),
`cpu_set_t` doesn't exist in `sched.h`.
We have to figure out another way to get the number of CPUs without this.

Reviewers: rnk

Reviewed By: rnk

Subscribers: kubamracek, llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D39867

llvm-svn: 317834
2017-11-09 21:26:07 +00:00
Kostya Kortchinsky 6458216b28 [scudo] Make getNumberOfCPUs Fuchsia compliant
Summary: This change allows Fuchsia to boot properly using the Scudo allocator.

Reviewers: cryptoad, alekseyshl, krytarowski

Reviewed By: cryptoad, krytarowski

Subscribers: rnk, krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39490

llvm-svn: 317822
2017-11-09 19:18:55 +00:00
Reid Kleckner f7fdac4508 Revert "[scudo] Make getNumberOfCPUs Fuchsia compliant"
This reverts commit r317604.

Android doesn't have cpu_set_t.

llvm-svn: 317655
2017-11-08 01:33:15 +00:00
Evgeniy Stepanov 0379d3f844 (NFC) Rename GetMax{,User}VirtualAddress.
Future change will introduce GetMaxVirtualAddress that will not take
the kernel area into account.

llvm-svn: 317638
2017-11-07 23:51:22 +00:00
Kostya Kortchinsky 4e8ce0225f [scudo] Make getNumberOfCPUs Fuchsia compliant
Summary: This change allows Fuchsia to boot properly using the Scudo allocator.

Reviewers: cryptoad, alekseyshl, krytarowski

Reviewed By: cryptoad, krytarowski

Subscribers: krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39490

llvm-svn: 317604
2017-11-07 19:30:08 +00:00
Kostya Kortchinsky c6f0ade6c3 Update sanitizer_allocator to use new API.
Summary:
Update sanitizer_allocator to use new API.
    
Second patch in a series.  First patch https://reviews.llvm.org/D39072
    
Updates MmapNoAccess / MmapFixed call sites in the saniziter_allocator
to use the new Init/Map APIs instead.


Reviewers: alekseyshl, cryptoad, phosek, mcgrathr, dvyukov

Reviewed By: alekseyshl, cryptoad

Subscribers: dvyukov, mcgrathr, kubamracek

Differential Revision: https://reviews.llvm.org/D38592

llvm-svn: 317586
2017-11-07 16:19:24 +00:00
Kostya Kortchinsky 96da9fa4ca Introduce ReservedAddressRange to sanitizer_common.
Summary:
Fixed version of https://reviews.llvm.org/D38437 (fixes Win/Fuchsia failures).

Creating a new revision, since the old one was getting a bit old/crowded.

    In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
    VMAR, which means that MmapNoAccess can only be called once. This works
    for the sanitizer allocator but *not* for the Scudo allocator.
    
    Hence, this changeset introduces a new ReservedAddressRange object to
    serve as the new API for these calls. In this changeset, the object
    still calls into the old Mmap implementations.
    
    The next changeset two changesets will convert the sanitizer and scudo
    allocators to use the new APIs, respectively. (ReservedAddressRange will
    replace the SecondaryHeader in Scudo.)
    
    Finally, a last changeset will update the Fuchsia implementation.


Reviewers: alekseyshl, cryptoad, phosek

Reviewed By: alekseyshl, cryptoad

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D39072

llvm-svn: 316934
2017-10-30 17:56:24 +00:00
Hans Wennborg eb2218ceae Revert r315533 "Reland "[sanitizer] Introduce ReservedAddressRange to sanitizer_common""
The SanitizerCommon.ReservedAddressRangeUnmap test fails on Windows:

FAIL: SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap (34003 of 35554)
******************** TEST 'SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap' FAILED ********************
Note: Google Test filter = SanitizerCommon.ReservedAddressRangeUnmap
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN      ] SanitizerCommon.ReservedAddressRangeUnmap
==3780==ERROR: SanitizerTool failed to deallocate 0x1000 (4096) bytes at address 0x0000000c3000 (error code: 487)
==3780==Sanitizer CHECK failed: E:\b\build\slave\win_upload_clang\build\src\third_party\llvm\projects\compiler-rt\lib\sanitizer_common\sanitizer_win.cc:129 (("unable to unmap" && 0)) != (0) (0, 0)
********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 299.76s
********************
Failing Tests (1):
    SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap

> In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
> VMAR, which means that MmapNoAccess can only be called once. This works
> for the sanitizer allocator but *not* for the Scudo allocator.
>
> Hence, this changeset introduces a new ReservedAddressRange object to
> serve as the new API for these calls. In this changeset, the object
> still calls into the old Mmap implementations.
>
> The next changeset two changesets will convert the sanitizer and scudo
> allocators to use the new APIs, respectively. (ReservedAddressRange will
> replace the SecondaryHeader in Scudo.)
>
> Finally, a last changeset will update the Fuchsia implementation.
>
> Patch by Julia Hansbrough
>
> Differential Revision: https://reviews.llvm.org/D38437

llvm-svn: 315553
2017-10-12 03:23:31 +00:00
Petr Hosek 4f213ae3a7 Reland "[sanitizer] Introduce ReservedAddressRange to sanitizer_common"
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.

Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.

The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)

Finally, a last changeset will update the Fuchsia implementation.

Patch by Julia Hansbrough

Differential Revision: https://reviews.llvm.org/D38437

llvm-svn: 315533
2017-10-11 23:41:32 +00:00
Petr Hosek c530f497b8 Revert "[sanitizer] Introduce ReservedAddressRange to sanitizer_common"
This reverts commit r315493 which is failing to build on sanitizer-windows.

llvm-svn: 315494
2017-10-11 19:29:14 +00:00
Petr Hosek 2bd0123afc [sanitizer] Introduce ReservedAddressRange to sanitizer_common
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.

Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.

The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)

Finally, a last changeset will update the Fuchsia implementation.

Patch by Julia Hansbrough

Differential Revision: https://reviews.llvm.org/D38759

llvm-svn: 315493
2017-10-11 19:17:35 +00:00
Francis Ricci daf210f7b6 Add support for custom loaders to the sanitizer symbolizer
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.

Reviewers: kubamracek, rnk, vitalybuka, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37269

llvm-svn: 314713
2017-10-02 20:22:16 +00:00
Francis Ricci b7055bb813 Revert "Add support for custom loaders to the sanitizer symbolizer"
This reverts commit r314671, which hangs on the gcc sanitizer buildbot.

llvm-svn: 314684
2017-10-02 15:55:11 +00:00
Francis Ricci e2aa5b2ace Add support for custom loaders to the sanitizer symbolizer
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.

Reviewers: kubamracek, rnk, vitalybuka, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37269

llvm-svn: 314671
2017-10-02 14:30:58 +00:00
Francis Ricci 345187338e Move LoadedModule list to a NoCtor vector and initialize on demand.
Unreverting this patch because llvm-clang-lld-x86_64-debian-fast started
passing again before the revert hit. Must've been just a flake.

llvm-svn: 314556
2017-09-29 20:55:06 +00:00
Francis Ricci 74c93de659 Revert "Move LoadedModule list to a NoCtor vector and initialize on demand."
I think this may have introduced a failure on
llvm-clang-lld-x86_64-debian-fast

This reverts commit r314533

llvm-svn: 314552
2017-09-29 20:04:29 +00:00
Francis Ricci 113708b7cb Move LoadedModule list to a NoCtor vector and initialize on demand.
llvm-svn: 314533
2017-09-29 16:47:02 +00:00
Francis Ricci 052ec5f92a Revert "Add support for custom loaders to the sanitizer symbolizer"
This causes the gcc sanitizer buildbot to timeout.

This reverts commit 81f388fe570e5b6460dd5bc9b9a36b72714eeb68.

llvm-svn: 314453
2017-09-28 19:37:17 +00:00
Francis Ricci b9a32d470a Add support for custom loaders to the sanitizer symbolizer
Summary:
Adds a fallback mode to procmaps when the symbolizer
fails to locate a module for a given address by using
dl_iterate_phdr.

Reviewers: kubamracek, rnk, vitalybuka, eugenis

Reviewed By: eugenis

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D37269

llvm-svn: 314431
2017-09-28 16:58:35 +00:00
Francis Ricci cb15e22b3b Revert "Add support for custom loaders to symbolizer"
This broke the windows buildbots, revert for now.

This reverts commit 24050b5ddef42f6f3306aa94d4a1f42a7893a9a7.

llvm-svn: 314347
2017-09-27 21:35:33 +00:00
Francis Ricci f80c42e589 Add support for custom loaders to symbolizer
Change-Id: I5594bd6b216deca2c73cf0a7001f9aec1e803c60
llvm-svn: 314342
2017-09-27 20:56:04 +00:00
Francis Ricci 6e4dfccc31 [sanitizer_common] Don't provide sanitizer_procmaps API functions where not defined
Summary:
Platforms that don't implement procmaps (primarily fuchsia and windows) still expose
the procmaps API when including sanitizer_procmaps.h, despite not implementing the functions
provided by that header. Ensure that the API is only exposed on platforms that implement it.

Reviewers: vitalybuka, alekseyshl, kubamracek

Subscribers: llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D38187

llvm-svn: 314149
2017-09-25 20:48:51 +00:00
Vitaly Buka 5b81dfc76e [ubsan] Replace CommonSanitizerReportMutex with ScopedErrorReportLock
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D38194

llvm-svn: 314053
2017-09-23 02:47:21 +00:00
Vitaly Buka 15d7e604d9 [lsan] Deadly signal handler for lsan
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, dberris, kubamracek, krytarowski

Differential Revision: https://reviews.llvm.org/D37608

llvm-svn: 314041
2017-09-22 22:57:48 +00:00
Vitaly Buka a33e156f81 [sanitizer] Replace thread id with GetThreadSelf
This allows to avoid constructor parameter

llvm-svn: 314040
2017-09-22 22:36:21 +00:00
Vitaly Buka d8e18a4cf6 [sanitizer] Move report locking code from asan into common
llvm-svn: 314008
2017-09-22 18:32:05 +00:00
Vitaly Buka 21ddc6219b [sanitizer] Move stack overflow and signal reporting from Asan into common.
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D37844

llvm-svn: 313310
2017-09-14 22:44:03 +00:00
Vitaly Buka 9a4c73e20c [sanitizer] Move IsStackOverflow into SignalContext
llvm-svn: 313227
2017-09-14 03:23:02 +00:00
Vitaly Buka 73c101613d [compiler-rt] Cleanup SignalContext initialization
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris

Differential Revision: https://reviews.llvm.org/D37827

llvm-svn: 313223
2017-09-14 02:48:41 +00:00
Vitaly Buka dbde6f337c [compiler-rt] Use SignalContext in ErrorStackOverflow and ErrorDeadlySignal
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl, filcab

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37793

llvm-svn: 313168
2017-09-13 18:30:16 +00:00
Vitaly Buka b215e90fb0 [compiler-rt] Add siginfo into SignalContext
Summary:
Information stored there is often been passed along with SignalContext.

Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37792

llvm-svn: 313167
2017-09-13 18:30:06 +00:00
Vitaly Buka ecc44ecdbc [compiler-rt] Move dump_instruction_bytes and dump_registers into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37766

llvm-svn: 313117
2017-09-13 06:24:59 +00:00
Vitaly Buka d9d2a89e50 [compiler-rt] Move *Sanitizer:DEADLYSIGNAL printing into common part
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, llvm-commits, dberris

Differential Revision: https://reviews.llvm.org/D37764

llvm-svn: 313115
2017-09-13 04:46:37 +00:00
Vitaly Buka f6fc68c7d1 [compiler-rt] Move IsStackOverflow from asan into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D37536

llvm-svn: 312987
2017-09-12 00:44:23 +00:00
Vitaly Buka 7af8e4b146 Revert "[compiler-rt] Move IsStackOverflow from asan into sanitizer_common"
Windows is broken.

This reverts commit r312951

llvm-svn: 312984
2017-09-12 00:14:33 +00:00
Vitaly Buka ecba46d0e6 [compiler-rt] Move IsStackOverflow from asan into sanitizer_common
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D37536

llvm-svn: 312951
2017-09-11 21:00:24 +00:00
Kostya Kortchinsky e1dde07640 [sanitizers] Add a blocking boolean to GetRandom prototype
Summary:
On platforms with `getrandom`, the system call defaults to blocking. This
becomes an issue in the very early stage of the boot for Scudo, when the RNG
source is not set-up yet: the syscall will block and we'll stall.

Introduce a parameter to specify that the function should not block, defaulting
to blocking as the underlying syscall does.

Update Scudo to use the non-blocking version.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D36399

llvm-svn: 310839
2017-08-14 14:53:47 +00:00
Francis Ricci 0d920c4418 Only scan global sections containing data in LSan on darwin
Summary:
__DATA segments on Darwin contain a large number of separate sections,
many of which cannot actually contain pointers, and contain const values or
objc metadata. Not scanning sections which cannot contain pointers significantly
improves performance.

On a medium-sized (~4000 files) internal project, I saw a speedup of about 30%
in standalone LSan's execution time (30% improvement in the time spent running
LSan, not the total program time).

Reviewers: kcc, kubamracek, alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35432

llvm-svn: 308999
2017-07-25 18:16:58 +00:00
Vitaly Buka d4abe9e7d3 [sanitizer_common] Move filesystem-related code out of sanitizer_common.cc
Summary:
This is a pure refactoring change.  It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}.  This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.

Submitted on behalf of Roland McGrath.

Reviewers: kcc, eugenis, alekseyshl

Reviewed By: alekseyshl

Subscribers: vitalybuka, llvm-commits, kubamracek, mgorny, phosek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35591

llvm-svn: 308819
2017-07-22 01:46:40 +00:00
Vitaly Buka 094684c155 Revert "[sanitizer_common] Move filesystem-related code out of sanitizer_common.cc"
Breaks Windows build.

This reverts commit r308640.

llvm-svn: 308648
2017-07-20 18:24:45 +00:00
Alex Shlyapnikov 89e85d1717 [sanitizer_common] Move filesystem-related code out of sanitizer_common.cc
This is a pure refactoring change. It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}. This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.

Commiting for mcgrathr.

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35591

llvm-svn: 308640
2017-07-20 17:48:42 +00:00
Francis Ricci 1b5f773859 Revert "Only scan global sections containing data in LSan on darwin"
This reverts commit 7e46d78d47832f03ce42adcf56417fbfd47cbaad.

llvm-svn: 308394
2017-07-18 23:51:44 +00:00
Francis Ricci 7096b08cd7 Only scan global sections containing data in LSan on darwin
Summary:
__DATA segments on Darwin contain a large number of separate sections,
most of which cannot actually contain pointers, and contain const values or
objc metadata. Only scanning sections which can contain pointers greatly improves
performance.

On a medium-sized (~4000 files) internal project, I saw a speedup of about 50%
in standalone LSan's execution time (50% improvement in the time spent running
LSan, not the total program time).

Reviewers: kcc, kubamracek, alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35432

llvm-svn: 308231
2017-07-17 23:03:03 +00:00
Kuba Mracek c1e903be19 [asan] For iOS/AArch64, if the dynamic shadow doesn't fit, restrict the VM space
On iOS/AArch64, the address space is very limited and has a dynamic maximum address based on the configuration of the device. We're already using a dynamic shadow, and we find a large-enough "gap" in the VM where we place the shadow memory. In some cases and some device configuration, we might not be able to find a large-enough gap: E.g. if the main executable is linked against a large number of libraries that are not part of the system, these libraries can fragment the address space, and this happens before ASan starts initializing.

This patch has a solution, where we have a "backup plan" when we cannot find a large-enough gap: We will restrict the address space (via MmapFixedNoAccess) to a limit, for which the shadow limit will fit.

Differential Revision: https://reviews.llvm.org/D35098

llvm-svn: 307865
2017-07-12 23:29:21 +00:00
Alex Shlyapnikov 01676883cd [Sanitizers] 64 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator64 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow
allocator_may_return_null=1 policy
(LargeMmapAllocator: D34243, SizeClassAllocator64: D34433).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34540

llvm-svn: 306342
2017-06-26 22:54:10 +00:00
Evgeniy Stepanov 90e5c30836 [asan] Add support for Android debug message.
Add ASan report to the "debug message" field in Android tombstones.

llvm-svn: 306184
2017-06-23 23:38:20 +00:00
Alex Shlyapnikov f3cc7cc3d8 [Sanitizers] 32 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).

Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34433

llvm-svn: 305972
2017-06-22 00:02:37 +00:00
Kostya Kortchinsky 2b053b1c28 [sanitizer] Add a function to gather random bytes
Summary:
AFAICT compiler-rt doesn't have a function that would return 'good' random
bytes to seed a PRNG. Currently, the `SizeClassAllocator64` uses addresses
returned by `mmap` to seed its PRNG, which is not ideal, and  
`SizeClassAllocator32` doesn't benefit from the entropy offered by its 64-bit
counterpart address space, so right now it has nothing. This function aims at
solving this, allowing to implement good 32-bit chunk randomization. Scudo also
has a function that does this for Cookie purposes, which would go away in a
later CL once this lands.

This function will try the `getrandom` syscall if available, and fallback to
`/dev/urandom` if not.

Unfortunately, I do not have a way to implement and test a Mac and Windows
version, so those are unimplemented as of now. Note that `kRandomShuffleChunks`
is only used on Linux for now.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: zturner, rnk, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34412

llvm-svn: 305922
2017-06-21 15:56:03 +00:00
Alex Shlyapnikov 9092fe6f4b [Sanitizers] Secondary allocator respects allocator_may_return_null=1.
Summary:
Context: https://github.com/google/sanitizers/issues/740.

Making secondary allocator to respect allocator_may_return_null=1 flag
and return nullptr when "out of memory" happens.

More changes in primary allocator and operator new will follow.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34243

llvm-svn: 305569
2017-06-16 18:48:08 +00:00
Kostya Kortchinsky 77f30c9c31 [sanitizer] Reverting D34152
Summary:
This broke thread_local_quarantine_pthread_join.cc on some architectures, due
to the overhead of the stashed regions. Reverting while figuring out the best
way to deal with it.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34213

llvm-svn: 305404
2017-06-14 17:32:26 +00:00