Commit Graph

3244 Commits

Author SHA1 Message Date
Dmitry Vyukov baf2f35ec4 sanitizers: Introduce ThreadType enum
Replace bool workerthread flag with ThreadType enum.
This change is preparation for fiber support.

[dvyukov: fixed build of sanitizer_thread_registry_test.cc]

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57839
Context: https://reviews.llvm.org/D54889

llvm-svn: 353390
2019-02-07 11:01:22 +00:00
Vitaly Buka 1665ea19e9 [sanitizer] Remove unneeded pointer check
Summary: unpoison_passwd and unpoison_group support nullptrs

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57784

llvm-svn: 353365
2019-02-07 01:54:59 +00:00
Vitaly Buka eb7031ecf6 [sanitizer] Don't unpoison buffer in getpw/getgr functions
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.

This fixes TSAN performance issue made us to disable this interceptors.

Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57731

llvm-svn: 353351
2019-02-07 00:08:14 +00:00
Evgeniy Stepanov 54c82886cb [sanitizer] Fix fuchsia and windows build.
llvm-svn: 353261
2019-02-06 01:58:23 +00:00
Evgeniy Stepanov 443c034391 [sanitizer] Decorate /proc/self/maps better.
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.

Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.

Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.

The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D57720

llvm-svn: 353255
2019-02-06 01:14:50 +00:00
Kamil Rytarowski 3349bd662a Update the ioctl(2) list in sanitizers with NetBSD 8.99.34
llvm-svn: 353224
2019-02-05 22:20:25 +00:00
Vitaly Buka 0ca702f8d5 [sanitizer] Fix unused function 'unpoison_passwd'
llvm-svn: 353205
2019-02-05 19:46:44 +00:00
Reid Kleckner 33706e3cbf [ubsan] Make suppressions.cpp test pass for me on Windows
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.

While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.

llvm-svn: 352921
2019-02-01 21:35:17 +00:00
Igor Kudrin 0f13000958 [compiler-rt] Fix reporting unrecognized flags in unit tests.
Previously, the warning messages might be issued within a wrong test case.

Differential Revision: https://reviews.llvm.org/D57318

llvm-svn: 352447
2019-01-29 02:31:57 +00:00
Dan Liew 225eb75a10 Make more allocator methods work correctly with in an out-of-process mode.
Summary:
This makes `GetBlockBegin()` and `GetBlockBeginFastLocked()` work correctly with `RemoteAddressSpaceView`.

This has a knock on effect of also making the `PointerIsMine()` and
`GetMetaData()` methods behave correctly when `RemoteAddressSpaceView`
is used to instantiate the allocators.

This will be used by future out-of-process allocator enumeration
patches.

rdar://problem/45284065

Reviewers: kcc, vitalybuka, dvyukov, cryptoad, eugenis, george.karpenkov, yln

Subscribers: #sanitizers, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D56964

llvm-svn: 352335
2019-01-27 22:45:11 +00:00
Martin Liska acc348aad0 Revert https://reviews.llvm.org/D56485.
llvm-svn: 352033
2019-01-24 08:46:06 +00:00
Martin Liska dc5c81330b Always compare C++ typeinfo (based on libstdc++ implementation).
Differential Revision: https://reviews.llvm.org/D56485.

llvm-svn: 352032
2019-01-24 08:25:36 +00:00
Jonathan Metzman dd467f4f43 [libFuzzer][MSVC] Make Sanitizer Coverage MSVC-compatible
Summary:
Make Sanitizer Coverage work when compiled work when compiler-rt
is compiled with MSVC.

The previous solution did not work for MSVC because MSVC tried to
align the .SCOV$CZ section even though we used
__declspec(align(1)) on its only symbol:
__stop___sancov_cntrs.
Because the counter array is composed
of 1 byte elements, it does not always end on an 8 or 4 byte
boundary. This means that padding was sometimes added to
added to align the next section, .SCOV$CZ.
Use a different strategy now: instead of only instructing
the compiler not to align the symbol, make the section
one byte long by making its only symbol a uint8_t, so that
the linker won't try to align it.

Reviewers: morehouse, rnk

Reviewed By: rnk

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D56866

llvm-svn: 351714
2019-01-21 02:15:29 +00:00
Dan Liew 5a3857c9b2 On Darwin add allocator address and size fields to
`sanitizer_malloc_introspection_t` and initialize them to zero.

We allow sanitizer implementations to perform different initialization
by defining `COMMON_MALLOC_HAS_EXTRA_INTROSPECTION_INIT` to be `1`
and providing an implementation of `mi_extra_init(...)`.

We use these changes in future patches to implement malloc zone enumeration.

rdar://problem/45284065

llvm-svn: 351712
2019-01-21 01:41:08 +00:00
Dan Liew 4dd0bf9487 On Darwin allow for sanitizer malloc implementations to provide a zone
enumerator.

This is done by defining `COMMON_MALLOC_HAS_ZONE_ENUMERATOR` to `1` and
then by providing an implementation of the `mi_enumerator(...)` function.
If a custom implementation isn't desired the macro is set to `0` which
causes a stub version (that fails) to be used.

Currently all Darwin sanitizers that have malloc implementations define
this to be `0` so there is no functionality change.

rdar://problem/45284065

llvm-svn: 351711
2019-01-21 01:41:01 +00:00
Chandler Carruth 57b08b0944 Update more file headers across all of the LLVM projects in the monorepo
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351648
2019-01-19 10:56:40 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Evgeniy Stepanov c089fbbaf2 [hwasan] Madvise away unused shadow.
Summary:
Whenever a large shadow region is tagged to zero, madvise(DONT_NEED)
as much of it as possible.

This reduces shadow RSS on Android by 45% or so, and total memory use
by 2-4%, probably even more on long running multithreaded programs.
CPU time seems to be in the noise.

Reviewers: kcc, pcc

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D56757

llvm-svn: 351620
2019-01-19 01:54:09 +00:00
Nico Weber 07d8b321b3 Make compiler-rt CMakeLists.txt formatting a bit more like LLVM's usual formatting
llvm-svn: 351363
2019-01-16 18:12:45 +00:00
David Carlier bd27e4c156 [Sanitizer] Intercept sl_add api on FreeBSD/NetBSD
Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56670

llvm-svn: 351189
2019-01-15 11:21:33 +00:00
Michal Gorny 965517fdad [test] Disable sunrpc tests when rpc/xdr.h is missing
Disable tests requiring sunrpc when the relevant headers are missing.
In order to accommodate that, move the header check
from sanitizer_common to base-config-ix, and define the check result
as a global variable there.  Use it afterwards both for definition
needed by sanitizer_common, and to control 'sunrpc' test feature.

While at it, remove the append_have_file_definition macro that was used
only once, and no longer fits the split check-definition.

Bug report: https://github.com/google/sanitizers/issues/974

Differential Revision: https://reviews.llvm.org/D47819

llvm-svn: 351109
2019-01-14 19:18:34 +00:00
Evgeniy Stepanov 0df20e893f [sanitizer] Move android's GetPageSize to a header (NFC)
No need to pay function call overhead for a function that returns a
constant.

llvm-svn: 350983
2019-01-12 00:09:24 +00:00
Peter Collingbourne c1fde4fa94 sanitizer_common: Change gen_dynamic_list.py to take a -o argument instead of writing to stdout.
This makes the script a little more gn friendly; gn does not support
redirecting the output of a script.

Differential Revision: https://reviews.llvm.org/D56579

llvm-svn: 350980
2019-01-11 23:30:24 +00:00
David Carlier d0de9890d3 [Sanitizer] Disable getusershell interception for Android
Reviewers: vitalybuka, pcc, eugenis

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D56583

llvm-svn: 350965
2019-01-11 20:09:24 +00:00
Dan Liew cb09a6f20c Provide storage for `true_type::value` and `false_type::value`.
Summary:
This fixes linker errors that occurs when the
`sanitizer_type_traits_test.cc` is built without optimizations.

The error occurs because the test tries to take a reference.
A possible workaround is to give the GTest macros take boolean rvalues
by doing something like:

```
ASSERT_TRUE(bool(is_same<uptr, uptr>::value));
```

However this only hides the problem. Unfortunately Using `constexpr`
won't fix the problem unless we are using C++17.

Reviewers: vitalybuka, kubamracek, george.karpenkov, yln

Subscribers: mgorny, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D56035

llvm-svn: 350940
2019-01-11 17:59:52 +00:00
David Carlier 8a81b29215 [Sanitizer] Intercept getusershell
- If entries are properly copied (there were a bug in FreeBSD implementation in earlier version), or list properly reset.

Reviewers: vitalybuka, krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56562

llvm-svn: 350919
2019-01-11 05:53:09 +00:00
Michal Gorny 1765839052 [sanitizer_common] Define __sanitizer_FILE on NetBSD
Differential Revision: https://reviews.llvm.org/D56109

llvm-svn: 350882
2019-01-10 19:56:57 +00:00
Michal Gorny 5b2b63e524 [sanitizer_common] Remove support for tirpc/rpc/xdr.h
Remove the partial support for rpc/xdr.h from libtirpc.  Since it is
an entirely external library, we ought to build it sanitized separately
and not attempt to intercept like the libc implementation.  Besides,
the existing code for tirpc support was neither complete nor working.

Noted by @krytarowski.

Differential Revision: https://reviews.llvm.org/D47817

llvm-svn: 350881
2019-01-10 19:56:50 +00:00
David Carlier c1ba1eec69 i[Sanitizer] Enable pututxline interception
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56495

llvm-svn: 350796
2019-01-10 04:19:30 +00:00
David Carlier bfce66fbc1 [Sanitizer] Intercept fdevname on FreeBSD
- Is a file descriptor flavor FreeBSD's specific.
- reentrant version included.

Reviewers: vitalybuka, krytarowski, emaste

Reviewed By: emaste

Differential Revision: https://reviews.llvm.org/D56268

llvm-svn: 350658
2019-01-08 21:49:37 +00:00
Evgeniy Stepanov 43cba99775 Fix Mac compilation.
Provide an implementation of MemoryMappingLayout::Error() for Mac.

llvm-svn: 350601
2019-01-08 02:02:27 +00:00
Evgeniy Stepanov e1caa4ac72 [asan] Support running without /proc
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.

This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.

Reviewers: vitalybuka, pcc, krytarowski

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D56141

llvm-svn: 350590
2019-01-08 01:07:34 +00:00
Petr Hosek 5c43496fa3 [compiler-rt][Fuchsia] Replace _zx_vmar_allocate_old call
This is the deprecated legacy interface, replace it with the current
_zx_vmar_allocate one.

Differential Revision: https://reviews.llvm.org/D56360

llvm-svn: 350488
2019-01-06 05:19:05 +00:00
Evgeniy Stepanov 045a83231d Revert "Revert "Switch Android from TLS_SLOT_TSAN(8) to TLS_SLOT_SANITIZER(6)""
This reapplies commit r348984.

llvm-svn: 350449
2019-01-05 00:45:14 +00:00
Evgeniy Stepanov 9fbc364e16 [sanitizer] Reduce stack depot size on Android.
Summary:
The default setting kTabSizeLog=20 results in an 8Mb global hash table,
almost all of it in private pages. That is not a sane setting in a
mobile, system-wide use case: with ~150 concurrent processes stack
depot will account for more than 1Gb of RAM.

Reviewers: kcc, pcc

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D56333

llvm-svn: 350443
2019-01-04 22:55:04 +00:00
Evgeniy Stepanov d398471a63 [hwasan] Switch to 64 allocator with a dense size class map.
Summary:
Replace the 32-bit allocator with a 64-bit one with a non-constant
base address, and reduce both the number of size classes and the maximum
size of per-thread caches.

As measured on [1], this reduces average weighted memory overhead
(MaxRSS) from 26% to 12% over stock android allocator. These numbers
include overhead from code instrumentation and hwasan shadow (i.e. not a
pure allocator benchmark).

This switch also enables release-to-OS functionality, which is not
implemented in the 32-bit allocator. I have not seen any effect from
that on the benchmark.

[1] https://android.googlesource.com/platform/system/extras/+/master/memory_replay/

Reviewers: vitalybuka, kcc

Subscribers: kubamracek, cryptoad, llvm-commits

Differential Revision: https://reviews.llvm.org/D56239

llvm-svn: 350370
2019-01-03 23:19:02 +00:00
David Carlier 205b709366 [Sanitizer] Enable funopen on FreeBSD
Reviewers: krytarowski

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56209

llvm-svn: 350248
2019-01-02 19:07:27 +00:00
Evgeniy Stepanov f5f1fb594b [sanitizer] Android does not provide <fstab.h>
llvm-svn: 350246
2019-01-02 19:05:26 +00:00
Michal Gorny 3d25e8d9f8 [sanitizer_common] Implement funopen*() interceptors for NetBSD
Differential Revision: https://reviews.llvm.org/D56158

llvm-svn: 350233
2019-01-02 17:37:14 +00:00
Michal Gorny 71a75307ae [sanitizer_common] Implement popen, popenve, pclose interceptors
Implement the interceptors for popen(), pclose() and popenve()
functions.  The first two are POSIX, the third one is specific
to NetBSD.  popen() spawns a process and creates a FILE object piping
data from/to that process.  pclose() closes the pipe and waits for
the process to terminate appropriately.

For the purpose of popen(), the COMMON_INTERCEPTOR_FILE_OPEN macro is
modified to allow null path parameter.

Differential Revision: https://reviews.llvm.org/D56157

llvm-svn: 350232
2019-01-02 17:37:09 +00:00
Michal Gorny bb1137da12 [sanitizer_common] Fix devname_r() return type on !NetBSD
Update the interceptor for devname_r() to account for correct return
types on different platforms.  This function returns int on NetBSD
but char* on FreeBSD/OSX.  Noticed by @krytarowski.

Differential Revision: https://reviews.llvm.org/D56150

llvm-svn: 350228
2019-01-02 17:36:50 +00:00
Kamil Rytarowski 646c130a18 Add support for background thread on NetBSD in ASan
Summary:
Change the point of calling MaybeStartBackgroudThread() from AsanInitInternal()
that is too early on NetBSD to a constructor (with aid of C++11 lambda construct).

Enable the code for background thread as is for NetBSD.

Rename test/sanitizer_common/TestCases/Linux/hard_rss_limit_mb_test.cc
to test/sanitizer_common/TestCases/hard_rss_limit_mb_test.cc and allow runs
on NetBSD. This tests passes correctly.

Reviewers: vitalybuka, joerg, eugenis

Reviewed By: eugenis

Subscribers: eugenis, kubamracek, fedor.sergeev, llvm-commits, mgorny, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D55887

llvm-svn: 350139
2018-12-29 00:32:07 +00:00
David Carlier adb5f1e3b5 [Sanitizer] Enable getfsent api for Darwin
Reviewers: vitalybuka, kubamracek

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D56138

llvm-svn: 350138
2018-12-29 00:17:21 +00:00
Dan Liew 8c11fb3ed4 Introduce `LocalAddressSpaceView::LoadWritable(...)` and make the `Load(...)` method return a const pointer.
Summary:
This is a follow-up to r346956 (https://reviews.llvm.org/D53975).

The purpose of this change to allow implementers of the
`AddressSpaceView` to be able to distinguish between when a caller wants
read-only memory and when a caller wants writable memory. Being able
distinguish these cases allows implementations to optimize for the
different cases and also provides a way to workaround possible platform
restrictions (e.g. the low level platform interface for reading
out-of-process memory may place memory in read-only pages).

For allocator enumeration in almost all cases read-only is sufficient so
we make `Load(...)` take on this new requirement and introduce the
`LoadWritable(...)` variants for cases where memory needs to be
writable.

The behaviour of `LoadWritable(...)` documented in comments are
deliberately very restrictive so that it will be possible in the future
to implement a simple write-cache (i.e. just a map from target address
to a writable region of memory). These restrictions can be loosened in
the future if necessary by implementing a more sophisticated
write-cache.

rdar://problem/45284065

Reviewers: kcc, cryptoad, eugenis, kubamracek, george.karpenkov

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D54879

llvm-svn: 350136
2018-12-28 19:30:51 +00:00
David Carlier f11fcc046e [Sanitizer] Intercept arc4random_buf / arc4random_addrandom on FreeBSD/NetBSD
- Disabled on purpose on Android and Darwin platform (for now).
- Darwin supports it, would need interception in its specific code before enabling it.
- Linux does not support it but only via third party library.
- Android supports it via bionic however it is known to have issue with older versions of the implementations. Can be enabled by an Android committer later on if necessary once there is more 'certainity'/been more tested.

Reviewers: krytarowski, vitalybuka

Reviewed By: krytarowski

Differential Revision: https://reviews.llvm.org/D56125

llvm-svn: 350123
2018-12-28 16:04:28 +00:00
Evgeniy Stepanov 3408b497f9 Revert "[asan] Support running without /proc.", +1
Revert r350104 "[asan] Fix build on windows."
Revert r350101 "[asan] Support running without /proc."

These changes break Mac build, too.

llvm-svn: 350112
2018-12-28 01:27:18 +00:00
Kamil Rytarowski 10828c39cb Fix a mistake in previous
Assign correct ioctl(2) operation number.

llvm-svn: 350110
2018-12-28 00:17:37 +00:00
Kamil Rytarowski 8e56064542 Update NetBSD ioctl(2) entries with 8.99.28
Add SIOCSETHERCAP.

Add commented out NVMM (NetBSD Virtual Machine Monitor) operations
as this interface is still WIP and a subject to change.

llvm-svn: 350107
2018-12-27 23:35:28 +00:00
Evgeniy Stepanov e9d00896bd Enable posix regex interceptors on linux.
Summary: They happen to work out of the box.

Reviewers: rtrieu, vitalybuka

Subscribers: kubamracek, fedor.sergeev, krytarowski, llvm-commits

Differential Revision: https://reviews.llvm.org/D56088

llvm-svn: 350103
2018-12-27 21:47:26 +00:00
Evgeniy Stepanov d9df65fd0d [asan] Support running without /proc.
Summary:
This patch lets ASan run when /proc is not accessible (ex. not mounted
yet). It includes a special test-only flag that emulates this condition
in an unpriviledged process.

This only matters on Linux, where /proc is necessary to enumerate
virtual memory mappings.

Reviewers: pcc, vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D55874

llvm-svn: 350101
2018-12-27 21:31:52 +00:00