Commit Graph

437 Commits

Author SHA1 Message Date
Kostya Kortchinsky 77f30c9c31 [sanitizer] Reverting D34152
Summary:
This broke thread_local_quarantine_pthread_join.cc on some architectures, due
to the overhead of the stashed regions. Reverting while figuring out the best
way to deal with it.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34213

llvm-svn: 305404
2017-06-14 17:32:26 +00:00
Kostya Kortchinsky eca926ab3a [sanitizer] MmapAlignedOrDie changes to reduce fragmentation
Summary:
The reasoning behind this change is explained in D33454, which unfortunately
broke the Windows version (due to the platform not supporting partial unmapping
of a memory region).

This new approach changes `MmapAlignedOrDie` to allow for the specification of
a `padding_chunk`. If non-null, and the initial allocation is aligned, this
padding chunk will hold the address of the extra memory (of `alignment` bytes).
This allows `AllocateRegion` to get 2 regions if the memory is aligned
properly, and thus help reduce fragmentation (and saves on unmapping
operations). As with the initial D33454, we use a stash in the 32-bit Primary
to hold those extra regions and return them on the fast-path.

The Windows version of `MmapAlignedOrDie` will always return a 0
`padding_chunk` if one was requested.

Reviewers: alekseyshl, dvyukov, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34152

llvm-svn: 305391
2017-06-14 15:32:17 +00:00
Kostya Serebryany 124c2ae4fa [sanitizer-coverage] nuke more stale code
llvm-svn: 304508
2017-06-02 01:17:04 +00:00
Pierre Gousseau 183d1368f3 [asan] Add strndup/__strndup interceptors.
Recommit of r302781 with Vitaly Buka's fix for non zero terminated strings.

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 304399
2017-06-01 09:37:22 +00:00
Vitaly Buka a05da1fca9 [compiler-rt] Replace allow_user_segv_handler=0 with kHandleSignalExclusive
Summary:
allow_user_segv_handler had confusing name did not allow to control behavior for
signals separately.

Reviewers: eugenis, alekseyshl, kcc

Subscribers: llvm-commits, dberris, kubamracek

Differential Revision: https://reviews.llvm.org/D33371

llvm-svn: 303941
2017-05-25 23:42:33 +00:00
Vitaly Buka 8018780d39 [compiler-rt] Switch handle_<signal> flags from bool to enum.
Summary: We are going to make it tri-state and remove allow_user_segv_handler.

Reviewers: eugenis, alekseys, kcc

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D33159

llvm-svn: 303464
2017-05-19 22:37:16 +00:00
Vitaly Buka 5934882576 [compiler-rt] Add negative test for boolean flags.
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D33344

llvm-svn: 303462
2017-05-19 22:37:13 +00:00
Simon Dardis 657899bad1 [compiler-rt][cmake] Build unit tests conditionally with _FILE_OFFSET_BITS=64 and _LARGEFILE_SOURCE
The sanitizer library unit tests for libc can get a different definition
of 'struct stat' to what the sanitizer library is built with for certain
targets.

For MIPS the size element of 'struct stat' is after a macro guarded
explicit padding element.

This patch resolves any possible inconsistency by adding the same
_FILE_OFFSET_BITS=64 and _LARGE_SOURCE with the same
conditions as the sanitizer library to the build flags for the unit tests.

This resolves a recurring build failure on the MIPS buildbots due to
'struct stat' defintion differences.

Reviewers: slthakur

Differential Revision: https://reviews.llvm.org/D33131

llvm-svn: 303350
2017-05-18 13:19:35 +00:00
Daniel Jasper f97310fb7a Revert r302781 and subsequent attempts to disable part of it.
The Msan unit tests are still broken and by this point, I think we
should start over.

llvm-svn: 303339
2017-05-18 09:31:37 +00:00
Kostya Serebryany 9aa8ef8504 temporary disable stndup interceptor, due to r302781 being buggy
llvm-svn: 303324
2017-05-18 03:00:07 +00:00
Kostya Kortchinsky dc646a0889 [sanitizer] Change SizeClassAllocator32 to accept just one template
Summary:
With rL279771, SizeClassAllocator64 was changed to accept only one template
instead of 5, for the following reasons: "First, this will make the mangled
names shorter. Second, this will make adding more parameters simpler". This
patch mirrors that work for SizeClassAllocator32.

This is in preparation for introducing the randomization of chunks in the
32-bit SizeClassAllocator in a later patch.

Reviewers: kcc, alekseyshl, dvyukov

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33141

llvm-svn: 303071
2017-05-15 14:47:19 +00:00
Pierre Gousseau 0550581070 [asan] Recommit of r301904: Add strndup/__strndup interceptors
Fix undeclared __interceptor_malloc in esan_interceptors.cc
Fix undeclared strnlen on OSX

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 302781
2017-05-11 08:53:24 +00:00
Pierre Gousseau 1c5550671d Revert r301904 causing tsan test failure in x86_64-linux-autoconf
llvm-svn: 301909
2017-05-02 10:22:05 +00:00
Pierre Gousseau b7101479a8 [asan] Add strndup/__strndup interceptors if targeting linux.
Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 301904
2017-05-02 09:01:02 +00:00
Kostya Serebryany 263722f40f [asan] replace std::random_shuffle with std::shuffle in tests since std::random_shuffle is being deprecated in C++17; NFC
llvm-svn: 294370
2017-02-07 23:13:10 +00:00
Kuba Mracek 5c88271528 Fixup of r293882: Forgot to update sanitizer_thread_registry.test.cc
llvm-svn: 293884
2017-02-02 13:07:22 +00:00
Dimitry Andric f2ac1d1df2 Recommit: Stop intercepting some malloc-related functions on FreeBSD and
macOS

Summary:
In https://bugs.freebsd.org/215125 I was notified that some configure
scripts attempt to test for the Linux-specific `mallinfo` and `mallopt`
functions by compiling and linking small programs which references the
functions, and observing whether that results in errors.

FreeBSD and macOS do not have the `mallinfo` and `mallopt` functions, so
normally these tests would fail, but when sanitizers are enabled, they
incorrectly succeed, because the sanitizers define interceptors for
these functions.  This also applies to some other malloc-related
functions, such as `memalign`, `pvalloc` and `cfree`.

Fix this by not intercepting `mallinfo`, `mallopt`, `memalign`,
`pvalloc` and `cfree` for FreeBSD and macOS, in all sanitizers.

Also delete the non-functional `cfree` wrapper for Windows, to fix the
test cases on that platform.

Reviewers: emaste, kcc, rnk

Subscribers: timurrrr, eugenis, hans, joerg, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D27654

llvm-svn: 293536
2017-01-30 19:06:13 +00:00
Evgeniy Stepanov 8bf0633288 Revert "Stop intercepting some malloc-related functions on FreeBSD and macOS"
This reverts r293337, which breaks tests on Windows:

malloc-no-intercept-499eb7.o : error LNK2019: unresolved external symbol _mallinfo referenced in function _main

llvm-svn: 293346
2017-01-27 23:53:20 +00:00
Dimitry Andric 47e0ef3148 Stop intercepting some malloc-related functions on FreeBSD and macOS
Summary:
In https://bugs.freebsd.org/215125 I was notified that some configure
scripts attempt to test for the Linux-specific `mallinfo` and `mallopt`
functions by compiling and linking small programs which references the
functions, and observing whether that results in errors.

FreeBSD and macOS do not have the `mallinfo` and `mallopt` functions, so
normally these tests would fail, but when sanitizers are enabled, they
incorrectly succeed, because the sanitizers define interceptors for
these functions.  This also applies to some other malloc-related
functions, such as `memalign`, `pvalloc` and `cfree`.

Fix this by not intercepting `mallinfo`, `mallopt`, `memalign`,
`pvalloc` and `cfree` for FreeBSD and macOS, in all sanitizers.

Reviewers: emaste, kcc

Subscribers: hans, joerg, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D27654

llvm-svn: 293337
2017-01-27 22:19:11 +00:00
Alex Shlyapnikov a8ba9c8e66 Whenever reasonable, merge ASAN quarantine batches to save memory.
Summary:
There are cases when thread local quarantine drains almost empty
quarantine batches into the global quarantine. The current approach leaves
them almost empty, which might create a huge memory overhead (each batch
is 4K/8K, depends on bitness).

Reviewers: eugenis

Subscribers: kubabrecka, llvm-commits

Differential Revision: https://reviews.llvm.org/D28068

llvm-svn: 292525
2017-01-19 22:15:54 +00:00
Mike Aizatsky 9700acba11 Revert "[sancov] moving sancov rt to sancov/ directory"
This reverts commit https://reviews.llvm.org/rL291734
Reason: mac breakage
http://lab.llvm.org:8080/green//job/clang-stage1-configure-RA_build/28798/consoleFull#1657087648e9a0fee5-ebcc-4238-a641-c5aa112c323e

llvm-svn: 291736
2017-01-12 01:37:35 +00:00
Mike Aizatsky 875572f358 [sancov] moving sancov rt to sancov/ directory
Subscribers: kubabrecka, mgorny

Differential Revision: https://reviews.llvm.org/D28541

llvm-svn: 291734
2017-01-12 01:19:34 +00:00
Francis Ricci 17781c71b0 Make cmake link flag naming consistent
Summary:
The build system was inconsistent in its naming conventions for
link flags. This patch changes all uses of LINKFLAGS to LINK_FLAGS,
for consistency with cmake's LINK_FLAGS property.

This patch should make it easier to search the source code for
uses of link flags, as well as providing the benefit of improved
style and consistency.

Reviewers: compnerd, beanz

Subscribers: kubabrecka, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D28506

llvm-svn: 291539
2017-01-10 04:33:04 +00:00
Francis Ricci ff567a8ba2 Enable weak hooks on darwin
Summary:
By default, darwin requires a definition for weak interface functions at
link time. Adding the '-U' link flag with each weak function allows these
weak interface functions to be used without definitions, which mirrors
behavior on linux and windows.

Reviewers: compnerd, eugenis

Subscribers: kubabrecka, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D28203

llvm-svn: 291417
2017-01-09 03:51:42 +00:00
Kuba Mracek b38f1ca2d5 [sanitizer] Use architecture/slice information when symbolizing fat Mach-O files on Darwin
This patch starts passing architecture information about a module to llvm-symbolizer and into text reports. This fixes the longstanding x86_64/x86_64h mismatch issue on Darwin.

Differential Revision: https://reviews.llvm.org/D27390

llvm-svn: 291287
2017-01-06 21:45:05 +00:00
Evgeniy Stepanov f787cf7ee6 [asan] Fix handling of %m in printf interceptor.
llvm-svn: 290632
2016-12-27 22:14:03 +00:00
Kuba Mracek 3eb98a1318 [sanitizer] Track architecture and UUID of modules in LoadedModule
When we enumerate loaded modules, we only track the module name and base address, which then has several problems on macOS. Dylibs and executables often have several architecture slices and not storing which architecture/UUID is actually loaded creates problems with symbolication: A file path + offset isn't enough to correctly symbolicate, since the offset can be valid in multiple slices. This is especially common for Haswell+ X86_64 machines, where x86_64h slices are preferred, but if one is not available, a regular x86_64 is loaded instead. But the same issue exists for i386 vs. x86_64 as well.

This patch adds tracking of arch and UUID for each LoadedModule. At this point, this information isn't used in reports, but this is the first step. The goal is to correctly identify which slice is loaded in symbolication, and also to output this information in reports so that we can tell which exact slices were loaded in post-mortem analysis.

Differential Revision: https://reviews.llvm.org/D26632

llvm-svn: 288537
2016-12-02 21:27:14 +00:00
Evgeniy Stepanov d3305afc75 Return memory to OS right after free (not in the async thread).
Summary:
In order to avoid starting a separate thread to return unused memory to
the system (the thread interferes with process startup on Android,
Zygota waits for all threads to exit before fork, but this thread never
exits), try to return it right after free.

Reviewers: eugenis

Subscribers: cryptoad, filcab, danalbert, kubabrecka, llvm-commits

Patch by Aleksey Shlyapnikov.

Differential Revision: https://reviews.llvm.org/D27003

llvm-svn: 288091
2016-11-29 00:22:50 +00:00
Mike Aizatsky 7594ec3355 rename InternalBinarySearch to InternalLowerBound
Summary: The new name better corresponds to its logic.

Reviewers: kcc

Subscribers: kubabrecka

Differential Revision: https://reviews.llvm.org/D26821

llvm-svn: 287377
2016-11-18 20:48:52 +00:00
Mike Aizatsky ff3bdbac35 fixing binary search for cases when element is not in array
Subscribers: kubabrecka

Differential Revision: https://reviews.llvm.org/D26707

llvm-svn: 287078
2016-11-16 04:03:27 +00:00
Anna Zaks be7ae6684f [sanitizers] Set Darwin specific linker and compiler flags for all tests
Looks like we are missing these flags only in tsan and sanitizer-common.

This results in linker warnings in some settings as it can cause the Unit
tests to be built with a different SDK version than that was used to build
the runtime. For example, we are not setting the minimal deployment target
on the tests but are setting the minimal deployment target for the sanitizer
library, which leads to the following warning on some bots: ld: warning:
object file (sanitizer_posix_test.cc.i386.o) was built for newer OSX version
(10.12) than being linked (10.11).

Differential Revision: https://reviews.llvm.org/D25860  https://reviews.llvm.org/D25352

llvm-svn: 285255
2016-10-26 23:23:38 +00:00
Evgeniy Stepanov 8652d6ac05 [asan] Reenable 64-bit allocator on android/aarch64.
This is a re-commit of r281371, with updated tests.

llvm-svn: 281674
2016-09-15 22:34:53 +00:00
Anna Zaks 691644f3ca [compiler-rt] Do not introduce __sanitizer namespace globally
The definitions in sanitizer_common may conflict with definitions from system headers because:

The runtime includes the system headers after the project headers (as per LLVM coding guidelines).
lib/sanitizer_common/sanitizer_internal_defs.h pollutes the namespace of everything defined after it, which is all/most of the sanitizer .h and .cc files and the included system headers with: using namespace __sanitizer; // NOLINT
This patch solves the problem by introducing the namespace only within the sanitizer namespaces as proposed by Dmitry.

Differential Revision: https://reviews.llvm.org/D21947

llvm-svn: 281657
2016-09-15 21:02:18 +00:00
Reid Kleckner 110ecbd336 [sanitizer] Test the allocator with the ASan win64 memory constants
These got out of sync and the tests were failing for me locally.  We
assume a 47 bit address space in ASan, so we should do the same in the
tests.

llvm-svn: 281622
2016-09-15 16:00:46 +00:00
Kostya Serebryany 58560a05ad [sanitizer] fix an allocator bug where the allocated memory may overlap with the free array (kudos to Kostya Korcthinsky). Also make sure that the allocator does not mmap more than requested. Test both.
llvm-svn: 281103
2016-09-09 21:42:33 +00:00
Kostya Serebryany 2700bd831a [sanitizer] extend SizeClassMap to take more template parameters, add VeryCompactSizeClassMap for testing purposes
llvm-svn: 280266
2016-08-31 17:52:55 +00:00
Kostya Serebryany 7c5ae7cbc6 [sanitizer] enable random shuffling the memory chunks inside the allocator, under a flag. Set this flag for the scudo allocator, add a test.
llvm-svn: 279793
2016-08-26 00:06:03 +00:00
Kostya Serebryany 15647b17f3 [sanitizer] change SizeClassAllocator64 to accept just one template parameter instead of 5. First, this will make the mangled names shorter. Second, this will make adding more parameters simpler.
llvm-svn: 279771
2016-08-25 20:23:08 +00:00
Kostya Serebryany 8e7ea9ddb5 [sanitizer] re-apply r279572 and r279595 reverted in r279643: change the 64-bit allocator to use a single array for free-d chunks instead of a lock-free linked list of tranfer batches. This change simplifies the code, makes the allocator more 'hardened', and will allow simpler code to release RAM to OS. This may also slowdown malloc stress tests due to lock contension, but I did not observe noticeable slowdown on various real multi-threaded benchmarks.
llvm-svn: 279664
2016-08-24 21:20:10 +00:00
Vitaly Buka 769ec705bc Revert r279572 "[sanitizer] change the 64-bit..." because of failures on ubsan
This reverts commit r279572 and r279595.

llvm-svn: 279643
2016-08-24 17:40:29 +00:00
Kostya Serebryany f46d50e360 [sanitizer] change the 64-bit allocator to use a single array for free-d chunks instead of a lock-free linked list of tranfer batches. This change simplifies the code, makes the allocator more 'hardened', and will allow simpler code to release RAM to OS. This may also slowdown malloc stress tests due to lock contension, but I did not observe noticeable slowdown on various real multi-threaded benchmarks.
llvm-svn: 279572
2016-08-23 21:19:47 +00:00
Kostya Serebryany 0549a39a17 [sanitizer] adding a threaded performance stress test for malloc (useful for manual analysis of malloc performance)
llvm-svn: 279570
2016-08-23 21:12:04 +00:00
Kostya Serebryany 1ab649649d [sanitizer] use 32-bit offset instead of 64-bit pointers in the 64-bit allocator's transfer batches. This saves 2x memory for the transfer batches (up to ~1.5% overall in some cases)
llvm-svn: 278179
2016-08-09 23:30:22 +00:00
Kostya Serebryany 4cd2845e6a [sanitizer] allocator: move TransferBatch into SizeClassAllocator64/SizeClassAllocator32 because we actually need different iplementations for the 64- and 32-bit case. NFC; the following patches will make the TransferBatch implementations differ
llvm-svn: 277899
2016-08-06 01:24:11 +00:00
Nico Weber 023bb97737 Disable a few more flaky asan64 tests.
llvm-svn: 277886
2016-08-05 22:51:10 +00:00
Nico Weber 0f6aad4ad2 Disable a few win asan64 tests that don't reliably pass, https://reviews.llvm.org/D23230
llvm-svn: 277882
2016-08-05 22:36:30 +00:00
Etienne Bergeron 27eb6d521e [compiler-rt] Fix memory allocator for dynamic address space
Summary:
The sanitizer allocators can works with a dynamic address space
(i.e. specified with ~0ULL).

Unfortunately, the code was broken on GetMetadata and GetChunkIdx.

The current patch is moving the Win64 memory test to a dynamic
address space. There is a migration to move every concept to a
dynamic address space on windows.

To have a better coverage, the unittest are now testing
dynamic address space on other platforms too.

Reviewers: rnk, kcc

Subscribers: kubabrecka, dberris, llvm-commits, chrisha

Differential Revision: https://reviews.llvm.org/D23170

llvm-svn: 277745
2016-08-04 18:15:38 +00:00
Dimitry Andric 75534e23d0 XFAIL one sanitizer symbolizer test for FreeBSD
Summary:
Due to a QoI issuse in FreeBSD's libcxxrt-based demangler, one sanitizer
symbolizer test consistently appears to fail:

    Value of: DemangleSwiftAndCXX("foo")
      Actual: "float"
    Expected: "foo"

This is because libcxxrt's __cxa_demangle() incorrectly demangles the "foo"
identifier to "float".  It should return an error instead.

For now, XFAIL this particular test for FreeBSD, until we can fix libcxxrt
properly (which might take some time to coordinate with upstream).

Reviewers: rnk, zaks.anna, emaste

Subscribers: emaste, llvm-commits, kubabrecka

Differential Revision: https://reviews.llvm.org/D23001

llvm-svn: 277297
2016-07-31 19:27:46 +00:00
Reid Kleckner cb42ea0b4f [sanitizer] Try to fix LargeMmapAllocator test on Windows
This test attempts to allocate 100 512MB aligned pages of memory. This
is implemented in the usual way by allocating size + alignment bytes and
aligning the result. As a result, this test allocates 51.2GB of memory.
Windows allocates swap for all memory allocated, and our bots do not
have this much swap available.

Avoid the failure by using a more reasonable alignment, like 16MB, as we
do on 32-bit.

llvm-svn: 276779
2016-07-26 17:59:09 +00:00
Reid Kleckner ff3ea5f4f8 Try to fix more Windows portability issues in sanitizer tests
Add a %stdcxx11 lit substitution for -std=c++11. Windows defaults to
-std=c++14 when VS 2015 is used because the STL requires it. Harcoding
-std=c++11 in the ASan tests actually downgrades the C++ standard level,
leading to test failures.

Relax a FileCheck pattern in use-after-scope-types.cc.

Disable the sanitizer_common OOM tests. They fail on bots with low swap,
and cause other concurrently running tests to OOM.

llvm-svn: 276454
2016-07-22 18:41:22 +00:00