Commit Graph

548 Commits

Author SHA1 Message Date
George Karpenkov 50dd3fe903 [sanitizers] Sanitizer tests CMake clean up: try #2
This patch addresses two issues:

Most of the time, hacks with `if/else` in order to get support for
multi-configuration builds are superfluous.
The variable `CMAKE_CFG_INTDIR` was created precisely for this purpose: it
expands to `.`  on all single-configuration builds, and to a configuration
name otherwise.
The `if/else` hacks for the library name generation should also not be
done, as CMake has `TARGET_FILE` generator expression precisely for this
purpose, as it expands to the exact filename of the resulting target.

Differential Revision: https://reviews.llvm.org/D35952

llvm-svn: 309341
2017-07-28 00:50:56 +00:00
George Karpenkov 1bf535daae Revert "[sanitizers] Sanitizer tests CMake clean up"
This reverts commit 0ab44db2aa1cd3710355ad79b04f954ce68c0b3a.

Fails on some bots, reverting until I can fix it.

llvm-svn: 309318
2017-07-27 20:44:33 +00:00
George Karpenkov 165a1edc85 [sanitizers] Sanitizer tests CMake clean up
This patch addresses two issues:

Most of the time, hacks with `if/else` in order to get support for
multi-configuration builds are superfluous.
The variable `CMAKE_CFG_INTDIR` was created precisely for this purpose: it
expands to `.`  on all single-configuration builds, and to a configuration
name otherwise.
The `if/else` hacks for the library name generation should also not be
done, as CMake has `TARGET_FILE` generator expression precisely for this
purpose, as it expands to the exact filename of the resulting target.

Differential Revision: https://reviews.llvm.org/D35952

llvm-svn: 309306
2017-07-27 18:40:38 +00:00
Vitaly Buka d4abe9e7d3 [sanitizer_common] Move filesystem-related code out of sanitizer_common.cc
Summary:
This is a pure refactoring change.  It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}.  This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.

Submitted on behalf of Roland McGrath.

Reviewers: kcc, eugenis, alekseyshl

Reviewed By: alekseyshl

Subscribers: vitalybuka, llvm-commits, kubamracek, mgorny, phosek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D35591

llvm-svn: 308819
2017-07-22 01:46:40 +00:00
Vitaly Buka 094684c155 Revert "[sanitizer_common] Move filesystem-related code out of sanitizer_common.cc"
Breaks Windows build.

This reverts commit r308640.

llvm-svn: 308648
2017-07-20 18:24:45 +00:00
Alex Shlyapnikov 89e85d1717 [sanitizer_common] Move filesystem-related code out of sanitizer_common.cc
This is a pure refactoring change. It just moves code that is
related to filesystem operations from sanitizer_common.{cc,h} to
sanitizer_file.{cc,h}. This makes it cleaner to disable the
filesystem-related code for a new port that doesn't want it.

Commiting for mcgrathr.

Reviewers: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35591

llvm-svn: 308640
2017-07-20 17:48:42 +00:00
Vitaly Buka 50648af623 [compiler-rt] Fix fix format specifies type in test
llvm-svn: 308117
2017-07-16 00:17:11 +00:00
Alex Shlyapnikov 01676883cd [Sanitizers] 64 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator64 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow
allocator_may_return_null=1 policy
(LargeMmapAllocator: D34243, SizeClassAllocator64: D34433).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34540

llvm-svn: 306342
2017-06-26 22:54:10 +00:00
Alex Shlyapnikov f3cc7cc3d8 [Sanitizers] 32 bit allocator respects allocator_may_return_null flag
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).

Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34433

llvm-svn: 305972
2017-06-22 00:02:37 +00:00
Kostya Kortchinsky 2b053b1c28 [sanitizer] Add a function to gather random bytes
Summary:
AFAICT compiler-rt doesn't have a function that would return 'good' random
bytes to seed a PRNG. Currently, the `SizeClassAllocator64` uses addresses
returned by `mmap` to seed its PRNG, which is not ideal, and  
`SizeClassAllocator32` doesn't benefit from the entropy offered by its 64-bit
counterpart address space, so right now it has nothing. This function aims at
solving this, allowing to implement good 32-bit chunk randomization. Scudo also
has a function that does this for Cookie purposes, which would go away in a
later CL once this lands.

This function will try the `getrandom` syscall if available, and fallback to
`/dev/urandom` if not.

Unfortunately, I do not have a way to implement and test a Mac and Windows
version, so those are unimplemented as of now. Note that `kRandomShuffleChunks`
is only used on Linux for now.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: zturner, rnk, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34412

llvm-svn: 305922
2017-06-21 15:56:03 +00:00
Alex Shlyapnikov ccab11b0e8 [Sanitizers] Move cached allocator_may_return_null flag to sanitizer_allocator
Summary:
Move cached allocator_may_return_null flag to sanitizer_allocator.cc and
provide API to consolidate and unify the behavior of all specific allocators.

Make all sanitizers using CombinedAllocator to follow
AllocatorReturnNullOrDieOnOOM() rules to behave the same way when OOM
happens.

When OOM happens, turn allocator_out_of_memory flag on regardless of
allocator_may_return_null flag value (it used to not to be set when
allocator_may_return_null == true).

release_to_os_interval_ms and rss_limit_exceeded will likely be moved to
sanitizer_allocator.cc too (later).

Reviewers: eugenis

Subscribers: srhines, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D34310

llvm-svn: 305858
2017-06-20 21:23:02 +00:00
Kostya Kortchinsky 77f30c9c31 [sanitizer] Reverting D34152
Summary:
This broke thread_local_quarantine_pthread_join.cc on some architectures, due
to the overhead of the stashed regions. Reverting while figuring out the best
way to deal with it.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34213

llvm-svn: 305404
2017-06-14 17:32:26 +00:00
Kostya Kortchinsky eca926ab3a [sanitizer] MmapAlignedOrDie changes to reduce fragmentation
Summary:
The reasoning behind this change is explained in D33454, which unfortunately
broke the Windows version (due to the platform not supporting partial unmapping
of a memory region).

This new approach changes `MmapAlignedOrDie` to allow for the specification of
a `padding_chunk`. If non-null, and the initial allocation is aligned, this
padding chunk will hold the address of the extra memory (of `alignment` bytes).
This allows `AllocateRegion` to get 2 regions if the memory is aligned
properly, and thus help reduce fragmentation (and saves on unmapping
operations). As with the initial D33454, we use a stash in the 32-bit Primary
to hold those extra regions and return them on the fast-path.

The Windows version of `MmapAlignedOrDie` will always return a 0
`padding_chunk` if one was requested.

Reviewers: alekseyshl, dvyukov, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D34152

llvm-svn: 305391
2017-06-14 15:32:17 +00:00
Kostya Serebryany 124c2ae4fa [sanitizer-coverage] nuke more stale code
llvm-svn: 304508
2017-06-02 01:17:04 +00:00
Pierre Gousseau 183d1368f3 [asan] Add strndup/__strndup interceptors.
Recommit of r302781 with Vitaly Buka's fix for non zero terminated strings.

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 304399
2017-06-01 09:37:22 +00:00
Vitaly Buka a05da1fca9 [compiler-rt] Replace allow_user_segv_handler=0 with kHandleSignalExclusive
Summary:
allow_user_segv_handler had confusing name did not allow to control behavior for
signals separately.

Reviewers: eugenis, alekseyshl, kcc

Subscribers: llvm-commits, dberris, kubamracek

Differential Revision: https://reviews.llvm.org/D33371

llvm-svn: 303941
2017-05-25 23:42:33 +00:00
Vitaly Buka 8018780d39 [compiler-rt] Switch handle_<signal> flags from bool to enum.
Summary: We are going to make it tri-state and remove allow_user_segv_handler.

Reviewers: eugenis, alekseys, kcc

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D33159

llvm-svn: 303464
2017-05-19 22:37:16 +00:00
Vitaly Buka 5934882576 [compiler-rt] Add negative test for boolean flags.
Reviewers: eugenis, alekseyshl

Subscribers: kubamracek, dberris, llvm-commits

Differential Revision: https://reviews.llvm.org/D33344

llvm-svn: 303462
2017-05-19 22:37:13 +00:00
Simon Dardis 657899bad1 [compiler-rt][cmake] Build unit tests conditionally with _FILE_OFFSET_BITS=64 and _LARGEFILE_SOURCE
The sanitizer library unit tests for libc can get a different definition
of 'struct stat' to what the sanitizer library is built with for certain
targets.

For MIPS the size element of 'struct stat' is after a macro guarded
explicit padding element.

This patch resolves any possible inconsistency by adding the same
_FILE_OFFSET_BITS=64 and _LARGE_SOURCE with the same
conditions as the sanitizer library to the build flags for the unit tests.

This resolves a recurring build failure on the MIPS buildbots due to
'struct stat' defintion differences.

Reviewers: slthakur

Differential Revision: https://reviews.llvm.org/D33131

llvm-svn: 303350
2017-05-18 13:19:35 +00:00
Daniel Jasper f97310fb7a Revert r302781 and subsequent attempts to disable part of it.
The Msan unit tests are still broken and by this point, I think we
should start over.

llvm-svn: 303339
2017-05-18 09:31:37 +00:00
Kostya Serebryany 9aa8ef8504 temporary disable stndup interceptor, due to r302781 being buggy
llvm-svn: 303324
2017-05-18 03:00:07 +00:00
Kostya Kortchinsky dc646a0889 [sanitizer] Change SizeClassAllocator32 to accept just one template
Summary:
With rL279771, SizeClassAllocator64 was changed to accept only one template
instead of 5, for the following reasons: "First, this will make the mangled
names shorter. Second, this will make adding more parameters simpler". This
patch mirrors that work for SizeClassAllocator32.

This is in preparation for introducing the randomization of chunks in the
32-bit SizeClassAllocator in a later patch.

Reviewers: kcc, alekseyshl, dvyukov

Reviewed By: alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D33141

llvm-svn: 303071
2017-05-15 14:47:19 +00:00
Pierre Gousseau 0550581070 [asan] Recommit of r301904: Add strndup/__strndup interceptors
Fix undeclared __interceptor_malloc in esan_interceptors.cc
Fix undeclared strnlen on OSX

Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 302781
2017-05-11 08:53:24 +00:00
Pierre Gousseau 1c5550671d Revert r301904 causing tsan test failure in x86_64-linux-autoconf
llvm-svn: 301909
2017-05-02 10:22:05 +00:00
Pierre Gousseau b7101479a8 [asan] Add strndup/__strndup interceptors if targeting linux.
Differential Revision: https://reviews.llvm.org/D31457

llvm-svn: 301904
2017-05-02 09:01:02 +00:00
Kostya Serebryany 263722f40f [asan] replace std::random_shuffle with std::shuffle in tests since std::random_shuffle is being deprecated in C++17; NFC
llvm-svn: 294370
2017-02-07 23:13:10 +00:00
Kuba Mracek 5c88271528 Fixup of r293882: Forgot to update sanitizer_thread_registry.test.cc
llvm-svn: 293884
2017-02-02 13:07:22 +00:00
Dimitry Andric f2ac1d1df2 Recommit: Stop intercepting some malloc-related functions on FreeBSD and
macOS

Summary:
In https://bugs.freebsd.org/215125 I was notified that some configure
scripts attempt to test for the Linux-specific `mallinfo` and `mallopt`
functions by compiling and linking small programs which references the
functions, and observing whether that results in errors.

FreeBSD and macOS do not have the `mallinfo` and `mallopt` functions, so
normally these tests would fail, but when sanitizers are enabled, they
incorrectly succeed, because the sanitizers define interceptors for
these functions.  This also applies to some other malloc-related
functions, such as `memalign`, `pvalloc` and `cfree`.

Fix this by not intercepting `mallinfo`, `mallopt`, `memalign`,
`pvalloc` and `cfree` for FreeBSD and macOS, in all sanitizers.

Also delete the non-functional `cfree` wrapper for Windows, to fix the
test cases on that platform.

Reviewers: emaste, kcc, rnk

Subscribers: timurrrr, eugenis, hans, joerg, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D27654

llvm-svn: 293536
2017-01-30 19:06:13 +00:00
Evgeniy Stepanov 8bf0633288 Revert "Stop intercepting some malloc-related functions on FreeBSD and macOS"
This reverts r293337, which breaks tests on Windows:

malloc-no-intercept-499eb7.o : error LNK2019: unresolved external symbol _mallinfo referenced in function _main

llvm-svn: 293346
2017-01-27 23:53:20 +00:00
Dimitry Andric 47e0ef3148 Stop intercepting some malloc-related functions on FreeBSD and macOS
Summary:
In https://bugs.freebsd.org/215125 I was notified that some configure
scripts attempt to test for the Linux-specific `mallinfo` and `mallopt`
functions by compiling and linking small programs which references the
functions, and observing whether that results in errors.

FreeBSD and macOS do not have the `mallinfo` and `mallopt` functions, so
normally these tests would fail, but when sanitizers are enabled, they
incorrectly succeed, because the sanitizers define interceptors for
these functions.  This also applies to some other malloc-related
functions, such as `memalign`, `pvalloc` and `cfree`.

Fix this by not intercepting `mallinfo`, `mallopt`, `memalign`,
`pvalloc` and `cfree` for FreeBSD and macOS, in all sanitizers.

Reviewers: emaste, kcc

Subscribers: hans, joerg, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D27654

llvm-svn: 293337
2017-01-27 22:19:11 +00:00
Alex Shlyapnikov a8ba9c8e66 Whenever reasonable, merge ASAN quarantine batches to save memory.
Summary:
There are cases when thread local quarantine drains almost empty
quarantine batches into the global quarantine. The current approach leaves
them almost empty, which might create a huge memory overhead (each batch
is 4K/8K, depends on bitness).

Reviewers: eugenis

Subscribers: kubabrecka, llvm-commits

Differential Revision: https://reviews.llvm.org/D28068

llvm-svn: 292525
2017-01-19 22:15:54 +00:00
Mike Aizatsky 9700acba11 Revert "[sancov] moving sancov rt to sancov/ directory"
This reverts commit https://reviews.llvm.org/rL291734
Reason: mac breakage
http://lab.llvm.org:8080/green//job/clang-stage1-configure-RA_build/28798/consoleFull#1657087648e9a0fee5-ebcc-4238-a641-c5aa112c323e

llvm-svn: 291736
2017-01-12 01:37:35 +00:00
Mike Aizatsky 875572f358 [sancov] moving sancov rt to sancov/ directory
Subscribers: kubabrecka, mgorny

Differential Revision: https://reviews.llvm.org/D28541

llvm-svn: 291734
2017-01-12 01:19:34 +00:00
Francis Ricci 17781c71b0 Make cmake link flag naming consistent
Summary:
The build system was inconsistent in its naming conventions for
link flags. This patch changes all uses of LINKFLAGS to LINK_FLAGS,
for consistency with cmake's LINK_FLAGS property.

This patch should make it easier to search the source code for
uses of link flags, as well as providing the benefit of improved
style and consistency.

Reviewers: compnerd, beanz

Subscribers: kubabrecka, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D28506

llvm-svn: 291539
2017-01-10 04:33:04 +00:00
Francis Ricci ff567a8ba2 Enable weak hooks on darwin
Summary:
By default, darwin requires a definition for weak interface functions at
link time. Adding the '-U' link flag with each weak function allows these
weak interface functions to be used without definitions, which mirrors
behavior on linux and windows.

Reviewers: compnerd, eugenis

Subscribers: kubabrecka, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D28203

llvm-svn: 291417
2017-01-09 03:51:42 +00:00
Kuba Mracek b38f1ca2d5 [sanitizer] Use architecture/slice information when symbolizing fat Mach-O files on Darwin
This patch starts passing architecture information about a module to llvm-symbolizer and into text reports. This fixes the longstanding x86_64/x86_64h mismatch issue on Darwin.

Differential Revision: https://reviews.llvm.org/D27390

llvm-svn: 291287
2017-01-06 21:45:05 +00:00
Evgeniy Stepanov f787cf7ee6 [asan] Fix handling of %m in printf interceptor.
llvm-svn: 290632
2016-12-27 22:14:03 +00:00
Kuba Mracek 3eb98a1318 [sanitizer] Track architecture and UUID of modules in LoadedModule
When we enumerate loaded modules, we only track the module name and base address, which then has several problems on macOS. Dylibs and executables often have several architecture slices and not storing which architecture/UUID is actually loaded creates problems with symbolication: A file path + offset isn't enough to correctly symbolicate, since the offset can be valid in multiple slices. This is especially common for Haswell+ X86_64 machines, where x86_64h slices are preferred, but if one is not available, a regular x86_64 is loaded instead. But the same issue exists for i386 vs. x86_64 as well.

This patch adds tracking of arch and UUID for each LoadedModule. At this point, this information isn't used in reports, but this is the first step. The goal is to correctly identify which slice is loaded in symbolication, and also to output this information in reports so that we can tell which exact slices were loaded in post-mortem analysis.

Differential Revision: https://reviews.llvm.org/D26632

llvm-svn: 288537
2016-12-02 21:27:14 +00:00
Evgeniy Stepanov d3305afc75 Return memory to OS right after free (not in the async thread).
Summary:
In order to avoid starting a separate thread to return unused memory to
the system (the thread interferes with process startup on Android,
Zygota waits for all threads to exit before fork, but this thread never
exits), try to return it right after free.

Reviewers: eugenis

Subscribers: cryptoad, filcab, danalbert, kubabrecka, llvm-commits

Patch by Aleksey Shlyapnikov.

Differential Revision: https://reviews.llvm.org/D27003

llvm-svn: 288091
2016-11-29 00:22:50 +00:00
Mike Aizatsky 7594ec3355 rename InternalBinarySearch to InternalLowerBound
Summary: The new name better corresponds to its logic.

Reviewers: kcc

Subscribers: kubabrecka

Differential Revision: https://reviews.llvm.org/D26821

llvm-svn: 287377
2016-11-18 20:48:52 +00:00
Mike Aizatsky ff3bdbac35 fixing binary search for cases when element is not in array
Subscribers: kubabrecka

Differential Revision: https://reviews.llvm.org/D26707

llvm-svn: 287078
2016-11-16 04:03:27 +00:00
Anna Zaks be7ae6684f [sanitizers] Set Darwin specific linker and compiler flags for all tests
Looks like we are missing these flags only in tsan and sanitizer-common.

This results in linker warnings in some settings as it can cause the Unit
tests to be built with a different SDK version than that was used to build
the runtime. For example, we are not setting the minimal deployment target
on the tests but are setting the minimal deployment target for the sanitizer
library, which leads to the following warning on some bots: ld: warning:
object file (sanitizer_posix_test.cc.i386.o) was built for newer OSX version
(10.12) than being linked (10.11).

Differential Revision: https://reviews.llvm.org/D25860  https://reviews.llvm.org/D25352

llvm-svn: 285255
2016-10-26 23:23:38 +00:00
Evgeniy Stepanov 8652d6ac05 [asan] Reenable 64-bit allocator on android/aarch64.
This is a re-commit of r281371, with updated tests.

llvm-svn: 281674
2016-09-15 22:34:53 +00:00
Anna Zaks 691644f3ca [compiler-rt] Do not introduce __sanitizer namespace globally
The definitions in sanitizer_common may conflict with definitions from system headers because:

The runtime includes the system headers after the project headers (as per LLVM coding guidelines).
lib/sanitizer_common/sanitizer_internal_defs.h pollutes the namespace of everything defined after it, which is all/most of the sanitizer .h and .cc files and the included system headers with: using namespace __sanitizer; // NOLINT
This patch solves the problem by introducing the namespace only within the sanitizer namespaces as proposed by Dmitry.

Differential Revision: https://reviews.llvm.org/D21947

llvm-svn: 281657
2016-09-15 21:02:18 +00:00
Reid Kleckner 110ecbd336 [sanitizer] Test the allocator with the ASan win64 memory constants
These got out of sync and the tests were failing for me locally.  We
assume a 47 bit address space in ASan, so we should do the same in the
tests.

llvm-svn: 281622
2016-09-15 16:00:46 +00:00
Kostya Serebryany 58560a05ad [sanitizer] fix an allocator bug where the allocated memory may overlap with the free array (kudos to Kostya Korcthinsky). Also make sure that the allocator does not mmap more than requested. Test both.
llvm-svn: 281103
2016-09-09 21:42:33 +00:00
Kostya Serebryany 2700bd831a [sanitizer] extend SizeClassMap to take more template parameters, add VeryCompactSizeClassMap for testing purposes
llvm-svn: 280266
2016-08-31 17:52:55 +00:00
Kostya Serebryany 7c5ae7cbc6 [sanitizer] enable random shuffling the memory chunks inside the allocator, under a flag. Set this flag for the scudo allocator, add a test.
llvm-svn: 279793
2016-08-26 00:06:03 +00:00
Kostya Serebryany 15647b17f3 [sanitizer] change SizeClassAllocator64 to accept just one template parameter instead of 5. First, this will make the mangled names shorter. Second, this will make adding more parameters simpler.
llvm-svn: 279771
2016-08-25 20:23:08 +00:00
Kostya Serebryany 8e7ea9ddb5 [sanitizer] re-apply r279572 and r279595 reverted in r279643: change the 64-bit allocator to use a single array for free-d chunks instead of a lock-free linked list of tranfer batches. This change simplifies the code, makes the allocator more 'hardened', and will allow simpler code to release RAM to OS. This may also slowdown malloc stress tests due to lock contension, but I did not observe noticeable slowdown on various real multi-threaded benchmarks.
llvm-svn: 279664
2016-08-24 21:20:10 +00:00
Vitaly Buka 769ec705bc Revert r279572 "[sanitizer] change the 64-bit..." because of failures on ubsan
This reverts commit r279572 and r279595.

llvm-svn: 279643
2016-08-24 17:40:29 +00:00
Kostya Serebryany f46d50e360 [sanitizer] change the 64-bit allocator to use a single array for free-d chunks instead of a lock-free linked list of tranfer batches. This change simplifies the code, makes the allocator more 'hardened', and will allow simpler code to release RAM to OS. This may also slowdown malloc stress tests due to lock contension, but I did not observe noticeable slowdown on various real multi-threaded benchmarks.
llvm-svn: 279572
2016-08-23 21:19:47 +00:00
Kostya Serebryany 0549a39a17 [sanitizer] adding a threaded performance stress test for malloc (useful for manual analysis of malloc performance)
llvm-svn: 279570
2016-08-23 21:12:04 +00:00
Kostya Serebryany 1ab649649d [sanitizer] use 32-bit offset instead of 64-bit pointers in the 64-bit allocator's transfer batches. This saves 2x memory for the transfer batches (up to ~1.5% overall in some cases)
llvm-svn: 278179
2016-08-09 23:30:22 +00:00
Kostya Serebryany 4cd2845e6a [sanitizer] allocator: move TransferBatch into SizeClassAllocator64/SizeClassAllocator32 because we actually need different iplementations for the 64- and 32-bit case. NFC; the following patches will make the TransferBatch implementations differ
llvm-svn: 277899
2016-08-06 01:24:11 +00:00
Nico Weber 023bb97737 Disable a few more flaky asan64 tests.
llvm-svn: 277886
2016-08-05 22:51:10 +00:00
Nico Weber 0f6aad4ad2 Disable a few win asan64 tests that don't reliably pass, https://reviews.llvm.org/D23230
llvm-svn: 277882
2016-08-05 22:36:30 +00:00
Etienne Bergeron 27eb6d521e [compiler-rt] Fix memory allocator for dynamic address space
Summary:
The sanitizer allocators can works with a dynamic address space
(i.e. specified with ~0ULL).

Unfortunately, the code was broken on GetMetadata and GetChunkIdx.

The current patch is moving the Win64 memory test to a dynamic
address space. There is a migration to move every concept to a
dynamic address space on windows.

To have a better coverage, the unittest are now testing
dynamic address space on other platforms too.

Reviewers: rnk, kcc

Subscribers: kubabrecka, dberris, llvm-commits, chrisha

Differential Revision: https://reviews.llvm.org/D23170

llvm-svn: 277745
2016-08-04 18:15:38 +00:00
Dimitry Andric 75534e23d0 XFAIL one sanitizer symbolizer test for FreeBSD
Summary:
Due to a QoI issuse in FreeBSD's libcxxrt-based demangler, one sanitizer
symbolizer test consistently appears to fail:

    Value of: DemangleSwiftAndCXX("foo")
      Actual: "float"
    Expected: "foo"

This is because libcxxrt's __cxa_demangle() incorrectly demangles the "foo"
identifier to "float".  It should return an error instead.

For now, XFAIL this particular test for FreeBSD, until we can fix libcxxrt
properly (which might take some time to coordinate with upstream).

Reviewers: rnk, zaks.anna, emaste

Subscribers: emaste, llvm-commits, kubabrecka

Differential Revision: https://reviews.llvm.org/D23001

llvm-svn: 277297
2016-07-31 19:27:46 +00:00
Reid Kleckner cb42ea0b4f [sanitizer] Try to fix LargeMmapAllocator test on Windows
This test attempts to allocate 100 512MB aligned pages of memory. This
is implemented in the usual way by allocating size + alignment bytes and
aligning the result. As a result, this test allocates 51.2GB of memory.
Windows allocates swap for all memory allocated, and our bots do not
have this much swap available.

Avoid the failure by using a more reasonable alignment, like 16MB, as we
do on 32-bit.

llvm-svn: 276779
2016-07-26 17:59:09 +00:00
Reid Kleckner ff3ea5f4f8 Try to fix more Windows portability issues in sanitizer tests
Add a %stdcxx11 lit substitution for -std=c++11. Windows defaults to
-std=c++14 when VS 2015 is used because the STL requires it. Harcoding
-std=c++11 in the ASan tests actually downgrades the C++ standard level,
leading to test failures.

Relax a FileCheck pattern in use-after-scope-types.cc.

Disable the sanitizer_common OOM tests. They fail on bots with low swap,
and cause other concurrently running tests to OOM.

llvm-svn: 276454
2016-07-22 18:41:22 +00:00
Kostya Serebryany bfd69e0580 [sanitizer] allocator: introduce kUseSeparateSizeClassForBatch (false by default). When true, it will cause all TransferBatches to be allocated on a separate dedicated size class, which improves security and may potentially simplify memory reclamation. However in the current state this may cause up to 3% extra memory usage. Subsequent changes should bring this overhead down
llvm-svn: 276377
2016-07-22 01:13:13 +00:00
Kostya Serebryany 35eeea707e [sanitizer] allocator: remove kPopulateSize and only use SizeClassMap::MaxCached; ensure that TransferBatch size is a power of two, refactor TransferBatch creation/destruction into separate functions.
llvm-svn: 276318
2016-07-21 18:47:53 +00:00
Kostya Serebryany 9835a81529 [sanitizers] split sanitizer_allocator.h into a number of smaller .h files; NFC
llvm-svn: 276195
2016-07-20 22:06:41 +00:00
Reid Kleckner cd03f6f4f6 [sanitizers] Allocate 12MB for stack instead of 134MB
The thread registry test was failing to allocate 25 threads with stack
size 134MB, which is pretty reasonable.

Also print the error code in our pthread wrappers in case this happens
again.

llvm-svn: 275209
2016-07-12 20:10:28 +00:00
Etienne Bergeron ab42f4ddba [compiler-rt] Fix VisualStudio virtual folders layout
Summary:
This patch is a refactoring of the way cmake 'targets' are grouped.
It won't affect non-UI cmake-generators.

Clang/LLVM are using a structured way to group targets which ease
navigation through Visual Studio UI. The Compiler-RT projects
differ from the way Clang/LLVM are grouping targets.

This patch doesn't contain behavior changes.

Reviewers: kubabrecka, rnk

Subscribers: wang0109, llvm-commits, kubabrecka, chrisha

Differential Revision: http://reviews.llvm.org/D21952

llvm-svn: 275111
2016-07-11 21:51:56 +00:00
Etienne Bergeron fb5a24b4f0 [compiler-rt] Fix minimal stack reserved for unittest on MSVC build
Summary:
Some unittest were not able to run in 64-bit because they need more than
2MB of stack and the default allocated stack with MSVC linker is 1MB.

Reviewers: rnk

Subscribers: wang0109, chrisha, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D22101

llvm-svn: 274785
2016-07-07 18:41:08 +00:00
Etienne Bergeron 9654f2afe3 [compiler-rt] Fix sanitizer memory allocator on win64.
Summary:
This patch is fixing unittests for sanitizer memory allocator.

There was two issues:
  1) The VirtualAlloc can't reserve twice a memory range.
     The memory space used by the SizeClass allocator is reserved
     with NoAccess and pages are commited on demand (using MmapFixedOrDie).

  2) The address space is allocated using two VirtualAlloc calls. The first one
     for the memory space, the second one for the AdditionnalSpace (after).

     On windows, they need to be freed separately.

Reviewers: rnk

Subscribers: llvm-commits, wang0109, kubabrecka, chrisha

Differential Revision: http://reviews.llvm.org/D21900

llvm-svn: 274772
2016-07-07 17:44:08 +00:00
Etienne Bergeron ec4b0e4ef4 [compiler-rt] Fix AllocatorCacheDeallocNewThread unittest
Summary:
The unittest was not freeing the mapped memory.


```
Repeating all tests (iteration 1) . . .

Note: Google Test filter = Allocator.AllocatorCacheDeallocNewThread
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Allocator
[ RUN      ] Allocator.AllocatorCacheDeallocNewThread
[       OK ] Allocator.AllocatorCacheDeallocNewThread (3 ms)
[----------] 1 test from Allocator (4 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (5 ms total)
[  PASSED  ] 1 test.

Repeating all tests (iteration 2) . . .

Note: Google Test filter = Allocator.AllocatorCacheDeallocNewThread
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from Allocator
[ RUN      ] Allocator.AllocatorCacheDeallocNewThread
==4504==WARNING: SanitizerTool failed to mprotect 0x010000003000 (1099511640064) bytes at 0x010000000000 (error code: 48
7)
==4504==Sanitizer CHECK failed: D:/src/llvm/llvm/projects/compiler-rt/lib\sanitizer_common/sanitizer_allocator.h:329 ((kSpaceBeg)) == ((reinterpret_cast<uptr>( MmapFixedNoAccess(kSpaceBeg, TotalSpaceSize)))) (1099511627776, 0)
```

Reviewers: rnk

Subscribers: llvm-commits, kubabrecka, chrisha

Differential Revision: http://reviews.llvm.org/D22094

llvm-svn: 274764
2016-07-07 15:52:28 +00:00
Etienne Bergeron 5220d4e760 [compiler-rt] Re-active unittest previously broken on windows.
Summary: These tests are working fine.

Reviewers: rnk

Subscribers: kubabrecka, wang0109, llvm-commits, chrisha

Differential Revision: http://reviews.llvm.org/D21905

llvm-svn: 274265
2016-06-30 20:06:29 +00:00
Etienne Bergeron fb4ba14b70 [compiler-rt] Fix TLS resource leaking in unittest
Summary:
The thread specific key wasn't not released.
Running the unittest in loop will fail after 1024 iteraions.

```
 ./projects/compiler-rt/lib/sanitizer_common/tests/Sanitizer-i386-Test --gtest_filter=SanitizerCommon.PthreadDestructorIterations --gtest_repeat=2000 --gtest_break_on_failure
```

```
Repeating all tests (iteration 1023) . . .

Note: Google Test filter = SanitizerCommon.PthreadDestructorIterations
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN      ] SanitizerCommon.PthreadDestructorIterations
/usr/local/google/home/etienneb/llvm/llvm/projects/compiler-rt/lib/sanitizer_common/tests/sanitizer_posix_test.cc:54: Failure
Value of: pthread_key_create(&key, &destructor)
  Actual: 11
Expected: 0
Aborted (core dumped)
```

Reviewers: rnk

Subscribers: kubabrecka, llvm-commits, chrisha

Differential Revision: http://reviews.llvm.org/D21902

llvm-svn: 274264
2016-06-30 19:56:14 +00:00
Etienne Bergeron 2d1938be0d [compiler-rt] Fix broken (flaky) unittests based on FlagParser.
Summary:
The FlagParser is populating a static global class with the
unrecognized flags when parsing. That global class has a 
dcheck that limit the number of unrecognized flag to 20.

```
class UnknownFlags {
  static const int kMaxUnknownFlags = 20;
  const char *unknown_flags_[kMaxUnknownFlags];
  int n_unknown_flags_;

  [...]

  void Report() {
    if (!n_unknown_flags_) return;
    Printf("WARNING: found %d unrecognized flag(s):\n", n_unknown_flags_);
    for (int i = 0; i < n_unknown_flags_; ++i)
      Printf("    %s\n", unknown_flags_[i]);
    n_unknown_flags_ = 0;
  }
};

UnknownFlags unknown_flags;
```

Unittests based on that class must reset the counter 'n_unknown_flags_' or
the next usage of that class may fail arbitrary. This can be done by
reporting the pending unknown flags.

Reviewers: rnk

Subscribers: llvm-commits, wang0109, kubabrecka, chrisha

Differential Revision: http://reviews.llvm.org/D21896

llvm-svn: 274234
2016-06-30 15:11:13 +00:00
Etienne Bergeron c8ee9d9614 [compiler-rt] Reset global variables in ThreadRegistryThreadedTest
Summary:
The unittest 'ThreadRegistryThreadedTest' is failing when running in loop.
There are global variables that need to be cleared.

To repro:
```
projects\compiler-rt\lib\sanitizer_common\tests\Release\Sanitizer-x86_64-Test.exe --gtest_filter=SanitizerCommon.ThreadRegistryThreadedTest  --gtest_repeat=2
```

Output:
```
Repeating all tests (iteration 1) . . .

Note: Google Test filter = SanitizerCommon.ThreadRegistryThreadedTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN      ] SanitizerCommon.ThreadRegistryThreadedTest
[       OK ] SanitizerCommon.ThreadRegistryThreadedTest (1 ms)
[----------] 1 test from SanitizerCommon (1 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (2 ms total)
[  PASSED  ] 1 test.

Repeating all tests (iteration 2) . . .

Note: Google Test filter = SanitizerCommon.ThreadRegistryThreadedTest
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN      ] SanitizerCommon.ThreadRegistryThreadedTest
C:/src/llvm/llvm/projects/compiler-rt/lib/sanitizer_common/tests/sanitizer_thread_registry_test.cc(216): error: Value of: num_created[0]
  Actual: 2
Expected: 1
C:/src/llvm/llvm/projects/compiler-rt/lib/sanitizer_common/tests/sanitizer_thread_registry_test.cc(217): error: Value of: num_started[0]
  Actual: 2
Expected: 1
C:/src/llvm/llvm/projects/compiler-rt/lib/sanitizer_common/tests/sanitizer_thread_registry_test.cc(220): error: Value of: num_created[i]
[...]
[  FAILED  ] SanitizerCommon.ThreadRegistryThreadedTest (294 ms)
[----------] 1 test from SanitizerCommon (294 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (299 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] SanitizerCommon.ThreadRegistryThreadedTest

 1 FAILED TEST
```

Reviewers: rnk

Subscribers: llvm-commits, wang0109, kubabrecka, chrisha

Differential Revision: http://reviews.llvm.org/D21886

llvm-svn: 274231
2016-06-30 14:37:26 +00:00
Etienne Bergeron 94500cb532 [compiler-rt] Fix Sanitizer-common Unittests on win64
Summary:
This patch is fixing unittests that are broken on windows (64-bits).

Tests under 'SANITIZER_CAN_USE_ALLOCATOR64' are disabled.
A custom allocator for windows based on WinHeap API will replace these tests.

Tested on Win32/Win64 (Ninja and MSVC).
Tested on Linux 32-bit/64-bit clang.

```
C:\src\llvm\build64\projects\compiler-rt>lib\sanitizer_common\tests\Release\Sanitizer-x86_64-Test.exe
[==========] Running 101 tests from 12 test cases.
[----------] Global test environment set-up.
[----------] 51 tests from SanitizerCommon
[ RUN      ] SanitizerCommon.DefaultSizeClassMap
[       OK ] SanitizerCommon.DefaultSizeClassMap (1 ms)
[ RUN      ] SanitizerCommon.CompactSizeClassMap
[       OK ] SanitizerCommon.CompactSizeClassMap (1 ms)
[ RUN      ] SanitizerCommon.InternalSizeClassMap
[       OK ] SanitizerCommon.InternalSizeClassMap (1 ms)
[ RUN      ] SanitizerCommon.SizeClassAllocator32Compact
[       OK ] SanitizerCommon.SizeClassAllocator32Compact (828 ms)
[ RUN      ] SanitizerCommon.SizeClassAllocator32CompactMetadataStress
[       OK ] SanitizerCommon.SizeClassAllocator32CompactMetadataStress (914 ms)
[ RUN      ] SanitizerCommon.SizeClassAllocator32MapUnmapCallback

[...]

[----------] 4 tests from Symbolizer
[ RUN      ] Symbolizer.ExtractToken
[       OK ] Symbolizer.ExtractToken (0 ms)
[ RUN      ] Symbolizer.ExtractInt
[       OK ] Symbolizer.ExtractInt (0 ms)
[ RUN      ] Symbolizer.ExtractUptr
[       OK ] Symbolizer.ExtractUptr (0 ms)
[ RUN      ] Symbolizer.ExtractTokenUpToDelimiter
[       OK ] Symbolizer.ExtractTokenUpToDelimiter (0 ms)
[----------] 4 tests from Symbolizer (24 ms total)

[----------] Global test environment tear-down
[==========] 101 tests from 12 test cases ran. (5090 ms total)
[  PASSED  ] 101 tests.
```

Reviewers: rnk

Subscribers: chrisha, wang0109, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21817

llvm-svn: 274174
2016-06-29 21:57:58 +00:00
Etienne Bergeron f1d4e44d61 [compiler-rt] Fix passing debug information to unittest.
Summary:
On windows, the debug information was not present in the unittest executables,
which make them hard to debug.

The Sanitizer Unittests are compiled with a local clang build.
The link pass is also done by calling clang.

This pass is adding the appropriate flags to bring the right debug information
to these executables.

Reviewers: rnk

Subscribers: kubabrecka, llvm-commits, wang0109, chrisha

Differential Revision: http://reviews.llvm.org/D21838

llvm-svn: 274153
2016-06-29 19:58:02 +00:00
Kuba Brecka 02478f4166 [sanitizer] Stop unwinding the stack when a close-to-zero PC is found
On OS X, we often get stack trace in a report that ends with a 0x0 frame.  To get rid of it, let's trim the stack trace when we find a close-to-zero value, which is obviously not a valid PC.

Differential Revision: http://reviews.llvm.org/D14656

llvm-svn: 273886
2016-06-27 15:32:18 +00:00
Etienne Bergeron 3df2879232 [compiler-rt] Fix multi-configuration output paths
Summary:
When using a multi-configuration build (i.e. MSVC) the output path where
libraries are dropped is incorrect.

Example:
```
C:\src\llvm\examples>d:\src\llvm\build\Release\bin\clang-cl.exe -fsanitize=address test.cc
LINK : fatal error LNK1181: cannot open input file 'd:\src\llvm\build\Release\bin\..\lib\clang\3.9.0\lib\windows\clang_rt.asan-i386.lib'
```

The dropped executable path contains the configuration 'Release':
```
'd:\src\llvm\build\Release\bin\..\lib\clang\3.9.0\lib\windows\Release\clang_rt.asan-i386.lib'
```


The variable 'RUNTIME_OUTPUT_DIRECTORY' is used to specify the output directory.
But CMAKE is appending the current configuration (i.e. Debug, Release).

see: https://cmake.org/cmake/help/v3.0/prop_tgt/RUNTIME_OUTPUT_DIRECTORY.html
```
"Multi-configuration generators (VS, Xcode) append a per-configuration subdirectory to the specified directory."
```

To avoid this problem, the configuration specific variable must be set:
'RUNTIME_OUTPUT_DIRECTORY_DEBUG', 'RUNTIME_OUTPUT_DIRECTORY_RELEASE', and so on.

Reviewers: ddunbar, chapuni, rnk

Subscribers: kubabrecka, llvm-commits

Differential Revision: http://reviews.llvm.org/D20261

llvm-svn: 269658
2016-05-16 14:58:07 +00:00
Kostya Serebryany 3884f1a5bd [sanitizer] add a function MmapNoAccess that mmaps a protected region *somewhere*; use MmapNoAccess in the Allocator when SpaceBeg is not a constant. In this mode the allocator will be a bit more hardened
llvm-svn: 267256
2016-04-23 00:05:24 +00:00
Renato Golin aa673486c4 Revert "[sanitizer] Allow the sanitizer allocator to use a non-fixed address range. An allocator with a non-fixed address range will be attack-resistan. NFC for the sanitizers at this point."
This reverts commit r267094, because it broke a lot of MSAN tests in AArch64.
Being NFC and all, this needs some deeper investigation before it goes in again.

llvm-svn: 267136
2016-04-22 11:18:14 +00:00
Kostya Serebryany d55d0875d2 [sanitizer] Allow the sanitizer allocator to use a non-fixed address range. An allocator with a non-fixed address range will be attack-resistan. NFC for the sanitizers at this point.
llvm-svn: 267094
2016-04-22 01:08:54 +00:00
Kostya Serebryany e63db09bea [sanitizer] update the sanitizer_allocator_testlib
llvm-svn: 266934
2016-04-20 23:28:06 +00:00
Anna Zaks 8f8d3db940 [sanitizers] Fixup to r266494; the Unit test should not run on Windows
The test is failing on Windows because we do not have a definition for
DemangleSwiftAndCXX nor DemangleCXXABI, which I am replacing, on Windows.

llvm-svn: 266499
2016-04-16 00:32:16 +00:00
Anna Zaks 60469e2ac0 [sanitizers] Teach the internal demangler about Swift names
Add support for Swift names when symbolicating sanitizer traces. This is
now relevant since TSan and ASan support have been added to Swift on OS X.

Differential Revision: http://reviews.llvm.org/D19135

llvm-svn: 266494
2016-04-15 23:51:00 +00:00
Marcin Koscielnicki 0ffa9eaa4a [sanitizer] [SystemZ] Add virtual space size.
This teaches sanitizer_common about s390 and s390x virtual space size.
s390 is unusual in that it has 31-bit virtual space.

Differential Revision: http://reviews.llvm.org/D18896

llvm-svn: 266296
2016-04-14 12:56:15 +00:00
Mohit K. Bhakkad ac4bd45f02 [Compiler-rt] Fix ioctl codes for MIPS in sanitizer_ioctl_test
Reviewers: samsonov

Subscribers: jaydeep, sagar, llvm-commits

Differential Revision: http://reviews.llvm.org/D17985

llvm-svn: 263199
2016-03-11 05:57:20 +00:00
Alexey Samsonov 8e3cbde27d [Sanitizer] Introduce ListOfModules object and use it to replace GetListOfModules().
Summary:
This removes the hard limit on the number of loaded modules (used to be
16K), and makes it easier to use LoadedModules w/o causing a memory
leak: ListOfModules owns the modules, and makes sure to properly clean
them in destructor.

Remove filtering functionality that is only needed in one place (LSan).

Reviewers: aizatsky

Subscribers: llvm-commits, kcc

Differential Revision: http://reviews.llvm.org/D17470

llvm-svn: 261554
2016-02-22 18:52:51 +00:00
Zia Ansari 3cc265458f Fixed non-NULL terminating array bug in SanitizerCommon.StartSubprocessTest that caused them to fail sometimes and act flaky.
Differential Revision: http://reviews.llvm.org/D17228

llvm-svn: 260876
2016-02-15 05:11:51 +00:00
Mike Aizatsky adc0a67a55 using /system/bin/sh on android in tests.
Subscribers: tberghammer, danalbert, srhines

Differential Revision: http://reviews.llvm.org/D16654

llvm-svn: 259001
2016-01-27 23:58:23 +00:00
Mike Aizatsky 4a93316c8c [sanitizers] extracted process management functions
Differential Revision: http://reviews.llvm.org/D16546

llvm-svn: 258849
2016-01-26 20:10:01 +00:00
Mike Aizatsky 92f0f7c2f3 Revert "[sanitizers] extracted process management functions"
This reverts commit e5b34d5a2bf4c882bc647dd26a8cea2adc76f63c.

llvm-svn: 258713
2016-01-25 19:52:25 +00:00
Mike Aizatsky 53849fee26 [sanitizers] extracted process management functions
Differential Revision: http://reviews.llvm.org/D16542

llvm-svn: 258710
2016-01-25 19:43:52 +00:00
Sumanth Gundapaneni b76bf106b1 Fix the cross compilation of unit tests. NFC (second attempt)
With COMPILER_RT_INCLUDE_TESTS turned ON and in a cross compiling
environment, the unit tests fail to link. This patch does the following changes

>Rename COMPILER_RT_TEST_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS to reflect the 
way it's used.
>Add COMPILER_RT_TEST_COMPILER_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS so 
that cross-compiler would be able to build/compile the unit tests
>Add COMPILER_RT_UNITTEST_LINKFLAGS to COMPILER_RT_UNITTEST_CFLAGS so 
that cross-compiler would be able to link the unit tests (if needed)

Differential Revision: http://reviews.llvm.org/D16165

llvm-svn: 257783
2016-01-14 18:18:49 +00:00
Hans Wennborg 7b9d2b6c87 Revert r257686 "With COMPILER_RT_INCLUDE_TESTS turned ON and in a cross compiling"
This broke the build. For example, from
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/1191/steps/cmake%20stage%201/logs/stdio:

	-- Compiler-RT supported architectures: aarch64
	CMake Error at projects/compiler-rt/cmake/Modules/AddCompilerRT.cmake:170 (string):
		string sub-command REPLACE requires at least four arguments.
	Call Stack (most recent call first):
		projects/compiler-rt/lib/CMakeLists.txt:4 (include)

llvm-svn: 257694
2016-01-13 22:50:24 +00:00
Sumanth Gundapaneni 2d2f2b5c80 With COMPILER_RT_INCLUDE_TESTS turned ON and in a cross compiling
environment, the unit tests fail to link. This patch does the following changes

>Rename COMPILER_RT_TEST_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS to reflect the 
way it's used.
>Add COMPILER_RT_TEST_COMPILER_CFLAGS to COMPILER_RT_UNITTEST_CFLAGS so that 
cross-compiler would be able to build/compile the unit tests
>Add COMPILER_RT_UNITTEST_LINKFLAGS to COMPILER_RT_UNITTEST_CFLAGS so that 
cross-compiler would be able to link the unit tests (if needed)

Differential Revision:http://reviews.llvm.org/D15082

llvm-svn: 257686
2016-01-13 22:09:47 +00:00
Alexander Potapenko b6a2537c60 [libsanitizer] Fix bugs and wiki links to point to GitHub.
llvm-svn: 254738
2015-12-04 17:50:03 +00:00
Anna Zaks a44def276c [compiler-rt] Do not pull in min and max macros from windows.h
This should fix Windows buildbot breakage triggered by r253690.

llvm-svn: 253701
2015-11-20 20:18:06 +00:00
Anna Zaks c54cf07fd7 [compiler-rt] Add internal implementations for strlcat and strlcpy
The compiler-rt should make use of strlcpy() rather than strncpy(). Using internal_strncpy() may be fine with appropriate bounds checking or enforcement of nul-termination elsewhere, but it's just good practice these days to avoid using strncpy() in new code.

A patch by Jeremy Sequoia!

Differential Revision: http://reviews.llvm.org/D14714

llvm-svn: 253690
2015-11-20 18:42:05 +00:00
Anna Zaks 2ce9f9447c Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
of the report to stderr. This patch changes this behavior and writes the full
text of the report to syslog before we terminate the process. It also calls
os_trace (Activity Tracing available on OS X and iOS) with a message saying
that the report is available in syslog. This is useful, because this message
will be shown in the crash log.

For this to work, the patch makes sure we store the full report into
error_message_buffer unconditionally, and it also strips out ANSI escape
sequences from the report (they are used when producing colored reports).

I've initially tried to log to syslog during printing, which is done on Android
right now. The advantage is that if we crash during error reporting or the
produced error does not go through ScopedInErrorReport, we would still get a
(partial) message in the syslog. However, that solution is very problematic on
OS X. One issue is that the logging routine uses GCD, which may spawn a new
thread on its behalf. In many cases, the reporting logic locks threadRegistry,
which leads to deadlocks.

Reviewed at http://reviews.llvm.org/D13452

(In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
build failures on Linux.)

llvm-svn: 253688
2015-11-20 18:41:44 +00:00
Juergen Ributzka d7dcffaea4 Revert "Reapply: [asan] On OS X, log reports to syslog and os_trace"
Looks like this commit is deadlocking the ASAN tests on the green dragon bot
(http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA/).

llvm-svn: 252076
2015-11-04 21:03:12 +00:00
Anna Zaks 7920eb0b1f Reapply: [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
        of the report to stderr. This patch changes this behavior and writes the full
        text of the report to syslog before we terminate the process. It also calls
        os_trace (Activity Tracing available on OS X and iOS) with a message saying
        that the report is available in syslog. This is useful, because this message
        will be shown in the crash log.

        For this to work, the patch makes sure we store the full report into
        error_message_buffer unconditionally, and it also strips out ANSI escape
        sequences from the report (they are used when producing colored reports).

        I've initially tried to log to syslog during printing, which is done on Android
        right now. The advantage is that if we crash during error reporting or the
        produced error does not go through ScopedInErrorReport, we would still get a
        (partial) message in the syslog. However, that solution is very problematic on
        OS X. One issue is that the logging routine uses GCD, which may spawn a new
        thread on its behalf. In many cases, the reporting logic locks threadRegistry,
        which leads to deadlocks.

        Reviewed at http://reviews.llvm.org/D13452

        (In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
         build failures on Linux.)

llvm-svn: 251577
2015-10-28 23:18:44 +00:00
Anna Zaks 55fbb6d7f5 Revert "[asan] On OS X, log reports to syslog and os_trace"
This reverts commit 251447.

(Which caused failures on a Linux bot.)

llvm-svn: 251467
2015-10-27 23:15:05 +00:00
Anna Zaks 9a95c9a633 [asan] On OS X, log reports to syslog and os_trace
When ASan currently detects a bug, by default it will only print out the text
of the report to stderr. This patch changes this behavior and writes the full
text of the report to syslog before we terminate the process. It also calls
os_trace (Activity Tracing available on OS X and iOS) with a message saying
that the report is available in syslog. This is useful, because this message
will be shown in the crash log.

For this to work, the patch makes sure we store the full report into
error_message_buffer unconditionally, and it also strips out ANSI escape
sequences from the report (they are used when producing colored reports).

I've initially tried to log to syslog during printing, which is done on Android
right now. The advantage is that if we crash during error reporting or the
produced error does not go through ScopedInErrorReport, we would still get a
(partial) message in the syslog. However, that solution is very problematic on
OS X. One issue is that the logging routine uses GCD, which may spawn a new
thread on its behalf. In many cases, the reporting logic locks threadRegistry,
which leads to deadlocks.

Reviewed at http://reviews.llvm.org/D13452

llvm-svn: 251447
2015-10-27 20:13:01 +00:00
Evgeniy Stepanov 8adbded6a4 [asan] Tune down SizeClassAllocator tests.
This should make the Android/x86 bot (running on a Nexus Player) less
flaky.

llvm-svn: 249044
2015-10-01 17:59:08 +00:00
Evgeniy Stepanov 640499b1fa [asan] Disable ThreadSelfTest on Android.
Also #ifdef out the implementation of ThreadSelf() and
ThreadSelfOffset() helper functions that are broken and unused on
Android.

llvm-svn: 247053
2015-09-08 19:32:37 +00:00
Alexey Samsonov ee03b5bbb9 [CMake] Unify build rules for sanitizer_common for Apple and non-Apple platforms.
Additionally, link safestack runtime on OS X with nolibc version of
sanitizer_common runtime, as we do on Linux.

llvm-svn: 246227
2015-08-27 22:23:27 +00:00
Reid Kleckner 255ee043cb Unit test the CRLF change to suppression parsing
It's a simpler, faster, and more portable.

llvm-svn: 246171
2015-08-27 18:05:33 +00:00
Chris Bieneman 5eae197ab0 [CMake] Fix building unit tests on Darwin
Summary:
There are a number of issues with unit tests on Darwin. These patches address the following:
* Unit tests should be passed -arch (-m32/-m64 isn't sufficient)
* Unit tests should be passed ${DARWIN_osx_CFLAGS} because they're being built for OS X
* Test architectures should be filtered based on base system capabilities (i.e. don't try running x86_64h tests on pre-haswell hardware).

Reviewers: bogner, filcab, kubabrecka

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D12174

llvm-svn: 245580
2015-08-20 17:32:06 +00:00
Alexey Samsonov 0a779d7e3d [Sanitizer] Initialize common flags to default values in unit tests.
llvm-svn: 245363
2015-08-18 21:54:32 +00:00
Reid Kleckner e96833e648 [Windows] Implement FileExists, ReadFromFile, and FindPathToBinary
Summary: These are needed to talk to llvm-symbolizer on Windows.

Reviewers: samsonov

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D11920

llvm-svn: 244533
2015-08-10 23:40:27 +00:00
Alexey Samsonov 1e156011d1 [Sanitizer] Try to fix sanitizer_libc_test on FreeBSD.
llvm-svn: 243797
2015-07-31 23:02:51 +00:00
Evgeniy Stepanov 7fb865ea8e [asan] Only link liblog on Android when it exists.
With r242975, liblog is only used on certain older platforms.

llvm-svn: 243139
2015-07-24 19:00:16 +00:00
Evgeniy Stepanov 5a268b10b3 [sanitizer] Implement logging to syslog.
Previously, Android target had a logic of duplicating all sanitizer
output to logcat. This change extends it to all posix platforms via
the use of syslog, controlled by log_to_syslog flag. Enabled by
default on Android, off everywhere else.

A bit of cmake magic is required to allow Printf() to call a libc
function. I'm adding a stub implementation to support no-libc builds
like dfsan and safestack.

This is a second attempt. I believe I've fixed all the issues that
prompted the revert: Mac build, and all kinds of non-CMake builds
(there are 3 of those).

llvm-svn: 243051
2015-07-23 22:05:20 +00:00
Evgeniy Stepanov ffbc4ec790 Revert r242975.
Breaks Mac build.

llvm-svn: 242978
2015-07-23 01:29:26 +00:00
Evgeniy Stepanov 4f03227fa2 [sanitizer] Implement logging to syslog.
Previously, Android target had a logic of duplicating all sanitizer
output to logcat. This change extends it to all posix platforms via
the use of syslog, controlled by log_to_syslog flag. Enabled by
default on Android, off everywhere else.

A bit of cmake magic is required to allow Printf() to call a libc
function. I'm adding a stub implementation to support no-libc builds
like dfsan and safestack.

llvm-svn: 242975
2015-07-22 23:57:19 +00:00
Evgeniy Stepanov fdcaafa2b8 [asan] Fix sanitizer_allocator_test on AArch64.
llvm-svn: 242582
2015-07-17 22:29:05 +00:00
Evgeniy Stepanov 856b11199f [asan] Fix SanitizerCommon.PthreadDestructorIterations test on Android L.
On Android L, TSD destructors run 8 times instead of 4.
Back to 4 times on the current master branch (as well as on K).

llvm-svn: 240992
2015-06-29 20:28:55 +00:00
Evgeniy Stepanov 609af9098c [asan] Link sanitizer_common tests on Android as -pie.
llvm-svn: 240825
2015-06-26 20:39:30 +00:00
Filipe Cabecinhas 215046bf98 [sanitizer_common] Added VS-style output for source locations
Summary:
With this patch, we have a flag to toggle displaying source locations in
the regular style:
file:line:column

or Visual Studio style:
file(line,column)

This way, they get picked up on the Visual Studio output window and one
can double-click them to get to that file location.

Reviewers: samsonov, rnk

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D10113

llvm-svn: 239000
2015-06-04 01:20:06 +00:00
Evgeniy Stepanov 8e9c70be7f Add descriptive names to sanitizer entries in /proc/self/maps. Helps debugging.
This is done by creating a named shared memory region, unlinking it
and setting up a private (i.e. copy-on-write) mapping of that instead
of a regular anonymous mapping. I've experimented with regular
(sparse) files, but they can not be scaled to the size of MSan shadow
mapping, at least on Linux/X86_64 and ext3 fs.

Controlled by a common flag, decorate_proc_maps, disabled by default.

This patch has a few shortcomings:
* not all mappings are annotated, especially in TSan.
* our handling of memset() of shadow via mmap() puts small anonymous
  mappings inside larger named mappings, which looks ugly and can, in
  theory, hit the mapping number limit.

llvm-svn: 238621
2015-05-29 22:31:28 +00:00
Reid Kleckner 60bdf6e4c0 Disable exceptions with Clang on Windows in lib/sanitizer-common/tests
While I'm here, fix a copy-paste bug so we get debug info for these
tests.

llvm-svn: 236505
2015-05-05 16:52:50 +00:00
Timur Iskhodzhanov ea1f332b79 Split Mprotect into MmapNoAccess and MprotectNoAccess to be more portable
On Windows, we have to know if a memory to be protected is mapped or not.
On POSIX, Mprotect was semantically different from mprotect most people know.

llvm-svn: 234602
2015-04-10 15:02:19 +00:00
Timur Iskhodzhanov 5df4d0471f Move more POSIX-specific functions to sanitizer_posix.h
llvm-svn: 234482
2015-04-09 12:54:06 +00:00
Timur Iskhodzhanov 864308a78d Introduce CloseFile to be used instead of internal_close on non-POSIX
llvm-svn: 234481
2015-04-09 12:37:05 +00:00
Timur Iskhodzhanov 3745303c5b Move some POSIX-specific functions from sanitizer_libc.h to a new sanitizer_posix.h
llvm-svn: 234418
2015-04-08 17:08:24 +00:00
Timur Iskhodzhanov daa9e2d4e8 [Sanitizers] Make OpenFile more portable
llvm-svn: 234410
2015-04-08 16:03:22 +00:00
Timur Iskhodzhanov b97bcc4981 [ASan] Unify handling of loaded modules between POSIX and Windows
Reviewed at http://reviews.llvm.org/D8805

llvm-svn: 234150
2015-04-06 12:49:30 +00:00
Alexander Potapenko f0581dda03 [sanitizer] Fix file access modes in SanitizerCommon.InternalMmapWithOffset and SanitizerCommon.FileOps
llvm-svn: 232972
2015-03-23 17:55:19 +00:00
Alexander Potapenko 141e420a81 [ASan] Distinguish between read, write and read-write file access modes in OpenFile.
This is to fix mapping coverage files into memory on OSX.

llvm-svn: 232936
2015-03-23 10:10:46 +00:00
Kuba Brecka 656e184f6c Adding the implementation of atos and dladdr symbolizers for OS X.
They are currently still *not* used, "llvm-symbolizer" is still the default symbolizer on OS X.

Reviewed at http://reviews.llvm.org/D6588

llvm-svn: 232026
2015-03-12 10:53:18 +00:00
Kostya Serebryany 4bdf5ada18 [asan] more fixes for x32, patches by H.J. Lu
llvm-svn: 231174
2015-03-03 23:46:40 +00:00
Kuba Brecka 59e8c3b342 Symbolizer refactoring: ExtractToken and friends
Reviewed at http://reviews.llvm.org/D7867

llvm-svn: 231027
2015-03-02 22:15:14 +00:00
Dmitry Vyukov b79ac88155 asan: fix signal handling during stoptheworld
The problem is that without SA_RESTORER flag, kernel ignores the handler. So tracer actually did not setup any handler.
Add SA_RESTORER flag when setting up handlers.
Add a test that causes SIGSEGV in stoptheworld callback.
Move SignalContext from asan to sanitizer_common to print better diagnostics about signal in the tracer thread.

http://reviews.llvm.org/D8005

llvm-svn: 230978
2015-03-02 17:36:02 +00:00
Alexey Samsonov d1c318657b [Sanitizer] Refactor SuppressionContext class.
SuppressionContext is no longer a singleton, shared by all sanitizers,
but a regular class. Each of ASan, LSan, UBSan and TSan now have their
own SuppressionContext, which only parses suppressions specific to
that sanitizer.

"suppressions" flag is moved away from common flags into tool-specific
flags, so the user now may pass
  ASAN_OPTIONS=suppressions=asan_supp.txt LSAN_OPIONS=suppressions=lsan_supp.txt
in a single invocation.

llvm-svn: 230026
2015-02-20 17:41:59 +00:00
Mohit K. Bhakkad 36f974d76b [LSan] [MIPS] adding support of LSan for mips64/mips64el arch
Patch by Sagar Thakur

Reviewers: petarj, earthdok, kcc.

Subscribers:  samsonov, dsanders, mohit.bhakkad, Anand.Takale, llvm-commits.

Differential Revision: http://reviews.llvm.org/D7013

llvm-svn: 229830
2015-02-19 07:30:39 +00:00
Alexey Samsonov e41a26e4b4 Revert "Remove unused function."
This reverts commit r227633. SetEnv was prematurely deleted.

llvm-svn: 227665
2015-01-31 01:27:18 +00:00
Alexey Samsonov df0b22c5a8 Remove unused function.
llvm-svn: 227633
2015-01-30 23:04:54 +00:00
Evgeniy Stepanov a2edd9159a [sanitizer] Make unrecognized flags not fatal.
Print a warning at verbosity=1 and higher instead of dying immediately.

llvm-svn: 226458
2015-01-19 12:22:57 +00:00
Evgeniy Stepanov f294d5b829 [sanitizer] Flag parser rewrite.
The new parser is a lot stricter about syntax, reports unrecognized
flags, and will make it easier to implemented some of the planned features.

llvm-svn: 226169
2015-01-15 15:13:43 +00:00
Alexey Samsonov a04ca22edf Fix memory leaks in GetListOfModules() users.
llvm-svn: 225472
2015-01-08 22:31:14 +00:00
Viktor Kutuzov e01a595dad [Sanitizers] Fix internal_lseek() to work on FreeBSD
Differential Revision: http://reviews.llvm.org/D6825

llvm-svn: 225443
2015-01-08 13:28:22 +00:00
Alexey Samsonov df3aeb8e71 Remove TSAN_DEBUG in favor of SANITIZER_DEBUG.
llvm-svn: 225111
2015-01-03 04:29:12 +00:00
Evgeniy Stepanov 9e609426a7 [sanitizer] Treat \r, \n, \t as flag separators.
llvm-svn: 224858
2014-12-26 16:09:15 +00:00
Evgeniy Stepanov 5a09b2cdeb [sanitizer] Disable InternalMmapWithOffset test on OSX.
llvm-svn: 224822
2014-12-24 13:59:35 +00:00
Evgeniy Stepanov 40b16f4037 [sanitizer] Disable InternalMmapWithOffset test on Windows.
llvm-svn: 224820
2014-12-24 13:05:21 +00:00
Evgeniy Stepanov 98211121ab [sanitizer] mmap2 syscall works with 4096-byte units instead of bytes.
Ouch.

llvm-svn: 224819
2014-12-24 12:58:09 +00:00
Viktor Kutuzov d4c3ffc5bf [Sanitizers] Intercept snprintf_l() on FreeBSD
Differential Revision: http://reviews.llvm.org/D6418

llvm-svn: 224700
2014-12-22 12:29:40 +00:00
Alexey Samsonov 2f8c8d59b2 [Sanitizer] Refactor CommonFlags interface. NFC.
Add CommonFlags::SetDefaults() and CommonFlags::ParseFromString(),
so that this object can be easily tested. Enforce
that ParseCommonFlagsFromString() and SetCommonFlagsDefaults()
work only with singleton CommonFlags, shared across all sanitizer
runtimes.

llvm-svn: 224617
2014-12-19 21:40:04 +00:00
Alexey Samsonov 6334f46db8 [Sanitizer] Introduce Allocator::may_return_null bool flag.
Summary:
Turn "allocator_may_return_null" common flag into an
Allocator::may_return_null bool flag. We want to make sure
that common flags are immutable after initialization. There
are cases when we want to change this flag in the allocator
at runtime: e.g. in unit tests and during ASan activation
on Android.

Test Plan: regression test suite, real-life applications

Reviewers: kcc, eugenis

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D6623

llvm-svn: 224148
2014-12-12 20:07:35 +00:00
Kumar Sukhani adb3864744 [mips] [test] Enable COMPILER_RT_INCLUDE_TESTS for MIPS
Enabling COMPILER_RT_INCLUDE_TESTS and updating tests/sanitizer_allocator_test.cc to remove Allocator64 related tests for MIPS.

Reviewed By: samsonov

llvm-svn: 224101
2014-12-12 07:08:12 +00:00
Kuba Brecka dde00302c7 Implement AddressSanitizer suppressions.
Adds 3 new suppression types, "interceptor_name", "interceptor_via_fun", "interceptor_via_lib".

Reviewed at http://reviews.llvm.org/D6280.

llvm-svn: 223508
2014-12-05 20:26:09 +00:00