Summary:
Set proper errno code on alloction failures and change some
implementations to satisfy their man-specified requirements:
LSan: valloc and memalign
ASan: pvalloc, memalign and posix_memalign
Changing both allocators in one patch since LSan depends on ASan allocator in some configurations.
Reviewers: vitalybuka
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D35440
llvm-svn: 308064
Set proper errno code on alloction failures and change valloc and
memalign implementations to satisfy their man-specified requirements.
llvm-svn: 308063
Summary:
Set proper errno code on alloction failure and change pvalloc and
posix_memalign implementation to satisfy their man-specified
requirements.
Reviewers: cryptoad
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35429
llvm-svn: 308053
This change implements 2 optimizations of sync clocks that reduce memory consumption:
Use previously unused first level block space to store clock elements.
Currently a clock for 100 threads consumes 3 512-byte blocks:
2 64-bit second level blocks to store clock elements
+1 32-bit first level block to store indices to second level blocks
Only 8 bytes of the first level block are actually used.
With this change such clock consumes only 2 blocks.
Share similar clocks differing only by a single clock entry for the current thread.
When a thread does several release operations on fresh sync objects without intervening
acquire operations in between (e.g. initialization of several fields in ctor),
the resulting clocks differ only by a single entry for the current thread.
This change reuses a single clock for such release operations. The current thread time
(which is different for different clocks) is stored in dirty entries.
We are experiencing issues with a large program that eats all 64M clock blocks
(32GB of non-flushable memory) and crashes with dense allocator overflow.
Max number of threads in the program is ~170 which is currently quite unfortunate
(consume 4 blocks per clock). Currently it crashes after consuming 60+ GB of memory.
The first optimization brings clock block consumption down to ~40M and
allows the program to work. The second optimization further reduces block consumption
to "modest" 16M blocks (~8GB of RAM) and reduces overall RAM consumption to ~30GB.
Measurements on another real world C++ RPC benchmark show RSS reduction
from 3.491G to 3.186G and a modest speedup of ~5%.
Go parallel client/server HTTP benchmark:
https://github.com/golang/benchmarks/blob/master/http/http.go
shows RSS reduction from 320MB to 240MB and a few percent speedup.
Reviewed in https://reviews.llvm.org/D35323
llvm-svn: 308018
Summary:
libsanitizer doesn't build against latest glibc anymore, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81066 for details.
One of the changes is that stack_t changed from typedef struct sigaltstack { ... } stack_t; to typedef struct { ... } stack_t; for conformance reasons.
And the other change is that the glibc internal __need_res_state macro is now ignored, so when doing
```
#define __need_res_state
#include <resolv.h>
```
the effect is now the same as just
```
#include <resolv.h>
```
and thus one doesn't get just the
```
struct __res_state { ... };
```
definition, but newly also the
```
extern struct __res_state *__res_state(void) __attribute__ ((__const__));
```
prototype. So __res_state is no longer a type, but a function.
Reviewers: kcc, ygribov
Reviewed By: kcc
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D35246
llvm-svn: 307969
Summary:
Secondary backed allocations do not require a cache. While it's not necessary
an issue when each thread has its cache, it becomes one with a shared pool of
caches (Android), as a Secondary backed allocation or deallocation holds a
cache that could be useful to another thread doing a Primary backed allocation.
We introduce an additional PRNG and its mutex (to avoid contention with the
Fallback one for Primary allocations) that will provide the `Salt` needed for
Secondary backed allocations.
I changed some of the code in a way that feels more readable to me (eg: using
some values directly rather than going through ternary assigned variables,
using directly `true`/`false` rather than `FromPrimary`). I will let reviewers
decide if it actually is.
An additional change is to mark `CheckForCallocOverflow` as `UNLIKELY`.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35358
llvm-svn: 307958
Summary:
We were missing many feature flags that newer gcc supports and we had our own set of feature flags that gcc didnt' support that were overlapping. Clang's implementation assumes gcc's features list so a mismatch here is problematic.
I've also matched the cpu type/subtype lists with gcc and removed all the cpus that gcc doesn't support. I've also removed the fallback autodetection logic that was taken from Host.cpp. It was the main reason we had extra feature flags relative to gcc. I don't think gcc does this in libgcc.
Once this support is in place we can consider implementing __builtin_cpu_is in clang. This could also be needed for function dispatching that Erich Keane is working on.
Reviewers: echristo, asbirlea, RKSimon, erichkeane, zvi
Reviewed By: asbirlea
Subscribers: dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D35214
llvm-svn: 307878
On iOS/AArch64, the address space is very limited and has a dynamic maximum address based on the configuration of the device. We're already using a dynamic shadow, and we find a large-enough "gap" in the VM where we place the shadow memory. In some cases and some device configuration, we might not be able to find a large-enough gap: E.g. if the main executable is linked against a large number of libraries that are not part of the system, these libraries can fragment the address space, and this happens before ASan starts initializing.
This patch has a solution, where we have a "backup plan" when we cannot find a large-enough gap: We will restrict the address space (via MmapFixedNoAccess) to a limit, for which the shadow limit will fit.
Differential Revision: https://reviews.llvm.org/D35098
llvm-svn: 307865
Add Fuchsia support to some builtings and avoid building builtins
that are not and will never be used on Fuchsia.
Differential Revision: https://reviews.llvm.org/D34075
llvm-svn: 307832
Summary:
This follows the addition of `GetRandom` with D34412. We remove our
`/dev/urandom` code and use the new function. Additionally, change the PRNG for
a slightly faster version. One of the issues with the old code is that we have
64 full bits of randomness per "next", using only 8 of those for the Salt and
discarding the rest. So we add a cached u64 in the PRNG that can serve up to
8 u8 before having to call the "next" function again.
During some integration work, I also realized that some very early processes
(like `init`) do not benefit from `/dev/urandom` yet. So if there is no
`getrandom` syscall as well, we have to fallback to some sort of initialization
of the PRNG.
Now a few words on why XoRoShiRo and not something else. I have played a while
with various PRNGs on 32 & 64 bit platforms. Some results are below. LCG 32 & 64
are usually faster but produce respectively 15 & 31 bits of entropy, meaning
that to get a full 64-bit, you would need to call them several times. The simple
XorShift is fast, produces 32 bits but is mediocre with regard to PRNG test
suites, PCG is slower overall, and XoRoShiRo is faster than XorShift128+ and
produces full 64 bits.
%%%
root@tulip-chiphd:/data # ./randtest.arm
[+] starting xs32...
[?] xs32 duration: 22431833053ns
[+] starting lcg32...
[?] lcg32 duration: 14941402090ns
[+] starting pcg32...
[?] pcg32 duration: 44941973771ns
[+] starting xs128p...
[?] xs128p duration: 48889786981ns
[+] starting lcg64...
[?] lcg64 duration: 33831042391ns
[+] starting xos128p...
[?] xos128p duration: 44850878605ns
root@tulip-chiphd:/data # ./randtest.aarch64
[+] starting xs32...
[?] xs32 duration: 22425151678ns
[+] starting lcg32...
[?] lcg32 duration: 14954255257ns
[+] starting pcg32...
[?] pcg32 duration: 37346265726ns
[+] starting xs128p...
[?] xs128p duration: 22523807219ns
[+] starting lcg64...
[?] lcg64 duration: 26141304679ns
[+] starting xos128p...
[?] xos128p duration: 14937033215ns
%%%
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: aemerson, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D35221
llvm-svn: 307798
1. Add SyncClock::ResetImpl which removes code
duplication between ctor and Reset.
2. Move SyncClock::Resize to SyncClock methods,
currently it's defined between ThreadClock methods.
llvm-svn: 307785
Don't create sync object if it does not exist yet. For example, an atomic
pointer is initialized to nullptr and then periodically acquire-loaded.
llvm-svn: 307778
The test should have been added in 289682
"tsan: allow Java VM iterate over allocated objects"
but I forgot to avn add.
Author: Alexander Smundak (asmundak)
Reviewed in https://reviews.llvm.org/D27720
llvm-svn: 307776
r307338 enabled new optimization reducing number of operation in tested functions.
There is no any performance regression detectable with TsanRtlTest DISABLED_BENCH.Mop* tests.
llvm-svn: 307739
Cleaner than using a while loop to copy the string character by character.
Reviewers: alekseyshl, glider
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35136
llvm-svn: 307696
Summary:
This function is only called once and is fairly simple. Inline to
keep API simple.
Reviewers: alekseyshl, kubamracek
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D35270
llvm-svn: 307695
Summary:
This is the first in a series of patches to refactor sanitizer_procmaps
to allow MachO section information to be exposed on darwin.
In addition, grouping all segment information in a single struct is
cleaner than passing it through a large set of output parameters, and
avoids the need for annotations of NULL parameters for unneeded
information.
The filename string is optional and must be managed and supplied by the
calling function. This is to allow the MemoryMappedSegment struct to be
stored on the stack without causing overly large stack sizes.
Reviewers: alekseyshl, kubamracek, glider
Subscribers: emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D35135
llvm-svn: 307688
Printing stacktrace from ASAN crashes with a segfault in DEDUP mode when
symbolication is missing.
Differential Revision: https://reviews.llvm.org/D34914
llvm-svn: 307577
This patch ports the assembly file implementing TSan's setjmp support to AArch64 on Darwin.
Differential Revision: https://reviews.llvm.org/D35143
llvm-svn: 307541
Revert "Copy arguments passed by value into explicit allocas for ASan."
Revert "[asan] Add end-to-end tests for overflows of byval arguments."
Build failure on lldb-x86_64-ubuntu-14.04-buildserver.
Test failure on clang-cmake-aarch64-42vma and sanitizer-x86_64-linux-android.
llvm-svn: 307345
This improves find_darwin_sdk_dir to cache the results of executing xcodebuild to find the SDK. Should significantly reduce the CMake re-configure time.
Differential Revision: https://reviews.llvm.org/D34736
llvm-svn: 307344
Included is one test for passing structs by value and one test for passing C++
objects by value.
Patch by Matt Morehouse.
Differential revision: https://reviews.llvm.org/D34827
llvm-svn: 307343
Adds a CMake option DARWIN_PREFER_PUBLIC_SDK, off by default. When on, this prefers to use the public SDK, even when an internal one is present. With this, it's easy to emulate a build that the public buildbots are doing.
Differential Revision: https://reviews.llvm.org/D35071
llvm-svn: 307330
We currently hardcode the maximum VM address on iOS/AArch64, which is not really correct and this value changes between device configurations. Let's use TASK_VM_INFO to retrieve the maximum VM address dynamically.
Differential Revision: https://reviews.llvm.org/D35032
llvm-svn: 307307
The logic in GetMaxVirtualAddress is already pretty complex, and I want to get rid of the hardcoded value for iOS/AArch64, which would need adding more Darwin-specific code, so let's split the implementation into sanitizer_linux.cc and sanitizer_mac.cc files. NFC.
Differential Revision: https://reviews.llvm.org/D35031
llvm-svn: 307281
On Darwin, sigprocmask changes the signal mask for the entire process. This has some unwanted consequences, because e.g. internal_start_thread wants to disable signals only in the current thread (to make the new thread inherit the signal mask), which is currently broken on Darwin. This patch switches to pthread_sigmask.
Differential Revision: https://reviews.llvm.org/D35016
llvm-svn: 307212
Summary:
An attempt to reland D34786 (which caused bot failres on Mac), now with
properly intercepted operators new() and delete().
LSan allocator used to always return nullptr on too big allocation requests
(the definition of "too big" depends on platform and bitness), now it
follows policy configured by allocator_may_return_null flag
Reviewers: eugenis
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34845
llvm-svn: 306845
This fixes an issue with the emission of lifetime markers for struct-returning Obj-C msgSend calls. When the result of a struct-returning call is ignored, the temporary storage is only marked with lifetime markers in one of the two branches of the nil-receiver-check. The check is, however, not required when the result is unused. If we still need to emit the check (due to consumer arguments), let's not emit the memset to zero out the result if it's unused. This fixes a use-after-scope false positive with AddressSanitizer.
Differential Revision: https://reviews.llvm.org/D34834
llvm-svn: 306838
Summary:
In `sanitizer_allocator_primary32.h`:
- rounding up in `MapWithCallback` is not needed as `MmapOrDie` does it. Note
that the 64-bit counterpart doesn't round up, this keeps the behavior
consistent;
- since `IsAligned` exists, use it in `AllocateRegion`;
- in `PopulateFreeList`:
- checking `b->Count` to be greater than 0 when `b->Count() == max_count` is
redundant when done more than once. Just check that `max_count` is greater
than 0 out of the loop; the compiler (at least on ARM) didn't optimize it;
- mark the batch creation failure as `UNLIKELY`;
In `sanitizer_allocator_primary64.h`:
- in `MapWithCallback`, mark the failure condition as `UNLIKELY`;
In `sanitizer_posix.h`:
- mark a bunch of Mmap related failure conditions as `UNLIKELY`;
- in `MmapAlignedOrDieOnFatalError`, we have `IsAligned`, so use it; rearrange
the conditions as one test was redudant;
- in `MmapFixedImpl`, 30 chars was not large enough to hold the message and a
full 64-bit address (or at least a 48-bit usermode address), increase to 40.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: aemerson, kubamracek, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D34840
llvm-svn: 306834
Summary:
Due to changes in semantics, CheckForCallocOverflow makes much more sense
now.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34799
llvm-svn: 306747
Users can specify the path a raw profile is written to by passing
-fprofile-instr-generate=<path>, but this functionality broke on Darwin
after __llvm_profile_filename was made weak [1], resulting in profiles
being written to "default.profraw" even when <path> is specified.
The situation is that instrumented programs provide a weak definition of
__llvm_profile_filename, which conflicts with a weak redefinition
provided by the profiling runtime.
The linker appears to pick the 'winning' definition arbitrarily: on
Darwin, it usually prefers the larger definition, which is probably why
the instrprof-override-filename.c test has been passing.
The fix is to move the runtime's definition into a separate object file
within the archive. This means that the linker won't "see" the runtime's
definition unless the user program has not provided one. I couldn't
think of a great way to test this other than to mimic the Darwin
failure: use -fprofile-instr-generate=<some-small-path>.
Testing: check-{clang,profile}, modified instrprof-override-filename.c.
[1] [Profile] deprecate __llvm_profile_override_default_filename
https://reviews.llvm.org/D22613https://reviews.llvm.org/D22614
Differential Revision: https://reviews.llvm.org/D34797
llvm-svn: 306710
Do this by removing SANITIZER_INTERCEPT_WCSLEN and intercept wcslen
everywhere. Before this change, we were already intercepting wcslen on
Windows, but the interceptor was in asan, not sanitizer_common. After
this change, we stopped intercepting wcslen on Windows, which broke
asan_dll_thunk.c, which attempts to thunk to __asan_wcslen in the ASan
runtime.
llvm-svn: 306706
Summary:
We were not following the `man` documented behaviors for invalid arguments to
`memalign` and associated functions. Using `CHECK` for those was a bit extreme,
so we relax the behavior to return null pointers as expected when this happens.
Adapt the associated test.
I am using this change also to change a few more minor performance improvements:
- mark as `UNLIKELY` a bunch of unlikely conditions;
- the current `CHECK` in `__sanitizer::RoundUpTo` is redundant for us in *all*
calls. So I am introducing our own version without said `CHECK`.
- change our combined allocator `GetActuallyAllocatedSize`. We already know if
the pointer is from the Primary or Secondary, so the `PointerIsMine` check is
redundant as well, and costly for the 32-bit Primary. So we get the size by
directly using the available Primary functions.
Finally, change a `int` to `uptr` to avoid a warning/error when compiling on
Android.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34782
llvm-svn: 306698
Summary:
LSan allocator used to always return nullptr on too big allocation requests
(the definition of "too big" depends on platform and bitness), now it
follows policy configured by allocator_may_return_null flag.
Reviewers: eugenis
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34786
llvm-svn: 306624
Summary:
Operator new interceptors behavior is now controlled by their nothrow
property as well as by allocator_may_return_null flag value:
- allocator_may_return_null=* + new() - die on allocation error
- allocator_may_return_null=0 + new(nothrow) - die on allocation error
- allocator_may_return_null=1 + new(nothrow) - return null
Ideally new() should throw std::bad_alloc exception, but that is not
trivial to achieve, hence TODO.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34731
llvm-svn: 306604
Summary:
This change introduces two files that show exaples of the
always/never instrument files that can be provided to clang. We don't
add these as defaults yet in clang, which we can do later on (in a
separate change).
We also add a test that makes sure that these apply in the compiler-rt
project tests, and that changes in clang don't break the expectations in
compiler-rt.
Reviewers: pelikan, kpw
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34669
llvm-svn: 306502
Summary: Cleaner than computing the intersection for each possible sanitizer
Reviewers: compnerd, beanz
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D34693
llvm-svn: 306453
Summary: This allows check-all to be used when only a subset of the sanitizers are built.
Reviewers: beanz, compnerd, rnk, pcc
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D34644
llvm-svn: 306450
Introduces a 'owner' struct to include the overridable write
method and the write context in C.
This allows easy introdution of new member API to help reduce
profile merge time in the follow up patch.
llvm-svn: 306432
Summary: This allows check-all to be used when only a subset of the sanitizers are built.
Reviewers: beanz, compnerd
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D34644
llvm-svn: 306415
Summary:
Make SizeClassAllocator32 return nullptr when it encounters OOM, which
allows the entire sanitizer's allocator to follow allocator_may_return_null=1
policy, even for small allocations (LargeMmapAllocator is already fixed
by D34243).
Will add a test for OOM in primary allocator later, when
SizeClassAllocator64 can gracefully handle OOM too.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34433
llvm-svn: 305972
Summary:
AFAICT compiler-rt doesn't have a function that would return 'good' random
bytes to seed a PRNG. Currently, the `SizeClassAllocator64` uses addresses
returned by `mmap` to seed its PRNG, which is not ideal, and
`SizeClassAllocator32` doesn't benefit from the entropy offered by its 64-bit
counterpart address space, so right now it has nothing. This function aims at
solving this, allowing to implement good 32-bit chunk randomization. Scudo also
has a function that does this for Cookie purposes, which would go away in a
later CL once this lands.
This function will try the `getrandom` syscall if available, and fallback to
`/dev/urandom` if not.
Unfortunately, I do not have a way to implement and test a Mac and Windows
version, so those are unimplemented as of now. Note that `kRandomShuffleChunks`
is only used on Linux for now.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: zturner, rnk, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34412
llvm-svn: 305922
Change some reinterpret_casts to c-style casts due to template instantiation
restrictions and build breakage due to missing paranthesises.
llvm-svn: 305899
Summary:
On Android we still need to reset preinstalled handlers and allow use handlers later.
This reverts commit r304039.
Reviewers: eugenis
Subscribers: kubamracek, dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D34434
llvm-svn: 305871
Summary:
Move cached allocator_may_return_null flag to sanitizer_allocator.cc and
provide API to consolidate and unify the behavior of all specific allocators.
Make all sanitizers using CombinedAllocator to follow
AllocatorReturnNullOrDieOnOOM() rules to behave the same way when OOM
happens.
When OOM happens, turn allocator_out_of_memory flag on regardless of
allocator_may_return_null flag value (it used to not to be set when
allocator_may_return_null == true).
release_to_os_interval_ms and rss_limit_exceeded will likely be moved to
sanitizer_allocator.cc too (later).
Reviewers: eugenis
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34310
llvm-svn: 305858
Summary:
ASan shadow memory on s390 is larger than other configurations, let's
disable this test for now (will revisit it later).
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34414
llvm-svn: 305822
Summary:
This is required for standalone LSan to work with libdispatch worker threads,
and is a slimmed down version of the functionality provided for ASan
in asan_mac.cc.
Re-commit of r305695 with use_stacks=0 to get around a racy lingering pointer.
Reviewers: alekseyshl, kubamracek, glider, kcc
Subscribers: mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D34247
llvm-svn: 305732
Summary:
This is required for standalone LSan to work with libdispatch worker threads,
and is a slimmed down version of the functionality provided for ASan
in asan_mac.cc.
Reviewers: alekseyshl, kubamracek, glider, kcc
Subscribers: mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D34247
llvm-svn: 305695
This test makes sure we can handle both arg0 and arg1 handling in the
same binary, and making sure that the XRay runtime calls the correct
trampoline when handlers for both of these cases are installed.
llvm-svn: 305660
Summary:
Since r298413, the NEW behavior of the CMake policy CMP0056 is followed.
However, it is only effective after the call to cmake_minimum_required.
This causes CMAKE_EXE_LINKER_FLAGS etc. to be unused when CMake tries to
check compilers for languages specified in the 'project' declaration.
Set cmake_minimum_required(VERSION) at the top of the file and ahead of
the project declaration.
Reviewers: beanz
Subscribers: mgorny, srhines, llvm-commits
Differential Revision: https://reviews.llvm.org/D34282
llvm-svn: 305593
Summary:
CombinedAllocator::Allocate cleared parameter is not used anywhere and
seem to be obsolete.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34289
llvm-svn: 305590
The first instruction of the new ucrtbase!strnlen implementation loads a
global, presumably to dispatch between SSE and non-SSE optimized strnlen
implementations.
Fixes PR32895 and probably
https://github.com/google/sanitizers/issues/818
llvm-svn: 305581
Summary:
Point of failure is different after D34243, hence the change of the
message.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34292
llvm-svn: 305580
Summary:
Context: https://github.com/google/sanitizers/issues/740.
Making secondary allocator to respect allocator_may_return_null=1 flag
and return nullptr when "out of memory" happens.
More changes in primary allocator and operator new will follow.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34243
llvm-svn: 305569
Summary:
This allows us to do more interesting things with the data available to
C++ methods, to log the `this` pointer.
Depends on D34050.
Reviewers: pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D34051
llvm-svn: 305545
The dynamic type check needs to inspect vtables, but could crash if it
encounters a vtable pointer to inaccessible memory. In the first attempt
to fix the issue (r304437), we performed a memory accessibility check on
the wrong range of memory. This should *really* fix the problem.
Patch by Max Moroz!
Differential Revision: https://reviews.llvm.org/D34215
llvm-svn: 305489
Summary:
After r303941 it was not possible to setup ASAN_OPTIONS to have the same
behavior for pre r303941 and post r303941 builds.
Pre r303941 Asan does not accept handle_sigbus=2.
Post r303941 Asan does not accept allow_user_segv_handler.
This fix ignores allow_user_segv_handler=1, but for allow_user_segv_handler=0
it will upgrade flags like handle_sigbus=1 to handle_sigbus=2. So user can set
ASAN_OPTIONS=allow_user_segv_handler=0 and have same behavior on old and new
clang builds (except range from r303941 to this revision).
In future users which need to prevent third party handlers should switch to
handle_sigbus=2 and remove allow_user_segv_handler as soon as suport of older
builds is not needed.
Related bugs:
https://github.com/google/oss-fuzz/issues/675https://bugs.chromium.org/p/chromium/issues/detail?id=731130
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34227
llvm-svn: 305433
Summary:
This broke thread_local_quarantine_pthread_join.cc on some architectures, due
to the overhead of the stashed regions. Reverting while figuring out the best
way to deal with it.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34213
llvm-svn: 305404
Summary:
The reasoning behind this change is explained in D33454, which unfortunately
broke the Windows version (due to the platform not supporting partial unmapping
of a memory region).
This new approach changes `MmapAlignedOrDie` to allow for the specification of
a `padding_chunk`. If non-null, and the initial allocation is aligned, this
padding chunk will hold the address of the extra memory (of `alignment` bytes).
This allows `AllocateRegion` to get 2 regions if the memory is aligned
properly, and thus help reduce fragmentation (and saves on unmapping
operations). As with the initial D33454, we use a stash in the 32-bit Primary
to hold those extra regions and return them on the fast-path.
The Windows version of `MmapAlignedOrDie` will always return a 0
`padding_chunk` if one was requested.
Reviewers: alekseyshl, dvyukov, kcc
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D34152
llvm-svn: 305391
Summary:
Move the OOM decision based on RSS limits out of generic allocator to
ASan allocator, where it makes more sense at the moment.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D34180
llvm-svn: 305342
This patch addresses PR 33206. There might be a situation when dynamic ASan runtime initializes later
than shared library which has malloc in static constructor (rtld doesn't provide an order of shared libs initialization).
In this case ASan hasn't yet initialized interceptors, but already intercepts malloc.
If malloc is too big to be handled by static local pool, ASan will die with error:
Sanitizer CHECK failed: lib/asan/asan_malloc_linux.cc:40 ((allocated_for_dlsym)) < ((kDlsymAllocPoolSize)) (1036, 1024)
Patch by Denis Khalikov.
Differential Revision: https://reviews.llvm.org/D33784
llvm-svn: 305058
This reverts commit r304941. Vitaly Buka writes:
"Actually it depends on return value.
Test is for char* version of function. It will probably fail for int
version."
llvm-svn: 304943
On Darwin, strerror_r returns an int, not a char*. I don't think this
test really depends on what strerror_r returns, so I've used something
else in place of the result of the call to strerror_r.
llvm-svn: 304941
GNU version of strerror_r returns a result pointer that doesn't match the input
buffer. The result pointer is in fact a pointer to some internal storage.
TSAN was recording a write to this location, which was incorrect.
Fixed https://github.com/google/sanitizers/issues/696
llvm-svn: 304858
Summary:
As mentioned in test/msan/fork.cc, if test output is redirected to a file
(as opposed to being piped directly to FileCheck), we may lose some "done"s due to
a kernel bug: https://lkml.org/lkml/2014/2/17/324, so let's pipe the
output of the test.
Reviewers: eugenis
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33915
llvm-svn: 304744
r304285 - [sanitizer] Avoid possible deadlock in child process after fork
r304297 - [sanitizer] Trying to fix MAC buildbots after r304285
These changes create deadlock when Tcl calls pthread_create from a
pthread_atfork child handler. More info in the original review at
https://reviews.llvm.org/D33325
llvm-svn: 304735
Summary:
halt_on_error-torture.cc intermittently fails on ppc64be, let's try to
collect more info on failures.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D33912
llvm-svn: 304731
Revert "Mark sancov test as unsupported on Darwin"
Revert "[LSan] Detect dynamic loader by its base address."
This reverts commit r304633.
This reverts commit r304673.
This reverts commit r304632.
Those commit have broken LOTS of ARM/AArch64 bots for two days.
llvm-svn: 304699
atos is apparently not able to resolve symbol addresses properly on
i386-darwin reliably any more. This is causing bot flakiness:
http://lab.llvm.org:8080/green/job/clang-stage1-cmake-RA-expensive/6841
There have not been any SDK changes on the bot as of late.
/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/llvm/projects/compiler-rt/test/asan/TestCases/Darwin/atos-symbolizer.cc:20:12: error: expected string not found in input
// CHECK: #1 0x{{.*}} in main {{.*}}atos-symbolizer.cc:[[@LINE-4]]
^
<stdin>:35:27: note: scanning from here
#0 0x112f56 in wrap_free (/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/clang-build/lib/clang/5.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:i386+0x56f56)
^
<stdin>:35:27: note: with expression "@LINE-4" equal to "16"
#0 0x112f56 in wrap_free (/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/clang-build/lib/clang/5.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib:i386+0x56f56)
^
<stdin>:36:168: note: possible intended match here
#1 0xb6f20 in main (/Users/buildslave/jenkins/sharedspace/clang-stage1-cmake-RA_workspace/clang-build/tools/clang/runtime/compiler-rt-bins/test/asan/I386DarwinConfig/TestCases/Darwin/Output/atos-symbolizer.cc.tmp:i386+0x1f20)
llvm-svn: 304674
This test has been failing on all Darwin bots since it was introduced:
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_check/32111
fatal error: error in backend: Global variable '__sancov_gen_' has an invalid section specifier '__DATA,__sancov_counters': mach-o section specifier requires a section whose length is between 1 and 16 characters.
Target: x86_64-apple-darwin15.6.0
llvm-svn: 304673
Summary:
Very recently, FreeBSD 12 has been updated to use 64-bit inode numbers:
<https://svnweb.freebsd.org/changeset/base/318737>. This entails many
user-visible changes, but for the sanitizers the modifications are
limited in scope:
* The `stat` and `lstat` syscalls were removed, and should be replaced
with calls to `fstatat`.
* The `getdents` syscall was removed, and should be replaced with calls
to `getdirentries`.
* The layout of `struct dirent` was changed to accomodate 64-bit inode
numbers, and a new `d_off` field was added.
* The system header <sys/_types.h> now contains a macro `__INO64` to
determine whether the system uses 64-bit inode numbers.
I tested these changes on both FreeBSD 12.0-CURRENT (after r318959,
which adds the `__INO64` macro), and FreeBSD 11.0-STABLE (which still
uses 32-bit inode numbers).
Reviewers: emaste, kcc, vitalybuka, kubamracek
Reviewed By: vitalybuka
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D33600
llvm-svn: 304658
Summary:
Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.
Reviewers: eugenis
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D33859
llvm-svn: 304633
There is can be a situation when vptr is not initializing
by constructor of the object, and has a junk data which should
be properly checked, because c++ standard says:
"if default constructor is not specified
16 (7.3) no initialization is performed."
Patch by Denis Khalikov!
Differential Revision: https://reviews.llvm.org/D33712
llvm-svn: 304437
This patch addresses https://github.com/google/sanitizers/issues/774. When we
fork a multi-threaded process it's possible to deadlock if some thread acquired
StackDepot or allocator internal lock just before fork. In this case the lock
will never be released in child process causing deadlock on following memory alloc/dealloc
routine. While calling alloc/dealloc routines after multi-threaded fork is not allowed,
most of modern allocators (Glibc, tcmalloc, jemalloc) are actually fork safe. Let's do the same
for sanitizers except TSan that has complex locking rules.
Differential Revision: https://reviews.llvm.org/D33325
llvm-svn: 304285
Summary:
D33521 addressed a memory ordering issue in BlockingMutex, which seems
to be the cause of a flakiness of a few ASan tests on PowerPC.
Reviewers: eugenis
Subscribers: kubamracek, nemanjai, llvm-commits
Differential Revision: https://reviews.llvm.org/D33611
llvm-svn: 304045
The test was meant for Darwin anyway, so I'm not even sure it's supposed
to run on Linux. If it was, then we need time to investigate, but since
the test is new, there's no point in reverting the whole patch because
of it.
llvm-svn: 304010
Summary:
Currently we are not enforcing the success of `pthread_once`, and
`pthread_setspecific`. Errors could lead to harder to debug issues later in
the thread's life. This adds checks for a 0 return value for both.
If `pthread_setspecific` fails in the teardown path, opt for an immediate
teardown as opposed to a fatal failure.
Reviewers: alekseyshl, kcc
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D33555
llvm-svn: 303998
Summary:
D33521 addressed a memory ordering issue in BlockingMutex, which seems
to be the cause of a flakiness of a few ASan tests on PowerPC.
Reviewers: eugenis
Subscribers: kubamracek, nemanjai, llvm-commits
Differential Revision: https://reviews.llvm.org/D33569
llvm-svn: 303995
Summary:
allow_user_segv_handler had confusing name did not allow to control behavior for
signals separately.
Reviewers: eugenis, alekseyshl, kcc
Subscribers: llvm-commits, dberris, kubamracek
Differential Revision: https://reviews.llvm.org/D33371
llvm-svn: 303941
Summary:
This required for any users who call exit() after creating
thread-specific data, as tls destructors are only called when
pthread_exit() or pthread_cancel() are used. This should also
match tls behavior on linux.
Getting the base address of the tls section is straightforward,
as it's stored as a section offset in %gs. The size is a bit trickier
to work out, as there doesn't appear to be any official documentation
or source code referring to it. The size used in this patch was determined
by taking the difference between the base address and the address of the
subsequent memory region returned by vm_region_recurse_64, which was
1024 * sizeof(uptr) on all threads except the main thread, where it was
larger. Since the section must be the same size on all of the threads,
1024 * sizeof(uptr) seemed to be a reasonable size to use, barring
a more programtic way to get the size.
1024 seems like a reasonable number, given that PTHREAD_KEYS_MAX
is 512 on darwin, so pthread keys will fit inside the region while
leaving space for other tls data. A larger size would overflow the
memory region returned by vm_region_recurse_64, and a smaller size
wouldn't leave room for all the pthread keys. In addition, the
stress test added here passes, which means that we are scanning at
least the full set of possible pthread keys, and probably
the full tls section.
Reviewers: alekseyshl, kubamracek
Subscribers: krytarowski, llvm-commits
Differential Revision: https://reviews.llvm.org/D33215
llvm-svn: 303887
The existing implementation ran CHECKs to assert that the thread state
was stored inside the tls. However, the mac implementation of tsan doesn't
store the thread state in tls, so these checks fail once darwin tls support
is added to the sanitizers. Only run these checks on platforms where
the thread state is expected to be contained in the tls.
llvm-svn: 303886
Summary:
Apparently Windows's `UnmapOrDie` doesn't support partial unmapping. Which
makes the new region allocation technique not Windows compliant.
Reviewers: alekseyshl, dvyukov
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33554
llvm-svn: 303883
Summary:
Currently, AllocateRegion has a tendency to fragment memory: it allocates
`2*kRegionSize`, and if the memory is aligned, will unmap `kRegionSize` bytes,
thus creating a hole, which can't itself be reused for another region. This
is exacerbated by the fact that if 2 regions get allocated one after another
without any `mmap` in between, the second will be aligned due to mappings
generally being contiguous.
An idea, suggested by @alekseyshl, to prevent such a behavior is to have a
stash of regions: if the `2*kRegionSize` allocation is properly aligned, split
it in two, and stash the second part to be returned next time a region is
requested.
At this point, I thought about a couple of ways to implement this:
- either an `IntrusiveList` of regions candidates, storing `next` at the
begining of the region;
- a small array of regions candidates existing in the Primary.
While the second option is more constrained in terms of size, it offers several
advantages:
- security wise, a pointer in a region candidate could be overflowed into, and
abused when popping an element;
- we do not dirty the first page of the region by storing something in it;
- unless several threads request regions simultaneously from different size
classes, the stash rarely goes above 1 entry.
I am not certain about the Windows impact of this change, as `sanitizer_win.cc`
has its own version of MmapAlignedOrDie, maybe someone could chime in on this.
MmapAlignedOrDie is effectively unused after this change and could be removed
at a later point. I didn't notice any sizeable performance gain, even though we
are saving a few `mmap`/`munmap` syscalls.
Reviewers: alekseyshl, kcc, dvyukov
Reviewed By: alekseyshl
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33454
llvm-svn: 303879
Summary:
Dmitry, seeking your expertise. I believe, the proper way to implement
Lock/Unlock here would be to use acquire/release semantics. Am I missing
something?
Reviewers: dvyukov
Subscribers: llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D33521
llvm-svn: 303869
Summary:
In FreeBSD we needed to add generic implementations for `__bswapdi2` and
`__bswapsi2`, since gcc 6.x for mips is emitting calls to these. See:
https://reviews.freebsd.org/D10838 and https://reviews.freebsd.org/rS318601
The actual mips code generated for these generic C versions is pretty
OK, as can be seen in the (FreeBSD) review.
I checked over gcc sources, and it seems that it can emit these calls on
more architectures, so maybe it's best to simply always add them to the
compiler-rt builtins library.
Reviewers: howard.hinnant, compnerd, petarj, emaste
Reviewed By: compnerd, emaste
Subscribers: mgorny, llvm-commits, arichardson
Differential Revision: https://reviews.llvm.org/D33516
llvm-svn: 303866
This test case occassionally fails when run on powerpc64 be.
asan/TestCases/Posix/halt_on_error-torture.cc
The failure causes false problem reports to be sent to developers whose
code had nothing to do with the failures. Reactivate it when the real
problem is fixed.
This could also be related to the same problems as with the tests
ThreadedOneSizeMallocStressTest, ThreadedMallocStressTest, ManyThreadsTest,
and several others that do not run reliably on powerpc.
llvm-svn: 303864
George Karpenkov