This patch reorganize the platform specific mapping information to
export the application mask on a external variable. This exported
variable will be used by intrumentation phase to create code to be
used on architecture with multiple VMA range.
The patch creates a new header, dfsan_platform.h, and move all the
mapping information and also create function accessors to the
mapping value.
Also for aarch64 it initialize application exported mask to the
value based on runtime VMA detection.
llvm-svn: 254197
This patch unify the 39 and 42-bit support for AArch64 by using an external
memory read to check the runtime detected VMA and select the better mapping
and transformation. Although slower, this leads to same instrumented binary
to be independent of the kernel.
Along with this change this patch also fix some 42-bit failures with
ALSR disable by increasing the upper high app memory threshold and also
the 42-bit madvise value for non large page set.
llvm-svn: 254151
Summary:
In r251553, I added a warning that proved to be super noisy,
especially in case of MSan. Removing it for now, and I will
probably add some additional diagnostics for symbolizer, when
I better understand how to make it properly.
Reviewers: eugenis
Differential Revision: http://reviews.llvm.org/D14998
llvm-svn: 254102
Due to a typo, atos-symbolizer-dyld-root-path.cc is currently being skipped all the time. There's a few more typos/copy-paste-errors. Let's fix them and enable this test.
Differential Revision: http://reviews.llvm.org/D14951
llvm-svn: 254058
This patch adds some fixes for MSAN with libc++ for aarch64:
1. Adds the libmsan_loadable name for aarch64.
2. Fixes some pthread_attr_setstacksize for aarch64, since glibc sets
the mininum stack size to be higher than the x86_64 default (16KB
vs 128KB).
3. Fixes a swprintf null char constant definition.
llvm-svn: 254015
We need to intercept libdispatch APIs (dispatch_sync, dispatch_async, etc.) to add synchronization between the code that submits the task and the code that gets executed (possibly on a different thread). This patch adds release+acquire semantics for dispatch_sync, and dispatch_async (plus their "_f" and barrier variants). The synchronization is done on malloc'd contexts (separate for each submitted block/callback). Added tests to show usage of dispatch_sync and dispatch_async, for cases where we expect no warnings and for cases where TSan finds races.
Differential Revision: http://reviews.llvm.org/D14745
llvm-svn: 253982
The test relies on two variables in different frames to end up being on the same address. For some reason, this isn't true on OS X. This patch adds `__attribute__((aligned(64)))` to the variables, which actually makes the variables occupy the same address. This is still not a guarantee, but it's more likely to work (the test looks very fragile already).
Differential Revision: http://reviews.llvm.org/D14925
llvm-svn: 253981
Pthread semaphores are not available on OS X. Let's replace sem_wait/sem_post with barrier_wait, which makes the test pass on OS X. I know that sem_wait/sem_post is intercepted by TSan, whereas barrier_wait is TSan-invisible, but the purpose of the test is not affected by this.
Also, let's properly initialize the mutex and cond variables.
Differential Revision: http://reviews.llvm.org/D14924
llvm-svn: 253980
On OS X, interceptors don't work in unit tests, so calloc() calls the system allocator. We need to use user_calloc() instead.
Differential Revision: http://reviews.llvm.org/D14918
llvm-svn: 253979
On OS X, __thread variables are lazily heap-allocated (with malloc). Therefore, they're recognized as heap blocks (which is what they are) and not as TLS variables in TSan reports. Figuring out if a heap block is a TLS or not is difficult (in malloc interceptor we could analyze the caller and then mark the object), so let's instead modify the tests so that we expect the report to say "Location is heap block" instead of "Location is TLS".
Differential Revision: http://reviews.llvm.org/D14873
llvm-svn: 253858
This patch fixes the __cxa_guard_acquire, __cxa_guard_release and __cxa_guard_abort interceptors on OS X. They apparently work on Linux just by having the same name, but on OS X, we actually need to use TSAN_INTERCEPTOR.
Differential Revision: http://reviews.llvm.org/D14868
llvm-svn: 253776
We need to call the intercepted version of pthread_detach. Secondly, PTHREAD_CREATE_JOINABLE and PTHREAD_CREATE_DETACHED are not 0 and 1 on OS X, so we need to properly pass these constants and not just a bool.
Differential Revision: http://reviews.llvm.org/D14837
llvm-svn: 253775
The tsan_test_util_posix.cc implementation of mutexes call pthread APIs directly, which on OS X don't end up calling the intercepted versions and we miss the synchronization. This patch changes the unit tests to directly call the intercepted versions. This fixes several test failures on OS X.
Differential Revision: http://reviews.llvm.org/D14835
llvm-svn: 253774
(patch suggested by silvas)
With this patch, the IO information is wrapped in struct
ProfDataIOVec, and interface of writerCallback takes a vector
of IOVec and a pointer to writer context pointer.
Differential Revision: http://reviews.llvm.org/D14859
llvm-svn: 253764
This test checks if we can print a backtrace from the death callback. On old
OS X versions, backtrace is not able to symbolicate the trace past the ASan
runtime because we build with -fomit-frame-pointer.
llvm-svn: 253729
There seems to be a problem in system header (stdint.h) of FreeBSD
where uint8_t nor uint16_t are defined. Explicitly define the key
types as done for FreeBSD i386.
llvm-svn: 253703
Value profile enumerator change to match LLVM code
ProfData new member field name change to match LLVM code
ProfData member type change to match LLVM code
Do not use lower case for types that are internal to implementation (not exposed to APIs)
There is no functional change. This is a preparation patch to enable more code sharing
in follow up patches
Differential Revision: http://reviews.llvm.org/D14841
llvm-svn: 253700
The Crash Report is going to add app specific info if it is stored in the magical buffer.
Differential Revision: http://reviews.llvm.org/D14791
llvm-svn: 253691
The compiler-rt should make use of strlcpy() rather than strncpy(). Using internal_strncpy() may be fine with appropriate bounds checking or enforcement of nul-termination elsewhere, but it's just good practice these days to avoid using strncpy() in new code.
A patch by Jeremy Sequoia!
Differential Revision: http://reviews.llvm.org/D14714
llvm-svn: 253690
[asan] On OS X, log reports to syslog and os_trace, has been reverted in r252076 due to deadlocks on earlier versions of OS X. Alexey has also noticed deadlocks in some corner cases on Linux. This patch, if applied on top of the logging patch (http://reviews.llvm.org/D13452), addresses the known deadlock issues.
(This also proactively removes the color escape sequences from the error report buffer since we have to copy the buffer anyway.)
Differential Revision: http://reviews.llvm.org/D14470
llvm-svn: 253689
When ASan currently detects a bug, by default it will only print out the text
of the report to stderr. This patch changes this behavior and writes the full
text of the report to syslog before we terminate the process. It also calls
os_trace (Activity Tracing available on OS X and iOS) with a message saying
that the report is available in syslog. This is useful, because this message
will be shown in the crash log.
For this to work, the patch makes sure we store the full report into
error_message_buffer unconditionally, and it also strips out ANSI escape
sequences from the report (they are used when producing colored reports).
I've initially tried to log to syslog during printing, which is done on Android
right now. The advantage is that if we crash during error reporting or the
produced error does not go through ScopedInErrorReport, we would still get a
(partial) message in the syslog. However, that solution is very problematic on
OS X. One issue is that the logging routine uses GCD, which may spawn a new
thread on its behalf. In many cases, the reporting logic locks threadRegistry,
which leads to deadlocks.
Reviewed at http://reviews.llvm.org/D13452
(In addition, add sanitizer_common_libcdep.cc to buildgo.sh to avoid
build failures on Linux.)
llvm-svn: 253688
MIPS build bots failed due to lack of 64bit atomic operations.
The fix is to disable VP for MIPS target until a better solution
is found.
llvm-svn: 253687
On Linux, we have internal_fork that forks without invoking user's pthread_atfork handlers, which is important for spawning external symbolizers. Let's implement this for OS X as well (using __fork). This patch also adds internal_forkpty which re-implements forkpty and uses __fork in it as well.
Differential Revision: http://reviews.llvm.org/D14869
llvm-svn: 253666
OS X doesn't support POSIX semaphores (but it does have the API for it, which returns ENOSYS - "Function not implemented").
Differential Revision: http://reviews.llvm.org/D14865
llvm-svn: 253665
On OS X, we don't have pthread spinlocks, let's just use a regular mutex instead. Secondly, pthread_rwlock_t is much larger (200 bytes), so `char padding_[64 - sizeof(pthread_rwlock_t)]` actually underflows.
Differential Revision: http://reviews.llvm.org/D14862
llvm-svn: 253659
Several tests rely on CLOCK_MONOTONIC, which doesn't exist on OS X. This patch fixes these tests by either disabling them (in case of cond_version.c which doesn't make sense on OS X), or by porting the test to also work on OS X.
Differential Revision: http://reviews.llvm.org/D14861
llvm-svn: 253658
Apparently check_c_compiler_flag isn't a thing everywhere. Not being
incredibly well-versed in cmake, I'm hoping that check_cxx_compiler_flag
serves a similar purpose. :)
llvm-svn: 253648
On OS X, this unit test (ThreadSpecificDtors) fails, because the new and delete operators actually call the overridden operators, which end up using TLVs and crash. Since C++'s new and delete is not important in this test, let's just replace them with a local variable. This fixes the test on OS X.
Differential Revision: http://reviews.llvm.org/D14826
llvm-svn: 253583
Summary:
Newer libstdc++ has global pool, which is filled with objects
allocated during libstdc++ initialization, and never released.
Using use_globals=0 in the lit tests results in these objects
being treated as leaks.
Fix this by porting several tests to plain C, and introducing
a simple sanity test case for __lsan::ScopedDisabler.
Reviewers: kcc, ygribov
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14798
llvm-svn: 253576
On OS X, the thread finalization is fragile due to thread-local variables destruction order. I've seen cases where the we destroy the ThreadState too early and subsequent thread-local values' destructors call interceptors again. Let's replace the TLV-based thread finalization method with libpthread hooks. The notification PTHREAD_INTROSPECTION_THREAD_TERMINATE is called *after* all TLVs have been destroyed.
Differential Revision: http://reviews.llvm.org/D14777
llvm-svn: 253560
On OS X, we build a dylib of the TSan runtime, which doesn't necessarily need to contain debugging symbols (and file and line information), so llvm-symbolizer might not be able to find file names for TSan internal frames. FrameIsInternal currently only considers filenames, but we should simply treat all frames within `libclang_rt.tsan_osx_dynamic.dylib` as internal. This patch treats all modules starting with `libclang_rt.tsan_` as internal, because there may be more runtimes for other platforms in the future.
Differential Revision: http://reviews.llvm.org/D14813
llvm-svn: 253559
Several testcases need pthread barriers (e.g. all bench_*.cc which use test/tsan/bench.h) which are not available on OS X. Let's mark them with "UNSUPPORTED: darwin".
Differential Revision: http://reviews.llvm.org/D14636
llvm-svn: 253558
Reimplement dispatch_once in an interceptor to solve these issues that may produce false positives with TSan on OS X:
1) there is a racy load inside an inlined part of dispatch_once,
2) the fast path in dispatch_once doesn't perform an acquire load, so we don't properly synchronize the initialization and subsequent uses of whatever is initialized,
3) dispatch_once is already used in a lot of already-compiled code, so TSan doesn't see the inlined fast-path.
This patch uses a trick to avoid ever taking the fast path (by never storing ~0 into the predicate), which means the interceptor will always be called even from already-compiled code. Within the interceptor, our own atomic reads and writes are not written into shadow cells, so the race in the inlined part is not reported (because the accesses are only loads).
Differential Revision: http://reviews.llvm.org/D14811
llvm-svn: 253552
Summary:
dlopen(NULL, ...) is intended to give you back a handle to the
executable for use with dlsym. Casting it to link_map and using it with
ForEachMappedRegion results in a crash.
We also shouldn't unpoison the globals of a DSO that is already in
memory. This ensures that we don't do it for the executable, but in
general, MSan may have false negatives if the DSO is already loaded.
Reviewers: eugenis
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14795
llvm-svn: 253530
1. Added missing public API decl in InstrProfiling.h
2. Clang formatting fix
3. Added more comments for new VP code
4. refactor the VP allocation code to make it more readable.
llvm-svn: 253508
This change adds extends the data structures and adds in the routines
for handling runtime calls for value profiling. The profile data format
is modified and the version number is incremented.
llvm-svn: 253483
Symbolizers can call malloc/realloc/free/..., which we don't want to intercept. This is already implemented on Linux, let's do it for OS X as well.
Differential Revision: http://reviews.llvm.org/D14710
llvm-svn: 253460
This patch adds assembly routines to enable setjmp/longjmp for aarch64
on linux. It fixes:
* test/tsan/longjmp2.cc
* test/tsan/longjmp3.cc
* test/tsan/longjmp4.cc
* test/tsan/signal_longjmp.cc
I also checked with perlbench from specpu2006 (it fails to run
with missing setjmp/longjmp intrumentation).
llvm-svn: 253205
- This is to handle a corner case where profile lib is linked
in but non of the modules are instrumented (On linux, since
we avoided the overhead to emit runtime hook use functions so
this is the side effect of that size optimization).
- Added a profile runtime test case to cover all scenarios of
shared library builds.
Differential Revision: http://reviews.llvm.org/D14468
llvm-svn: 253098
I noticed that when a symbol is named just "x", it gets demangled to "long long". On POSIX, AFAIK, mangled names always start with "_Z", so lets just require that.
Differential Revision: http://reviews.llvm.org/D14637
llvm-svn: 253080
Currently, we crash on finalization of detached threads, because we'll try to clear the ThreadState twice.
Differential Revision: http://reviews.llvm.org/D14644
llvm-svn: 253079
On OS X, memcpy and memmove are actually aliases of the same function, so the memmove interceptor can be invoked on a call to memcpy. This patch updates the tests to expect either memmove or memcpy on a stack trace.
Differential Revision: http://reviews.llvm.org/D14638
llvm-svn: 253077
The OS X symbolizers (namely AtosSymbolizer) don't return full file paths, only file names. This patch modifies `mutexset*.cc` tests not to require a path to be present in the symbol on the stack trace.
Differential Revision: http://reviews.llvm.org/D14642
llvm-svn: 253075
Turns out that there are some checks in the backend that change code generation for armv7 if you are building against an iOS sys root. For the macho_embedded builtins we really don't want that.
llvm-svn: 253064
This patch adds support for symbolication of globals (implements `SymbolizeData`) for `AtosSymbolizer` on OS X.
Differential Revision: http://reviews.llvm.org/D14618
llvm-svn: 253015
We need to add -fPIC to the flags for the builtins in case PIC was turned off at a higher level. We also want to set ENABLE_PIC to Off when building the macho_embedded builtins so the top-level settings don't impact that build.
llvm-svn: 252966
Go build does not link in whatever library provides these symbols:
# runtime/race
race_windows_amd64.syso:gotsan.cc:(.text+0x578f): undefined reference to `__sanitizer::DumpProcessMap()'
race_windows_amd64.syso:gotsan.cc:(.text+0xee33): undefined reference to `EnumProcessModules'
race_windows_amd64.syso:gotsan.cc:(.text+0xeeb9): undefined reference to `GetModuleInformation'
llvm-svn: 252922
`DlAddrSymbolizer` is used on OS X when we're running inside a sandbox that prevents us from spawning an external symbolizer. This patch adds support for symbolication of globals (implements `SymbolizeData`) for `DlAddrSymbolizer`.
Differential Revision: http://reviews.llvm.org/D14613
llvm-svn: 252899
The default symbolizer, `llvm-symbolizer` provides sizes for global symbols. On OS X, we want to also allow using `atos` (because it's available everywhere and users don't need to copy/install it) and `dladdr` (it's the only available option when running in a sandbox). However, these symbolizers do not supply the symbol sizes, only names and starting addresses. This patch changes the reporting functions to hide the size of the symbol when this value is unavailable, and modifies tests to make this part of the report "optional".
Differential Revision: http://reviews.llvm.org/D14608
llvm-svn: 252896
This seems to be dead code – `system_malloc_zone` is initialized as null and we never assign to it. This code is apparently currently never executed (in ASan), but is causes trouble for the TSan OS X port. Let's replace the checks with `COMMON_MALLOC_ENTER` (`ENSURE_ASAN_INITED()`).
Differential Revision: http://reviews.llvm.org/D14334
llvm-svn: 252876
The TSan unit test build currently fails if we're also building the iOS parts of compiler-rt, because `TSAN_SUPPORTED_ARCH` contains ARM64. For unit tests, we need to filter this only to host architecture(s).
Differential Revision: http://reviews.llvm.org/D14604
llvm-svn: 252873
Setting CMAKE_*_FLAGS isn't sufficient here because CMAKE_*_FLAGS_${CMAKE_BUILD_TYPE} can override the flags, and there is no way to safely clear that because it is a cached variable (<sarcasm> YAY! </sarcasm>).
llvm-svn: 252807
Summary: This is enough to get the asan static_tls.cc test case working.
Reviewers: eugenis, samsonov
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14574
llvm-svn: 252738
Summary:
On PowerPC64 Linux PTRACE_GETREGS is a #define and PT_GETREGS is not.
On other systems it's the other way round. Extend the #ifs to check for
both PTRACE_* and PT_*.
This fixes test/sanitizer_common/TestCases/Linux/ptrace.cc when msan is
enabled for PowerPC64.
Reviewers: zatrazz, kcc, eugenis, samsonov
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14564
llvm-svn: 252730
Atomic RMW is not necessary in InitializeGuardArray.
It is supposed to run when no user code runs.
And if user code runs concurrently, then the atomic
RMW won't help anyway. So replace it with non-atomic RMW.
InitializeGuardArray takes more than 50% of time during re2 fuzzing:
real 0m47.215s
51.56% a.out a.out [.] __sanitizer_reset_coverage
6.68% a.out a.out [.] __sanitizer_cov
3.41% a.out a.out [.] __sanitizer::internal_bzero_aligned16(void*, unsigned long)
1.79% a.out a.out [.] __asan::Allocator::Allocate(unsigned long, unsigned long,
With this change:
real 0m31.661s
26.21% a.out a.out [.] sanitizer_reset_coverage
10.12% a.out a.out [.] sanitizer_cov
5.38% a.out a.out [.] __sanitizer::internal_bzero_aligned16(void*, unsigned long)
2.53% a.out a.out [.] __asan::Allocator::Allocate(unsigned long, unsigned long,
That's 33% speedup.
Reviewed in http://reviews.llvm.org/D14537
llvm-svn: 252715
Summary:
On Darwin, interposed functions are prefixed with "wrap_". On Linux,
they are prefixed with "__interceptor_".
Reviewers: dvyukov, samsonov, glider, kcc, kubabrecka
Subscribers: zaks.anna, llvm-commits
Differential Revision: http://reviews.llvm.org/D14512
llvm-svn: 252695
Summary:
First batch of sancov.py rewrite in C++.
Supports "-print" and "-covered_functions" commands.
Differential Revision: http://reviews.llvm.org/D14356
llvm-svn: 252683
* Setting CMAKE_*_FLAGS_${BUILD_TYPE} isn't really needed since we're setting the same value everywhere
* functions sanitize variables differently from macros, darwin_add_embedded_builtin_libraries should be a macro otherwise it won't alter the variables.
llvm-svn: 252618
Adhemerval Zanella