Fixes an esan workingset-memset test failure by switching to malloc to
avoid a shadow mapping issue with mmap in certain situations that will be
fully fixed separately.
llvm-svn: 270949
The max warning check was masking the "return 0" codepath.
See the thread "Warnings and compile-time failure on 458.sjeng" for more
info.
llvm-svn: 270762
After r270617 I am getting an error when building:
projects/compiler-rt/lib/profile/InstrProfilingFile.c:33:36:
error: missing field 'PidChars' initializer
[-Werror,-Wmissing-field-initializers]
lprofFilename lprofCurFilename = {0};
^
Fix the aggregate initializer.
llvm-svn: 270697
Summary:
Adds the base runtime library for the working set tool.
Adds slowpath code for updating the shadow memory.
To be added in the future:
+ Scan memory and report the total size.
+ Take samples for intermediate values.
Reviewers: aizatsky
Subscribers: kubabrecka, vitalybuka, zhaoqin, kcc, eugenis, llvm-commits
Differential Revision: http://reviews.llvm.org/D20485
llvm-svn: 270650
Summary: Adds a new test struct-simple.cpp for testing the
cache-fragmentation tool with a multi-compilation-unit application.
Patch by Qin Zhao.
Reviewers: bruening
Subscribers: kubabrecka, kcc, vitalybuka, eugenis, aizatsky, llvm-commits, zhaoqin
Differential Revision: http://reviews.llvm.org/D20599
llvm-svn: 270631
Summary:
Adds processCompilationUnitInit and processCompilationUnitExit
for compilation unit init/exit.
Adds a tool-specific argument passed to esan_init.
Adds the dtor and esan_exit called from the dtor.
A test will be added separately (adding it here results in failure until
the corresponding compilation patch is in place).
Reviewers: aizatsky
Subscribers: kubabrecka, bruening, kcc, vitalybuka, eugenis, llvm-commits
Differential Revision: http://reviews.llvm.org/D20486
llvm-svn: 270624
Using -fomit-frame-pointer sometimes makes a crash log miss some frames. Let's not use this optimization in debug builds.
Differential Revision: http://reviews.llvm.org/D20425
llvm-svn: 270376
In one of the already existing apps that I'm testing TSan on, I really see a mutex path that is longer than 10 (but not by much, something like 11-13 actually). Let's raise this to 20 and weaken the assertion so we don't crash.
Differential Revision: http://reviews.llvm.org/D20427
llvm-svn: 270319
Summary:
Adds custom flag support to EfficiencySanitizer's runtime library.
Adds an initial flag cache_line_size which will be used by multiple tools.
Reviewers: aizatsky, vitalybuka
Subscribers: llvm-commits, eugenis, kcc, zhaoqin, aizatsky, kubabrecka
Differential Revision: http://reviews.llvm.org/D20478
llvm-svn: 270256
Summary:
Adds support for app libraries starting slightly below 0x7f00'00000000 (the
mmap ASLR range extends down to 0x7efb'f8000000 for reasonable stack
limits) by switching to a shadow mapping offset of 0x1300'00000000.
Reviewers: aizatsky
Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka
Differential Revision: http://reviews.llvm.org/D20479
llvm-svn: 270255
We're missing interceptors for dispatch_after and dispatch_after_f. Let's add them to avoid false positives. Added a test case.
Differential Revision: http://reviews.llvm.org/D20426
llvm-svn: 270071
Summary:
Adds *fstat to the common interceptors.
Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.
Resubmit of http://reviews.llvm.org/D20318 with ios build fixes.
Reviewers: eugenis, vitalybuka, aizatsky
Subscribers: zaks.anna, kcc, bruening, kubabrecka, srhines, danalbert, tberghammer
Differential Revision: http://reviews.llvm.org/D20350
llvm-svn: 269981
The previous patch (r269291) was reverted (commented out) because the patch caused leaks that
were detected by LSan and they broke some lit tests. The actual reason was that dlsym allocates
an error string buffer in TLS, and some LSan lit tests are intentionally not scanning TLS for
root pointers. This patch simply makes LSan ignore the allocation from dlsym, because it's
not interesting anyway.
llvm-svn: 269917
There is no frame validity check in the slow unwinder like there is in the fast unwinder due to which lsan reports a leak even for heap allocated coroutine in the test swapcontext.cc. Since mips/linux uses slow unwindwer instead of fast unwinder, the test fails for mips/linux. Therefore adding the checks before unwinding fixes the test for mips/linux.
Reviewed by aizatsky.
Differential: http://reviews.llvm.org/D19961
llvm-svn: 269882
Summary:
Adds *fstat to the common interceptors.
Removes the now-duplicate fstat interceptor from msan/tsan
This adds fstat to asan/esan, which previously did not intercept it.
Reviewers: eugenis, vitalybuka, aizatsky
Subscribers: tberghammer, danalbert, srhines, kubabrecka, bruening, kcc
Differential Revision: http://reviews.llvm.org/D20318
llvm-svn: 269856
The ignore_interceptors_accesses setting did not have an effect on mmap, so
let's change that. It helps in cases user code is accessing the memory
written to by mmap when the synchronization is ensured by the code that
does not get rebuilt.
(This effects Swift interoperability since it's runtime is mapping memory
which gets accessed by the code emitted into the Swift application by the
compiler.)
Differential Revision: http://reviews.llvm.org/D20294
llvm-svn: 269855
Fixes a bug in checking the endpoint of a shadow region and removes an
invalid check (both introduced in http://reviews.llvm.org/rL269198).
llvm-svn: 269834
http://reviews.llvm.org/rL269291 introduced a memory leak.
Disabling offending call temprorary rather than rolling back the chain
of CLs.
llvm-svn: 269799
Fix https://llvm.org/bugs/show_bug.cgi?id=27673.
Currenty ASan checks the return value of real recv/recvfrom to see if the written bytes fit in the buffer. That works fine most of time.
However, there is an exception: (from the RECV(2) man page)
MSG_TRUNC (since Linux 2.2)
... return the real length of the packet or datagram, even when it was longer than the passed buffer. ...
Some programs combine MSG_TRUNC, MSG_PEEK and a single-byte buffer to peek the incoming data size without reading (much of) them. In this case,
the return value is usually longer than what's been written and ASan raises a false alarm here. To avoid such false positive reports,
we can use min(res, len) in COMMON_INTERCEPTOR_WRITE_RANGE checks.
Differential Revision: http://reviews.llvm.org/D20280
llvm-svn: 269749
licensees actually see in the toolchain we deliver to them. This will
reduce the set of local patches we have to maintain. The triple is
not changing. (The term ORBIS is an internal code name for PS4.)
llvm-svn: 269672
Summary:
When using a multi-configuration build (i.e. MSVC) the output path where
libraries are dropped is incorrect.
Example:
```
C:\src\llvm\examples>d:\src\llvm\build\Release\bin\clang-cl.exe -fsanitize=address test.cc
LINK : fatal error LNK1181: cannot open input file 'd:\src\llvm\build\Release\bin\..\lib\clang\3.9.0\lib\windows\clang_rt.asan-i386.lib'
```
The dropped executable path contains the configuration 'Release':
```
'd:\src\llvm\build\Release\bin\..\lib\clang\3.9.0\lib\windows\Release\clang_rt.asan-i386.lib'
```
The variable 'RUNTIME_OUTPUT_DIRECTORY' is used to specify the output directory.
But CMAKE is appending the current configuration (i.e. Debug, Release).
see: https://cmake.org/cmake/help/v3.0/prop_tgt/RUNTIME_OUTPUT_DIRECTORY.html
```
"Multi-configuration generators (VS, Xcode) append a per-configuration subdirectory to the specified directory."
```
To avoid this problem, the configuration specific variable must be set:
'RUNTIME_OUTPUT_DIRECTORY_DEBUG', 'RUNTIME_OUTPUT_DIRECTORY_RELEASE', and so on.
Reviewers: ddunbar, chapuni, rnk
Subscribers: kubabrecka, llvm-commits
Differential Revision: http://reviews.llvm.org/D20261
llvm-svn: 269658
This patch tries to fix https://llvm.org/bugs/show_bug.cgi?id=27310 by using the same hack for malloc as we use for calloc: allocate corresponding memory from internal buffer when ASan is not initialized.
This way we could avoid nasty '==6987==AddressSanitizer CHECK failed: ../../../../libsanitizer/asan/asan_rtl.cc:556 "((!asan_init_is_running && "ASan init calls itself!")) != (0)" (0x0, 0x0)' errors in
environments with glibc 2.23+ in use, where _dl_signal_error, called from dlsym for undefined symbols calls malloc in order to get a buffer for error message.
Differential Revision: http://reviews.llvm.org/D20235
llvm-svn: 269633
This is part-3 of the effort to eliminate dependency on
libc allocator in instr profiler runtime. With this change,
the profile dumper is completely free of malloc/calloc.
Value profile instr API implementation is the only remaining
piece with calloc dependency.
llvm-svn: 269576
This reverts commit r269493 as the corresponding LLVM commit was
reverted due to lots of warnings. See the review thread for the original
LLVM commit (r269491) for details.
llvm-svn: 269550
With this change, dynamic memory allocation is only used
for testing purpose. This change is one of the many steps to
make instrument profiler dynamic allocation free.
llvm-svn: 269453
The introduction of the Swift demangler now causes an assertion failure when we
try to demangle nullptr, but we used to allow that (and return nullptr back).
This situation is rare, but it can still happen. Let's allow nullptr.
llvm-svn: 269302
Summary:
On a 32-bit MIPS, the `ld` instruction does not exist. However, GAS has an `ld`
macro that expands to a pair of `lw` instructions which load to a pair of
registers (reg, and reg+1). This macro is not available in the Integrated
Assembler and its use causes -fintegrated-as builds to fail. Even if it were
available, the behaviour on 32-bit MIPS would be incorrect since the current
usage of `ld` causes the code to clobber $5 (which is supposed to hold
child_stack). It also clobbers $k0 which is reserved for kernel use.
Aside from enabling builds with the integrated assembler, there is no functional
change since internal_clone() is only used by StopTheWorld() which is only used
by 64-bit sanitizers.
Reviewers: kcc, sagar
Subscribers: mohit.bhakkad, jaydeep, sagar, llvm-commits
Differential Revision: http://reviews.llvm.org/D18753
llvm-svn: 269297
To invoke the Swift demangler, we use dlsym to locate swift_demangle. However, dlsym malloc's storage and stores it in thread-local storage. Since allocations from the symbolizer are done with the system allocator (at least in TSan, interceptors are skipped when inside the symbolizer), we will crash when we try to deallocate later using the sanitizer allocator again.
To fix this, let's just not call dlsym from the demangler, and call it during initialization. The dlsym function calls malloc, so it needs to be only used after our allocator is initialized. Adding a Symbolizer::LateInitialize call that is only invoked after all other initializations.
Differential Revision: http://reviews.llvm.org/D20015
llvm-svn: 269291
We're using forkpty to spawn the atos symbolizer. In some cases, login_tty (part of forkpty) can fail due to security measures (sandboxing). In this case, we should exit with a status code instead of completely crashing the spawned process. Even processing a failed CHECK() is problematic here, because we're post-fork and pre-exec where a lot of things don't work (for multithreaded processes, for OS X GUI apps, etc.).
Differential Revision: http://reviews.llvm.org/D20048
llvm-svn: 269289
While debugging ASan and TSan, I sometimes get a recursion during a failed CHECK processing. CheckFailed can call a lot of code (printing, unwinding a stack trace, symbolicating, ...) and this can fail another CHECK. This means I sometimes see a crash due to a infinite recursion stack overflow. Let's stop after 10 failed CHECKs and just kill the process immediately. I also added a Sleep(2) call before the trap, so that other threads still get a chance to print their failed CHECKs.
Differential Revision: http://reviews.llvm.org/D20047
llvm-svn: 269288
This is needed for proper operation of the fast unwinder, see the discussion
on D18895.
Differential Revision: http://reviews.llvm.org/D19898
llvm-svn: 269277
ASan runtime library used libcorkscrew from Android platform for
stack unwinding. Since Android L, this is both unnecessary (the
libgcc unwinder has been fixed) and impossible (the library is not
there any more). Don't even try.
This should have not effect on modern Android devices other than
removing a message about failing to open the library with
ASAN_OPTIONS=verbosity=1.
llvm-svn: 269233
Adds *stat to the common interceptors.
Removes the now-duplicate *stat interceptor from msan/tsan/esan.
This adds *stat to asan, which previously did not intercept it.
Patch by Qin Zhao.
llvm-svn: 269223
Summary:
Adds shadow memory mapping support common to all tools to the new
Efficiencysanitizer ("esan") family of tools. This includes:
+ Shadow memory layout and mapping support for 64-bit Linux for any
power-of-2 scale-down (1x, 2x, 4x, 8x, 16x, etc.) that ensures that
shadow(shadow(address)) does not overlap shadow or application
memory.
+ Mmap interception to ensure the application does not map on top of
our shadow memory.
+ Init-time sanity checks for shadow regions.
+ A test of the mmap conflict mechanism.
Reviewers: aizatsky, filcab
Subscribers: filcab, kubabrecka, llvm-commits, vitalybuka, eugenis, kcc, zhaoqin
Differential Revision: http://reviews.llvm.org/D19921
llvm-svn: 269198
Another stack where we try to free sync objects,
but don't have a processors is:
// ResetRange
// __interceptor_munmap
// __deallocate_stack
// start_thread
// clone
Again, it is a latent bug that lead to memory leaks.
Also, increase amount of memory we scan in MetaMap::ResetRange.
Without that the test does not fail, as we fail to free
the sync objects on stack.
llvm-svn: 269041
Summary:
This patch adds support for building lib/builtins without a fully functioning toolchain. It allows you to bootstrap a cross-compiler, which previously couldn't be done with CMake.
This patch contains the following specific changes:
* Split builtin-specific code out of config-ix.cmake into builtin-config-ix.cmake
* Split some common CMake functionality needed by both builtins and sanitizers into base-config-ix.cmake
* Made lib/builtins/CMakeLists.txt able to be a top-level CMake configuration
I have tested this on Darwin targeting embedded Darwin, and on FreeBSD x86_64 targeting FreeBSD AArch64.
This patch depends on http://reviews.llvm.org/D19692, and is the last part of http://reviews.llvm.org/D16653.
Reviewers: samsonov, iains, jroelofs
Subscribers: compnerd, aemerson, tberghammer, danalbert, srhines, emaste, llvm-commits
Differential Revision: http://reviews.llvm.org/D19742
llvm-svn: 268977
This reverts commit r268840, as it breaks Thumb2 self-hosting. There is something
unstable in the profiling for Thumb2 that needs to be sorted out before we continue
implementing these changes to the profiler. See PR27667.
llvm-svn: 268864
Fixes crash reported in:
https://bugs.chromium.org/p/v8/issues/detail?id=4995
The problem is that we don't have a processor in a free interceptor
during thread exit.
The crash was introduced by introduction of Processors.
However, previously we silently leaked memory which wasn't any better.
llvm-svn: 268782
To invoke the Swift demangler, we use dlsym to locate swift_demangle. However, dlsym malloc's storage and stores it in thread-local storage. Since allocations from the symbolizer are done with the system allocator (at least in TSan, interceptors are skipped when inside the symbolizer), we will crash when we try to deallocate later using the sanitizer allocator again.
To fix this, let's just not call dlsym from the demangler, and call it during initialization.
Differential Revision: http://reviews.llvm.org/D19974
llvm-svn: 268716
In recovery mode, when ASan detects stack overflow (say, when infinite recursion detected),
it tries to continue program execution and hangs on repetitive error reports. There isn't any
sense to do it, we can just bail out on stack overflow error, because the program would crash soon anyway.
Differential Revision: http://reviews.llvm.org/D19958
llvm-svn: 268713
Summary:
Adds stat/__xstat to the common interceptors.
Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.
Resubmit of http://reviews.llvm.org/D19875 with win build fixes.
Reviewers: aizatsky, eugenis
Subscribers: tberghammer, llvm-commits, danalbert, vitalybuka, bruening, srhines, kubabrecka, kcc
Differential Revision: http://reviews.llvm.org/D19890
llvm-svn: 268466
Summary:
Adds stat/__xstat to the common interceptors.
Removes the now-duplicate stat/__xstat interceptor from msan/tsan/esan.
This adds stat/__xstat to asan, which previously did not intercept it.
Reviewers: aizatsky, eugenis
Subscribers: tberghammer, danalbert, srhines, kubabrecka, llvm-commits, vitalybuka, eugenis, kcc, bruening
Differential Revision: http://reviews.llvm.org/D19875
llvm-svn: 268440
Summary:
One of the big limitations we have in the compiler-rt build system today is that we cannot bootstrap building the builtins because you need a fully functional toolchain to pass CMake's tests.
This change adds support for compile only tests.
It is NFC because nothing is using the compile-only tests yet.
I believe this is the last separable part of D16653.
Reviewers: samsonov
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D19692
llvm-svn: 268427
Summary:
Replaces {} with a do..while sequence in esan's empty interceptors to allow
natural use with a trailing semicolon. The sequence uses each argument to
avoid warnings.
Reviewers: filcab
Subscribers: kubabrecka, llvm-commits, zhaoqin
Differential Revision: http://reviews.llvm.org/D19832
llvm-svn: 268426
We used to depend on host gcc. But some distributions got
new gcc recently which broke the check. Generally, we can't
depend that an arbitrary host gcc generates something stable.
Switch to clang.
This has an additional advantage of catching regressions in
clang codegen.
llvm-svn: 268382
Summary:
On Windows (already fixed) and FreeBSD we have stacks traces without
operator().
Reviewers: eugenis
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D19427
llvm-svn: 268332
Summary:
Use after scope is not detected if array larger then 8 bytes.
Subscribers: kubabrecka
Differential Revision: http://reviews.llvm.org/D19572
llvm-svn: 268330
Summary:
Hello,
Building a recent gcc on a powerpc-linux system advertsing:
Red Hat Enterprise Linux Server release 5.10 (Tikanga)
we stumbled on a compilation error on a file originating
from compiler-rt/lib/sanitizer-common.
sanitizer_platform_limits_linux.cc #includes asm/posix_types.h,
which, on our system, uses __kernel_fd_set and associated macros.
These aren't defined at the point of their use, and the compilation
fails with symptoms like:
In file included from ../../../../src/libsanitizer/sanitizer_common/sanitizer_platform_limits_linux.cc:29:0:
/usr/include/asm/posix_types.h:72:51: error: '__kernel_fd_set' has not been declared
static __inline__ void __FD_SET(unsigned long fd, __kernel_fd_set *fdsetp)
...
The attached patch is a suggestion to fix this, by including linux/posix_types.h
instead of asm/posix_types.h. linux/posix_types defines the necessary types and
macros, then #includes asm/posix_types.h.
We have been using it locally for gcc without problems for a couple of years
on powerpc, x86 and x86_64-linux platforms. It is still needed for gcc-6 on
our powerpc host and applies cleanly on the compiler-rt trunk.
Comments ?
Thanks much in advance for your feedback,
With Kind Regards,
Olivier
Reviewers: llvm-commits, kcc
Subscribers: kcc, kubabrecka
Differential Revision: http://reviews.llvm.org/D19799
llvm-svn: 268283
This happens on a 64-bit platform that uses SizeClassAllocator32 (e.g. ASan on AArch64). When querying a large invalid pointer, `__sanitizer_get_allocated_size(0xdeadbeefdeadbeef)`, an assertion will fail. This patch changes PointerIsMine to return false if the pointer is outside of [kSpaceBeg, kSpaceBeg + kSpaceSize).
Differential Revision: http://reviews.llvm.org/D15008
llvm-svn: 268243
There is a hard-to-reproduce crash happening on OS X that involves terminating the main thread (dispatch_main does that, see discussion at http://reviews.llvm.org/D18496) and later reusing the main thread's ThreadContext. This patch disables reuse of the main thread. I believe this problem exists only on OS X, because on other systems the main thread cannot be terminated without exiting the process.
Differential Revision: http://reviews.llvm.org/D19722
llvm-svn: 268238
In http://reviews.llvm.org/D19100, I introduced a bug: On OS X, existing programs rely on malloc_size() to detect whether a pointer comes from heap memory (malloc_size returns non-zero) or not. We have to distinguish between a zero-sized allocation (where we need to return 1 from malloc_size, due to other binary compatibility reasons, see http://reviews.llvm.org/D19100), and pointers that are not returned from malloc at all.
Differential Revision: http://reviews.llvm.org/D19653
llvm-svn: 268157
Summary:
This (partially) implements the check mentioned at
http://kristerw.blogspot.co.uk/2016/04/dangling-pointers-and-undefined-behavior.html
(via John Regehr)
Quoting:
"That the behavior is undefined follows from C11 6.2.4 "Storage
durations of objects"
The lifetime of an object is the portion of program execution during
which storage is guaranteed to be reserved for it. An object exists, has
a constant address, and retains its last-stored value throughout its
lifetime. If an object is referred to outside of its lifetime, the
behavior is undefined. The value of a pointer becomes indeterminate when
the object it points to (or just past) reaches the end of its lifetime.
and 7.22.3 "Memory management functions" that says that free ends the
lifetime of objects
The lifetime of an allocated object extends from the allocation until
the deallocation.
"
We can probably implement this for stack variables too, but I think this
is a good start to see if there's interest in this check.
We can also hide this behind a flag, too.
Reviewers: samsonov, kcc, rsmith, regehr
Subscribers: kubabrecka, llvm-commits
Differential Revision: http://reviews.llvm.org/D19691
llvm-svn: 268097
On s390*-linux, sigcontext just doesn't contain any information that could
be used to recover the type of access, so there's no way to fix this, short
of emulating the faulting instruction.
Differential Revision: http://reviews.llvm.org/D19655
llvm-svn: 267960
Since __builtin_setjmp has been fixed by rL267943, the test now works
on PowerPC. Enable it.
On the other hand, the SystemZ backend doesn't currently support
__builtin_setjmp. Disable it.
Differential Revision: http://reviews.llvm.org/D19657
llvm-svn: 267946
This happens to be working now because the includes exist in another CMake file that is included before this one. That will change with upcoming refactoring.
llvm-svn: 267912
On Darwin, MAP_ANONYMOUS is a synonym for MAP_ANON. However, some SDK's
don't define MAP_ANONYMOUS. Use MAP_ANON to work around this.
(As a point of interest, the situation is exactly reversed on Linux.)
llvm-svn: 267907
Recent TSan changes (r267678) which factor out parts of ThreadState into a Processor structure broke worker threads on OS X. This fixes it by properly calling ProcCreate for GCD worker threads and by replacing some CHECKs with RAW_CHECK in early process initialization. CHECK() in TSan calls the allocator, which requires a valid Processor.
llvm-svn: 267864
On linux, some architectures had an ABI transition from 64-bit long double
(ie. same as double) to 128-bit long double. On those, glibc symbols
involving long doubles come in two versions, and we need to pass the
correct one to dlvsym when intercepting them.
A few more functions we intercept are also versioned (all printf, scanf,
strtold variants), but there's no need to fix these, as the REAL() versions
are never called.
Differential Revision: http://reviews.llvm.org/D19555
llvm-svn: 267794
Summary:
The strchr and strrchr interceptors are sometimes invoked too early
for their REAL() counterparts to be initialized. We have seen this in
hooks invoked from tcmalloc on the dlsym() used in initializing
interceptors. A special check is added to use internal_ routines for
this situation.
Reviewers: vitalybuka, aizatsky, filcab
Subscribers: filcab, llvm-commits, eugenis, kcc, zhaoqin, aizatsky, kubabrecka
Differential Revision: http://reviews.llvm.org/D19607
llvm-svn: 267793
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork(). ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected. Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.
Unfortunately, there's no reliable way to detect the fix without crashing
the kernel. So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work. In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.
Differential Revision: http://reviews.llvm.org/D19576
llvm-svn: 267747
UnmapOrDie used to do MEM_DECOMMIT and so worked
on partial regions. But r263160 changed it to use
MEM_RELEASE and MEM_RELEASE can only work with
whole regions mapped by VirtualAlloc. This broke
windows as:
FATAL: ThreadSanitizer CHECK failed: gotsan.cc:8296 "((mbi.AllocationBase == addr && "Windows cannot unmap part of a previous mapping")) != (0)" (0x0, 0x0)
Restore the previous behavior.
llvm-svn: 267730
os_trace turns out to be a macro that creates static object.
Function-static objects use __cxa_atexit and __dso_handle
which are not present in Go runtime.
llvm-svn: 267720
Ifdef out global variables with destructors.
This requires runtime support that is not provided by Go runtime
(in particular _dso_handle symbol).
llvm-svn: 267709
Current interface assumes that Go calls ProcWire/ProcUnwire
to establish the association between thread and proc.
With the wisdom of hindsight, this interface does not work
very well. I had to sprinkle Go scheduler with wire/unwire
calls, and any mistake leads to hard to debug crashes.
This is not something one wants to maintian.
Fortunately, there is a simpler solution. We can ask Go
runtime as to what is the current Processor, and that
question is very easy to answer on Go side.
Switch to such interface.
llvm-svn: 267703
tsan_debugging.cc: In function ‘void* __tsan_get_current_report()’:
tsan_debugging.cc:61:18: warning: cast from type ‘const __tsan::ReportDesc*’
to type ‘void*’ casts away qualifiers [-Wcast-qual]
return (void *)rep;
llvm-svn: 267679
This is reincarnation of http://reviews.llvm.org/D17648 with the bug fix pointed out by Adhemerval (zatrazz).
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.
llvm-svn: 267678
Summary:
On windows platform assert() call creates two distinct CFG edges
which are coverage-instrumented. Simply calling assert would
change coverage numbers on the platform.
Subscribers: kubabrecka
Differential Revision: http://reviews.llvm.org/D19514
llvm-svn: 267610
This change modifies find_darwin_sdk_dir to set a variable if a Darwin "Internal" SDK is present which allows CMake to disable components that require internal-only APIs.
This mechanism is then used to disable TSan when an internal SDK is not present.
llvm-svn: 267575
This reverts commit r267477.
It broke our bots that enables the AArch64 backends, it seems that
this code is using a Darwin *X86 specific* field.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 267526
This fixes fails in test/msan/dlerror.cc - when real dlerror calls strcmp,
our strcmp interceptor now skips poison checking, since it's called in
interceptor context. Strictly speaking, only the dlerror change is
necessary to fix the fail, but let's also change the other two just in case.
Differential Revision: http://reviews.llvm.org/D19499
llvm-svn: 267486
The interception context is not used by esan, but the compiler complains
about it being uninitialized all the same. We set it to null to avoid the
warning.
llvm-svn: 267376
Summary:
Adds libc interceptors to the runtime library for the new
EfficiencySanitizer ("esan") family of tools. The interceptors cover
the memory operations in most common library calls and will be shared
among all esan tools.
Reviewers: aizatsky
Subscribers: zhaoqin, tberghammer, danalbert, srhines, llvm-commits, vitalybuka, eugenis, kcc
Differential Revision: http://reviews.llvm.org/D19411
llvm-svn: 267293
This reverts commit r267094, because it broke a lot of MSAN tests in AArch64.
Being NFC and all, this needs some deeper investigation before it goes in again.
llvm-svn: 267136
Summary: When using 32-bit python with 64-bit asan the pc array in sancov.py cannot fit in 64-bit pc's because the type-code 'L' for
arrays in python corresponds to the C type long which is only of 4 bytes. Because of this some of the coverage tool tests fail on
mips. To fix these test possible solutions are to use 64-bit python or use struct.unpack with the 'Q' type-code. We have used
struct.unpack with 'Q' type code since it is not appropriate to have a 64-bit python on all hosts.
Reviewed by kcc, aizatsky
Differential: http://reviews.llvm.org/D18817
llvm-svn: 267126
Fix and enable working stack-use-after-scope tests.
Add more failing tests for the feature, for fix later.
PR27453.
Patch by Vitaly Buka.
llvm-svn: 267084
Summary:
Adds the initial version of a runtime library for the new
EfficiencySanitizer ("esan") family of tools. The library includes:
+ Slowpath code via callouts from the compiler instrumentation for
each memory access.
+ Registration of atexit() to call finalization code.
+ Runtime option flags controlled by the environment variable
ESAN_OPTIONS. The common sanitizer flags are supported such as
verbosity and log_path.
+ An initial simple test.
Still TODO: common code for libc interceptors and shadow memory mapping,
and tool-specific code for shadow state updating.
Reviewers: eugenis, vitalybuka, aizatsky, filcab
Subscribers: filcab, vkalintiris, kubabrecka, llvm-commits, zhaoqin, kcc
Differential Revision: http://reviews.llvm.org/D19168
llvm-svn: 267060
The field "pid" in ReportThread is used to store the OS-provided thread ID (pthread_self or gettid). The name "pid" suggests it's a process ID, which it isn't. Let's rename it.
Differential Revision: http://reviews.llvm.org/D19365
llvm-svn: 266994
Let's use pthread_threadid_np which returns a more reasonable ID than pthread_self (which is actually a stack pointer). The numbers from pthread_threadid_np are already used in other tools, e.g. in LLDB, and often appear in logs, so it's much more useful than pthread_self.
Differential Revision: http://reviews.llvm.org/D18951
llvm-svn: 266991
Not sure what changed, but on my machine this is literally one byte
short. Only happens when malloc_context_size <= 2 due to the special
case in GET_STACK_TRACE definition (see asan_stack.h):
StackTrace::GetCurrentPc() on the right (context size > 2) branch
returns the address that is 200-something bytes from the return
address it is later matched to, while the same call on the left
branch is 321 bytes away from it.
This fixes the double-free test on my machine.
llvm-svn: 266932
Instead of calling a sanitizer_common function, implement GetPageSize in the
test directly. MSan runtime does not export __sanitizer::* symbols, and the
current code breaks when the test and the runtime library are in the separate
link units (ex. when the test is built as a shared library).
llvm-svn: 266910
Windows does not honour the __attribute__((pcs)) on ARM. Although this will
result in ABI mismatches, compiler-rt should largely be unneeded for resolving
dependencies as we generate MS ABI compliant library calls now for the most
part.
llvm-svn: 266891
The real problem is that sanitizer_print_stack_trace obtains current PC and
expects the PC to be in the stack trace after function calls. We don't
prevent tail calls in sanitizer runtimes, so this assumption does not
necessary hold.
We add "always inline" attribute on PrintCurrentStackSlow to address this
issue, however this solution is not reliable enough, but unfortunately, we
don't see any simple, reliable solution.
Reviewers: samsonov hfinkel kbarton tjablin dvyukov kcc
http://reviews.llvm.org/D19148
Thanks Hal, dvyukov, and kcc for invaluable discussion, I have even borrowed
part of dvyukov's summary as my commit message!
llvm-svn: 266869
This patch fixes https://github.com/google/sanitizers/issues/669. On older Darwin systems (in particular, Darwin 10),
dyld doesn't export '_dyldVersionNumber' symbol so we would have 'undefined reference' error in sanitzer library. Although
sanitizers support was added to LLVM on OS X 10.7+ where '_dyldVersionNumber' symbol is already exported, GCC users still
may want use them on older systems.
Differential Revision: http://reviews.llvm.org/D19218
llvm-svn: 266868
Summary: There is no frame validity check in the slow unwinder like there is in the fast unwinder due to which lsan reports a leak even for heap allocated coroutine in the test swapcontext.cc. Since mips/linux uses slow unwindwer instead of fast unwinder, the test fails for mips/linux. Therefore adding the checks before unwinding fixes the test for mips/linux.
Reviewers: samsonov, earthdok, kcc
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D18690
llvm-svn: 266716
This breaks the valloc test on PowerPC, which has 64kiB pages. Since
getting page size portably is nontrivial, and there's already a function
for that in __sanitizer, just use it. Unfortunately, sanitizer_common.h
conflicts with the interface headers inclucded by msan_test.cc (and a few
of its own macros), so we have to declare it manually.
Differential Revision: http://reviews.llvm.org/D19227
llvm-svn: 266688
Summary: Test that asan detects access to the dead variable captured by lambda.
Reviewers: aizatsky, kcc
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D19238
llvm-svn: 266676
On s390, siginfo reports the faulting address with page granularity -
we need to mask off the low bits of sp before comparison.
Differential Revision: http://reviews.llvm.org/D19112
llvm-svn: 266593
At the moment almost every lit.site.cfg.in contains two lines comment:
## Autogenerated by LLVM/Clang configuration.
# Do not edit!
The patch adds variable LIT_SITE_CFG_IN_HEADER, that is replaced from
configure_lit_site_cfg with the note and some useful information.
llvm-svn: 266520
The test is failing on Windows because we do not have a definition for
DemangleSwiftAndCXX nor DemangleCXXABI, which I am replacing, on Windows.
llvm-svn: 266499
Add support for Swift names when symbolicating sanitizer traces. This is
now relevant since TSan and ASan support have been added to Swift on OS X.
Differential Revision: http://reviews.llvm.org/D19135
llvm-svn: 266494
sanitizer_common is now in good enough shape on s390x to support UBSan
- all tests passing. Let's enable it.
Differential Revision: http://reviews.llvm.org/D19157
llvm-svn: 266483
This file will contain s390-specific code. For now, let's move the s390
version of internal_mmap here.
Differential Revision: http://reviews.llvm.org/D19174
llvm-svn: 266482
Clang's StaticAnalyzer seems to (correctly) complain about code like:
T *p = calloc(sizeof(U), N);
...Where T and U are different types.
This patch removes some instances of this pattern from compiler-rt.
Patch by Apelete Seketeli.
Differential Revision: http://reviews.llvm.org/D19085
llvm-svn: 266388
On s390, the return address is in %r14, which is saved 14 words from
the frame pointer.
Unfortunately, there's no way to do a proper fast backtrace on SystemZ
with current LLVM - the saved %r15 in fixed-layout register save
area points to the containing frame itself, and not to the next one.
Likewise for %r11 - it's identical to %r15, unless alloca is used
(and even if it is, it's still useless). There's just no way to
determine frame size / next frame pointer. -mbackchain would fix that
(and make the current code just work), but that's not yet supported
in LLVM. We will thus need to XFAIL some asan tests
(Linux/stack-trace-dlclose.cc, deep_stack_uaf.cc).
Differential Revision: http://reviews.llvm.org/D18895
llvm-svn: 266371
This is the first part of upcoming asan support for s390 and s390x.
Note that there are bits for 31-bit support in this and subsequent
patches - while LLVM itself doesn't support it, gcc should be able
to make use of it just fine.
Differential Revision: http://reviews.llvm.org/D18888
llvm-svn: 266370
The PS_STRINGS constant can easily be incorrect with mismatched
kernel/userland - e.g. when building i386 sanitizers on FreeBSD/amd64
with -m32. The kern.ps_strings sysctl was introduced over 20 years ago
as the supported way to fetch the environment and argument string
addresses from the kernel, so the fallback is never used.
Differential Revision: http://reviews.llvm.org/D19027
llvm-svn: 266305
In short, CVE-2016-2143 will crash the machine if a process uses both >4TB
virtual addresses and fork(). ASan, TSan, and MSan will, by necessity, map
a sizable chunk of virtual address space, which is much larger than 4TB.
Even worse, sanitizers will always use fork() for llvm-symbolizer when a bug
is detected. Disable all three by aborting on process initialization if
the running kernel version is not known to contain a fix.
Unfortunately, there's no reliable way to detect the fix without crashing
the kernel. So, we rely on whitelisting - I've included a list of upstream
kernel versions that will work. In case someone uses a distribution kernel
or applied the fix themselves, an override switch is also included.
Differential Revision: http://reviews.llvm.org/D18915
llvm-svn: 266297
This teaches sanitizer_common about s390 and s390x virtual space size.
s390 is unusual in that it has 31-bit virtual space.
Differential Revision: http://reviews.llvm.org/D18896
llvm-svn: 266296
mmap on s390 is quite a special snowflake: since it has too many
parameters to pass them in registers, it passes a pointer to a struct
with all the parameters instead.
Differential Revision: http://reviews.llvm.org/D18889
llvm-svn: 266295
The custom zone implementation for OS X must not return 0 (even for 0-sized allocations). Returning 0 indicates that the pointer doesn't belong to the zone. This can break existing applications. The underlaying allocator allocates 1 byte for 0-sized allocations anyway, so returning 1 in this case is okay.
Differential Revision: http://reviews.llvm.org/D19100
llvm-svn: 266283
With -fsized-deallocation, new[] vs delete mismatch is reported as
new-delete-type-mismatch. This is technically true, but
alloc-dealloc-mismatch describes it better.
llvm-svn: 266246
We need to handle the case when handler is NULL in dispatch_source_set_cancel_handler and similar interceptors.
Differential Revision: http://reviews.llvm.org/D18968
llvm-svn: 266080
In `AtosSymbolizer`, we're using `forkpty()` to create a new pseudo-terminal to communicate with the `atos` tool (we need that to avoid output buffering in interactive mode). This however redirects both stdout and stderr into a single stream, so when we read the output, we can't distinguish between errors and standard replies. Let's save&restore stderr to avoid that.
Differential Revision: http://reviews.llvm.org/D15073
llvm-svn: 265923
glibc can use one of 2 layouts for semaphores: architectures that
don't HAVE_64B_ATOMIC use an uint32_t field with semaphore value,
then a private field, then a waiting thread count field - this is
the layout currently assumed by the test. However, HAVE_64B_ATOMIC
arches use a fused uint64_t field that contains the value in low bits
and waiting thread count in high bits, followed by a private field.
This resulted in taking private field from the wrong offset on 64-bit
atomic platforms (the test still passed, but didn't actually test
the private field). On big-endian platforms, this resulted in a fail,
since the first 4 bytes overlay the thread count field, and not
the value field.
Found while porting ASan to s390x.
Patch by Marcin Kościelnicki.
llvm-svn: 265715
Summary:
The strlen interceptor is sometimes invoked too early for REAL(strlen) to
be initialized. A special check is added to use internal_strlen for this
situation.
Reviewers: dim
Subscribers: llvm-commits, samsonov
Differential Revision: http://reviews.llvm.org/D18851
Change-Id: I3acc58f4abbae1904f25324abd84efea67aad0a2
llvm-svn: 265705
On one of our testing machines, we're running the tests under heavy load, and especially in the fork-based TSan tests, we're seeing timeouts when a test uses sleep(10), assuming that calling fork() on another thread will finish sooner than that. This patch removes a timeout and makes another one longer.
Differential Revision: http://reviews.llvm.org/D18476
llvm-svn: 265666
OS X provides atomic functions in libkern/OSAtomic.h. These provide atomic guarantees and they have alternatives which have barrier semantics. This patch adds proper TSan support for the functions from libkern/OSAtomic.h.
Differential Revision: http://reviews.llvm.org/D18500
llvm-svn: 265665
To avoid using the public header (tsan_interface_atomic.h), which has different data types, let's add all the __tsan_atomic* functions to tsan_interface.h.
Differential Revision: http://reviews.llvm.org/D18543
llvm-svn: 265663
Adding an interceptor with two more release+acquire pairs to avoid false positives with dispatch_apply.
Differential Revision: http://reviews.llvm.org/D18722
llvm-svn: 265662
XPC APIs have async callbacks, and we need some more happen-before edges to avoid false positives. This patch add them, plus a test case (sorry for the long boilerplate code, but XPC just needs all that).
Differential Revision: http://reviews.llvm.org/D18493
llvm-svn: 265661
GCD has APIs for event sources, we need some more release-acquire pairs to avoid false positives in TSan.
Differential Revision: http://reviews.llvm.org/D18515
llvm-svn: 265660
In the interceptor for dispatch_sync, we're currently missing synchronization between the callback and the code *after* the call to dispatch_sync. This patch fixes this by adding an extra release+acquire pair to dispatch_sync() and similar APIs. Added a testcase.
Differential Revision: http://reviews.llvm.org/D18502
llvm-svn: 265659
A little embarrassing, but we're missing the call to FileCheck in several Darwin tests. Let's fix this.
Differential Revision: http://reviews.llvm.org/D18503
llvm-svn: 265658
If we don't create the target, don't try to add it as a dependency.
After r265595, we were only creating the SanitizerLintCheck when
`CMAKE_HOST_UNIX` was true.
CMake was emitting a warning:
The dependency target "SanitizerLintCheck" of target "check-ubsan" does not
exist.
llvm-svn: 265613
Summary:
After patch https://lkml.org/lkml/2015/12/21/340 is introduced in
linux kernel, the random gap between stack and heap is increased
from 128M to 36G on 39-bit aarch64. And it is almost impossible
to cover this big range. So we need to disable randomized virtual
space on aarch64 linux.
Reviewers: llvm-commits, zatrazz, dvyukov, rengolin
Subscribers: aemerson, rengolin, tberghammer, danalbert, srhines
Differential Revision: http://reviews.llvm.org/D18526
llvm-svn: 265366
We've reset thr->ignore_reads_and_writes, but forget to do
thr->fast_state.ClearIgnoreBit(). So ignores were not effective
reset and fast_state.ignore_bit was corrupted if signal handler
itself uses ignores.
Properly reset/restore fast_state.ignore_bit around signal handlers.
llvm-svn: 265288
For debugging it is useful to be able to generate dSYM files but not strip the executables. This change adds the ability to skip stripping by setting COMPILER_RT_EXTERNALIZE_DEBUGINFO_SKIP_STRIP=On.
llvm-svn: 265057
This test case is meant to check whether verbose error logging works. It
does this by _failing_ to write a profile to a path corresponding to a
directory. Make this clearer by writing to a temporary directory.
Patch suggested by Sean Silva!
llvm-svn: 264817
This also makes it work on PS4 a bit better.
For running the libprofile tests on PS4, we are sneaking environment
variables through on the command line, so the extra argument for
LLVM_PROFILE_FILE was messing up the simple `argc < 2` check.
llvm-svn: 264812
This addresses PR27077. For some historical reason Darwin wasn't shipping multi3 in the compiler builtin library or in the OS builtin library. This caused building ffmpeg to fail because Polly was generating calls to multi3. It is easy enough to just add the builtin.
llvm-svn: 264750
to function names
Summary:
Hopefully this will make it easier for the next person to figure all
this out...
Reviewers: bogner, davidxl
Subscribers: davidxl, cfe-commits
Differential Revision: http://reviews.llvm.org/D18489
llvm-svn: 264680
This is implicitly needed at least by gcc-flag-compatibility.test
The thing that needs it is the `\` preceding the "default.profraw"
appended internally by clang when doing `-fprofile-use=`.
Clang uses `\` because is uses sys::path::append which will use `\` on a
Windows host. This is wrong, but I don't think there's an easy way to
solve it (maybe just always using `/` since places that accept `\` also
tend to accept `/`, but not the other way around).
llvm-svn: 264665
This change introduces routines that register and unregister all
instrumented globals in a loaded executable image.
These routines are only implemented on Darwin, where globals metadata
is expected to be placed in the __DATA,__asan_globals section.
Review: http://reviews.llvm.org/D16841
llvm-svn: 264644
This patch fixes the custom ThreadState destruction on OS X to avoid crashing when dispatch_main calls pthread_exit which quits the main thread.
Differential Revision: http://reviews.llvm.org/D18496
llvm-svn: 264627
Summary:
Hopefully this will make it easier for the next person to figure all
this out...
Reviewers: bogner, davidxl
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D18490
llvm-svn: 264612
On PS4, we have to fake environment variables by passing extra command
line arguments, so the dummy test `argc > 1` was failing.
The condition is just a dummy condition that the compiler can't fold
away, so the number is arbitrary as long as the condition is false.
Increase the number it compares against.
llvm-svn: 264491
Summary:
Currently, sanitizer_common_interceptors.inc has an implicit, undocumented
assumption that the sanitizer including it has previously declared
interceptors for memset and memmove. Since the memset, memmove, and memcpy
routines require interception by many sanitizers, we add them to the
set of common interceptions, both to address the undocumented assumption
and to speed future tool development. They are intercepted under a new
flag intercept_intrin.
The tsan interceptors are removed in favor of the new common versions. The
asan and msan interceptors for these are more complex (they incur extra
interception steps and their function bodies are exposed to the compiler)
so they opt out of the common versions and keep their own.
Reviewers: vitalybuka
Subscribers: zhaoqin, llvm-commits, kcc
Differential Revision: http://reviews.llvm.org/D18465
llvm-svn: 264451
ucrtbase.dll appears to be built with some kind of cross-module
inlining, because there are calls to imported Heap* routines sprinkled
throughout the code. This inlining defeats our attempts to hotpatch
malloc, _malloc_base, and related functions. Failing to intercept an
allocation or deallocation results in a crash when the program attempts
to deallocate or reallocate memory with the wrong allocator.
This change patches the IAT of ucrtbase.dll to replace the addresses of
the imported Heap* functions with implementations provided by ASan. We
don't globally intercept the win32 Heap* functions because they are
typically used by system DLLs that run before ASan initializes.
Eventually, we may want to intercept them, but for now I think this is
the minimal change that will keep ASan stable.
Reviewers: samsonov
Differential Revision: http://reviews.llvm.org/D18413
llvm-svn: 264327
On OS X, fork() under TSan asserts (in debug builds only) because REAL(fork) calls some intercepted functions, which check that no internal locks are held via CheckNoLocks(). But the wrapper of fork intentionally holds some locks. This patch fixes that by using ScopedIgnoreInterceptors during the call to REAL(fork). After that, all the fork-based tests seem to pass on OS X, so let's just remove all the UNSUPPORTED: darwin annotations we have.
Differential Revision: http://reviews.llvm.org/D18409
llvm-svn: 264261
On OS X, internal_mmap just uses mmap, which can invoke callbacks into libmalloc (e.g. when MallocStackLogging is enabled). This can subsequently call other intercepted functions, and this breaks our Darwin-specific ThreadState initialization. Let's use direct syscalls in internal_mmap and internal_munmap. Added a testcase.
Differential Revision: http://reviews.llvm.org/D18431
llvm-svn: 264259
Summary:
Adds strnlen to the common interceptors, under the existing flag
intercept_strlen.
Removes the now-duplicate strnlen interceptor from asan and msan.
This adds strnlen to tsan, which previously did not intercept it.
Adds a new test of strnlen to the sanitizer_common test cases.
Reviewers: samsonov
Subscribers: zhaoqin, llvm-commits, kcc
Differential Revision: http://reviews.llvm.org/D18397
llvm-svn: 264195
This is necessary to support the dynamic CRT (/MD) with VS2015. In
VS2015, these symbols are no longer imported from a DLL, they provided
statically by msvcrt.lib. This means our approach of hotpatching the DLL
no longer works.
By exporting the symbols, we end up relying on the same mechanism that
we use to intercept symbols in the static CRT (/MT) case. The ASan
runtime always needs to appear first on the link line, and the linker
searches for symbol definitions from left to right. This means we can
stop hotpatching operator new and delete in the CRT, which is nice.
I think that the only reason we weren't exporting the symbols already is
because MSVC doesn't allow you to do it directly with
__declspec(dllexport). Instead, we can use
`#pragma comment(linker, "/export:foo")`, which is most of what the
attribute does under the hood. It does mean we have to write down the
mangled names of the operators, but that's not too bad.
llvm-svn: 264190
This reverts commits r264068 and r264079, and they were breaking the build and
weren't reverted in time, nor they exhibited expected behaviour from the
reviewers. There is more to discuss than just a test fix.
llvm-svn: 264150
Summary:
After patch https://lkml.org/lkml/2015/12/21/340 is introduced in
linux kernel, the random gap between stack and heap is increased
from 128M to 36G on 39-bit aarch64. And it is almost impossible
to cover this big range. So I think we need to disable randomized
virtual space on aarch64 linux.
Reviewers: kcc, llvm-commits, eugenis, zatrazz, dvyukov, rengolin
Subscribers: rengolin, aemerson, tberghammer, danalbert, srhines, enh
Differential Revision: http://reviews.llvm.org/D18003
llvm-svn: 264068
Some unit tests were failing because we didn't intercept strdup. It
turns out it works just fine on 2013 and 2015 with a small patch to the
interception logic.
llvm-svn: 264013
printf is an inline function in VS 2015, giving these tests an
unexpected extra point of coverage. This change works around that by
avoiding printf.
llvm-svn: 264010
VS 2015 moved the priority of their exception filter initializer from
XIY to XCAA. We now set ours to XCAB, which makes it run after both CRT
versions but before user constructors, as it should.
Fixes null_deref.cc and a variety of related tests with VS 2015. Only 4
failures remain.
llvm-svn: 264006
There are some places in the CRT (such as mbctype) that directly call
_malloc_base. If you are incrementally linking a binary with ASan from
before this change, this change appears to result in a linker error.
Retrying the link succeeds for some reason.
llvm-svn: 264005
Adds strchr, strchrnul, and strrchr to the common interceptors, under a new
common flag intercept_strchr.
Removes the now-duplicate strchr interceptor from asan and all 3
interceptors from tsan. Previously, asan did not intercept strchrnul, but
does now; previously, msan did not intercept strchr, strchrnul, or strrchr,
but does now.
http://reviews.llvm.org/D18329
Patch by Derek Bruening!
llvm-svn: 263992
ASan interceptors contain local statics which run before the CRT is
initialized. Thread-safe statics appear to rely on CRT-internal state,
and will crash without this flag.
llvm-svn: 263979
In VS 2015, the memset fill parameter is zero extended from one byte
instead of being copied wholesale.
The issue reproduces with existing tests if you use VS2015.
llvm-svn: 263966
`__tsan_get_report_thread` and others can crash if a stack trace is missing, let's add the missing checks.
Differential Revision: http://reviews.llvm.org/D18306
llvm-svn: 263939
Update the compiler-rt cmake to build TSan dylibs for iOS-style simulators when the
corresponding COMPILER_RT_ENABLE_FOO_OS setting is enabled.
Differential Revision: http://reviews.llvm.org/D18277
Part of rdar://problem/24048382
llvm-svn: 263910
__clear_cache on Android is identical to the version on Linux. Use __linux__
instead of __ANDROID__ as __linux__ is defined for Linux and Android.
llvm-svn: 263833
Summary:
Introducing InitializeCommonFlags accross all sanitizers to simplify
common flags management.
Setting coverage=1 when html_cov_report is requested.
Differential Revision: http://reviews.llvm.org/D18273
llvm-svn: 263820
On OS X, we have pthread_cond_timedwait_relative_np. TSan needs to intercept this API to avoid false positives when using condition variables.
Differential Revision: http://reviews.llvm.org/D18184
llvm-svn: 263782
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.
TSan unit tests run with a statically linked runtime, where interceptors don't work. To avoid aborting the process in this case, the patch replaces `DisableReexec()` with a weak `ReexecDisabled()` function which is defined to return true in unit tests.
Differential Revision: http://reviews.llvm.org/D18212
llvm-svn: 263695
This patch adds a new TSan report type, ReportTypeMutexInvalidAccess, which is triggered when pthread_mutex_lock or pthread_mutex_unlock returns EINVAL (this means the mutex is invalid, uninitialized or already destroyed).
Differential Revision: http://reviews.llvm.org/D18132
llvm-svn: 263641
On OS X 10.11+, we have "automatic interceptors", so we don't need to use DYLD_INSERT_LIBRARIES when launching instrumented programs. However, non-instrumented programs that load TSan late (e.g. via dlopen) are currently broken, as TSan will still try to initialize, but the program will crash/hang at random places (because the interceptors don't work). This patch adds an explicit check that interceptors are working, and if not, it aborts and prints out an error message suggesting to explicitly use DYLD_INSERT_LIBRARIES.
Differential Revision: http://reviews.llvm.org/D18121
llvm-svn: 263551
That change did:
-#if defined(__BIG_ENDIAN__)
+#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
If __BYTE_ORDER__ and __ORDER_BIG_ENDIAN__ aren't defined, like
they are with MSVC, this condition is true (0 == 0).
Fixes PR26919.
llvm-svn: 263324
Summary:
Adds strlen to the common interceptors, under a new common flag
intercept_strlen. This provides better sharing of interception code among
sanitizers and cleans up the inconsistent type declarations of the
previously duplicated interceptors.
Removes the now-duplicate strlen interceptor from asan, msan, and tsan.
The entry check semantics are normalized now for msan and asan, whose
private strlen interceptors contained multiple layers of checks that
included impossible-to-reach code. The new semantics are identical to the
old: bypass interception if in the middle of init or if both on Mac and not
initialized; else, call the init routine and proceed.
Patch by Derek Bruening!
Reviewers: samsonov, vitalybuka
Subscribers: llvm-commits, kcc, zhaoqin
Differential Revision: http://reviews.llvm.org/D18020
llvm-svn: 263177
Summary:
Use InternalScopedString more extensively. This reduces the number of
write() syscalls, and reduces the chance that UBSan output will be
mixed with program output.
Reviewers: vitalybuka
Subscribers: kcc, llvm-commits
Differential Revision: http://reviews.llvm.org/D18068
llvm-svn: 263176
Now ASan can return virtual memory to the underlying OS. Portable
sanitizer runtime code needs to be aware that UnmapOrDie cannot unmap
part of previous mapping.
In particular, this required changing how we implement MmapAlignedOrDie
on Windows, which is what Allocator32 uses.
The new code first attempts to allocate memory of the given size, and if
it is appropriately aligned, returns early. If not, it frees the memory
and attempts to reserve size + alignment bytes. In this region there
must be an aligned address. We then free the oversized mapping and
request a new mapping at the aligned address immediately after. However,
a thread could allocate that virtual address in between our free and
allocation, so we have to retry if that allocation fails. The existing
thread creation stress test managed to trigger this condition, so the
code isn't totally untested.
Reviewers: samsonov
Differential Revision: http://reviews.llvm.org/D17431
llvm-svn: 263160
Summary:
Recently I saw the test `TestCases/Posix/print_cmdline.cc` failing on
FreeBSD, with "expected string not found in input". This is because
asan could not retrieve the command line arguments properly.
In `lib/sanitizer_common/sanitizer_linux.cc`, this is taken care of by
the `GetArgsAndEnv()` function, but it uses `__libc_stack_end` to get at
the required data. This variable does not exist on BSDs; the regular
way to retrieve the arguments and environment information is via the
`kern.ps_strings` sysctl.
I added this functionality in sanitizer_linux.cc, as a separate #ifdef
block in `GetArgsAndEnv()`. Also, `ReadNullSepFileToArray()` becomes
unused due to this change. (It won't work on FreeBSD anyway, since
`/proc` is not mounted by default.)
Reviewers: kcc, emaste, joerg, davide
Subscribers: llvm-commits, emaste
Differential Revision: http://reviews.llvm.org/D17832
llvm-svn: 263157
Summary: This is an initial setup in order to move some additional tests from Linux onto Posix.
I also moved decorate_proc_maps onto the Linux directory
Finally added msan's definition for "stable-runtime".
Only a test requires it, and its commit message (r248014) seems to imply
that AArch64 is problematic with MSan.
Reviewers: samsonov, rengolin, t.p.northover, eugenis
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D17928
llvm-svn: 263142
Currently, TSan only reports everything in a formatted textual form. The idea behind this patch is to provide a consistent API that can be used to query information contained in a TSan-produced report. User can use these APIs either in a debugger (via a script or directly), or they can use it directly from the process (e.g. in the __tsan_on_report callback). ASan already has a similar API, see http://reviews.llvm.org/D4466.
Differential Revision: http://reviews.llvm.org/D16191
llvm-svn: 263126
Summary:
__BIG_ENDIAN__ and __LITTLE_ENDIAN__ are not supported by gcc, which
eg. for ubsan Value::getFloatValue will silently fall through to
the little endian branch, breaking display of float values by ubsan.
Use __BYTE_ORDER__ == __ORDER_BIG/LITTLE_ENDIAN__ as the condition
instead, which is supported by both clang and gcc.
Noticed while porting ubsan to s390x.
Patch by Marcin Kościelnicki!
Differential Revision: http://reviews.llvm.org/D17660
llvm-svn: 263077
Until now the only exception APIs supported by gcc_personality_v0
are DWARF EH and SJLJ. This adds support for ARM EHABI as well.
This is achieved by
a) changing the function signature on ARM EHABI,
b) unwinding the stack before returning _URC_CONTINUE_UNWIND.
See "Exception Handling ABI for the ARM Architecture" for details
(http://infocenter.arm.com/help/topic/com.arm.doc.ihi0038b/IHI0038B_ehabi.pdf).
Patch by Timon Van Overveldt.
llvm-svn: 263010
The GCDA writer closed the arc file before unlocking it. This causes an
EBADF while unlocking the file, and opens us up to racy behavior.
Fixes PR26847.
llvm-svn: 262779
Summary:
Adds another global to asan's odr_c_test to help force the target global to
not lie at the start of bss with the gold linker where it is always
aligned.
Patch by Derek Bruening!
llvm-svn: 262678
Summary:
Hi David, SCE folks,
What is implemented in this patch is enough for the upstream libprofile to
work for PGO with the PS4 game codebase I tested ("game7" for you SCE
folks; this is with a standalone build of compiler-rt).
The first change, which is simple, is to stub out gethostname. PS4
doesn't have a simple analog for this that doesn't bring in extra
OS libraries, so for now we do not support `%h` expansion.
This is consistent with internal B#136272.
The second change implies future work, but is a simple change at present.
PS4 does not have `getenv`, so for now we will introduce a shim.
This obviously makes it impossible for many of the tests to be run since
they require setting `LLVM_PROFILE_FILE=`.
I see two paths forward:
1. In the tests we are already wrapping execution with `%run` and so by
setting a PS4-specific expansion for `%run` we can pass the information
in another way We can adapt the getenv shim as appropriate.
We will need to experiment with this internally.
Maggie, Phillip, Filipe? Any ideas? Maybe ping me internally since we
may need to get into some PS4 vagaries. I'm thinking a fake getenv
library that uses some side channel for communication.
2. Another possibility which is more verbose is to use a separate clang
invocation with `-profile-generate=<filename>` to set the filename in
each test.
This might require redundant clang invocations though which may be
undesirable for upstream. David, thoughts?
Also, this is a fairly libprofile-specific workaround, so it e.g.
doesn't help Filipe's ASan work.
Overall, this approach sounds like a bit of a hack to me.
Small detail:
InstrProfilingPort.h seems like the natural place for the getenv shim,
but GCDAProfiling.c needs it as well. InstrProfilingUtil.h is currently
the only header common between InstrProfilingFile.c and GCDAProfiling.c.
I can move the shim to InstrProfilingPort.h and add an include to
GCDAProfiling.c as per your preference David.
Reviewers: davidxl, MaggieYi, phillip.power, filcab
Subscribers: simon.f.whittaker, slingn, probinson, llvm-commits
Differential Revision: http://reviews.llvm.org/D17676
llvm-svn: 262527
Incremented the pc for each architecture in accordance with StackTrace:GetPreviousInstructionPC
Reviewers: samsonov, dvyukov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D17802
llvm-svn: 262483
This code is actually never executed because all RUN lines trigger an
earlier heap-use-after-free, but there is still a compiler warning.
llvm-svn: 262276
This testcase failed on sanitizer-x86_64-linux buildbot in large parallel build due to race on
port 1234 between AddressSanitizer-i386-linux and AddressSanitizer-x86_64-linux instances of recvfrom.cc.
This patch tries to resolve the issue by relying on kernel to choose available port instead of hardcoding
its number in testcase.
Differential Revision: http://reviews.llvm.org/D17639
llvm-svn: 262204
Summary:
iOS on ARM64 doesn't unique RTTI.
Ref: clang's iOS64CXXABI::shouldRTTIBeUnique()
Due to this, pointer-equality will not necessarily work in this
architecture, across dylib boundaries.
dynamic_cast<>() will (as expected) still work, since Apple ships with
one prepared for this, but we can't rely on the type names being
pointer-equal.
I've limited the expensive strcmp check to the specific architecture
which needs it.
Example which triggers this bug:
lib.h:
struct X {
virtual ~X() {}
};
X *libCall();
lib.mm:
X *libCall() {
return new X;
}
prog.mm:
int main() {
X *px = libCall();
delete px;
}
Expected output: Nothing
Actual output:
<unknown>: runtime error: member call on address 0x00017001ef50 which does not point to an object of type 'X'
0x00017001ef50: note: object is of type 'X'
00 00 00 00 60 00 0f 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
^~~~~~~~~~~~~~~~~~~~~~~
vptr for ‘X’
Reviewers: kubabrecka, samsonov, eugenis, rsmith
Subscribers: aemerson, llvm-commits, rengolin
Differential Revision: http://reviews.llvm.org/D11502
llvm-svn: 262147
Currently ThreadState holds both logical state (required for race-detection algorithm, user-visible)
and physical state (various caches, most notably malloc cache). Move physical state in a new
Process entity. Besides just being the right thing from abstraction point of view, this solves several
problems:
1. Cache everything on P level in Go. Currently we cache on a mix of goroutine and OS thread levels.
This unnecessary increases memory consumption.
2. Properly handle free operations in Go. Frees are issue by GC which don't have goroutine context.
As the result we could not do anything more than just clearing shadow. For example, we leaked
sync objects and heap block descriptors.
3. This will allow to get rid of libc malloc in Go (now we have Processor context for internal allocator cache).
This in turn will allow to get rid of dependency on libc entirely.
4. Potentially we can make Processor per-CPU in C++ mode instead of per-thread, which will
reduce resource consumption.
The distinction between Thread and Processor is currently used only by Go, C++ creates Processor per OS thread,
which is equivalent to the current scheme.
llvm-svn: 262037
This test expects pthread_mutex_init in the frame #0 of thread T1 but we
get memset at frame #0 because memset that is called from pthread_init_mutex
is being intercepted by TSan
llvm-svn: 261986
Summary: Msan was intercepting version 2.1 of the pthread_create function which was making it to crash in libc because __pthread_create_2_1 modifies the stack attributes of the thread. Intercepting the correct version fixes the test SmallPreAllocatedStackThread.
Reviewers: eugenis, samsonov
Subscribers: llvm-commits, mohit.bhakkad, jaydeep
Differential: http://reviews.llvm.org/D17603
llvm-svn: 261980
Compiler-rt miscalculates the number of entries in the __llvm_prf_data section
on i386 Darwin. This results in a number of test failures (which we started
catching after r261344).
The fix we attempted earlier is insufficient (r261683). It caused some tests to
start passing again, but that hid the fact that we drop some data entries.
This patch should fix the real problem. It fixes the way we compute DataSize by
taking into account the way the Darwin linker lays out __llvm_prf_data.
Differential Revision: http://reviews.llvm.org/D17623
llvm-svn: 261957
Pass res instead of len as third parameter to COMMON_INTERCEPTOR_WRITE_RANGE,
because otherwise we can write to unrelated memory (in MSan) or get wrong report (in ASan).
Differential Revision: http://reviews.llvm.org/D17608
llvm-svn: 261898
This patch moves recv and recvfrom interceptors from MSan and TSan to
sanitizer_common to enable them in ASan.
Differential Revision: http://reviews.llvm.org/D17479
llvm-svn: 261841
Adjust the clobbers list. This use to work with older versions of gcc, but now
will error on newer versions (tested against 5.3) (as well as clang).
Patch by Tee Hao Wei!
llvm-svn: 261821
Summary: Building the sanitizer libraries without rpaths causes all sorts of problems when you try to use them. This simple fix should make it all work.
Reviewers: samsonov, zaks.anna
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D17556
llvm-svn: 261797
Summary: As per the test the 4th element of both arrays are not initialized and hence will contain garbage values. Memcmp returns the difference between the garbage values of the 4th element which will be different on every run of the test. And since the return value of memcmp is returned from main, we are getting random exit code every time.
Reviewers: kcc, eugenis
Subscribers: mohit.bhakkad, jaydeep, llvm-commits
Differential: http://reviews.llvm.org/D17534
llvm-svn: 261739
This makes it so that component-based installations will include resource files (i.e. blacklists). My next patch will add support for component-based installations.
llvm-svn: 261699
Fix a crash when gathering value profile data on i386 Darwin.
The Darwin linker shrinks sections containing aligned structures when
padding is not explicitly added to the end of the structure. When
iterating over these structures, be sure to not walk past the end of the
section.
No tests added, since running `ninja check-profile` on i386 Darwin is
enough to reproduce the original crash.
llvm-svn: 261683
The first issue is that we longjmp from ScopedInterceptor scope
when called from an ignored lib. This leaves thr->in_ignored_lib set.
This, in turn, disables handling of sigaction. This, in turn,
corrupts tsan state since signals delivered asynchronously.
Another issue is that we can ignore synchronization in asignal
handler, if the signal is delivered into an IgnoreSync region.
Since signals are generally asynchronous, they should ignore
memory access/synchronization/interceptor ignores.
This could lead to false positives in signal handlers.
llvm-svn: 261658
Test cases definitely should not care about the complete set of architectures
supported by compiler-rt - they should only care about current
architecture that the test suite was configured for.
Introduce new lit feature to reflect this, and convert tests to use it.
llvm-svn: 261603
Summary:
This removes the hard limit on the number of loaded modules (used to be
16K), and makes it easier to use LoadedModules w/o causing a memory
leak: ListOfModules owns the modules, and makes sure to properly clean
them in destructor.
Remove filtering functionality that is only needed in one place (LSan).
Reviewers: aizatsky
Subscribers: llvm-commits, kcc
Differential Revision: http://reviews.llvm.org/D17470
llvm-svn: 261554
I ran the test suite yesterday and when I came back this morning the
queue_user_work_item.cc test was hung. This could be why the
sanitizer-windows buildbot keeps randomly timing out. I updated all the
usages of WaitForSingleObject involving threading events. I'm assuming
the API can reliably wait for subprocesses, which is what the majority
of call sites use it for.
While I'm at it, we can simplify some EH tests now that clang can
compile C++ EH.
llvm-svn: 261338
Sean Silva