There's a TSan startup crash on Linux when used in Swift programs, where MonotonicNanoTime will try to call real_clock_gettime and then jump to NULL because interceptors are not yet initialized. This is on Ubuntu 18.04. Looks like TSan's main Initialize() function is called at a point where __progname is already set, but interceptors aren't yet set up. Let's fix this by checking whether interceptors are initialized in MonotonicNanoTime.
Differential Revision: https://reviews.llvm.org/D53528
llvm-svn: 345174
Summary:
At compile-time, create an array of {PC,HumanReadableStackFrameDescription}
for every function that has an instrumented frame, and pass this array
to the run-time at the module-init time.
Similar to how we handle pc-table in SanitizerCoverage.
The run-time is dummy, will add the actual logic in later commits.
Reviewers: morehouse, eugenis
Reviewed By: eugenis
Subscribers: srhines, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D53227
llvm-svn: 344985
Some Darwin functions have pairs like dispatch_apply and dispatch_apply_f so the added _f to interceptor types causes a clash. Let's add _type suffix instead.
Differential Revision: https://reviews.llvm.org/D53167
llvm-svn: 344954
Change the assumption when releasing memory to a buffer queue that new
generations might not be able to re-use the memory mapped addresses.
llvm-svn: 344882
Summary:
This change updates the buffer queue implementation to support using a
generation number to identify the lifetime of buffers. This first part
introduces the notion of the generation number, without changing the way
we handle the buffers yet.
What's missing here is the cleanup of the buffers. Ideally we'll keep
the two most recent generations. We need to ensure that before we do any
writes to the buffers, that we check the generation number(s) first.
Those changes will follow-on from this change.
Depends on D52588.
Reviewers: mboerger, eizan
Subscribers: llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D52974
llvm-svn: 344881
Summary:
This change allows us to handle allocator exhaustion properly in the
segmented array implementation. Before this change, we relied on the
caller of the `trim` function to provide a valid number of elements to
trim. This change allows us to do the right thing in case the elements
to trim is greater than the size of the container.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D53484
llvm-svn: 344880
Summary: This patch adds additional features and cpus from libgcc. Unfortunately we've overflowed the existing 32-bits of features so we had to add a new __cpu_features2 variable to hold the additional bits. This matches libgcc as far as I can tell.
Reviewers: echristo
Reviewed By: echristo
Subscribers: dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D53461
llvm-svn: 344830
Summary:
C++ flags should not be used for not-C++ files as it may trigger
-Werror=unused-command-line-argument. CMake will use CMAKE_C_FLAGS,
CMAKE_CXX_FLAGS, and CMAKE_ASM_FLAGS as appropriate implicitly, so
this does not need to be explicitly handled here.
This change depends on https://reviews.llvm.org/D53301, since one of
the builders depended on this behavior because it was not configuring
CMAKE_ASM_FLAGS.
Reviewers: eugenis, vitalybuka
Reviewed By: eugenis, vitalybuka
Subscribers: dberris, mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53335
llvm-svn: 344751
Summary:
This change updates the buffer queue implementation to support using a
generation number to identify the lifetime of buffers. This first part
introduces the notion of the generation number, without changing the way
we handle the buffers yet.
What's missing here is the cleanup of the buffers. Ideally we'll keep
the two most recent generations. We need to ensure that before we do any
writes to the buffers, that we check the generation number(s) first.
Those changes will follow-on from this change.
Depends on D52588.
Reviewers: mboerger, eizan
Subscribers: llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D52974
llvm-svn: 344670
This abstracts away the file descriptor related logic which makes it
easier to port XRay to platform that don't use file descriptors or
file system for writing the log data, such as Fuchsia.
Differential Revision: https://reviews.llvm.org/D52161
llvm-svn: 344578
Summary:
This change implements a controller for abstracting away the details of
what happens when tracing with FDR mode. This controller type allows us
to test in isolation the various cases where we're encountering function
entry, exit, and other kinds of events we are handling when FDR mode is
enabled.
This change introduces a number of testing facilities we've needed to
better support expressing the conditions we need for the unit tests. We
leave some TODOs for moving those utilities into the LLVM project,
sitting in the `Testing` library, to make matching conditions on XRay
`Trace` instances through googlemock more manageable and declarative.
We don't wire in the controller right away, to allow us to incrementally
update the implementation(s) as we increase testing coverage of the
controller type. There's a need to re-think the way we're managing
buffers in a multi-threaded environment, which is more invasive than
this implementation.
This step in the process allows us to encode our assumptions in the
implementation of the controller, and then evolve the buffer queue
implementation to support generational buffer management to ensure we
can continue to support the cases we're already supporting with the
controller.
Reviewers: mboerger, eizan
Subscribers: mgorny, llvm-commits, jfb
Differential Revision: https://reviews.llvm.org/D52588
llvm-svn: 344488
Summary:
Silence warning when linking unittest binary by not passing
-lstdc++ to the linker since it is ignored.
Reviewers: morehouse
Reviewed By: morehouse
Subscribers: mgorny
Differential Revision: https://reviews.llvm.org/D53225
llvm-svn: 344480
Summary:
The previous value looks Linux specific so that has been guarded with
the host OS being Linux.
On Apple platforms `%shared_libasan` expands to the absolute path of the
ASan dylib.
Previously on Linux `%shared_libasan` expanded to just the file name
of the shared library rather than the absolute path to the library.
This is likely a bug because it would rely on the OS's dynamic linker
to find the shared library which could accidentally pick up a system copy
rather than the shared library that was just built.
For other platforms we emit a warning if `config.asan_dynamic` is true.
This patch also only defines the substitution when `config.asan_dynamic`
is true because using this substitution only makes sense when the
dynamic library is available.
Reviewers: kubamracek, george.karpenkov, mgorny, phosek, etienneb, samsonov, kcc
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53111
llvm-svn: 344434
Summary:
Linking with the /OPT:REF linker flag when building COFF files causes
the linker to strip SanitizerCoverage's constructors. Prevent this by
giving the constructors WeakODR linkage and by passing the linker a
directive to include sancov.module_ctor.
Include a test in compiler-rt to verify libFuzzer can be linked using
/OPT:REF
Reviewers: morehouse, rnk
Reviewed By: morehouse, rnk
Subscribers: rnk, morehouse, hiraditya
Differential Revision: https://reviews.llvm.org/D52119
llvm-svn: 344391
This is a follow-up patch to r342541. After further investigations, only
48bits VMA size can be supported. As this is enforced in function
InitializePlatformEarly from lib/rt1/tsan_platform_linux.cc, the access
to the global variable vmaSize variable + switch can be removed. This
also addresses a comment from https://reviews.llvm.org/D52167.
vmaSize of 39 or 42bits are not compatible with a Go program memory
layout as the Go heap will not fit in the shadow memory area.
Patch by: Fangming Fang <Fangming.Fang@arm.com>
llvm-svn: 344329
`config.asan_dynamic` should actually be `True` because dylibs are the
only supported form of the ASan runtime on Apple platforms.
Reviewers: kubamracek, george.karpenkov, samsonov
Subscribers: srhines, mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D53183
llvm-svn: 344324
Summary:
The goal of the patch in D49853 is to display counter on the line of function definition.
So some tests need to be fixed.
Reviewers: marco-c, davidxl
Reviewed By: marco-c
Subscribers: sylvestre.ledru, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49854
llvm-svn: 344229
This allows users of static libraries (such as ubsan) to link without
knowing about this transitive dependency, if using the psapi functions
that require linking to a separate psapi library. Since Windows 7,
these functions (EnumProcessModules, GetModuleInformation,
GetProcessMemoryInfo) are remapped to K32- prefixed ones, available in
kernel32.dll.
Differential Revision: https://reviews.llvm.org/D53012
llvm-svn: 344126
Some seem fragile, some fail, and some just take a really long time to run.
It does not seem to make sense to support some subset of value profiling tests.
Differential Revision: https://reviews.llvm.org/D53047
llvm-svn: 344105
Without this change, tests in coverage.test and dump_coverage.test are
failing on non-x86_64 platforms.
The diff is copied from sanitizer_common library, an alternative would
be to link it together with libFuzzer.
Differential Revision: https://reviews.llvm.org/D53040
llvm-svn: 344104
Summary: This is just a minor cleanup to the allocateOneNode interface. The formals are no-longer used, so I just removed them.
Reviewers: davidxl, void
Reviewed By: davidxl
Subscribers: dberris, llvm-commits
Differential Revision: https://reviews.llvm.org/D52509
llvm-svn: 344073
PTHREAD_STACK_MIN is not available on NetBSD as it's not
clear what the value shall be represented by this constant
on a multiplatform OS.
llvm-svn: 344034
Summary:
The NetBSD version of internal routines no longer call
the indirect syscall interfaces, as these functions were
switched to lib calls.
Remove the remnant code complication that is no
longer needed after this change. Remove the variations
of internal_syscall, as they were NetBSD specific.
No functional change intended.
Reviewers: vitalybuka, joerg, javed.absar
Reviewed By: vitalybuka
Subscribers: kubamracek, fedor.sergeev, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D52955
llvm-svn: 343988
Summary:
The dl library does not exist on all system and in particular
this breaks build on NetBSD. Make it conditional and
enable only for Linux, following the approach from other
test suites in the same repository.
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D52994
llvm-svn: 343987
NetBSD returns a different type as a return value of
devname_r(3) than FreeBSD and Darwin (int vs char*).
This implies that checking for successful completion of this
function has to be handled differently.
This test used to work well, but was switched to fix Darwin,
which broke NetBSD.
Add a dedicated ifdef for NetBSD and make it functional again
for this OS.
llvm-svn: 343980
Data involving struct accesses accounting work (plan to support only efficiency-cache-frag flag in the frontend side).
Reviewers: krytarowski, vitalybuka, jfb
Reviewed By : vitalybuka
Differential Revision: https://reviews.llvm.org/D52608
llvm-svn: 343812
Summary:
GetStackTrace treats top PC as a return address from an error reporting
function, and adjusts it down by 1 instruction. This is not necessary in
a signal handler, so adjust PC up to compensate.
Reviewers: kcc, vitalybuka, jfb
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D52802
llvm-svn: 343638
This change fixes two aspects of logging on Fuchsia:
1. When logging to stdout, the output file descriptor should not be
closed in ExecuteCommand, as it prevent fdio_spawn_etc from succeeding
in subsequent calls.
2. When logging to a file in anything other than standalone Zircon, the
log file needs to be created in mutable storage. The best indicator
where this is relative to the Fuchsia component will be given by
'-artifact_prefix=...', so save the log file relative to that path.
Patch by: aarongreen
Differential Revision: https://reviews.llvm.org/D52562
llvm-svn: 343607
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
Differential Revision: https://reviews.llvm.org/D52091
llvm-svn: 343606
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.
New clients can read until they get empty string.
Reviewers: eugenis, morehouse
Subscribers: kubamracek, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D52743
llvm-svn: 343605
This patch removes the instruction set check to make the msr APSR_nzcvq,
ip instruction only execute if Thumb2 is used.
The APSR is a subset of the bits of the CPSR
(B.1.3.3 of the Arm v7 A and R ARM [1]) and is only available for A and
R profiles.
However in section B.9.3.11 of the same document we see that:
"In the A and R profiles, APSR_nzcvq is the same as CPSR_f"
"ARM recommends the APSR forms when only the N, Z, C, V, Q, and GE[3:0]
bits are being written."
This patch also make those files assemble for Armv8-M Mainline
architecture profile.
The builtins were cross-compiled for Arm, Aarch64 and Armv6-M, Armv7-M
and Armv7E-M targets.
Cross-compiled tests were executed for Arm target.
[1]: https://developer.arm.com/docs/ddi0406/latest/arm-architecture-reference-manual-armv7-a-and-armv7-r-edition
Patch by hug-dev (Hugues de Valon).
Differential Revision: https://reviews.llvm.org/D51854
llvm-svn: 343601
Summary:
This essentially reverts r337010 since it breaks UBSan, which is used
for a few platform libraries. The "-z global" flag is now added for
Scudo as well. The only other sanitizer shared libraries are for asan
and hwasan, which have also been reinstated to use the global flag.
Reviewers: cryptoad, eugenis
Reviewed By: cryptoad
Subscribers: kubamracek, mgorny, delcypher, #sanitizers, nickdesaulniers, chh, kongyi, pirama, llvm-commits
Differential Revision: https://reviews.llvm.org/D52770
llvm-svn: 343599
Summary:
Behavior for existing used is not changing as the first line is going
to be the same, and it was invalid to try to read more lines.
New clients can read until they get empty string.
Reviewers: eugenis, morehouse
Subscribers: kubamracek, eraman, llvm-commits
Differential Revision: https://reviews.llvm.org/D52743
llvm-svn: 343554
Summary:
When possible, switch syscall(2)/__syscall(2) calls
to direct calls of internal libc symbols.
Add a new function to detect address of a libc
symbol of a function that could be intercepted.
With the address detector in GetRealLibcAddress(),
an optional interceptor of libc call will be bypassed.
The original approach with syscall(2)/__syscall(2)
wasn't portable across supported ABIs and CPU
architectures. The indirect syscall interface is
also a candidate for removal in future revisions
of NetBSD, as the C language ABI is not a good
domain for serialization of arbitrary functions
arguments.
Switch the following functions to libc calls:
- internal_mmap()
- internal_munmap()
- internal_mprotect()
- internal_close()
- internal_open()
- internal_read()
- internal_write()
- internal_ftruncate()
- internal_stat()
- internal_lstat()
- internal_fstat()
- internal_dup2()
- internal_readlink()
- internal_unlink()
- internal_rename()
- internal_sched_yield()
- internal__exit()
- internal_sleep()
- internal_execve()
- NanoTime()
- internal_clock_gettime()
- internal_waitpid()
- internal_getpid()
- internal_getppid()
- internal_getdents()
- internal_lseek()
- internal_sigaltstack()
- internal_fork()
- internal_sigprocmask()
- internal_sysctl()
- internal_sigemptyset()
- internal_sigfillset()
- GetTid()
- TgKill()
This revision leaves room for refactoring in subsequent commits.
Reviewers: vitalybuka, kcc, joerg
Reviewed By: vitalybuka
Subscribers: mgorny, fedor.sergeev, jfb, loverszhaokai, devnexen, kubamracek, llvm-commits, ro, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D51419
llvm-svn: 343523
Summary:
Stop using directly sysctl(3) routines in sanitizer_procmaps_bsd
and replace it with internal_sysctl().
This will allow to install interceptors for sysctl(3).
Reviewers: joerg, vitalybuka
Reviewed By: vitalybuka
Subscribers: kubamracek, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D52646
llvm-svn: 343370
This seems to cause the thread's exit code to be clobbered, breaking
Chromium tests.
Also revert follow-up r342654.
> In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
>
> Differential Revision: https://reviews.llvm.org/D52091
llvm-svn: 343322
Summary:
Currently,
cd test/xray/TestCases/Posix
$build/bin/llvm-lit fdr-thread-order.cc
fails because `rm fdr-thread-order.*` deletes the .cc file.
This patch uses:
* %t as temporary directory name containing log files
* %t.exe as executable name
It does not delete %t after the test finishes for debugging convenience.
This matches the behavior of tests of various other LLVM components.
Log files will not clog up because the temporary directory (unique among
test files but the same among multiple invocations of a test) is cleaned
at the beginning of the test.
Reviewers: dberris, mboerger, eizan
Reviewed By: dberris
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D52638
llvm-svn: 343295
Summary:
This is for coding standard conformance, and for fixing an ODR violation
issue: __xray::ThreadLocalData is defined twice and differently in
xray_fdr_logging.cc and xray_basic_logging.cc
Reviewers: dberris, mboerger, eizan
Reviewed By: dberris
Subscribers: delcypher, jfb, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D52639
llvm-svn: 343289
Summary:
This change allows us to use the library path from which the LLVM
libraries are installed, in case the LLVM installation generates shared
libraries.
This should address llvm.org/PR39070.
Reviewers: mboerger, eizan
Subscribers: mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52597
llvm-svn: 343280
Summary:
The default values used for Space/Size for the new SizeClassMap do not work
with Android. The Compact map appears to be in the same boat.
Disable the test on Android for now to turn the bots green, but there is no
reason Compact & Dense should not have an Android test.
Added a FIXME, I will revisit this soon.
Reviewers: eugenis
Subscribers: srhines, kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52623
llvm-svn: 343252
Summary:
_Note_: I am not attached to the name `DenseSizeClassMap`, so if someone has a
better idea, feel free to suggest it.
The current pre-defined `SizeClassMap` hold a decent amount of cached entries,
either in cheer number of, or in amount of memory cached.
Empirical testing shows that more compact per-class arrays (whose sizes are
directly correlated to the number of cached entries) are beneficial to
performances, particularly in highly threaded environments.
The new proposed `SizeClassMap` has the following properties:
```
c00 => s: 0 diff: +0 00% l 0 cached: 0 0; id 0
c01 => s: 16 diff: +16 00% l 4 cached: 8 128; id 1
c02 => s: 32 diff: +16 100% l 5 cached: 8 256; id 2
c03 => s: 48 diff: +16 50% l 5 cached: 8 384; id 3
c04 => s: 64 diff: +16 33% l 6 cached: 8 512; id 4
c05 => s: 80 diff: +16 25% l 6 cached: 8 640; id 5
c06 => s: 96 diff: +16 20% l 6 cached: 8 768; id 6
c07 => s: 112 diff: +16 16% l 6 cached: 8 896; id 7
c08 => s: 128 diff: +16 14% l 7 cached: 8 1024; id 8
c09 => s: 144 diff: +16 12% l 7 cached: 7 1008; id 9
c10 => s: 160 diff: +16 11% l 7 cached: 6 960; id 10
c11 => s: 176 diff: +16 10% l 7 cached: 5 880; id 11
c12 => s: 192 diff: +16 09% l 7 cached: 5 960; id 12
c13 => s: 208 diff: +16 08% l 7 cached: 4 832; id 13
c14 => s: 224 diff: +16 07% l 7 cached: 4 896; id 14
c15 => s: 240 diff: +16 07% l 7 cached: 4 960; id 15
c16 => s: 256 diff: +16 06% l 8 cached: 4 1024; id 16
c17 => s: 320 diff: +64 25% l 8 cached: 3 960; id 49
c18 => s: 384 diff: +64 20% l 8 cached: 2 768; id 50
c19 => s: 448 diff: +64 16% l 8 cached: 2 896; id 51
c20 => s: 512 diff: +64 14% l 9 cached: 2 1024; id 48
c21 => s: 640 diff: +128 25% l 9 cached: 1 640; id 49
c22 => s: 768 diff: +128 20% l 9 cached: 1 768; id 50
c23 => s: 896 diff: +128 16% l 9 cached: 1 896; id 51
c24 => s: 1024 diff: +128 14% l 10 cached: 1 1024; id 48
c25 => s: 1280 diff: +256 25% l 10 cached: 1 1280; id 49
c26 => s: 1536 diff: +256 20% l 10 cached: 1 1536; id 50
c27 => s: 1792 diff: +256 16% l 10 cached: 1 1792; id 51
c28 => s: 2048 diff: +256 14% l 11 cached: 1 2048; id 48
c29 => s: 2560 diff: +512 25% l 11 cached: 1 2560; id 49
c30 => s: 3072 diff: +512 20% l 11 cached: 1 3072; id 50
c31 => s: 3584 diff: +512 16% l 11 cached: 1 3584; id 51
c32 => s: 4096 diff: +512 14% l 12 cached: 1 4096; id 48
c33 => s: 5120 diff: +1024 25% l 12 cached: 1 5120; id 49
c34 => s: 6144 diff: +1024 20% l 12 cached: 1 6144; id 50
c35 => s: 7168 diff: +1024 16% l 12 cached: 1 7168; id 51
c36 => s: 8192 diff: +1024 14% l 13 cached: 1 8192; id 48
c37 => s: 10240 diff: +2048 25% l 13 cached: 1 10240; id 49
c38 => s: 12288 diff: +2048 20% l 13 cached: 1 12288; id 50
c39 => s: 14336 diff: +2048 16% l 13 cached: 1 14336; id 51
c40 => s: 16384 diff: +2048 14% l 14 cached: 1 16384; id 48
c41 => s: 20480 diff: +4096 25% l 14 cached: 1 20480; id 49
c42 => s: 24576 diff: +4096 20% l 14 cached: 1 24576; id 50
c43 => s: 28672 diff: +4096 16% l 14 cached: 1 28672; id 51
c44 => s: 32768 diff: +4096 14% l 15 cached: 1 32768; id 48
c45 => s: 40960 diff: +8192 25% l 15 cached: 1 40960; id 49
c46 => s: 49152 diff: +8192 20% l 15 cached: 1 49152; id 50
c47 => s: 57344 diff: +8192 16% l 15 cached: 1 57344; id 51
c48 => s: 65536 diff: +8192 14% l 16 cached: 1 65536; id 48
c49 => s: 81920 diff: +16384 25% l 16 cached: 1 81920; id 49
c50 => s: 98304 diff: +16384 20% l 16 cached: 1 98304; id 50
c51 => s: 114688 diff: +16384 16% l 16 cached: 1 114688; id 51
c52 => s: 131072 diff: +16384 14% l 17 cached: 1 131072; id 48
c53 => s: 64 diff: +0 00% l 0 cached: 8 512; id 4
Total cached: 864928 (152/432)
```
It holds a bit less of 1MB of cached entries at most, and the cache fits in a
page.
The plan is to use this map by default for Scudo once we make sure that there
is no unforeseen impact for any of current use case.
Benchmarks give the most increase in performance (with Scudo) when looking at
highly threaded/contentious environments. For example, rcp2-benchmark
experiences a 10K QPS increase (~3%), and a decrease of 50MB for the max RSS
(~10%). On platforms like Android where we only have a couple of caches,
performance remain similar.
Reviewers: eugenis, kcc
Reviewed By: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52371
llvm-svn: 343246
Summary:
-lm is needed for these tests on Linux, but the lit config for this package automatically adds it for Linux and excludes it for Windows. So we should be able to get these tests running again by just dropping -lm and let the lit config add it when possible.
I was under the impression that -lm worked across platforms because it exists in other tests without and 'UNSUPPORTED: windows' commands (e.g. divsc3_test.c), but those are actually excluded because they 'REQUIRES: c99-complex' which is excluded from windows platforms (also by the local lit config).
I don't have easy access to a windows machine to verify this patch, but I can trigger a build bot run on clang-x64-ninja-win7 shortly after submitting.
Reviewers: hans
Subscribers: dberris, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D52563
llvm-svn: 343245
The `--target` option is rejected by GCC starting from version 7.0.
It's shorter and more portable to use `mabi` option to select MIPS ABI
for testing target architecture. Clang supports that starting from r343169.
llvm-svn: 343182
sancov.py relies on objdump -d to obtain the number of instrumented PCs.
The i386 %dynamiclib will now include .plt entries that are not recognized by objdump,
"sancov.py: found 0 instrumented PCs in *.so",
causing AddressSanitizer-i386-linux to fail.
Change it back to x86-target-arch after %sancov switches to a more robust approach.
llvm-svn: 343178
A version of AndroidGetApiLevel for static executables that is
completely compile-time.
The generic version uses dl_iterate_phdr which, even though it works
in static executables, crashes if called before libc is initialized.
llvm-svn: 343153
In MinGW mode, the /export: directives in object files are interpreted
differently; the symbols are not assumed to be decorated.
Since the GNU linker doesn't support the /alternatename and /include
directives, there's no such distinction for them in lld. This assumes
that the resulting sanitizer libraries will be linked by lld.
Differential Revision: https://reviews.llvm.org/D51882
llvm-svn: 343015
Instead provide manual declarations of the used types, to avoid
pulling in conflicting declarations of some of the functions that
are to be overridden.
Differential Revision: https://reviews.llvm.org/D51914
llvm-svn: 343014
The MinGW headers do define _M_IX86 (contrary to clang-cl and
MSVC where it is a compiler predefined macro), but the headers that
define it aren't included here.
Also check these defines for setting the symbol prefix, instead of
inconsistently using _WIN64 for that.
Differential Revision: https://reviews.llvm.org/D51883
llvm-svn: 343013
Summary: The gcda need to be delete before running the binary to avoid to have an increasing "# of Runs" when a test is failing
Reviewers: vitalybuka, eugenis, marco-c
Reviewed By: marco-c
Subscribers: delcypher, llvm-commits, #sanitizers, sylvestre.ledru, marco-c
Differential Revision: https://reviews.llvm.org/D52456
llvm-svn: 342963
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.
The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.
Developed in collaboration with Kostya Serebryany.
Reviewers: kcc
Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52249
llvm-svn: 342923
Summary:
Display a list of recent stack frames (not a stack trace!) when
tag-mismatch is detected on a stack address.
The implementation uses alignment tricks to get both the address of
the history buffer, and the base address of the shadow with a single
8-byte load. See the comment in hwasan_thread_list.h for more
details.
Developed in collaboration with Kostya Serebryany.
Reviewers: kcc
Subscribers: srhines, kubamracek, mgorny, hiraditya, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52249
llvm-svn: 342921
Summary:
The complex division builtins (div?c3) use logb methods from libm to scale numbers during division and avoid rounding issues. However, these come from libm, meaning anyone that uses --rtlib=compiler-rt also has to include -lm. Implement logb* methods for standard ieee 754 floats so we can avoid -lm on those platforms, falling back to the old behavior (using either logb() or `__builtin_logb()`) when not supported.
These new methods are defined internally as `__compiler_rt_logb` so as not to conflict with the libm definitions in any way.
This fixes just the libm methods mentioned in PR32279 and PR28652. libc is still required, although that seems to not be an issue.
Note: this is proposed as an alternative to just adding -lm: D49330.
Reviewers: efriedma, compnerd, scanon, echristo
Reviewed By: echristo
Subscribers: jsji, echristo, nemanjai, dberris, mgorny, kbarton, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49514
llvm-svn: 342917
Summary:
Previously we'd just show the exception and not the output from the
executed script. This is unhelpful in the case that the script actually
reports some useful information on the failure.
Now we print the output and re-raise the exception.
Reviewers: kubamracek, george.karpenkov
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52350
llvm-svn: 342869
changing the value of `SANITIZER_MMAP_RANGE_SIZE` to something more
sensible. The available VMA is at most 64GiB and not 256TiB that
was previously being used.
This change gives us several wins:
* Drastically improves LeakSanitizer performance on
Darwin ARM64 devices. On a simple synthentic benchmark
this took leak detection time from ~30 seconds to 0.5 seconds
due to the `ForEachChunk(...)` method enumerating a much smaller
number of regions. Previously we would pointlessly iterate
over a large portion of the SizeClassAllocator32's ByteMap
that would could never be set due it being configured for a much
larger VM space than is actually availble.
* Decreases the memory required for the Primary allocator.
Previously the ByteMap inside the the allocator used
an array of pointers that took 512KiB of space. Now the required
space for the array is 128 bytes.
rdar://problem/43509428
Differential Revision: https://reviews.llvm.org/D51173
llvm-svn: 342868
Summary:
This change spans both LLVM and compiler-rt, where we do the following:
- Add XRay to the LLVMBuild system, to allow for distributing the XRay
trace loading library along with the LLVM distributions.
- Use `llvm-config` better in the compiler-rt XRay implementation, to
depend on the potentially already-distributed LLVM XRay library.
While this is tested with the standalone compiler-rt build, it does
require that the LLVMXRay library (and LLVMSupport as well) are
available during the build. In case the static libraries are available,
the unit tests will build and work fine. We're still having issues with
attempting to use a shared library version of the LLVMXRay library since
the shared library might not be accessible from the standard shared
library lookup paths.
The larger change here is the inclusion of the LLVMXRay library in the
distribution, which allows for building tools around the XRay traces and
profiles that the XRay runtime already generates.
Reviewers: echristo, beanz
Subscribers: mgorny, hiraditya, mboerger, llvm-commits
Differential Revision: https://reviews.llvm.org/D52349
llvm-svn: 342859
Summary:
pthread_join() can return before a thread finishes exit()ing in the
kernel and a subsequent tgkill() can report the thread still alive.
Update the pthread-cleanup.c test to sleep and retry if it hits this
possible flake.
Thanks to Jeremy Morse for reporting.
Reviewers: jmorse, eugenis, vitalybuka
Reviewed By: jmorse, vitalybuka
Subscribers: delcypher, jfb, llvm-commits, #sanitizers, kcc
Differential Revision: https://reviews.llvm.org/D52330
llvm-svn: 342763
Summary:
`__sanitizer_set_death_callback` is a public interface function wrapping
`SetUserDieCallback`. Said function is defined in `sanitizer_termination.cc`,
which is not included in all the RT. Moving the interface function to that
file allows to not have a spurious public fuinction in RT that don't use it.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52363
llvm-svn: 342747
Summary:
The implementation of `internal_mmap(...)` deviates from the contract of
`mmap(...)` -- i.e. error returns are actually the equivalent of `errno`
results. We update how XRay uses `internal_mmap(...)` to better handle
these error conditions.
In the process, we change the default pointers we're using from `char*`
to `uint8_t*` to prevent potential usage of the pointers in the string
library functions that expect to operate on `char*`.
We also take the chance to "promote" sizes of individual `internal_mmap`
requests to at least page size bytes, consistent with the expectations
of calls to `mmap`.
Reviewers: cryptoad, mboerger
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D52361
llvm-svn: 342745
r342698 removed the 'UNSUPPORTED: windows' flags from a number of fuzzer
tests, however too many lines were removed from 'fuzzer-flags.test',
including the run-line that generates the test binary, which breaks that
test for me (and the clang-x64-ninja-win7 buildbot). This patch just
re-adds that line.
llvm-svn: 342720
Summary:
The goal of D49916 is to remove some hit counters from lines like '}'.
So as a consequence, the tests must be fixed.
FYI, after holidays, I'll add more tests.
Reviewers: marco-c, davidxl
Reviewed By: marco-c
Subscribers: delcypher, llvm-commits, #sanitizers, sylvestre.ledru
Differential Revision: https://reviews.llvm.org/D49917
llvm-svn: 342718
Summary:
Replace FuzzerExtFunctionsDlsymWin.cpp with FuzzerExtFunctionsWeakAlias.cpp
to get externally defined functions (eg: LLVMFuzzerInitialize,
LLVMFuzzerCustomMutator, etc) working again.
Also enable tests that depended on these functions (on windows)
Reviewers: rnk, morehouse
Reviewed By: rnk, morehouse
Subscribers: rnk, morehouse, mgorny
Differential Revision: https://reviews.llvm.org/D51700
llvm-svn: 342698
Summary: Some tests are broken if patch in D49659 is accepted so this patch fixes them.
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: dberris, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D49721
llvm-svn: 342661
In long-running builds we've seen some ASan complaints during thread creation that we suspect are due to leftover poisoning from previous threads whose stacks occupied that memory. This patch adds a hook that unpoisons the stack just before the NtTerminateThread syscall.
Differential Revision: https://reviews.llvm.org/D52091
llvm-svn: 342652
Summary:
As far as I can tell, there is no reason why `__sanitizer::CheckFailed` should
be exported. Looking back in time, it was added with the FIXME with the
following by @timurrrr:
```
[*San/RTL] Fix minor breakage
Grumbling: this hasn't been caught by running 'make check-{a,l,t}san check-sanitizer'
```
I can't find any detail about the breakage, all tests seem to work for me, so
maybe Windows (@rnk?) or something I have no setup for.
The reason to make it private (past the FIXME) is that Scudo defines its own
(without callback) and I am trying to make the .so be loadable with the UBsan
one (that has its own public `CheckFailed`) with as little drama as possible.
Reviewers: eugenis, rnk
Reviewed By: eugenis, rnk
Subscribers: kubamracek, delcypher, #sanitizers, timurrrr, rnk, llvm-commits
Differential Revision: https://reviews.llvm.org/D52279
llvm-svn: 342651
Same idea as r310419: The 8 byte nop is a suffix of the 9 byte nop, and we need at most 6 bytes.
Differential Revision: https://reviews.llvm.org/D51788
llvm-svn: 342649
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.
The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.
In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.
We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module. We also add
the terminfo library detection along with inclusion of the appropriate
compiler flags for header include lookup.
Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.
Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.
Reviewers: mboerger, eizan
Subscribers: mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52220
llvm-svn: 342617
This is called by Bionic on dlclose to delete the emutls pthread key.
The return value of pthread_key_delete is unchecked and behaviour of
multiple calls to the method is dependent on the implementation of
pthread_key_delete.
Differential Revision: https://reviews.llvm.org/D52251
llvm-svn: 342608
Summary:
Destroy and close a range's vmar if all its memory was unmapped.
This addresses some performance regression due to the proliferation of vmars
when Secondary backed allocations are concerned with Scudo on Fuchsia.
When a Secondary backed allocation was freed, the associated
`ReservedAddressRange` was going away after unmapping the entirety of the
mapping, but without getting rid of the associated vmar properly (which
was created specifically for that mapping). This resulted in an increase of
defunct vmars, that in turn slowed down further new vmar allocations.
This appears to solve ZX-2560/ZX-2642, at least on QEMU.
Reviewers: flowerhack, mcgrathr, phosek, mseaborn
Reviewed By: mcgrathr
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52242
llvm-svn: 342584
of a darwin platform was in the list of `UBSAN_SUPPORTED_ARCH`.
This is a follow up to r341306.
The typo meant that if an architecture was a prefix to another
architecture in the list (e.g. `armv7` is a prefix of `armv7k`) then
this would trigger a match which is not the intended behaviour.
rdar://problem/41126835
llvm-svn: 342553
Summary:
This patch adds TSan runtime support for Go on linux-aarch64
platforms. This enables people working on golang to implement their
platform/language part of the TSan support.
Basic testing is done with lib/tsan/go/buildgo.sh. Additional testing will be
done as part of the work done in the Go project.
It is intended to support other VMA sizes, except 39 which does not
have enough bits to support the Go heap requirements.
Patch by Fangming Fang <Fangming.Fang@arm.com>.
Reviewers: kubamracek, dvyukov, javed.absar
Subscribers: mcrosier, dberris, mgorny, kristof.beyls, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D52167
llvm-svn: 342541
Instead of assuming `-ltinfo` works, check whether there's terminfo
support on the host where LLVMSupport is compiled.
Follow-up to D52220.
llvm-svn: 342523
Summary:
This change introduces an `FDRLogWriter` type which is responsible for
serialising metadata and function records to character buffers. This is
the first step in a refactoring of the implementation of the FDR runtime
to allow for more granular testing of the individual components of the
implementation.
The main contribution of this change is a means of hiding the details of
how specific records are written to a buffer, and for managing the
extents of these buffers. We make use of C++ features (templates and
some metaprogramming) to reduce repetition in the act of writing out
specific kinds of records to the buffer.
In this process, we make a number of changes across both LLVM and
compiler-rt to allow us to use the `Trace` abstraction defined in the
LLVM project in the testing of the runtime implementation. This gives us
a closer end-to-end test which version-locks the runtime implementation
with the loading implementation in LLVM.
We also allow using gmock in compiler-rt unit tests, by adding the
requisite definitions in the `AddCompilerRT.cmake` module.
Finally, we've gone ahead and updated the FDR logging implementation to
use the FDRLogWriter for the lowest-level record-writing details.
Following patches will isolate the state machine transitions which
manage the set-up and tear-down of the buffers we're using in multiple
threads.
Reviewers: mboerger, eizan
Subscribers: mgorny, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52220
llvm-svn: 342518
The CMAKE_<LANG>_ARCHIVE_FINISH rule doesn't need to be cleared for Darwin
static libraries. Avoid resetting the variables in the SIP case. If
CMAKE_RANLIB is cached, then CMake's Ninja generator will invoke ranlib during
installation, not due to the CMAKE_<LANG>_ARCHIVE_FINISH rule.
llvm-svn: 342511
On sparc64/Linux, sparc64 isn't defined; the canonical way of
checking for sparc64 is sparc && arch64, which also works on the
BSDs and Solaris. Since this problem does not occur on 32-bit
architectures, riscv32 can be ignored. This fixes and refines rL324593.
Patch by jrtc27 (James Clarke)
Differential Revision: https://reviews.llvm.org/D43146
llvm-svn: 342504
When building static fat libraries, we need to ensure that we use libtool rather
than llvm-ar to create the library. Duplicate the rules from LLVM to ensure
that we correctly build the fat libraries when building compiler-rt standalone.
This also requires that we duplicate the workaround for the `DYLD_LIBRARY_PATH`
for SIP. Additionally, ensure that we set the `CMAKE_*_ARCHIVE_FINISH` variable
to ensure that we do not try to use `ranlib` on that target.
llvm-svn: 342425
Summary:
In order for this test to work the log file needs to be removed from both
from the host and device. To fix this the `rm` `RUN` lines have been
replaced with `RUN: rm` followed by `RUN: %device_rm`.
Initially I tried having it so that `RUN: %run rm` implicitly runs `rm`
on the host as well so that only one `RUN` line is needed. This
simplified writing the test however that had two large drawbacks.
* It's potentially very confusing (e.g. for use of the device scripts outside
of the lit tests) if asking for `rm` to run on device also causes files
on the host to be deleted.
* This doesn't work well with the glob patterns used in the test.
The host shell expands the `%t.log.*` glob pattern and not on the
device so we could easily miss deleting old log files from previous
test runs if the corresponding file doesn't exist on the host.
So instead deletion of files on the device and host are explicitly
separate commands.
The command to delete files from a device is provided by a new
substitution `%device_rm` as suggested by Filipe Cabecinhas.
The semantics of `%device_rm` are that:
* It provides a way remove files from a target device when
the host is not the same as the target. In the case that the
host and target are the same it is a no-op.
* It interprets shell glob patterns in the context of the device
file system instead of the host file system.
This solves the globbing problem provided the argument is quoted so
that lit's underlying shell doesn't try to expand the glob pattern.
* It supports the `-r` and `-f` flags of the `rm` command,
with the same semantics.
Right now an implementation of `%device_rm` is provided only for
ios devices. For all other devices a lit warning is emitted and
the `%device_rm` is treated as a no-op. This done to avoid changing
the behaviour for other device types but leaves room for others
to implement `%device_rm`.
The ios device implementation uses the `%run` wrapper to do the work
of removing files on a device.
The `iossim_run.py` script has been fixed so that it just runs `rm`
on the host operating system because the device and host file system
are the same.
rdar://problem/41126835
Reviewers: vsk, kubamracek, george.karpenkov, eugenis
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D51648
llvm-svn: 342391
Support for .preinit_array has been implemented in Fuchsia's libc,
add Fuchsia to the list of platforms that support this feature.
Differential Revision: https://reviews.llvm.org/D52155
llvm-svn: 342357
Summary:
This change makes XRay FDR mode use a single backing store for the
buffer queue, and have indexes into that backing store instead. We also
remove the reliance on the internal allocator implementation in the FDR
mode logging implementation.
In the process of making this change we found an inconsistency with the
way we're returning buffers to the queue, and how we're setting the
extents. We take the chance to simplify the way we're managing the
extents of each buffer. It turns out we do not need the indirection for
the extents, so we co-host the atomic 64-bit int with the buffer object.
It also seems that we've not been returning the buffers for the thread
running the flush functionality when writing out the files, so we can
run into a situation where we could be missing data.
We consolidate all the allocation routines now into xray_allocator.h,
where we used to have routines defined in xray_buffer_queue.cc.
Reviewers: mboerger, eizan
Subscribers: jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D52077
llvm-svn: 342356
This API has been deprecated three months ago and shouldn't be used
anymore, all clients should migrate to the new string based API.
Differential Revision: https://reviews.llvm.org/D51606
llvm-svn: 342318
Since we changed our inlining parameters, this test case was failing
on SystemZ, as the two tests were now both inlined into the main
function, which the test didn't expect. Fixed by adding a few more
noinline attributes.
llvm-svn: 342236
Summary:
Before this change, we only emit the XRay attributes in LLVM IR when the
-fxray-instrument flag is provided. This may cause issues with thinlto
when the final binary is being built/linked with -fxray-instrument, and
the constitutent LLVM IR gets re-lowered with xray instrumentation.
With this change, we can honour the "never-instrument "attributes
provided in the source code and preserve those in the IR. This way, even
in thinlto builds, we retain the attributes which say whether functions
should never be XRay instrumented.
This change addresses llvm.org/PR38922.
Reviewers: mboerger, eizan
Subscribers: mehdi_amini, dexonsmith, cfe-commits, llvm-commits
Differential Revision: https://reviews.llvm.org/D52015
llvm-svn: 342200
Summary:
This change has a number of fixes for FDR mode in compiler-rt along with
changes to the tooling handling the traces in llvm.
In the runtime, we do the following:
- Advance the "last record" pointer appropriately when writing the
custom event data in the log.
- Add XRAY_NEVER_INSTRUMENT in the rewinding routine.
- When collecting the argument of functions appropriately marked, we
should not attempt to rewind them (and reset the counts of functions
that can be re-wound).
In the tooling, we do the following:
- Remove the state logic in BlockIndexer and instead rely on the
presence/absence of records to indicate blocks.
- Move the verifier into a loop associated with each block.
Reviewers: mboerger, eizan
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D51965
llvm-svn: 342122
Similarly to before, D51985 again reduced the number of registers
required for the read/write routines causing this test to fail on
sanitizer-x86_64-linux-autoconf.
llvm-svn: 342092
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase, and matches how these
headers are included elsewhere in compiler-rt.
Also include these headers with angle brackets, as they are system
headers.
Differential Revision: https://reviews.llvm.org/D51913
llvm-svn: 341983
Right now, the counters are added in regards of the number of successors
for a given BasicBlock: it's good when we've only 1 or 2 successors (at
least with BranchInstr). But in the case of a switch statement, the
BasicBlock after switch has several predecessors and we need know from
which BB we're coming from.
So the idea is to revert what we're doing: add a PHINode in each block
which will select the counter according to the incoming BB. They're
several pros for doing that:
- we fix the "switch" bug
- we remove the function call to "__llvm_gcov_indirect_counter_increment"
and the lookup table stuff
- we replace by PHINodes, so the optimizer will probably makes a better
job.
Patch by calixte!
Differential Revision: https://reviews.llvm.org/D51619
llvm-svn: 341977
This function isn't declared with a const parameter anywhere; neither
in MSVC (neither in ucrt or in older msvcrt versions) nor in mingw-w64.
Differential Revision: https://reviews.llvm.org/D51876
llvm-svn: 341903
This fixes building on a case sensitive filesystem with mingw-w64
headers, where all headers are lowercase. This header actually also
is named with a lowercase name in the Windows SDK as well.
Differential Revision: https://reviews.llvm.org/D51877
llvm-svn: 341857
Summary:
In this change we apply `XRAY_NEVER_INSTRUMENT` to more functions in the
profiling implementation to ensure that these never get instrumented if
the compiler used to build the library is capable of doing XRay
instrumentation.
We also consolidate all the allocators into a single header
(xray_allocator.h) which sidestep the use of the internal allocator
implementation in sanitizer_common.
This addresses more cases mentioned in llvm.org/PR38577.
Reviewers: mboerger, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D51776
llvm-svn: 341647
Kuba Mracek