AddressSanitizer-Unit :: ./Asan-i386-calls-Test/AddressSanitizer.LongJmpTest
AddressSanitizer-Unit :: ./Asan-i386-calls-Test/AddressSanitizer.SigLongJmpTest
AddressSanitizer-Unit :: ./Asan-i386-inline-Test/AddressSanitizer.LongJmpTest
AddressSanitizer-Unit :: ./Asan-i386-inline-Test/AddressSanitizer.SigLongJmpTest
These failures will be examined properly when time permits.
rdar://problem/62141412
Add ThreadClock:: global_acquire_ which is the last time another thread
has done a global acquire of this thread's clock.
It helps to avoid problem described in:
https://github.com/golang/go/issues/39186
See test/tsan/java_finalizer2.cpp for a regression test.
Note the failuire is _extremely_ hard to hit, so if you are trying
to reproduce it, you may want to run something like:
$ go get golang.org/x/tools/cmd/stress
$ stress -p=64 ./a.out
The crux of the problem is roughly as follows.
A number of O(1) optimizations in the clocks algorithm assume proper
transitive cumulative propagation of clock values. The AcquireGlobal
operation may produce an inconsistent non-linearazable view of
thread clocks. Namely, it may acquire a later value from a thread
with a higher ID, but fail to acquire an earlier value from a thread
with a lower ID. If a thread that executed AcquireGlobal then releases
to a sync clock, it will spoil the sync clock with the inconsistent
values. If another thread later releases to the sync clock, the optimized
algorithm may break.
The exact sequence of events that leads to the failure.
- thread 1 executes AcquireGlobal
- thread 1 acquires value 1 for thread 2
- thread 2 increments clock to 2
- thread 2 releases to sync object 1
- thread 3 at time 1
- thread 3 acquires from sync object 1
- thread 1 acquires value 1 for thread 3
- thread 1 releases to sync object 2
- sync object 2 clock has 1 for thread 2 and 1 for thread 3
- thread 3 releases to sync object 2
- thread 3 sees value 1 in the clock for itself
and decides that it has already released to the clock
and did not acquire anything from other threads after that
(the last_acquire_ check in release operation)
- thread 3 does not update the value for thread 2 in the clock from 1 to 2
- thread 4 acquires from sync object 2
- thread 4 detects a false race with thread 2
as it should have been synchronized with thread 2 up to time 2,
but because of the broken clock it is now synchronized only up to time 1
The global_acquire_ value helps to prevent this scenario.
Namely, thread 3 will not trust any own clock values up to global_acquire_
for the purposes of the last_acquire_ optimization.
Reviewed-in: https://reviews.llvm.org/D80474
Reported-by: nvanbenschoten (Nathan VanBenschoten)
Some testcases are unexpectedly passing with NPM.
This is because the target functions are inlined in NPM.
I think we should add noinline attribute to keep these test points.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D79648
Summary:
This patch implements dynamic stack allocation for the VE target. Changes:
* compiler-rt: `__ve_grow_stack` to request stack allocation on the VE.
* VE: base pointer support, dynamic stack allocation.
Differential Revision: https://reviews.llvm.org/D79084
A few testcases are still using deprecated options.
warning: argument '-fsanitize-coverage=[func|bb|edge]' is deprecated,
use '-fsanitize-coverage=[func|bb|edge],[trace-pc-guard|trace-pc]'
instead [-Wdeprecated]
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D79741
This adds the family/model returned by CPUID for some Intel
Comet Lake CPUs. Instruction set and tuning wise these are
the same as "skylake".
These are not in the Intel SDM yet, but these should be correct.
Per target runtime dir may change the suffix of shared libs.
We can not assume we are always building with per_target_runtime_dir on.
Reviewed By: cryptoad
Differential Revision: https://reviews.llvm.org/D80243
Summary:
The previous code tries to strip out parentheses and anything in between
them. I'm guessing the idea here was to try to drop any listed arguments
for the function being symbolized. Unfortunately this approach is broken
in several ways.
* Templated functions may contain parentheses. The existing approach
messes up these names.
* In C++ argument types are part of a function's signature for the
purposes of overloading so removing them could be confusing.
Fix this simply by not trying to adjust the function name that comes
from `atos`.
A test case is included.
Without the change the test case produced output like:
```
WRITE of size 4 at 0x6060000001a0 thread T0
#0 0x10b96614d in IntWrapper<void >::operator=> const&) asan-symbolize-templated-cxx.cpp:10
#1 0x10b960b0e in void writeToA<IntWrapper<void > >>) asan-symbolize-templated-cxx.cpp:30
#2 0x10b96bf27 in decltype>)>> >)) std::__1::__invoke<void >), IntWrapper<void > >>), IntWrapper<void >&&) type_traits:4425
#3 0x10b96bdc1 in void std::__1::__invoke_void_return_wrapper<void>::__call<void >), IntWrapper<void > >>), IntWrapper<void >&&) __functional_base:348
#4 0x10b96bd71 in std::__1::__function::__alloc_func<void >), std::__1::allocator<void >)>, void >)>::operator>&&) functional:1533
#5 0x10b9684e2 in std::__1::__function::__func<void >), std::__1::allocator<void >)>, void >)>::operator>&&) functional:1707
#6 0x10b96cd7b in std::__1::__function::__value_func<void >)>::operator>&&) const functional:1860
#7 0x10b96cc17 in std::__1::function<void >)>::operator>) const functional:2419
#8 0x10b960ca6 in Foo<void >), IntWrapper<void > >::doCall>) asan-symbolize-templated-cxx.cpp:44
#9 0x10b96088b in main asan-symbolize-templated-cxx.cpp:54
#10 0x7fff6ffdfcc8 in start (in libdyld.dylib) + 0
```
Note how the symbol names for the frames are messed up (e.g. #8, #1).
With the patch the output looks like:
```
WRITE of size 4 at 0x6060000001a0 thread T0
#0 0x10005214d in IntWrapper<void (int)>::operator=(IntWrapper<void (int)> const&) asan-symbolize-templated-cxx.cpp:10
#1 0x10004cb0e in void writeToA<IntWrapper<void (int)> >(IntWrapper<void (int)>) asan-symbolize-templated-cxx.cpp:30
#2 0x100057f27 in decltype(std::__1::forward<void (*&)(IntWrapper<void (int)>)>(fp)(std::__1::forward<IntWrapper<void (int)> >(fp0))) std::__1::__invoke<void (*&)(IntWrapper<void (int)>), IntWrapper<void (int)> >(void (*&)(IntWrapper<void (int)>), IntWrapper<void (int)>&&) type_traits:4425
#3 0x100057dc1 in void std::__1::__invoke_void_return_wrapper<void>::__call<void (*&)(IntWrapper<void (int)>), IntWrapper<void (int)> >(void (*&)(IntWrapper<void (int)>), IntWrapper<void (int)>&&) __functional_base:348
#4 0x100057d71 in std::__1::__function::__alloc_func<void (*)(IntWrapper<void (int)>), std::__1::allocator<void (*)(IntWrapper<void (int)>)>, void (IntWrapper<void (int)>)>::operator()(IntWrapper<void (int)>&&) functional:1533
#5 0x1000544e2 in std::__1::__function::__func<void (*)(IntWrapper<void (int)>), std::__1::allocator<void (*)(IntWrapper<void (int)>)>, void (IntWrapper<void (int)>)>::operator()(IntWrapper<void (int)>&&) functional:1707
#6 0x100058d7b in std::__1::__function::__value_func<void (IntWrapper<void (int)>)>::operator()(IntWrapper<void (int)>&&) const functional:1860
#7 0x100058c17 in std::__1::function<void (IntWrapper<void (int)>)>::operator()(IntWrapper<void (int)>) const functional:2419
#8 0x10004cca6 in Foo<void (IntWrapper<void (int)>), IntWrapper<void (int)> >::doCall(IntWrapper<void (int)>) asan-symbolize-templated-cxx.cpp:44
#9 0x10004c88b in main asan-symbolize-templated-cxx.cpp:54
#10 0x7fff6ffdfcc8 in start (in libdyld.dylib) + 0
```
rdar://problem/58887175
Reviewers: kubamracek, yln
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79597
Summary:
This is collaboration between Marcel Boehme @ Monash, Australia and Valentin Manès plus Sang Kil Cha @ KAIST, South Korea.
We have made a few modifications to boost LibFuzzer performance by changing how weights are assigned to the seeds in the corpus. Essentially, seeds that reveal more "information" about globally rare features are assigned a higher weight. Our results on the Fuzzer Test Suite seem quite promising. In terms of bug finding, our Entropic patch usually finds the same errors much faster and in more runs. In terms of coverage, our version Entropic achieves the same coverage in less than half the time for the majority of subjects. For the lack of space, we shared more detailed performance results directly with @kcc. We'll publish the preprint with all the technical details as soon as it is accepted. Happy to share if you drop us an email.
There should be plenty of opportunities to optimise further. For instance, while Entropic achieves the same coverage in less than half the time, Entropic has a much lower #execs per second. We ran the perf-tool and found a few performance bottlenecks.
Thanks for open-sourcing LibFuzzer (and the entire LLVM Compiler Infrastructure)! This has been such a tremendous help to my research.
Patch By: Marcel Boehme
Reviewers: kcc, metzman, morehouse, Dor1s, vitalybuka
Reviewed By: kcc
Subscribers: dgg5503, Valentin, llvm-commits, kcc
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73776
When build in runtime bulid mode with LLVM_ENABLE_RUNTIMES,
the base-config-ix.cmake will complain about two errors.
One is empty string in replace, the other one is unknown `TEST_BIG_ENDIAN ` command.
This patch fix it so that we can test runtime build.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D80040
Since setting COMPILER_RT_USE_BUILTINS_LIBRARY would remove -z,defs
flag, missing builtins library would continue to build unnoticed.
Explicitly emit an error in such case.
Differential Revision: https://reviews.llvm.org/D79470
Only add CMAKE_EXE_LINKER_FLAGS when in a standalone bulid.
Or else CMAKE_EXE_LINKER_FLAGS contains flags for build compiler of Clang/llvm.
This might not be the same as what the COMPILER_RT_TEST_COMPILER supports.
eg: the build compiler use lld linker and we use it to build clang with
default ld linker then to be tested clang will complain about lld
options like --color-diagnostics.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D78373
In a previous change I added a shim for fork(), but when compiled from InstrProfiling.c, the
required header file was not included, so pid_t was undefined. This change adds that include.
Summary:
Synchronize the function definition with the LLVM documentation.
https://llvm.org/docs/Atomics.html#libcalls-atomic
GCC also returns bool for the same atomic builtin.
Reviewers: theraven
Reviewed By: theraven
Subscribers: theraven, dberris, jfb, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79845
Fixes PR45673
The commit 9180c14fe4 (D76206) resolved only a part of the problem
of concurrent .gcda file creation. It ensured that only one process
creates the file but did not ensure that the process locks the
file first. If not, the process which created the file may clobber
the contents written by a process which locked the file first.
This is the cause of PR45673.
This commit prevents the clobbering by revising the assumption
that a process which creates the file locks the file first.
Regardless of file creation, a process which locked the file first
uses fwrite (new_file==1) and other processes use mmap (new_file==0).
I also tried to keep the creation/first-lock process same by using
mkstemp/link/unlink but the code gets long. This commit is more
simple.
Note: You may be confused with other changes which try to resolve
concurrent file access. My understanding is (may not be correct):
D76206: Resolve race of .gcda file creation (but not lock)
This one: Resolve race of .gcda file creation and lock
D54599: Same as D76206 but abandoned?
D70910: Resolve race of multi-threaded counter flushing
D74953: Resolve counter sharing between parent/children processes
D78477: Revision of D74953
Differential Revision: https://reviews.llvm.org/D79556
In a big-endian .gcda file, the first four bytes are "gcda" instead of "adcg".
All 32-bit values are in big-endian.
With this change, libclang_rt.profile can hopefully produce gcov
compatible output.
D49132 is partially correct. For 64-bit values, the lower 32-bit part comes
before the higher 32-bit part (in a little-endian manner).
For 32-bit values, libgcov reads/writes 32-bit values in native endianness.
gcov 9 (r264462) started to use GCOV_TAG_OBJECT_SUMMARY. Before,
GCOV_TAG_PROGRAM_SUMMARY was used.
libclang_rt.profile should emit just one tag according to the version.
Another bug introduced by rL194499 is that the wrong runcount field was
selected.
Fix the two bugs so that gcov can correctly decode "Runs:" from
libclang_rt.profile produced .gcda files, and llvm-cov gcov can
correctly decode "Runs:" from libgcov produced .gcda files.
Summary:
Fix hwasan allocator not respecting the requested alignment when it is
higher than a page, but still within primary (i.e. [2048, 65536]).
Reviewers: pcc, hctim, cryptoad
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79656
Create a sanitizer_ptrauth.h header that #includes <ptrauth> when
available and defines just the required macros as "no ops" otherwise.
This should avoid the need for excessive #ifdef'ing.
Follow-up to and discussed in: https://reviews.llvm.org/D79132
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D79540
Defaulting to -Xclang -coverage-version='407*' makes .gcno/.gcda
compatible with gcov [4.7,8)
In addition, delete clang::CodeGenOptionsBase::CoverageExtraChecksum and GCOVOptions::UseCfgChecksum.
We can infer the information from the version.
With this change, .gcda files produced by `clang --coverage a.o` linked executable can be read by gcov 4.7~7.
We don't need other -Xclang -coverage* options.
There may be a mismatching version warning, though.
(Note, GCC r173147 "split checksum into cfg checksum and line checksum"
made gcov 4.7 incompatible with previous versions.)
rL144865 incorrectly wrote function names for GCOV_TAG_FUNCTION
(this might be part of the reasons the header says
"We emit files in a corrupt version of GCOV's "gcda" file format").
rL176173 and rL177475 realized the problem and introduced -coverage-no-function-names-in-data
to work around the issue. (However, the description is wrong.
libgcov never writes function names, even before GCC 4.2).
In reality, the linker command line has to look like:
clang --coverage -Xclang -coverage-version='407*' -Xclang -coverage-cfg-checksum -Xclang -coverage-no-function-names-in-data
Failing to pass -coverage-no-function-names-in-data can make gcov 4.7~7
either produce wrong results (for one gcov-4.9 program, I see "No executable lines")
or segfault (gcov-7).
(gcov-8 uses an incompatible format.)
This patch deletes -coverage-no-function-names-in-data and the related
function names support from libclang_rt.profile
https://reviews.llvm.org/D63616 added `-fsanitize-coverage-whitelist`
and `-fsanitize-coverage-blacklist` for clang.
However, it was done only for legacy pass manager.
This patch enable it for new pass manager as well.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D79653
Summary:
This is necessary to handle calls to free() after __hwasan_thread_exit,
which is possible in glibc.
Also, add a null check to GetCurrentThread, otherwise the logic in
GetThreadByBufferAddress turns it into a non-null value. This means that
all of the checks for GetCurrentThread() != nullptr do not have any
effect at all right now!
Reviewers: pcc, hctim
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79608
Summary: The new pass manager symbolizes the location as ~Simple instead of Simple::~Simple.
Reviewers: rnk, leonardchan, vitalybuka
Subscribers: #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79594
Summary:
When forking in several threads, the counters were written out in using the same global static variables (see GCDAProfiling.c): that leads to crashes.
So when there is a fork, the counters are resetted in the child process and they will be dumped at exit using the interprocess file locking.
When there is an exec, the counters are written out and in case of failures they're resetted.
Reviewers: jfb, vsk, marco-c, serge-sans-paille
Reviewed By: marco-c, serge-sans-paille
Subscribers: llvm-commits, serge-sans-paille, dmajor, cfe-commits, hiraditya, dexonsmith, #sanitizers, marco-c, sylvestre.ledru
Tags: #sanitizers, #clang, #llvm
Differential Revision: https://reviews.llvm.org/D78477
When reporting diagnostics from ASan's (and other sanitizer's) signal
handlers we should strip the "invalid signature" bit before printing
addresses. This makes the report less confusing and let's the user
focus on the real issue.
rdar://62615826
Reviewed By: kubamracek, delcypher
Differential Revision: https://reviews.llvm.org/D79132
Summary:
Currently, realloc is marked as "discard" in done_abilist.txt. As discussed in PR#45583, this is probably not the expected behavior; a custom wrapper seems to be required. Since this wrapper has not been implemented yet, realloc should not be in the done_abilist.txt file so that a warning is displayed when it is called.
Reviewers: kcc, pcc, vitalybuka
Reviewed By: vitalybuka
Subscribers: #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78379
It looks like some bots are failing with os log not giving any
output. This might be due to the system under test being heavy
load so the 2 minute window might not be large enough. This
patch makes the window larger in the hope that this test will
be more reliable.
rdar://problem/62141527
Summary:
Implement pattern initialization of memory (excluding the secondary
allocator because it already has predictable memory contents).
Expose both zero and pattern initialization through the C API.
Reviewers: pcc, cryptoad
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79133
Summary:
If this is called before the malloc call in a thread (or in the whole
program), the lazy initialization of the allocation can overwrite
Options.
Reviewers: pcc, cryptoad
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D79130
Newer iOS SDK introduce accessors to retrieve the register values
(arm_thread_state64_get_*) and disallows direct access to fields. If
arm_thread_state64_get_sp is defined, the accessors are available.
Summary: Fixes an ABI violation in the trampoline code for AArch64 that causes the indirect result register to get overwritten if the XRay handler function is complex enough to use it.
Reviewers: MaskRay, dberris, johnislarry
Subscribers: kristof.beyls, danielkiss, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78596
This is the first patch in a series to add support for the AVR target.
This patch includes changes to make compiler-rt more target independent
by not relying on the width of an int or long.
Differential Revision: https://reviews.llvm.org/D78662
Summary:
Following up the discussion on D77638 (and following rGd6cfed6060c283dc4a6bf9ca294dcd732e8b9f72
as example), defining `__sanitizer_cov_bool_flag_init` as the weak interface
functions in various compiler-rt/ files.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: dberris, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D77857
Follow-up of D78082 and D78590.
Otherwise, because xray_instr_map is now read-only, the absolute
relocation used for Sled.Function will cause a text relocation.
Follow-up of D78082 (x86-64).
This change avoids dynamic relocations in `xray_instr_map` for ARM/AArch64/powerpc64le.
MIPS64 cannot use 64-bit PC-relative addresses because R_MIPS_PC64 is not defined.
Because MIPS32 shares the same code, for simplicity, we don't use PC-relative addresses for MIPS32 as well.
Tested on AArch64 Linux and ppc64le Linux.
Reviewed By: ianlevesque
Differential Revision: https://reviews.llvm.org/D78590
Summary: Switch to pc-relative lookup of the xray handler function to avoid text relocations.
Reviewers: MaskRay, dberris, johnislarry
Subscribers: kristof.beyls, danielkiss, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78595
Summary: We load multiple copies of the trampolines into memory when instrumenting DSOs. Hidden visibility prevents conflicts in this scenario.
Reviewers: MaskRay, dberris, johnislarry
Subscribers: #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78593
Summary:
While working on rdar://problem/62083617 I noticed this call was
missing.
This is a no-op for all platforms except Darwin. For Darwin this
means the `use_xnu_fast_mmap` flag is initialized as it was intended
when using UBSan in standalone mode.
Reviewers: vitalybuka, vsk, kubamracek, yln, samsonov
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78532
Summary:
This fixes symbolization in Standalone UBSan mode for the Darwin simulators.
861b69faee (rdar://problem/58789439) tried to fix
symbolization for all sanitizers on Darwin simulators but unfortunately it only
fixed the problem for TSan.
For UBSan in standalone mode the fix wasn't sufficient because UBSan's
standalone init doesn't call `Symbolizer::LateInitialize()` like ASan
and TSan do. This meant that `AtosSymbolizerProcess::LateInitialize()`
was never being called before
`AtosSymbolizerProcess::StartSymbolizerSubprocess()` which breaks an
invariant we expect to hold.
The missing call to `Symbolizer::LateInitialize()` during UBSan's
standalone init seems like an accidently omission so this patch simply
adds it.
rdar://problem/62083617
Reviewers: vitalybuka, kubamracek, yln, samsonov
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78530
* Changing source lines seems to cause us to hit rdar://problem/62132428.
* Even if I workaround the above issue sometimes the source line in the dylib reported by atos is off by one.
It's simpler to just disable the test for now.
rdar://problem/61793759
xray_instr_map contains absolute addresses of sleds, which are relocated
by `R_*_RELATIVE` when linked in -pie or -shared mode.
By making these addresses relative to PC, we can avoid the dynamic
relocations and remove the SHF_WRITE flag from xray_instr_map. We can
thus save VM pages containg xray_instr_map (because they are not
modified).
This patch changes x86-64 and bumps the sled version to 2. Subsequent
changes will change powerpc64le and AArch64.
Reviewed By: dberris, ianlevesque
Differential Revision: https://reviews.llvm.org/D78082
Summary:
861b69faee (rdar://problem/58789439) while
fixing symbolization for TSan completely broke ASan's runtime for the
simulators.
The problem with the previous patch is that the memory passed to
`putenv()` was poisoned and when passed to `putenv()` it tripped
an interceptor for `strchr()` which saw the memory was poisoned and
raised an ASan issue.
The memory was poisoned because `AtosSymbolizerProcess` objects
are created using ASan's internal allocator. Memory from this
allocator gets poisoned with `kAsanInternalHeapMagic`.
To workaround this, this patch makes the memory for the environment
variable entry a global variable that isn't poisoned.
This pass also adds a `DCHECK(getenv(K_ATOS_ENV_VAR))` because the
following DCHECK would crash because `internal_strcmp()` doesn't
work on nullptr.
rdar://problem/62067724
Reviewers: kubamracek, yln
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78525
We can use `simctl spawn --standalone` to enable running tests without
the need for an already-booted simulator instance. This also side-steps
the problem of not having a good place to shutdown the instance after
we are finished with testing.
rdar://58118442
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D78409
This variable is being used, but it's not being set (it's only set
for ubsan_minimal, but not ubsan). This addresses a regression that
was introduced in D78325.
Differential Revision: https://reviews.llvm.org/D78410
Introduce a function __scudo_get_error_info() that may be called to interpret
a crash resulting from a memory error, potentially in another process,
given information extracted from the crashing process. The crash may be
interpreted as a use-after-free, buffer overflow or buffer underflow.
Also introduce a feature to optionally record a stack trace for each
allocation and deallocation. If this feature is enabled, a stack trace for
the allocation and (if applicable) the deallocation will also be available
via __scudo_get_error_info().
Differential Revision: https://reviews.llvm.org/D77283
Summary:
Due to sandbox restrictions in the recent versions of the simulator runtime the
atos program is no longer able to access the task port of a parent process
without additional help.
This patch fixes this by registering a task port for the parent process
before spawning atos and also tells atos to look for this by setting
a special environment variable.
This patch is based on an Apple internal fix (rdar://problem/43693565) that
unfortunately contained a bug (rdar://problem/58789439) because it used
setenv() to set the special environment variable. This is not safe because in
certain circumstances this can trigger a call to realloc() which can fail
during symbolization leading to deadlock. A test case is included that captures
this problem.
The approach used to set the necessary environment variable is as
follows:
1. Calling `putenv()` early during process init (but late enough that
malloc/realloc works) to set a dummy value for the environment variable.
2. Just before `atos` is spawned the storage for the environment
variable is modified to contain the correct PID.
A flaw with this approach is that if the application messes with the
atos environment variable (i.e. unsets it or changes it) between the
time its set and the time we need it then symbolization will fail. We
will ignore this issue for now but a `DCHECK()` is included in the patch
that documents this assumption but doesn't check it at runtime to avoid
calling `getenv()`.
The issue reported in rdar://problem/58789439 manifested as a deadlock
during symbolization in the following situation:
1. Before TSan detects an issue something outside of the runtime calls
setenv() that sets a new environment variable that wasn't previously
set. This triggers a call to malloc() to allocate a new environment
array. This uses TSan's normal user-facing allocator. LibC stores this
pointer for future use later.
2. TSan detects an issue and tries to launch the symbolizer. When we are in the
symbolizer we switch to a different (internal allocator) and then we call
setenv() to set a new environment variable. When this happen setenv() sees
that it needs to make the environment array larger and calls realloc() on the
existing enviroment array because it remembers that it previously allocated
memory for it. Calling realloc() fails here because it is being called on a
pointer its never seen before.
The included test case closely reproduces the originally reported
problem but it doesn't replicate the `((kBlockMagic)) ==
((((u64*)addr)[0])` assertion failure exactly. This is due to the way
TSan's normal allocator allocates the environment array the first time
it is allocated. In the test program addr[0] accesses an inaccessible
page and raises SIGBUS. If TSan's SIGBUS signal handler is active, the
signal is caught and symbolication is attempted again which results in
deadlock.
In the originally reported problem the pointer is successfully derefenced but
then the assert fails due to the provided pointer not coming from the active
allocator. When the assert fails TSan tries to symbolicate the stacktrace while
already being in the middle of symbolication which results in deadlock.
rdar://problem/58789439
Reviewers: kubamracek, yln
Subscribers: jfb, #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78179
Now that write data continously into the memory mapping, we don't need
to keep the VMO handle around after it has been mapped. This change also
ensures that the VMO is always closed on error.
Differential Revision: https://reviews.llvm.org/D76963
Summary:
Do not reexport libgcc.a symbols and random sanitizer internal symbols
by applying a version script to the shared library build.
This fixes unwinder conflicts on Android that are created by reexporting
the unwinder interface from libgcc_real.a. The same is already done in
asan and hwasan.
Reviewers: vitalybuka, srhines
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78325
Summary:
This is implemented by adding a `Symbolizer::LateInitializeTools()`
method that iterates over the registered tools and calls the
`LateInitialize()` method on them.
`Symbolizer::LateInitializeTools()` is now called from the various
`Symbolizer::LateInitialize()` implementations.
The default implementation of `SymbolizerTool::LateInitialize()`
does nothing so this change should be NFC.
This change allows `SymbolizerTool` implementations to perform
any initialization that they need to perform at the
LateInitialize stage of a sanitizer runtime init.
rdar://problem/58789439
Reviewers: kubamracek, yln, vitalybuka, cryptoad, phosek, rnk
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78178
Summary:
These tests pass with clang, but fail if gcc was used.
gcc build creates similar but not the same stacks.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: dvyukov, llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78114
Fixes:
1. Setting the number of entries in a thread's clock to max between
the thread and the SyncClock the thread is acquiring from
2. Setting last_acquire_
Unit- and stress-test for releaseStoreAcquire added to
tests/unit/tsan_clock_test.cpp
The following declarations were missing a prototype:
FE_ROUND_MODE __fe_getround();
int __fe_raise_inexact();
Discovered while fixing a bug in Clang related to unprototyped function
calls (see the previous commit).
Differential Revision: https://reviews.llvm.org/D78205
Summary:
The function used to log on Android will cut the message past
a certain amount of characters, which mostly materializes when
dumping the size class map on OOM.
This change splits the log message at newline boundaries.
Reviewers: pcc, cferris, hctim, eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D78018
@kamleshbhalui reported that when the Standard Extension M
(Multiplication and Division) is disabled for RISC-V,
`__udivdi3` will call __udivmodti4 which will in turn calls `__udivdi3`.
This patch moves __udivsi3 (shift and subtract) to int_div_impl.inc
`__udivXi3`, optimize a bit, add a `__umodXi3`, and use `__udivXi3` and
`__umodXi3` to define `__udivsi3` `__umodsi3` `__udivdi3` `__umoddi3`.
Reviewed By: kamleshbhalui
Differential Revision: https://reviews.llvm.org/D77912
Summary:
Previously `AtosSymbolizer` would set the PID to examine in the
constructor which is called early on during sanitizer init. This can
lead to incorrect behaviour in the case of a fork() because if the
symbolizer is launched in the child it will be told examine the parent
process rather than the child.
To fix this the PID is determined just before the symbolizer is
launched.
A test case is included that triggers the buggy behaviour that existed
prior to this patch. The test observes the PID that `atos` was called
on. It also examines the symbolized stacktrace. Prior to this patch
`atos` failed to symbolize the stacktrace giving output that looked
like...
```
#0 0x100fc3bb5 in __sanitizer_print_stack_trace asan_stack.cpp:86
#1 0x10490dd36 in PrintStack+0x56 (/path/to/print-stack-trace-in-code-loaded-after-fork.cpp.tmp_shared_lib.dylib:x86_64+0xd36)
#2 0x100f6f986 in main+0x4a6 (/path/to/print-stack-trace-in-code-loaded-after-fork.cpp.tmp_loader:x86_64+0x100001986)
#3 0x7fff714f1cc8 in start+0x0 (/usr/lib/system/libdyld.dylib:x86_64+0x1acc8)
```
After this patch stackframes `#1` and `#2` are fully symbolized.
This patch is also a pre-requisite refactor for rdar://problem/58789439.
Reviewers: kubamracek, yln
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D77623
Summary:
In preparation for writing a test for a bug fix we need to be able to
see the command used to launch the symbolizer process. This feature
will likely be useful for debugging how the Sanitizers use the
symbolizer in general.
This patch causes the command line used to launch the process to be
shown at verbosity level 3 and higher.
A small test case is included.
Reviewers: kubamracek, yln, vitalybuka, eugenis, kcc
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D77622
Summary:
Fuchsia's gcc uses this, which in turn prevents us to compile successfully
due to a few `memset`'ing some non-trivial classes in some `init`.
Change those `memset` to members initialization.
Reviewers: pcc, hctim
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D77902
For targets where char is unsigned (like PowerPC), something like
char c = fgetc(...) will never produce a char that will compare
equal to EOF so this loop does not terminate.
Change the type to int (which appears to be the POSIX return type
for fgetc).
This allows the test case to terminate normally on PPC.
Buildbots say:
[126/127] Running lint check for sanitizer sources...
FAILED: projects/compiler-rt/lib/CMakeFiles/SanitizerLintCheck
cd /home/buildbots/ppc64be-clang-multistage-test/clang-ppc64be-multistage/stage1/projects/compiler-rt/lib && env LLVM_CHECKOUT=/home/buildbots/ppc64be-clang-multistage-test/clang-ppc64be-multistage/llvm/llvm SILENT=1 TMPDIR= PYTHON_EXECUTABLE=/usr/bin/python COMPILER_RT=/home/buildbots/ppc64be-clang-multistage-test/clang-ppc64be-multistage/llvm/compiler-rt /home/buildbots/ppc64be-clang-multistage-test/clang-ppc64be-multistage/llvm/compiler-rt/lib/sanitizer_common/scripts/check_lint.sh
/home/buildbots/ppc64be-clang-multistage-test/clang-ppc64be-multistage/llvm/compiler-rt/test/tsan/fiber_cleanup.cpp:71: Could not find a newline character at the end of the file. [whitespace/ending_newline] [5]
ninja: build stopped: subcommand failed.
Somehow this check is not part of 'ninja check-tsan'.
When creating and destroying fibers in tsan a thread state is created and destroyed. Currently, a memory mapping is leaked with each fiber (in __tsan_destroy_fiber). This causes applications with many short running fibers to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to ThreadSignalContext for handling signals. The initialization and destruction of it is tied to platform specific events in tsan_interceptors_posix and missed when destroying a fiber (specifically, SigCtx is used to lazily create the ThreadSignalContext in tsan_interceptors_posix). This patch cleans up the memory by makinh the ThreadState create and destroy the ThreadSignalContext.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
Author: Florian
Reviewed-in: https://reviews.llvm.org/D76073
Summary:
This commit adds two command-line options to clang.
These options let the user decide which functions will receive SanitizerCoverage instrumentation.
This is most useful in the libFuzzer use case, where it enables targeted coverage-guided fuzzing.
Patch by Yannis Juglaret of DGA-MI, Rennes, France
libFuzzer tests its target against an evolving corpus, and relies on SanitizerCoverage instrumentation to collect the code coverage information that drives corpus evolution. Currently, libFuzzer collects such information for all functions of the target under test, and adds to the corpus every mutated sample that finds a new code coverage path in any function of the target. We propose instead to let the user specify which functions' code coverage information is relevant for building the upcoming fuzzing campaign's corpus. To this end, we add two new command line options for clang, enabling targeted coverage-guided fuzzing with libFuzzer. We see targeted coverage guided fuzzing as a simple way to leverage libFuzzer for big targets with thousands of functions or multiple dependencies. We publish this patch as work from DGA-MI of Rennes, France, with proper authorization from the hierarchy.
Targeted coverage-guided fuzzing can accelerate bug finding for two reasons. First, the compiler will avoid costly instrumentation for non-relevant functions, accelerating fuzzer execution for each call to any of these functions. Second, the built fuzzer will produce and use a more accurate corpus, because it will not keep the samples that find new coverage paths in non-relevant functions.
The two new command line options are `-fsanitize-coverage-whitelist` and `-fsanitize-coverage-blacklist`. They accept files in the same format as the existing `-fsanitize-blacklist` option <https://clang.llvm.org/docs/SanitizerSpecialCaseList.html#format>. The new options influence SanitizerCoverage so that it will only instrument a subset of the functions in the target. We explain these options in detail in `clang/docs/SanitizerCoverage.rst`.
Consider now the woff2 fuzzing example from the libFuzzer tutorial <https://github.com/google/fuzzer-test-suite/blob/master/tutorial/libFuzzerTutorial.md>. We are aware that we cannot conclude much from this example because mutating compressed data is generally a bad idea, but let us use it anyway as an illustration for its simplicity. Let us use an empty blacklist together with one of the three following whitelists:
```
# (a)
src:*
fun:*
# (b)
src:SRC/*
fun:*
# (c)
src:SRC/src/woff2_dec.cc
fun:*
```
Running the built fuzzers shows how many instrumentation points the compiler adds, the fuzzer will output //XXX PCs//. Whitelist (a) is the instrument-everything whitelist, it produces 11912 instrumentation points. Whitelist (b) focuses coverage to instrument woff2 source code only, ignoring the dependency code for brotli (de)compression; it produces 3984 instrumented instrumentation points. Whitelist (c) focuses coverage to only instrument functions in the main file that deals with WOFF2 to TTF conversion, resulting in 1056 instrumentation points.
For experimentation purposes, we ran each fuzzer approximately 100 times, single process, with the initial corpus provided in the tutorial. We let the fuzzer run until it either found the heap buffer overflow or went out of memory. On this simple example, whitelists (b) and (c) found the heap buffer overflow more reliably and 5x faster than whitelist (a). The average execution times when finding the heap buffer overflow were as follows: (a) 904 s, (b) 156 s, and (c) 176 s.
We explain these results by the fact that WOFF2 to TTF conversion calls the brotli decompression algorithm's functions, which are mostly irrelevant for finding bugs in WOFF2 font reconstruction but nevertheless instrumented and used by whitelist (a) to guide fuzzing. This results in longer execution time for these functions and a partially irrelevant corpus. Contrary to whitelist (a), whitelists (b) and (c) will execute brotli-related functions without instrumentation overhead, and ignore new code paths found in them. This results in faster bug finding for WOFF2 font reconstruction.
The results for whitelist (b) are similar to the ones for whitelist (c). Indeed, WOFF2 to TTF conversion calls functions that are mostly located in SRC/src/woff2_dec.cc. The 2892 extra instrumentation points allowed by whitelist (b) do not tamper with bug finding, even though they are mostly irrelevant, simply because most of these functions do not get called. We get a slightly faster average time for bug finding with whitelist (b), which might indicate that some of the extra instrumentation points are actually relevant, or might just be random noise.
Reviewers: kcc, morehouse, vitalybuka
Reviewed By: morehouse, vitalybuka
Subscribers: pratyai, vitalybuka, eternalsakura, xwlin222, dende, srhines, kubamracek, #sanitizers, lebedev.ri, hiraditya, cfe-commits, llvm-commits
Tags: #clang, #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D63616
Looks like this test fails on Darwin x86_64 as well:
http://green.lab.llvm.org/green/job/clang-stage1-RA/8593/
Command Output (stderr):
--
fatal error: error in backend: Global variable '__sancov_gen_' has an invalid section specifier '__DATA,__sancov_bool_flag': mach-o section specifier requires a section whose length is between 1 and 16 characters.
Summary:
Commit b684c1a50f ("Add a `Symbolizer::GetEnvP()` method that allows
symbolizer implementations to customise the environment of the
symbolizer binary.") exposed a latent ARM issue, and that broke
http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-full-sh
This coincided with breakage caused by my commit 5f5fb56c68
("[compiler-rt] Intercept the uname() function"), so I had to
investigate.
The issue is that GetArgsAndEnv does not work on ARM: there glibc's
_start overwrites argc value stored at __libc_start_end, breaking the
existing argv/envp parsing logic.
Fix by inferring argc from argv.
Reviewers: eugenis, vitalybuka
Reviewed By: eugenis
Subscribers: dberris, kristof.beyls, danielkiss, #sanitizers, delcypher
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D77400
Summary:
MSan not implementing COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED looks
like an omission - this macro makes it possible for those intercepted
functions, which libc needs very early, to work before interceptors are
initialized (i.e. before REAL() is usable).
While currently there are no observable practical problems in this
area, changes in libc or in MSan runtime may provoke them. Therefore,
change MSan to work like ASan and TSan already do - use internal
functions in certain interceptors when initialization is not complete.
Reviewers: eugenis, vitalybuka
Reviewed By: eugenis
Subscribers: #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76969
Fix similar to https://reviews.llvm.org/D77111 but fow Windows.
gotsan.cpp:14071:16: error: missing terminating ' character [-Werror]
case 0b10'010: // c.lwsp (rd != x0)
^
Reported-by: Keith Randall
The intent of the `llvm_gcda_start_file` function is that only
one process create the .gcda file and initialize it to be updated
by other processes later.
Before this change, if multiple processes are started simultaneously,
some of them may initialize the file because both the first and
second `open` calls may succeed in a race condition and `new_file`
becomes 1 in those processes. This leads incorrect coverage counter
values. This often happens in MPI (Message Passing Interface) programs.
The test program added in this change is a simple reproducer.
This change ensures only one process creates/initializes the file by
using the `O_EXCL` flag.
Differential Revision: https://reviews.llvm.org/D76206
Summary:
When doing cross-compilation from Linux to MacOS we don't have
access to have access to `xcodebuild` and therefore need a way
to set the SDK version from the outside.
Fixes https://reviews.llvm.org/D68292#1853594 for me.
Reviewers: delcypher, yln
Reviewed By: delcypher
Subscribers: #julialang, mgorny, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D77026
The above change used a binary literal that is not supported in c++11 mode when
using gcc. It was formalized into the c++14 standard and works when using that
mode to compile, so change the script to use c++14 instead.
Reviewed by: dvyukov
Differential Revision: https://reviews.llvm.org/D77111
Summary:
Commit 5f5fb56c68 ("[compiler-rt] Intercept the uname() function")
broke sanitizer-x86_64-linux and clang-cmake-thumbv7-full-sh (again)
builds:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/26313http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-full-sh/builds/4324
The reason is that uname() can be called as early as
__pthread_initialize_minimal_internal(). When intercepted, this
triggers ASan initialization, which eventually calls dlerror(), which
in turn uses pthreads, causing all sorts of issues.
Fix by falling back to internal_uname() when interceptor runs before
ASan is initialized. This is only for Linux at the moment.
Reviewers: eugenis, vitalybuka
Reviewed By: eugenis
Subscribers: dberris, #sanitizers, pcc
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76919
Summary:
Follow up fix to 445b810fbd. The `log show` command only works for
privileged users so run a quick test of the command during lit config to
see if the command works and only add the `darwin_log_cmd` feature if
this is the case.
Unfortunately this means the `asan/TestCases/Darwin/duplicate_os_log_reports.cpp`
test and any other tests in the future that use this feature won't run
for unprivileged users which is likely the case in CI.
rdar://problem/55986279
Reviewers: kubamracek, yln, dcoughlin
Subscribers: Charusso, #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76899
This patch follows the approach also used for MIPS, where we decode the
offending instruction to determine if the fault was caused by a read or
write operation, as that seems to be the only relevant information we have
in the signal context structure to determine that.
Differential Revision: https://reviews.llvm.org/D75168
Summary:
When ASan reports an issue the contents of the system log buffer
(`error_message_buffer`) get flushed to the system log (via
`LogFullErrorReport()`). After this happens the buffer is not cleared
but this is usually fine because the process usually exits soon after
reporting the issue.
However, when ASan runs in `halt_on_error=0` mode execution continues
without clearing the buffer. This leads to problems if more ASan
issues are found and reported.
1. Duplicate ASan reports in the system log. The Nth (start counting from 1)
ASan report will be duplicated (M - N) times in the system log if M is the
number of ASan issues reported.
2. Lost ASan reports. Given a sufficient
number of reports the buffer will fill up and consequently cannot be appended
to. This means reports can be lost.
The fix here is to reset `error_message_buffer_pos` to 0 which
effectively clears the system log buffer.
A test case is included but unfortunately it is Darwin specific because
querying the system log is an OS specific activity.
rdar://problem/55986279
Reviewers: kubamracek, yln, vitalybuka, kcc, filcab
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76749
Summary:
There's no unwinding functionality on Android that allows for line
numbers to be retrieved in-process. As a result, we can't have
this backtrace test run on Android.
Cleanup the test to use optnone functions instead, which is more stable
than line numbers anyway.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: #sanitizers, morehouse, cferris
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76807
Commit 5f5fb56c68 ("[compiler-rt] Intercept the uname() function")
broke clang-cmake-thumbv7-full-sh build:
http://lab.llvm.org:8011/builders/clang-cmake-thumbv7-full-sh/builds/4296
This also affects i386.
The reason is that intercepted uname() is called by GetKernelAreaSize()
during ASAN initialization on 32-bit platforms, but the respective
interceptor is not initialized yet at this point, leading to null
pointer dereference.
Introduce internal_uname() wrapper around uname syscall, and use it in
GetKernelAreaSize() and in FixedCVE_2016_2143().
Author: Ilya Leoshkevich
Reviewed By: Evgenii Stepanov
Differential Revision: https://reviews.llvm.org/D76776
Disable symbolization of results, since llvm-symbolizer cannot start
due to restricted readlink(), causing the test to die with SIGPIPE.
Author: Ilya Leoshkevich
Reviewed By: Evgenii Stepanov
Differential Revision: https://reviews.llvm.org/D76576
Temporarily revert "tsan: fix leak of ThreadSignalContext for fibers"
because it breaks the LLDB bot on GreenDragon.
This reverts commit 93f7743851.
This reverts commit d8a0f76de7.
When creating and destroying fibers in tsan a thread state
is created and destroyed. Currently, a memory mapping is
leaked with each fiber (in __tsan_destroy_fiber).
This causes applications with many short running fibers
to crash or hang because of linux vm.max_map_count.
The root of this is that ThreadState holds a pointer to
ThreadSignalContext for handling signals. The initialization
and destruction of it is tied to platform specific events
in tsan_interceptors_posix and missed when destroying a fiber
(specifically, SigCtx is used to lazily create the
ThreadSignalContext in tsan_interceptors_posix). This patch
cleans up the memory by inverting the control from the
platform specific code calling the generic ThreadFinish to
ThreadFinish calling a platform specific clean-up routine
after finishing a thread.
The relevant code causing the leak with fibers is the fiber destruction:
void FiberDestroy(ThreadState *thr, uptr pc, ThreadState *fiber) {
FiberSwitchImpl(thr, fiber);
ThreadFinish(fiber);
FiberSwitchImpl(fiber, thr);
internal_free(fiber);
}
I would appreciate feedback if this way of fixing the leak is ok.
Also, I think it would be worthwhile to more closely look at the
lifecycle of ThreadState (i.e. it uses no constructor/destructor,
thus requiring manual callbacks for cleanup) and how OS-Threads/user
level fibers are differentiated in the codebase. I would be happy to
contribute more if someone could point me at the right place to
discuss this issue.
Reviewed-in: https://reviews.llvm.org/D76073
Author: Florian (Florian)
tsan while used by golang's race detector was not working on alpine
linux, since it is using musl-c instead of glibc. Since alpine is very
popular distribution for container deployments, having working race
detector would be nice. This commits adds some ifdefs to get it working.
It fixes https://github.com/golang/go/issues/14481 on golang's issue tracker.
Reviewed-in: https://reviews.llvm.org/D75849
Author: graywolf-at-work (Tomas Volf)
Summary: Separate class definition and actual methods implementation. The main
goal is to keep the list of available methods in a concise readable form inside
the class definition.
Reviewers: hctim, metzman
Subscribers: dberris, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76651
This avoids the test failure that was introduced in rG32bddad where
this function pulls in the rest of InstrProfilingFile.c which is
undesirable in use cases when profile runtime is being used without
the rest of libc.
This also allows additional cleanup by eliminating another variable
from platforms that don't need it.
Differential Revision: https://reviews.llvm.org/D76750
On Fuchsia, we always use the continuous mode with runtime counter
relocation, so there's no need for atexit hook or support for dumping
the profile manually.
Differential Revision: https://reviews.llvm.org/D76556
Summary:
This change introduces the `Symbolizer::GetEnvP()` method that returns a
pointer to environment array used for spawning the symbolizer process.
The motivation is to allow implementations to customise the environment
if required. The default implementation just returns
`__sanitizer::GetEnviron()` which (provided it's implemented) should
preserve the existing behaviours of the various implementations.
This change has been plumbed through the `internal_spawn(...)` and
`StartSubprocess(...)` process spawning implementations.
For the `StartSubprocess()` implementation we need to call `execve()`
rather than `execv()` to pass the environment. However, it appears that
`internal_execve(...)` exists in sanitizer_common so this patch use that
which seems like a nice clean up.
Support in the Windows implementation of
`SymbolizerProcess:StartSymbolizerSubprocess()` has not been added
because the Windows sanitizer runtime doesn't implement `GetEnviron()`.
rdar://problem/58789439
Reviewers: kubamracek, yln, dvyukov, vitalybuka, eugenis, phosek, aizatsky, rnk
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76666
Summary:
Backtrace() returns the number of frames that are *available*, rather
than the number of frames stored. When we pack, we supply the number of
frames we have stored. The number of available frames can exceed the
number of stored frames, leading to stack OOB read.
Fix up this problem.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: #sanitizers, morehouse, cferris, pcc
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76722
realeaseAcquire() is a new function added to TSan in support of the Go data-race detector.
It's semantics is:
void ThreadClock::releaseAcquire(SyncClock *sc) const {
for (int i = 0; i < kMaxThreads; i++) {
tmp = clock[i];
clock[i] = max(clock[i], sc->clock[i]);
sc->clock[i] = tmp;
}
}
For context see: https://go-review.googlesource.com/c/go/+/220419
Reviewed-in: https://reviews.llvm.org/D76322
Author: dfava (Daniel Fava)
For MTE error reporting we will need to expose interfaces for crash handlers
to use to interpret scudo headers and metadata. The intent is that these
interfaces will live in scudo/interface.h.
Move the existing interface.h into an include/scudo directory and make it
independent of the internal headers, so that we will be able to add the
interfaces there.
Differential Revision: https://reviews.llvm.org/D76648
struct stack_t on Linux x86_64 has internal padding which may be left
uninitialized. The check should be replaced with multiple checks for
individual fields of the struct. For now, remove the check altogether.
Summary:
Move interceptor from msan to sanitizer_common_interceptors.inc, so that
other sanitizers could benefit.
Adjust FixedCVE_2016_2143() to deal with the intercepted uname().
Patch by Ilya Leoshkevich.
Reviewers: eugenis, vitalybuka, uweigand, jonpa
Reviewed By: eugenis, vitalybuka
Subscribers: dberris, krytarowski, #sanitizers, stefansf, Andreas-Krebbel
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76578
Summary:
We introduced a way to fallback to the immediately larger size class for
the Primary in the event a region was full, but in the event of the largest
size class, we would just fail.
This change allows to fallback to the Secondary when the last region of
the Primary is full. We also expand the trick to all platforms as opposed
to being Android only, and update the test to cover the new case.
Reviewers: hctim, cferris, eugenis, morehouse, pcc
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76430
The VMO size is always page-rounded, but Zircon now provides
a way to publish the precise intended size.
Patch By: mcgrathr
Differential Revision: https://reviews.llvm.org/D76437
While the VMO size is always page aligned, we can record the content
size as a property and then use this metadata when writing the data to
a file.
Differential Revision: https://reviews.llvm.org/D76462
While the VMO size is always page aligned, we can record the content
size as a property and then use this metadata when writing the profile
to a file.
Differential Revision: https://reviews.llvm.org/D76402
Summary:
Some kernels can provide 16EiB worth of mappings to each process, which
causes mmap test to run for a very long time. In order to make it stop
after a few seconds, make mmap_interceptor() fail when the original
mmap() returns an address which is outside of the application range.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: #sanitizers, Andreas-Krebbel, stefansf, jonpa, uweigand
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D76426
arm64e adds support for pointer authentication, which was adopted by
libplatform to harden setjmp/longjmp and friends. We need to teach
the TSan interceptors for those functions about this.
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D76257
This test case fails due to different handling of weak items between
LLD and LD on PPC. The issue only occurs when the default linker is LLD
and the test case is run on a system where ASLR is enabled.
https://github.com/apple/swift/pull/30112/ makes the Swift standard
library for iOS build for arm64e. If you're building Swift against your
own LLVM, this in turn requires having the builtins built for arm64e,
otherwise you won't be able to use the builtins (which will in turn lead
to an undefined symbol for `__isOSVersionAtLeast`). Make the builtins
build for arm64e to fix this.
Differential Revision: https://reviews.llvm.org/D76041
Summary:
The `ElfW()` macro is not provided by `<link.h>` on some
systems (e.g., FreeBSD). On these systems the data structures are
just called `Elf_XXX`. Define `ElfW()` locally.
(This fix is taken from [libunwind](9b05596eff/libunwind/src/AddressSpace.hpp (L144-L157)).)
Reviewers: compnerd
Differential revision: https://reviews.llvm.org/D75907
Note that in C++ the static keyword is implicit for const objects.
In C however it is not, and we get clashes at link time after
including the header from more than one C file:
lib.a(bar.o):(.rodata+0x0): multiple definition of `__tsan_mutex_linker_init'
lib.a(foo.o):(.rodata+0x0): first defined here
lib.a(bar.o):(.rodata+0xc): multiple definition of `__tsan_mutex_not_static'
lib.a(foo.o):(.rodata+0xc): first defined here
<snip>
Indeed both foo.o and bar.o define the clashing symbols:
$ nm foo.o
<snip>
0000000000000000 R __tsan_mutex_linker_init
000000000000000c R __tsan_mutex_not_static
<snip>
Fix it by explicitly making the constants static.
Reviewed-in: https://reviews.llvm.org/D75820
Author: cota (Emilio G. Cota)
A recent change to MemorySSA caused LLVM to start optimizing the call to
'f(x)' into just 'x', despite the 'noinline' attribute. So try harder to
prevent this optimization from firing.
Patch by Zhizhou Yang!
In his own words:
"""
Similar change to CMakeLists as r372312.
After r372209, compiler command line may include argument with quotes:
```
-fprofile-instr-use="/foo/bar.profdata"
```
And it causes a hidden failure with execute_process later: Could not
read profile "/foo/bar.profdata": No such file or directory.
In this particular case, the check for .init_array will fail silently
and creates a PGO-ed binary with bad .init_array section in compiler-rt.
Bug details can be found in PR45022
"""
Differential Revision: https://reviews.llvm.org/D75065
Patch by Zhizhou Yang!
In his own words:
"""
Currently compiler-rt doesn't officially support either PGO
instrumentation or use PGO profdata to build it.
PGO related flags are passed into compiler-rt since rL372209, and
causing bugs: 45022, crbug:1018840
This patch adds several checks in compiler-rt to disable PGO related
flags and provides a flag to turn on PGO for compiler-rt if needed.
"""
Differential Revision: https://reviews.llvm.org/D75499
After a first attempt to fix the test-suite failures, my first recommit
caused the same failures again. I had updated CMakeList.txt files of
tests that needed -fcommon, but it turns out that there are also
Makefiles which are used by some bots, so I've updated these Makefiles
now too.
See the original commit message for more details on this change:
0a9fc9233e
This includes fixes for:
- test-suite: some benchmarks need to be compiled with -fcommon, see D75557.
- compiler-rt: one test needed -fcommon, and another a change, see D75520.
Implements `__clear_cache` for RISC-V Linux. We can't just use `fence.i` on
Linux, because the Linux thread might be scheduled on another hart, and the
`fence.i` instruction only flushes the icache of the current hart.
Summary:
For the 32b primary, whenever we created a region, we would fill it
all at once (eg: create all the transfer batches for all the blocks
in that region). This wasn't ideal as all the potential blocks in
a newly created region might not be consummed right away, and it was
using extra memory (and release cycles) to keep all those free
blocks.
So now we keep track of the current region for a given class, and
how filled it is, carving out at most `MaxNumBatches` worth of
blocks at a time.
Additionally, lower `MaxNumBatches` on Android from 8 to 4. This
lowers the randomness of blocks, which isn't ideal for security, but
keeps things more clumped up for PSS/RSS accounting purposes.
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75551
llvm/Support/Base64, fix its implementation and provide a decent test suite.
Previous implementation code was using + operator instead of | to combine
results, which is a problem when shifting signed values. (0xFF << 16) is
implicitly converted to a (signed) int, and thus results in 0xffff0000,
h is
negative. Combining negative numbers with a + in that context is not what we
want to do.
This is a recommit of 5a1958f267 with UB removved.
This fixes https://github.com/llvm/llvm-project/issues/149.
Differential Revision: https://reviews.llvm.org/D75057
and follow-ups:
a2ca1c2d "build: disable zlib by default on Windows"
2181bf40 "[CMake] Link against ZLIB::ZLIB"
1079c68a "Attempt to fix ZLIB CMake logic on Windows"
This changed the output of llvm-config --system-libs, and more
importantly it broke stand-alone builds. Instead of piling on more fix
attempts, let's revert this to reduce the risk of more breakages.
After the format change from D69471, there can be more than one section
in an object that contains coverage function records. Look up each of
these sections and concatenate all the records together.
This re-enables the instrprof-merging.cpp test, which previously was
failing on OSes which use comdats.
Thanks to Jeremy Morse, who very kindly provided object files from the
bot I broke to help me debug.
Move Base64 implementation from clangd/SemanticHighlighting to
llvm/Support/Base64, fix its implementation and provide a decent test suite.
Previous implementation code was using + operator instead of | to combine some
results, which is a problem when shifting signed values. (0xFF << 16) is
implicitly converted to a (signed) int, and thus results in 0xffff0000, which is
negative. Combining negative numbers with a + in that context is not what we
want to do.
This fixes https://github.com/llvm/llvm-project/issues/149.
Differential Revision: https://reviews.llvm.org/D75057
An execution count goes missing for a constructor, this needs
investigation:
http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/45132/
```
/home/buildbots/ppc64be-clang-test/clang-ppc64be/llvm/compiler-rt/test/profile/instrprof-merging.cpp:28:16:
error: V1: expected string not found in input
A() {} // V1: [[@LINE]]{{ *}}|{{ *}}1
<stdin>:28:32: note: possible intended match here
28| | A() {} // V1: [[@LINE]]{{ *}}|{{ *}}1
```
Try again with an up-to-date version of D69471 (99317124 was a stale
revision).
---
Revise the coverage mapping format to reduce binary size by:
1. Naming function records and marking them `linkonce_odr`, and
2. Compressing filenames.
This shrinks the size of llc's coverage segment by 82% (334MB -> 62MB)
and speeds up end-to-end single-threaded report generation by 10%. For
reference the compressed name data in llc is 81MB (__llvm_prf_names).
Rationale for changes to the format:
- With the current format, most coverage function records are discarded.
E.g., more than 97% of the records in llc are *duplicate* placeholders
for functions visible-but-not-used in TUs. Placeholders *are* used to
show under-covered functions, but duplicate placeholders waste space.
- We reached general consensus about giving (1) a try at the 2017 code
coverage BoF [1]. The thinking was that using `linkonce_odr` to merge
duplicates is simpler than alternatives like teaching build systems
about a coverage-aware database/module/etc on the side.
- Revising the format is expensive due to the backwards compatibility
requirement, so we might as well compress filenames while we're at it.
This shrinks the encoded filenames in llc by 86% (12MB -> 1.6MB).
See CoverageMappingFormat.rst for the details on what exactly has
changed.
Fixes PR34533 [2], hopefully.
[1] http://lists.llvm.org/pipermail/llvm-dev/2017-October/118428.html
[2] https://bugs.llvm.org/show_bug.cgi?id=34533
Differential Revision: https://reviews.llvm.org/D69471
Revise the coverage mapping format to reduce binary size by:
1. Naming function records and marking them `linkonce_odr`, and
2. Compressing filenames.
This shrinks the size of llc's coverage segment by 82% (334MB -> 62MB)
and speeds up end-to-end single-threaded report generation by 10%. For
reference the compressed name data in llc is 81MB (__llvm_prf_names).
Rationale for changes to the format:
- With the current format, most coverage function records are discarded.
E.g., more than 97% of the records in llc are *duplicate* placeholders
for functions visible-but-not-used in TUs. Placeholders *are* used to
show under-covered functions, but duplicate placeholders waste space.
- We reached general consensus about giving (1) a try at the 2017 code
coverage BoF [1]. The thinking was that using `linkonce_odr` to merge
duplicates is simpler than alternatives like teaching build systems
about a coverage-aware database/module/etc on the side.
- Revising the format is expensive due to the backwards compatibility
requirement, so we might as well compress filenames while we're at it.
This shrinks the encoded filenames in llc by 86% (12MB -> 1.6MB).
See CoverageMappingFormat.rst for the details on what exactly has
changed.
Fixes PR34533 [2], hopefully.
[1] http://lists.llvm.org/pipermail/llvm-dev/2017-October/118428.html
[2] https://bugs.llvm.org/show_bug.cgi?id=34533
Differential Revision: https://reviews.llvm.org/D69471
Hope this fixes:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-android/builds/27977/steps/run%20lit%20tests%20%5Bi686%2Ffugu-userdebug%2FN2G48C%5D/logs/stdio
```
: 'RUN: at line 8'; UBSAN_OPTIONS=suppressions=/var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/compiler_rt_build_android_i686/test/ubsan/Standalone-i386/TestCases/Misc/Output/nullability.c.tmp.supp /var/lib/buildbot/sanitizer-buildbot6/sanitizer-x86_64-linux-android/build/compiler_rt_build_android_i686/test/ubsan/Standalone-i386/TestCases/Misc/Output/nullability.c.tmp 2>&1 | count 0
--
Exit Code: 1
Command Output (stderr):
--
Expected 0 lines, got 2.
```
Not sure what this would be printing though, a sanitizer initialization message?
Summary:
When -dfsan-event-callbacks is specified, insert a call to
__dfsan_mem_transfer_callback on every memcpy and memmove.
Reviewers: vitalybuka, kcc, pcc
Reviewed By: kcc
Subscribers: eugenis, hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75386
Summary:
For now just insert the callback for stores, similar to how MSan tracks
origins. In the future we may want to add callbacks for loads, memcpy,
function calls, CMPs, etc.
Reviewers: pcc, vitalybuka, kcc, eugenis
Reviewed By: vitalybuka, kcc, eugenis
Subscribers: eugenis, hiraditya, #sanitizers, llvm-commits, kcc
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75312
Summary:
Sanitizer tests don't entirely pass on an R device. Fix up all the
incompatibilities with the new system.
Reviewers: eugenis, pcc
Reviewed By: eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75303
Summary:
This patch includes several changes to reduce the overall footprint
of the allocator:
- for realloc'd chunks: only keep the same chunk when lowering the size
if the delta is within a page worth of bytes;
- when draining a cache: drain the beginning, not the end; we add pointers
at the end, so that meant we were draining the most recently added
pointers;
- change the release code to account for an freed up last page: when
scanning the pages, we were looking for pages fully covered by blocks;
in the event of the last page, if it's only partially covered, we
wouldn't mark it as releasable - even what follows the last chunk is
all 0s. So now mark the rest of the page as releasable, and adapt the
test;
- add a missing `setReleaseToOsIntervalMs` to the cacheless secondary;
- adjust the Android classes based on more captures thanks to pcc@'s
tool.
Reviewers: pcc, cferris, hctim, eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75142
Fails with the following message in the error case:
```
CMake Error at /path/to/llvm-project/compiler-rt/lib/tsan/CMakeLists.txt:119 (message):
Building the TSan runtime requires at least macOS SDK 10.12
```
Fixes#44682.
https://bugs.llvm.org/show_bug.cgi?id=44682
Reviewed By: dmajor, delcypher
Differential Revision: https://reviews.llvm.org/D74501
Summary:
For builtin compiler-rt, it is built with visibility hidden by default
to avoid the client exporting symbols from libclang static library. The
compiler option -fvisibility=hidden doesn't work on the aliases in c files
because they are created with inline assembly. On Darwin platform,
thoses aliases are exported by default if they are reference by the client.
Fix the issue by adding ".private_extern" to all the aliases if the
library is built with visibility hidden.
rdar://problem/58960296
Reviewers: dexonsmith, arphaman, delcypher, kledzik
Reviewed By: delcypher
Subscribers: dberris, jkorous, ributzka, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73577
See discussion on PR44792.
This reverts commit 02ce9d8ef5.
It also reverts the follow-up commits
8f46269f0 "[profile] Don't dump counters when forking and don't reset when calling exec** functions"
62c7d8402 "[profile] gcov_mutex must be static"
Generally we ignore interceptors coming from called_from_lib-suppressed libraries.
However, we must not ignore critical interceptors like e.g. pthread_create,
otherwise runtime will lost track of threads.
pthread_detach is one of these interceptors we should not ignore as it affects
thread states and behavior of pthread_join which we don't ignore as well.
Currently we can produce very obscure false positives. For more context see:
https://groups.google.com/forum/#!topic/thread-sanitizer/ecH2P0QUqPs
The added test captures this pattern.
While we are here rename ThreadTid to ThreadConsumeTid to make it clear that
it's not just a "getter", it resets user_id to 0. This lead to confusion recently.
Reviewed in https://reviews.llvm.org/D74828
Summary:
There is no need to write out gcdas when forking because we can just reset the counters in the parent process.
Let say a counter is N before the fork, then fork and this counter is set to 0 in the child process.
In the parent process, the counter is incremented by P and in the child process it's incremented by C.
When dump is ran at exit, parent process will dump N+P for the given counter and the child process will dump 0+C, so when the gcdas are merged the resulting counter will be N+P+C.
About exec** functions, since the current process is replaced by an another one there is no need to reset the counters but just write out the gcdas since the counters are definitely lost.
To avoid to have lists in a bad state, we just lock them during the fork and the flush (if called explicitely) and lock them when an element is added.
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: hiraditya, cfe-commits, #sanitizers, llvm-commits, sylvestre.ledru
Tags: #clang, #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74953
Like was done before in D67999 for `logbf`, this patch fixes the tests for
the internal compiler-rt implementations of `logb` and `logbl` to consider
all NaNs equivalent. Not doing so was resulting in test failures for
riscv64, since the the NaNs had different signs, but the spec doesn't
specify the NaN signedness or payload.
Fixes bug 44244.
Reviewers: rupprecht, delcypher
Reviewed By: rupprecht, delcypher
Differential Revision: https://reviews.llvm.org/D74826
Summary:
This substitution expands to the appropriate minimum deployment target
flag where thread local storage (TLS) was first introduced on Darwin
platforms. For all other platforms the substitution expands to an empty
string.
E.g. for macOS the substitution expands to `-mmacosx-version-min=10.12`
This patch adds support for the substitution (and future substitutions)
by doing a minor refactor and then uses the substitution in the relevant
TSan tests.
rdar://problem/59568956
Reviewers: yln, kubamracek, dvyukov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74802
Summary:
The number of "inputs have the Data Flow Trace" cannot be greater than
the number of inputs touching the focus function. The existing message is rather
confusing as the same log would mention a greater total number of traces a few
lines above.
Reviewers: kcc, metzman
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74779
The removed loop clears reused for entries at the tail of a SyncClock.
The loop is redundant since those entries were already cleared by the
immediately preceding loop, which iterates over all entries in the
SyncClock (including the tail entries).
Summary:
Most of our larger data is dynamically allocated (via `map`) but it
became an hindrance with regard to init time, for a cost to benefit
ratio that is not great. So change the `TSD`s, `RegionInfo`, `ByteMap`
to be static.
Additionally, for reclaiming, we used mapped & unmapped a buffer each
time, which is costly. It turns out that we can have a static buffer,
and that there isn't much contention on it.
One of the other things changed here, is that we hard set the number
of TSDs on Android to the maximum number, as there could be a
situation where cores are put to sleep and we could miss some.
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74696
This change makes the following lit substitutions expand to the correct
thing for macOS, iOS, tvOS, and watchOS.
%darwin_min_target_with_full_runtime_arc_support
%macos_min_target_10_11
rdar://problem/59463146
This patch defines `config.apple_platform_min_deployment_target_flag`
in the ASan, LibFuzzer, TSan, and UBSan lit test configs.
rdar://problem/59463146
platforms.
For the simulators this changes the following compilation flags (
used both for building the runtime and lit tests).
iOS simulator: `-miphoneos-version-min` -> `-mios-simulator-version-min`
watchOS simulator: `-mwatchos-version-min` -> `-mwatchos-simulator-version-min`
tvOS simulator: `-mtvos-version-min` -> `-mtvos-simulator-version-min`
rdar://problem/59463146
Summary:
Add a method to set the release to OS value as the system runs,
and allow this to be set differently in the primary and the secondary.
Also, add a default value to use for primary and secondary. This
allows Android to have a default that is different for
primary/secondary.
Update mallopt to support setting the release to OS value.
Reviewers: pcc, cryptoad
Reviewed By: cryptoad
Subscribers: cryptoad, jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74448
Summary: It complains about reaching the end of a non-void returning function.
Reviewers: eugenis, hctim, morehouse
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74578
Summary:
Due to Unity, we had to reduce our region sizes, but in some rare
situations, some programs (mostly tests AFAICT) manage to fill up
a region for a given size class.
So this adds a workaround for that attempts to allocate the block
from the immediately larger size class, wasting some memory but
allowing the application to keep going.
Reviewers: pcc, eugenis, cferris, hctim, morehouse
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74567
Summary:
A number of testcases in TSAN are designed to deal with intermittent problems
not exist in all executions of the tested program. A script called deflake.bash
runs the executable up to 10 times to deal with the intermittent nature of the tests.
The purpose of this patch is to parameterize the hard-coded threshold above via
--cmake_variables=-DTSAN_TEST_DEFLAKE_THRESHOLD=SomeIntegerValue
When this cmake var is not set, the default value of 10 will be used.
Reviewer: dvyukov (Dmitry Vyukov), eugenis (Evgenii Stepanov), rnk (Reid Kleckner), hubert.reinterpretcast (Hubert Tong), vitalybuka (Vitaly Buka)
Reviewed By: vitalybuka (Vitaly Buka)
Subscribers: mgorny (Michal Gorny), jfb (JF Bastien), steven.zhang (qshanz), llvm-commits (Mailing List llvm-commits), Sanitizers
Tag: LLVM, Sanitizers
Differential Revision: https://reviews.llvm.org/D73707
The interceptor uses thread-local variables, which (until very recently)
are emu-tls. An access to such variable may call malloc which can
deadlock the runtime library.
For CleanseCrashInput, discards stdout output anyway since it is not used.
These changes are to defend against aggressive PID recycle on windows to reduce the chance of contention on files.
Using pipe instead of file also workaround the problem that when the
process is spawned by llvm-lit, the aborted process keeps a handle to the
output file such that the output file can not be removed. This will
cause random test failures.
https://devblogs.microsoft.com/oldnewthing/20110107-00/?p=11803
Reviewers: kcc, vitalybuka
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D73329
Add an optional table lookup after the existing logarithm computation
for MidSize < Size <= MaxSize during size -> class lookups. The lookup is
O(1) due to indexing a precomputed (via constexpr) table based on a size
table. Switch to this approach for the Android size class maps.
Other approaches considered:
- Binary search was found to have an unacceptable (~30%) performance cost.
- An approach using NEON instructions (see older version of D73824) was found
to be slightly slower than this approach on newer SoCs but significantly
slower on older ones.
By selecting the values in the size tables to minimize wastage (for example,
by passing the malloc_info output of a target program to the included
compute_size_class_config program), we can increase the density of allocations
at a small (~0.5% on bionic malloc_sql_trace as measured using an identity
table) performance cost.
Reduces RSS on specific Android processes as follows (KB):
Before After
zygote (median of 50 runs) 26836 26792 (-0.2%)
zygote64 (median of 50 runs) 30384 30076 (-1.0%)
dex2oat (median of 3 runs) 375792 372952 (-0.8%)
I also measured the amount of whole-system idle dirty heap on Android by
rebooting the system and then running the following script repeatedly until
the results were stable:
for i in $(seq 1 50); do grep -A5 scudo: /proc/*/smaps | grep Pss: | cut -d: -f2 | awk '{s+=$1} END {print s}' ; sleep 1; done
I did this 3 times both before and after this change and the results were:
Before: 365650, 356795, 372663
After: 344521, 356328, 342589
These results are noisy so it is hard to make a definite conclusion, but
there does appear to be a significant effect.
On other platforms, increase the sizes of all size classes by a fixed offset
equal to the size of the allocation header. This has also been found to improve
density, since it is likely for allocation sizes to be a power of 2, which
would otherwise waste space by pushing the allocation into the next size class.
Differential Revision: https://reviews.llvm.org/D73824
This lets us remove two pointer indirections (one by removing the pointer,
and another by making the AllocatorPtr declaration hidden) in the C++ wrappers.
Differential Revision: https://reviews.llvm.org/D74356
Summary:
Instead of hand-crafting an offset into the structure returned by
dlopen(3) to get at the link map, use the documented API. This is
described in dlinfo(3): by calling it with `RTLD_DI_LINKMAP`, the
dynamic linker ensures the right address is returned.
This is a recommit of 92e267a94d, with
dlinfo(3) expliclity being referenced only for FreeBSD, non-Android
Linux, NetBSD and Solaris. Other OSes will have to add their own
implementation.
Reviewers: devnexen, emaste, MaskRay, krytarowski
Reviewed By: krytarowski
Subscribers: krytarowski, vitalybuka, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73990
Summary:
The refactoring has caused a failure in
http://lab.llvm.org:8011/builders/clang-ppc64le-linux/builds/29265
The idea of failing the symbolization when the symbolizer bufer is too small
was incorrect. The symbolizer can be invoked for other frames that may fit into
the buffer and get symbolized.
Reviewers: vitalybuka, eugenis
Subscribers: dberris, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74343
Summary:
Instead of hand-crafting an offset into the structure returned by
dlopen(3) to get at the link map, use the documented API. This is
described in dlinfo(3): by calling it with `RTLD_DI_LINKMAP`, the
dynamic linker ensures the right address is returned.
Reviewers: devnexen, emaste, MaskRay, krytarowski
Reviewed By: krytarowski
Subscribers: krytarowski, vitalybuka, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73990
Summary:
Nothing critical, just a few potential improvements I've noticed while reading
the code:
- return `false` when symbolizer buffer is too small to read all data
- invert some conditions to reduce indentation
- prefer `nullptr` over `0` for pointers; init some pointers on stack;
- remove minor code duplication
Reviewers: eugenis, vitalybuka
Subscribers: dberris, #sanitizers, llvm-commits, kcc
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74137
Summary:
This tweaks some behaviors of the allocator wrt 32-bit, notably
tailoring the size-class map.
I had to remove a `printStats` from `__scudo_print_stats` since when
within Bionic they share the same slot so they can't coexist at the
same time. I have to find a solution for that later, but right now we
are not using the Svelte configuration.
Reviewers: rengolin
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74178
EXCLUDE_FROM_ALL means something else for add_lit_testsuite as it does
for something like add_executable. Distinguish between the two by
renaming the variable and making it an argument to add_lit_testsuite.
Differential revision: https://reviews.llvm.org/D74168
Summary:
New shard out of the Check() function for GWP-ASan uses
android_set_abort_message. This is happily present on bionic Android,
but not Android for glibc host x86. Fix up to use the weak definition
always, so we don't have to worry.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: #sanitizers, llvm-commits, pcc, cferris
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74142
Summary:
GWP-ASan's PRNG didn't use Initial-Exec TLS. Fix that to ensure that we don't
have infinite recursion, and also that we don't allocate a DTV on Android when
GWP-ASan is touched.
Test coverage ensuring that the sample counter is UINT32_MAX for an
uninitialised GWP-ASan is provided by gwp_asan/tests/late_init.cpp.
Reviewers: pcc, cferris
Reviewed By: pcc
Subscribers: #sanitizers, llvm-commits, rprichard, eugenis
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74135
The class is only used in SizeClassAllocator32 in 64-bit mode, but we don't
use that class in 64-bit mode.
Differential Revision: https://reviews.llvm.org/D74099
Summary:
Forewarning: This patch looks big in #LOC changed. I promise it's not that bad, it just moves a lot of content from one file to another. I've gone ahead and left inline comments on Phabricator for sections where this has happened.
This patch:
1. Introduces the crash handler API (crash_handler_api.h).
2. Moves information required for out-of-process crash handling into an AllocatorState. This is a trivially-copied POD struct that designed to be recovered from a deceased process, and used by the crash handler to create a GWP-ASan report (along with the other trivially-copied Metadata struct).
3. Implements the crash handler API using the AllocatorState and Metadata.
4. Adds tests for the crash handler.
5. Reimplements the (now optionally linked by the supporting allocator) in-process crash handler (i.e. the segv handler) using the new crash handler API.
6. Minor updates Scudo & Scudo Standalone to fix compatibility.
7. Changed capitalisation of errors (e.g. /s/Use after free/Use After Free).
Reviewers: cryptoad, eugenis, jfb
Reviewed By: eugenis
Subscribers: merge_guards_bot, pcc, jfb, dexonsmith, mgorny, cryptoad, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73557
Summary:
I tried to move the `madvise` calls outside of one of the secondary
mutexes, but this backfired. There is situation when a low release
interval is set combined with secondary pressure that leads to a race:
a thread can get a block from the cache, while another thread is
`madvise`'ing that block, resulting in a null header.
I changed the secondary race test so that this situation would be
triggered, and moved the release into the cache mutex scope.
Reviewers: cferris, pcc, eugenis, hctim, morehouse
Subscribers: jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74072
XRay builds uses llvm-config to obtain the ldflags and libs and then
passes those to CMake. Unfortunately, this breaks on Windows because
CMake tries to interpret backslashes followed by certain characters
as flags. We need to rewrite these into forward slashes that are used
by CMake (even on Windows).
Differential Revision: https://reviews.llvm.org/D73523
Summary:
`sigaltstack` interception is implemented D73816. This updated OpenBSD and Solaris bits to fix the build errors.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: fedor.sergeev, krytarowski, emaste, eugenis, dberris, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Patch by Igor Sugak.
Differential Revision: https://reviews.llvm.org/D73976
By subtracting 1 from Size at the beginning we can simplify the
subsequent calculations. This also saves 4 instructions on aarch64
and 9 instructions on x86_64, but seems to be perf neutral.
Differential Revision: https://reviews.llvm.org/D73936
Summary:
Fix parsing of mangled stack trace lines where the address has been
replaced with "0x", literally.
Reviewers: vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73823
Summary:
An implementation for `sigaltstack` to make its side effect be visible to MSAN.
```
ninja check-msan
```
Reviewers: vitalybuka, eugenis
Reviewed By: eugenis
Subscribers: dberris, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73816
Patch by Igor Sugak.
As a result of recent changes to the Android size classes, the malloc_free_loop
benchmark started exhausting the 8192 size class at 32768 iterations. To avoid
this problem (and to make the test more realistic), change the benchmark to
use a variety of size classes.
Differential Revision: https://reviews.llvm.org/D73918
Summary:
This patch allows for late initialisation of the GWP-ASan allocator. Previously, if late initialisation occurred, the sample counter was never updated, meaning we would end up having to wait for 2^32 allocations before getting a sampled allocation.
Now, we initialise the sampling mechanism in init() as well. We require init() to be called single-threaded, so this isn't a problem.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: merge_guards_bot, mgorny, #sanitizers, llvm-commits, cferris
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73896
The configuration for -fPIC in the builtins library when built standalone
is unconditional, stating that the flags would "normally be added... by
the llvm cmake step"
This is untrue, as the llvm cmake step checks LLVM_ENABLE_PIC, which allows
a client to turn off -fPIC.
I've added an option when compiler-rt builtins are configured standalone, such
as when built as part of the LLVM runtimes system, to guard the application of
-fPIC for users that want it.
Patch By: JamesNagurne
Differential Revision: https://reviews.llvm.org/D72950
Summary:
This changes a couple of parameters in the default Android config to
address some performance and memory footprint issues (well to be closer
to the default Bionic allocator numbers).
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73750
Summary:
The Secondary's cache needs to be released when the Combined's
`releaseToOS` function is called (via `M_PURGE`) for example,
which this CL adds.
Additionally, if doing a forced release, we'll release the
transfer batch class as well since now we can do that.
There is a couple of other house keeping changes as well:
- read the page size only once in the Secondary Cache `store`
- remove the interval check for `CanRelease`: we are going to
make that configurable via `mallopt` so this needs not be
set in stone there.
Reviewers: cferris, hctim, pcc, eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73730
Summary: `.cmd` is interpreted as script in windows console.
Reviewers: davidxl, rnk
Reviewed By: davidxl
Differential Revision: https://reviews.llvm.org/D73327
Summary:
A couple of seemingly innocuous changes ended up having a large impact
on the 32-bit performance. I still have to make those configurable at
some point, but right now it will have to do.
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73658
Summary:
Since commit c299d1981d scudo
standalone can't be built without including sys/auxv.h.
I do not have that file on my system, and my builds have failed
when trying to simply build "all" runtimes. Assuming that "all"
means "all possible given the current environment" we need to
guard the setting of COMPILER_RT_HAS_SCUDO_STANDALONE=TRUE by
first checking if sys/auxv.h can be found.
Reviewers: pcc, cryptoad
Reviewed By: pcc
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73631
mlockall and munlockall were introduced in Android API 17, so avoid
referencing them on prior versions.
Differential Revision: https://reviews.llvm.org/D73515
Summary:
Zygote & children's stderr is lost, so use Bionic's provided allocation
free syslog function for `outputRaw`. Get rid of the mutex as it's not
vital and could cause issues with `fork`.
Reviewers: cferris, pcc, eugenis, hctim, morehouse
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73561
Summary:
This CL changes multiple things to improve performance (notably on
Android).We introduce a cache class for the Secondary that is taking
care of this mechanism now.
The changes:
- change the Secondary "freelist" to an array. By keeping free secondary
blocks linked together through their headers, we were keeping a page
per block, which isn't great. Also we know touch less pages when
walking the new "freelist".
- fix an issue with the freelist getting full: if the pattern is an ever
increasing size malloc then free, the freelist would fill up and
entries would not be used. So now we empty the list if we get to many
"full" events;
- use the global release to os interval option for the secondary: it
was too costly to release all the time, particularly for pattern that
are malloc(X)/free(X)/malloc(X). Now the release will only occur
after the selected interval, when going through the deallocate path;
- allow release of the `BatchClassId` class: it is releasable, we just
have to make sure we don't mark the batches containing batches
pointers as free.
- change the default release interval to 1s for Android to match the
current Bionic allocator configuration. A patch is coming up to allow
changing it through `mallopt`.
- lower the smallest class that can be released to `PageSize/64`.
Reviewers: cferris, pcc, eugenis, morehouse, hctim
Subscribers: phosek, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73507
Summary:
This is an Android-specific interface for iterating over all live
allocations in a memory range.
Reviewers: hctim, cferris
Subscribers: mgorny, mgrang, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73305
This is a small refactoring to prepare for porting LSan to Fuchsia.
Factor out parts of lsan_thread.{cpp,h} that don't apply to Fuchsia.
Since existing supported systems are POSIX-based, the affected code
is moved to lsan_posix.{cpp.h}.
Patch By: mcgrathr
Differential Revision: https://reviews.llvm.org/D73309
This is a small refactoring to prepare for porting LSan to Fuchsia.
On Fuchsia, the system supplies a unified API for suspending threads and
enumerating roots from OS-specific places like thread state and global data
ranges. So its LockStuffAndStopTheWorld implementation will make specific
callbacks for all the OS-specific root collection work before making the
common callback that includes the actual leak-checking logic.
Patch By: mcgrathr
Differential Revision: https://reviews.llvm.org/D72988
Summary:
* Implement enable() and disable() in GWP-ASan.
* Setup atfork handler.
* Improve test harness sanity and re-enable GWP-ASan in Scudo.
Scudo_standalone disables embedded GWP-ASan as necessary around fork().
Standalone GWP-ASan sets the atfork handler in init() if asked to. This
requires a working malloc(), therefore GWP-ASan initialization in Scudo
is delayed to the post-init callback.
Test harness changes are about setting up a single global instance of
the GWP-ASan allocator so that pthread_atfork() does not create
dangling pointers.
Test case shamelessly stolen from D72470.
Reviewers: cryptoad, hctim, jfb
Subscribers: mgorny, jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73294
Summary:
In order to do this `FUZZER_SUPPORTED_OS` had to be pulled out of
`lib/fuzzer/CMakeLists.txt` and into the main config so we can use it
from the `test/fuzzer/CMakeList.txt`. `FUZZER_SUPPORTED_OS` currently
has the same value of `SANITIZER_COMMON_SUPPORTED_OS` which preserves
the existing behaviour but this allows us in the future to adjust the
supported platforms independent of `SANITIZER_COMMON_SUPPORTED_OS`. This
mirrors the other sanitizers.
For non-Apple platforms `FUZZER_SUPPORTED_OS` is not defined and
surprisingly this was the behaviour before this patch because
`SANITIZER_COMMON_SUPPORTED_OS` was actually empty. This appears to
not matter right now because the functions that take an `OS` as an
argument seem to ignore it on non-Apple platforms.
While this change tries to be NFC it is technically not because we
now generate an iossim config whereas previously we didn't. This seems
like the right thing to do because the build system was configured to
compile LibFuzzer for iossim but previously we weren't generating a lit
test config for it. The device/simulator testing configs don't run by
default anyway so this shouldn't break testing.
This change relies on the get_capitalized_apple_platform() function
added in a previous commit.
rdar://problem/58798733
Reviewers: kubamracek, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73243
Summary:
The previous code hard-coded platform names but compiler-rt's CMake
build system actually already knows which Apple platforms TSan supports.
This change uses this information to enumerate the different Apple
platforms.
This change relies on the `get_capitalized_apple_platform()` function
added in a previous commit.
rdar://problem/58798733
Reviewers: kubamracek, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73238
Summary:
The previous code hard-coded platform names but compiler-rt's CMake
build system actually already knows which Apple platforms ASan supports.
This change uses this information to enumerate the different Apple
platforms.
rdar://problem/58798733
Reviewers: kubamracek, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73232
Summary:
there is an ongoing work on interchangeable custom mutators
(https://github.com/google/clusterfuzz/pull/1333/files#r367706283)
and having some sort of signalling from libFuzzer that it has loaded
a custom mutator would be helpful.
The initial idea was to make the mutator to print something, but given
the anticipated variety of different mutators, it does not seem possible
to make all of them print the same message to signal their execution.
Reviewers: kcc, metzman
Reviewed By: metzman
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73136
Summary:
Unity is making irresponsible assumptions as to how clumped up memory
should be. With larger regions, we break those, resulting in errors
like:
"Using memoryadresses from more that 16GB of memory"
This is unfortunately one of those situations where we have to bend to
existing code because we doubt it's going to change any time soon.
128MB should be enough, but we could be flirting with OOMs in the
higher class sizes.
Reviewers: cferris, eugenis, hctim, morehouse, pcc
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73143
Fix clear_cache_test to work on NetBSD with PaX MPROTECT enabled, that
is when creating W+X mmaps is prohibited. Use the recommended solution:
create two mappings for the same memory area, make one of them RW, while
the other RX. Copy the function into the RW area but run it from the RX
area.
In order to implement this, I've split the pointer variables to
'write_buffer' and 'execution_buffer'. Both are separate pointers
on NetBSD, while they have the same value on other systems.
I've also split the memcpy_f() into two: new memcpy_f() that only takes
care of copying memory and discards the (known) result of memcpy(),
and realign_f() that applies ARM realignment to the given pointer.
Again, there should be no difference on non-NetBSD systems but on NetBSD
copying is done on write_buffer, while realignment on pointer
to the execution_buffer.
I have tested this change on NetBSD and Linux.
Differential Revision: https://reviews.llvm.org/D72578
Summary:
In some configuration, `sched_getaffinity` can fail. Some reasons for
that being the lack of `CAP_SYS_NICE` capability or some syscall
filtering and so on.
This should not be fatal to the allocator, so in this situation, we
will fallback to the `MaxTSDCount` value specified in the allocator
configuration.
Reviewers: cferris, eugenis, hctim, morehouse, pcc
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73055
This is an alternative to the continous mode that was implemented in
D68351. This mode relies on padding and the ability to mmap a file over
the existing mapping which is generally only available on POSIX systems
and isn't suitable for other platforms.
This change instead introduces the ability to relocate counters at
runtime using a level of indirection. On every counter access, we add a
bias to the counter address. This bias is stored in a symbol that's
provided by the profile runtime and is initially set to zero, meaning no
relocation. The runtime can mmap the profile into memory at abitrary
location, and set bias to the offset between the original and the new
counter location, at which point every subsequent counter access will be
to the new location, which allows updating profile directly akin to the
continous mode.
The advantage of this implementation is that doesn't require any special
OS support. The disadvantage is the extra overhead due to additional
instructions required for each counter access (overhead both in terms of
binary size and performance) plus duplication of counters (i.e. one copy
in the binary itself and another copy that's mmapped).
Differential Revision: https://reviews.llvm.org/D69740
Summary:
This commit modifies the way `ExecuteCommand` works in fuchsia by adding
special logic to handle `/dev/null`.
The FuzzerCommand interface does not have a way to "discard" the output,
so other parts of the code just set the output file to `getDevNull()`.
The problem is that fuchsia does not have a named file that is
equivalent to `/dev/null`, so opening that file just fails.
This commit detects whether the specified output file is `getDevNull`,
and if that's the case, it will not copy the file descriptor for stdout
in the spawned process.
NOTE that modifying `FuzzerCommand` to add a "discardOutput" function
involves a significant refactor of all the other platforms, as they all
rely on the `toString()` method of `FuzzerCommand`.
This allows libfuzzer in fuchsia to run with `fork=1`, as the merge
process (`FuzzerMerge.cpp`) invoked `ExecuteCommand` with `/dev/null` as the
output.
Reviewers: aarongreen, phosek
Reviewed By: aarongreen
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D72894
When the hardware and operating system support the ARM Memory Tagging
Extension, tag primary allocation granules with a random tag. The granules
either side of the allocation are tagged with tag 0, which is normally
excluded from the set of tags that may be selected randomly. Memory is
also retagged with a random tag when it is freed, and we opportunistically
reuse the new tag when the block is reused to reduce overhead. This causes
linear buffer overflows to be caught deterministically and non-linear buffer
overflows and use-after-free to be caught probabilistically.
This feature is currently only enabled for the Android allocator
and depends on an experimental Linux kernel branch available here:
https://github.com/pcc/linux/tree/android-experimental-mte
All code that depends on the kernel branch is hidden behind a macro,
ANDROID_EXPERIMENTAL_MTE. This is the same macro that is used by the Android
platform and may only be defined in non-production configurations. When the
userspace interface is finalized the code will be updated to use the stable
interface and all #ifdef ANDROID_EXPERIMENTAL_MTE will be removed.
Differential Revision: https://reviews.llvm.org/D70762
Summary:
The IBM XL compiler uses `-qfuncsect` for `-ffunction-sections`.
The comment about sanitizers and `-f[no-]function-sections` is corrected
also, as it is pertinent to this patch.
The sanitizer-related use of `-fno-function-sections` is associated with
powerpc64le, a target for which there is an IBM XL compiler, so that use
is updated in this patch to apply `-qnofuncsect` in case a build using
the XL compiler is viable on that platform.
This patch has been verified with the XL compiler on AIX only.
Reviewers: daltenty, stevewan
Reviewed By: daltenty
Subscribers: mgorny, steven.zhang, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D72335
Summary:
fork() wasn't well (or at all) supported in Scudo. This materialized
in deadlocks in children.
In order to properly support fork, we will lock the allocator pre-fork
and unlock it post-fork in parent and child. This is done via a
`pthread_atfork` call installing the necessary handlers.
A couple of things suck here: this function allocates - so this has to
be done post initialization as our init path is not reentrance, and it
doesn't allow for an extra pointer - so we can't pass the allocator we
are currently working with.
In order to work around this, I added a post-init template parameter
that gets executed once the allocator is initialized for the current
thread. Its job for the C wrappers is to install the atfork handlers.
I reorganized a bit the impacted area and added some tests, courtesy
of cferris@ that were deadlocking prior to this fix.
Subscribers: jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D72470
The executable acquires an advisory record lock (`fcntl(fd, F_SETLKW, *)`) on a profile file.
Merge pool size >= 10 may be beneficial when the concurrency is large.
Also fix a small problem about snprintf. It can cause the filename to be truncated after %m.
Reviewed By: davidxl
Differential Revision: https://reviews.llvm.org/D71970
__sanitizer_stat_init is called for the executable first, then the
shared object. In WriterModuleReport(), the information for the shared
object will be recorded first. It'd be nice to get rid of the order
requirement of static constructors. (This should make .ctors platforms
work.)
Summary:
Qsort interceptor suppresses all checks by unpoisoning the data in the
wrapper of a comparator function, and then unpoisoning the output array
as well.
This change adds an explicit run of the comparator on all elements of
the input array to catch any sanitizer bugs.
Reviewers: vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71780
Rather than handling zlib handling manually, use `find_package` from CMake
to find zlib properly. Use this to normalize the `LLVM_ENABLE_ZLIB`,
`HAVE_ZLIB`, `HAVE_ZLIB_H`. Furthermore, require zlib if `LLVM_ENABLE_ZLIB` is
set to `YES`, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.
This restores 68a235d07f,
e6c7ed6d21. The problem with the windows
bot is a need for clearing the cache.
This reverts commit 68a235d07f.
This commit broke the clang-x64-windows-msvc build bot and a follow-up
commit did not fix it. Reverting to fix the bot.
Rather than handling zlib handling manually, use `find_package` from CMake
to find zlib properly. Use this to normalize the `LLVM_ENABLE_ZLIB`,
`HAVE_ZLIB`, `HAVE_ZLIB_H`. Furthermore, require zlib if `LLVM_ENABLE_ZLIB` is
set to `YES`, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.
Pass -static so that clang will not pass -Wl,--dynamic-linker,... to the
linker. The test is not expected to run under a ld.so. (Technically it
works under a ld.so but glibc expects to see a PT_DYNAMIC. lld
intentionally does not follow GNU ld's complex rules regarding
PT_DYNAMIC.)
This allows commit 1417558e4a to be
relanded.
This reverts commit 7a9ebe9512, and
dependent commit 54c5224203, which
disables qsort interception for some iOS platforms.
After this change, the -Nolibc sanitizer common test binary crashes on
startup on my regular Linux workstation, as well as on our bots:
https://ci.chromium.org/p/chromium/builders/try/linux_upload_clang/740
********************
Failing Tests (1):
SanitizerCommon-Unit ::
./Sanitizer-x86_64-Test/SanitizerCommon.NolibcMain
Loading it up in gdb shows that it crashes during relocation processing,
which suggests that some glibc loader versions do not support the
THREADLOCAL data added in this interceptor.
Summary:
Add support for NetBSD 9.0 and newer versions of interceptors
operating on struct statvfs: fstatvfs, fstatvfs1, getmntinfo,
getvfsstat, statvfs, statvfs1.
The default promoted interceptors are for NetBSD 9.99.26. Older
ones (currently 9.0) are kept in a new NetBSD specific file:
/sanitizer_common_interceptors_netbsd_compat.inc. This file
defines compat interceptors and mangles `INIT_*` macros,
concatenating the current interceptors and the compat ones.
This redefinition is not elegant, but it avoids preprocessor madness.
Define struct_statvfs90_sz for the compat purposes.
Reviewers: mgorny, kcc, vitalybuka, joerg
Reviewed By: mgorny
Subscribers: dberris, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71700
as it's failing the netbsd specific linter parts of the sanitizer linter:
llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors_netbsd_compat.inc:23: Lines should be <= 80 characters long [whitespace/line_length]
llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_platform_limits_netbsd.cpp:2450: Do not use variable-length arrays. Use an appropriately named ('k' followed by CamelCase) compile-time constant for the size.
This reverts commit 78f714f824.
Summary:
Add support for NetBSD 9.0 and newer versions of interceptors
operating on struct statvfs: fstatvfs, fstatvfs1, getmntinfo,
getvfsstat, statvfs, statvfs1.
The default promoted interceptors are for NetBSD 9.99.26. Older
ones (currently 9.0) are kept in a new NetBSD specific file:
/sanitizer_common_interceptors_netbsd_compat.inc. This file
defines compat interceptors and mangles `INIT_*` macros,
concatenating the current interceptors and the compat ones.
This redefinition is not elegant, but it avoids preprocessor madness.
Define struct_statvfs90_sz for the compat purposes.
Reviewers: mgorny, kcc, vitalybuka, joerg
Reviewed By: mgorny
Subscribers: dberris, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71700
This change breaks LLVM bootstrap with ASan and MSan.
FAILED: lib/ToolDrivers/llvm-lib/Options.inc
OptParser.td:137:1: error: Option is equivalent to
def INPUT : Option<[], "<input>", KIND_INPUT>;
^
OptParser.td:137:1: error: Other defined here
def INPUT : Option<[], "<input>", KIND_INPUT>;
This reverts commit caa48a6b88.
Building the sanitizers for watchOS currently fails with
sanitizer_common_interceptors.inc:9656:8: error: thread-local storage is not supported for the current target
static THREADLOCAL SIZE_T qsort_size;
I've also speculatively disabled QSORT interception for tvOS to unblock
failing builds. I'll ask someone with more sanitizer knowledge to check
after the holidays.
Summary:
Qsort interceptor suppresses all checks by unpoisoning the data in the
wrapper of a comparator function, and then unpoisoning the output array
as well.
This change adds an explicit run of the comparator on all elements of
the input array to catch any sanitizer bugs.
Reviewers: vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71780
Summary:
This fixes qsort-related false positives with glibc-2.27.
I'm not entirely sure why they did not show up with the earlier
versions; the code seems similar enough.
Reviewers: vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71740
Recent versions of the iOS simulator require that a "simulator device"
is booted before we can use `simctl spawn` (see iossim_run.py) to start
processes.
We can use `simctl bootstatus` to ensure that the simulator device
is booted before we run any tests via lit. The `-b` option starts the
device if necessary.
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D71449
Summary:
Some Android builds that we are interested in define `__BIONIC__`
but not `__ANDROID__`, so expand `SCUDO_ANDROID` to encompass those.
Reviewers: cferris, hctim, pcc, eugenis, morehouse
Subscribers: krytarowski, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71772
Summary:
Qsort interceptor suppresses all checks by unpoisoning the data in the
wrapper of a comparator function, and then unpoisoning the output array
as well.
This change adds an explicit run of the comparator on all elements of
the input array to catch any sanitizer bugs.
Reviewers: vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71780
Summary:
This fixes qsort-related false positives with glibc-2.27.
I'm not entirely sure why they did not show up with the earlier
versions; the code seems similar enough.
Reviewers: vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71740
Introduce a new %run_nomprotect substitution to run tests that do not
work with MPROTECT enabled. This uses paxctl via a wrapper on NetBSD,
and evaluates to plain %run on other systems.
Differential Revision: https://reviews.llvm.org/D71513
Summary:
In order to implement `malloc_{enable|disable}` we were just disabling
(or really locking) the Primary and the Secondary. That meant that
allocations could still be serviced from the TSD as long as the cache
wouldn't have to be filled from the Primary.
This wasn't working out for Android tests, so this change implements
registry disabling (eg: locking) so that `getTSDAndLock` doesn't
return a TSD if the allocator is disabled. This also means that the
Primary doesn't have to be disabled in this situation.
For the Shared Registry, we loop through all the TSDs and lock them.
For the Exclusive Registry, we add a `Disabled` boolean to the Registry
that forces `getTSDAndLock` to use the Fallback TSD instead of the
thread local one. Disabling the Registry is then done by locking the
Fallback TSD and setting the boolean in question (I don't think this
needed an atomic variable but I might be wrong).
I clang-formatted the whole thing as usual hence the couple of extra
whiteline changes in this CL.
Reviewers: cferris, pcc, hctim, morehouse, eugenis
Subscribers: jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71719
Add a missing %run substitution to fread_fwrite test. This fixes
the test on NetBSD where %run disables ASLR as necessary for MSAN
to function.
Differential Revision: https://reviews.llvm.org/D71623
Summary: Also add an error case when targetting an unimplement architecture.
Subscribers: nemanjai, jsji, shchenz, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D71670
Summary:
Remove REQUIRES-ANY alias lit directive since it is hardly used and can
be easily implemented using an OR expression using REQUIRES. Fixup
remaining testcases still using REQUIRES-ANY.
Reviewers: probinson, jdenny, gparker42
Reviewed By: gparker42
Subscribers: eugenis, asb, rbar, johnrusso, simoncook, sabuasal, niosHD, delcypher, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, lenary, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, cfe-commits, #sanitizers, llvm-commits
Tags: #llvm, #clang, #sanitizers
Differential Revision: https://reviews.llvm.org/D71408
Dan Liew