Summary:
For now just insert the callback for stores, similar to how MSan tracks
origins. In the future we may want to add callbacks for loads, memcpy,
function calls, CMPs, etc.
Reviewers: pcc, vitalybuka, kcc, eugenis
Reviewed By: vitalybuka, kcc, eugenis
Subscribers: eugenis, hiraditya, #sanitizers, llvm-commits, kcc
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75312
Summary:
Sanitizer tests don't entirely pass on an R device. Fix up all the
incompatibilities with the new system.
Reviewers: eugenis, pcc
Reviewed By: eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75303
Summary:
This patch includes several changes to reduce the overall footprint
of the allocator:
- for realloc'd chunks: only keep the same chunk when lowering the size
if the delta is within a page worth of bytes;
- when draining a cache: drain the beginning, not the end; we add pointers
at the end, so that meant we were draining the most recently added
pointers;
- change the release code to account for an freed up last page: when
scanning the pages, we were looking for pages fully covered by blocks;
in the event of the last page, if it's only partially covered, we
wouldn't mark it as releasable - even what follows the last chunk is
all 0s. So now mark the rest of the page as releasable, and adapt the
test;
- add a missing `setReleaseToOsIntervalMs` to the cacheless secondary;
- adjust the Android classes based on more captures thanks to pcc@'s
tool.
Reviewers: pcc, cferris, hctim, eugenis
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D75142
Fails with the following message in the error case:
```
CMake Error at /path/to/llvm-project/compiler-rt/lib/tsan/CMakeLists.txt:119 (message):
Building the TSan runtime requires at least macOS SDK 10.12
```
Fixes#44682.
https://bugs.llvm.org/show_bug.cgi?id=44682
Reviewed By: dmajor, delcypher
Differential Revision: https://reviews.llvm.org/D74501
Summary:
For builtin compiler-rt, it is built with visibility hidden by default
to avoid the client exporting symbols from libclang static library. The
compiler option -fvisibility=hidden doesn't work on the aliases in c files
because they are created with inline assembly. On Darwin platform,
thoses aliases are exported by default if they are reference by the client.
Fix the issue by adding ".private_extern" to all the aliases if the
library is built with visibility hidden.
rdar://problem/58960296
Reviewers: dexonsmith, arphaman, delcypher, kledzik
Reviewed By: delcypher
Subscribers: dberris, jkorous, ributzka, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73577
See discussion on PR44792.
This reverts commit 02ce9d8ef5.
It also reverts the follow-up commits
8f46269f0 "[profile] Don't dump counters when forking and don't reset when calling exec** functions"
62c7d8402 "[profile] gcov_mutex must be static"
Generally we ignore interceptors coming from called_from_lib-suppressed libraries.
However, we must not ignore critical interceptors like e.g. pthread_create,
otherwise runtime will lost track of threads.
pthread_detach is one of these interceptors we should not ignore as it affects
thread states and behavior of pthread_join which we don't ignore as well.
Currently we can produce very obscure false positives. For more context see:
https://groups.google.com/forum/#!topic/thread-sanitizer/ecH2P0QUqPs
The added test captures this pattern.
While we are here rename ThreadTid to ThreadConsumeTid to make it clear that
it's not just a "getter", it resets user_id to 0. This lead to confusion recently.
Reviewed in https://reviews.llvm.org/D74828
Summary:
There is no need to write out gcdas when forking because we can just reset the counters in the parent process.
Let say a counter is N before the fork, then fork and this counter is set to 0 in the child process.
In the parent process, the counter is incremented by P and in the child process it's incremented by C.
When dump is ran at exit, parent process will dump N+P for the given counter and the child process will dump 0+C, so when the gcdas are merged the resulting counter will be N+P+C.
About exec** functions, since the current process is replaced by an another one there is no need to reset the counters but just write out the gcdas since the counters are definitely lost.
To avoid to have lists in a bad state, we just lock them during the fork and the flush (if called explicitely) and lock them when an element is added.
Reviewers: marco-c
Reviewed By: marco-c
Subscribers: hiraditya, cfe-commits, #sanitizers, llvm-commits, sylvestre.ledru
Tags: #clang, #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74953
Like was done before in D67999 for `logbf`, this patch fixes the tests for
the internal compiler-rt implementations of `logb` and `logbl` to consider
all NaNs equivalent. Not doing so was resulting in test failures for
riscv64, since the the NaNs had different signs, but the spec doesn't
specify the NaN signedness or payload.
Fixes bug 44244.
Reviewers: rupprecht, delcypher
Reviewed By: rupprecht, delcypher
Differential Revision: https://reviews.llvm.org/D74826
Summary:
This substitution expands to the appropriate minimum deployment target
flag where thread local storage (TLS) was first introduced on Darwin
platforms. For all other platforms the substitution expands to an empty
string.
E.g. for macOS the substitution expands to `-mmacosx-version-min=10.12`
This patch adds support for the substitution (and future substitutions)
by doing a minor refactor and then uses the substitution in the relevant
TSan tests.
rdar://problem/59568956
Reviewers: yln, kubamracek, dvyukov, vitalybuka
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74802
Summary:
The number of "inputs have the Data Flow Trace" cannot be greater than
the number of inputs touching the focus function. The existing message is rather
confusing as the same log would mention a greater total number of traces a few
lines above.
Reviewers: kcc, metzman
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74779
The removed loop clears reused for entries at the tail of a SyncClock.
The loop is redundant since those entries were already cleared by the
immediately preceding loop, which iterates over all entries in the
SyncClock (including the tail entries).
Summary:
Most of our larger data is dynamically allocated (via `map`) but it
became an hindrance with regard to init time, for a cost to benefit
ratio that is not great. So change the `TSD`s, `RegionInfo`, `ByteMap`
to be static.
Additionally, for reclaiming, we used mapped & unmapped a buffer each
time, which is costly. It turns out that we can have a static buffer,
and that there isn't much contention on it.
One of the other things changed here, is that we hard set the number
of TSDs on Android to the maximum number, as there could be a
situation where cores are put to sleep and we could miss some.
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74696
This change makes the following lit substitutions expand to the correct
thing for macOS, iOS, tvOS, and watchOS.
%darwin_min_target_with_full_runtime_arc_support
%macos_min_target_10_11
rdar://problem/59463146
This patch defines `config.apple_platform_min_deployment_target_flag`
in the ASan, LibFuzzer, TSan, and UBSan lit test configs.
rdar://problem/59463146
platforms.
For the simulators this changes the following compilation flags (
used both for building the runtime and lit tests).
iOS simulator: `-miphoneos-version-min` -> `-mios-simulator-version-min`
watchOS simulator: `-mwatchos-version-min` -> `-mwatchos-simulator-version-min`
tvOS simulator: `-mtvos-version-min` -> `-mtvos-simulator-version-min`
rdar://problem/59463146
Summary:
Add a method to set the release to OS value as the system runs,
and allow this to be set differently in the primary and the secondary.
Also, add a default value to use for primary and secondary. This
allows Android to have a default that is different for
primary/secondary.
Update mallopt to support setting the release to OS value.
Reviewers: pcc, cryptoad
Reviewed By: cryptoad
Subscribers: cryptoad, jfb, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74448
Summary: It complains about reaching the end of a non-void returning function.
Reviewers: eugenis, hctim, morehouse
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74578
Summary:
Due to Unity, we had to reduce our region sizes, but in some rare
situations, some programs (mostly tests AFAICT) manage to fill up
a region for a given size class.
So this adds a workaround for that attempts to allocate the block
from the immediately larger size class, wasting some memory but
allowing the application to keep going.
Reviewers: pcc, eugenis, cferris, hctim, morehouse
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74567
Summary:
A number of testcases in TSAN are designed to deal with intermittent problems
not exist in all executions of the tested program. A script called deflake.bash
runs the executable up to 10 times to deal with the intermittent nature of the tests.
The purpose of this patch is to parameterize the hard-coded threshold above via
--cmake_variables=-DTSAN_TEST_DEFLAKE_THRESHOLD=SomeIntegerValue
When this cmake var is not set, the default value of 10 will be used.
Reviewer: dvyukov (Dmitry Vyukov), eugenis (Evgenii Stepanov), rnk (Reid Kleckner), hubert.reinterpretcast (Hubert Tong), vitalybuka (Vitaly Buka)
Reviewed By: vitalybuka (Vitaly Buka)
Subscribers: mgorny (Michal Gorny), jfb (JF Bastien), steven.zhang (qshanz), llvm-commits (Mailing List llvm-commits), Sanitizers
Tag: LLVM, Sanitizers
Differential Revision: https://reviews.llvm.org/D73707
The interceptor uses thread-local variables, which (until very recently)
are emu-tls. An access to such variable may call malloc which can
deadlock the runtime library.
For CleanseCrashInput, discards stdout output anyway since it is not used.
These changes are to defend against aggressive PID recycle on windows to reduce the chance of contention on files.
Using pipe instead of file also workaround the problem that when the
process is spawned by llvm-lit, the aborted process keeps a handle to the
output file such that the output file can not be removed. This will
cause random test failures.
https://devblogs.microsoft.com/oldnewthing/20110107-00/?p=11803
Reviewers: kcc, vitalybuka
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D73329
Add an optional table lookup after the existing logarithm computation
for MidSize < Size <= MaxSize during size -> class lookups. The lookup is
O(1) due to indexing a precomputed (via constexpr) table based on a size
table. Switch to this approach for the Android size class maps.
Other approaches considered:
- Binary search was found to have an unacceptable (~30%) performance cost.
- An approach using NEON instructions (see older version of D73824) was found
to be slightly slower than this approach on newer SoCs but significantly
slower on older ones.
By selecting the values in the size tables to minimize wastage (for example,
by passing the malloc_info output of a target program to the included
compute_size_class_config program), we can increase the density of allocations
at a small (~0.5% on bionic malloc_sql_trace as measured using an identity
table) performance cost.
Reduces RSS on specific Android processes as follows (KB):
Before After
zygote (median of 50 runs) 26836 26792 (-0.2%)
zygote64 (median of 50 runs) 30384 30076 (-1.0%)
dex2oat (median of 3 runs) 375792 372952 (-0.8%)
I also measured the amount of whole-system idle dirty heap on Android by
rebooting the system and then running the following script repeatedly until
the results were stable:
for i in $(seq 1 50); do grep -A5 scudo: /proc/*/smaps | grep Pss: | cut -d: -f2 | awk '{s+=$1} END {print s}' ; sleep 1; done
I did this 3 times both before and after this change and the results were:
Before: 365650, 356795, 372663
After: 344521, 356328, 342589
These results are noisy so it is hard to make a definite conclusion, but
there does appear to be a significant effect.
On other platforms, increase the sizes of all size classes by a fixed offset
equal to the size of the allocation header. This has also been found to improve
density, since it is likely for allocation sizes to be a power of 2, which
would otherwise waste space by pushing the allocation into the next size class.
Differential Revision: https://reviews.llvm.org/D73824
This lets us remove two pointer indirections (one by removing the pointer,
and another by making the AllocatorPtr declaration hidden) in the C++ wrappers.
Differential Revision: https://reviews.llvm.org/D74356
Summary:
Instead of hand-crafting an offset into the structure returned by
dlopen(3) to get at the link map, use the documented API. This is
described in dlinfo(3): by calling it with `RTLD_DI_LINKMAP`, the
dynamic linker ensures the right address is returned.
This is a recommit of 92e267a94d, with
dlinfo(3) expliclity being referenced only for FreeBSD, non-Android
Linux, NetBSD and Solaris. Other OSes will have to add their own
implementation.
Reviewers: devnexen, emaste, MaskRay, krytarowski
Reviewed By: krytarowski
Subscribers: krytarowski, vitalybuka, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73990
Summary:
The refactoring has caused a failure in
http://lab.llvm.org:8011/builders/clang-ppc64le-linux/builds/29265
The idea of failing the symbolization when the symbolizer bufer is too small
was incorrect. The symbolizer can be invoked for other frames that may fit into
the buffer and get symbolized.
Reviewers: vitalybuka, eugenis
Subscribers: dberris, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D74343
Summary:
Instead of hand-crafting an offset into the structure returned by
dlopen(3) to get at the link map, use the documented API. This is
described in dlinfo(3): by calling it with `RTLD_DI_LINKMAP`, the
dynamic linker ensures the right address is returned.
Reviewers: devnexen, emaste, MaskRay, krytarowski
Reviewed By: krytarowski
Subscribers: krytarowski, vitalybuka, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D73990
Matt Morehouse