The Solaris/amd64 buildbot
<https://lab.llvm.org/staging/#/builders/101/builds/2845> has recently been
broken several times, at least one of those remains unfixed:
[63/446] Generating Sanitizer-x86_64-Test
[...]
Undefined first referenced
symbol in file
_ZN11__sanitizer15internal_usleepEy /opt/llvm-buildbot/home/solaris11-amd64/clang-solaris11-amd64/stage1/projects/compiler-rt/lib/sanitizer_common/tests/libRTSanitizerCommon.test.x86_64.a(sanitizer_common.cpp.o)
ld: fatal: symbol referencing errors
Thist patch fixes it by defining the missing `internal_usleep`.
Tested on `amd64-pc-solaris2.11.`
Differential Revision: https://reviews.llvm.org/D105878
Currently we allocate MemoryMapper per size class.
MemoryMapper mmap's and munmap's internal buffer.
This results in 50 mmap/munmap calls under the global
allocator mutex. Reuse MemoryMapper and the buffer
for all size classes. This radically reduces number of
mmap/munmap calls. Smaller size classes tend to have
more objects allocated, so it's highly likely that
the buffer allocated for the first size class will
be enough for all subsequent size classes.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105778
This moves logic for setting kAliasRegionStart into hwasan_allocator.cpp
so other platforms that do not support aliasing mode will not need to define
kAliasRegionStart.
Differential Revision: https://reviews.llvm.org/D105725
It turns out that COMPILER_RT_TEST_COMPILER_CFLAGS is actually a string
that is being appended to and not a list.
Therefore, append the thread-safety flags to the string. Because CMake
separates list elements by ';' when turning into a string, also
substitute ';' with ' '.
Reviewed By: hctim
Differential Revision: https://reviews.llvm.org/D105829
This reverts commit 0726695214.
This causes the following build failure with gcc 10.3.0:
/home/nikic/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_allocator_primary64.h:114:31: error: declaration of ‘typedef class __sanitizer::MemoryMapper<__sanitizer::SizeClassAllocator64<Params> > __sanitizer::SizeClassAllocator64<Params>::MemoryMapper’ changes meaning of ‘MemoryMapper’ [-fpermissive]
114 | typedef MemoryMapper<ThisT> MemoryMapper;
Currently we allocate MemoryMapper per size class.
MemoryMapper mmap's and munmap's internal buffer.
This results in 50 mmap/munmap calls under the global
allocator mutex. Reuse MemoryMapper and the buffer
for all size classes. This radically reduces number of
mmap/munmap calls. Smaller size classes tend to have
more objects allocated, so it's highly likely that
the buffer allocated for the first size class will
be enough for all subsequent size classes.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105778
Another follow-up to 0da172b176
compiler-rt/lib/scudo/scudo_tsd_shared.cpp:103:1: error: mutex 'CandidateTSD->Mutex' is not held on every path through here [-Werror,-Wthread-safety-analysis]
}
^
compiler-rt/lib/scudo/scudo_tsd_shared.cpp:95:21: note: mutex acquired here
CandidateTSD->lock();
^
compiler-rt/lib/scudo/scudo_tsd_shared.cpp:103:1: error: mutex 'TSD->Mutex' is not held on every path through here [-Werror,-Wthread-safety-analysis]
}
^
compiler-rt/lib/scudo/scudo_tsd_shared.cpp:101:8: note: mutex acquired here
TSD->lock();
^
compiler-rt/lib/scudo/scudo_tsd_shared.cpp:103:1: error: mutex 'TSDs[Index].Mutex' is not held on every path through here [-Werror,-Wthread-safety-analysis]
}
^
compiler-rt/lib/scudo/scudo_tsd_shared.cpp:80:23: note: mutex acquired here
if (TSDs[Index].tryLock()) {
^
Looks like an oversight in 0da172b176
compiler-rt/lib/scudo/scudo_tsd_shared.inc:53:1: error: mutex 'TSD->Mutex' is not held on every path through here [-Werror,-Wthread-safety-analysis]
}
^
compiler-rt/lib/scudo/scudo_tsd_shared.inc:49:12: note: mutex acquired here
if (TSD->tryLock())
^
__m128i is vector SSE type used in tsan.
It's handy to be able to print it for debugging.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D102167
The memcpy interceptor is the only one that uses COMMON_INTERCEPTOR_ENTER
more than once in a single function. This does not allow COMMON_INTERCEPTOR_ENTER
to use labels, because they are global for the whole function (not block scoped).
Don't include COMMON_INTERCEPTOR_ENTER code twice.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105774
sem_trywait never blocks.
Use REAL instead of COMMON_INTERCEPTOR_BLOCK_REAL.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105775
The internal allocator adds 8-byte header for debugging purposes.
The problem with it is that it's not possible to allocate nicely-sized
objects without a significant overhead. For example, if we allocate
512-byte objects, that will be rounded up to 768 or something.
This logic migrated from tsan where it was added during initial development,
I don't remember that it ever caught anything (we don't do bugs!).
Remove it so that it's possible to allocate nicely-sized objects
without overheads.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105777
We use 0 for empty stack id from stack depot.
Deadlock detector 1 is the only place that uses -1
as a special case. Use 0 because there is a number
of checks of the form "if (stack id) ...".
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105776
Enable clang Thread Safety Analysis for sanitizers:
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
Thread Safety Analysis can detect inconsistent locking,
deadlocks and data races. Without GUARDED_BY annotations
it has limited value. But this does all the heavy lifting
to enable analysis and allows to add GUARDED_BY incrementally.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105716
We have 3 different mutexes (RWMutex, BlockingMutex __tsan::Mutex),
each with own set of downsides. I want to unify them under a name Mutex.
But it will conflict with Mutex in the deadlock detector,
which is a way too generic name. Rename it to MutexState.
Reviewed By: melver
Differential Revision: https://reviews.llvm.org/D105773
Currently ThreadRegistry is overcomplicated because of tsan,
it needs tid quarantine and reuse counters. Other sanitizers
don't need that. It also seems that no other sanitizer now
needs max number of threads. Asan used to need 2^24 limit,
but it does not seem to be needed now. Other sanitizers blindly
copy-pasted that without reasons. Lsan also uses quarantine,
but I don't see why that may be potentially needed.
Add a ThreadRegistry ctor that does not require any sizes
and use it in all sanitizers except for tsan.
In preparation for new tsan runtime, which won't need
any of these parameters as well.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D105713
We have SleepForSeconds, SleepForMillis and internal_sleep.
Some are implemented in terms of libc functions, some -- in terms
of syscalls. Some are implemented in per OS files,
some -- in libc/nolibc files. That's unnecessary complex
and libc functions cause crashes in some contexts because
we intercept them. There is no single reason to have calls to libc
when we have syscalls (and we have them anyway).
Add internal_usleep that is implemented in terms of syscalls per OS.
Make SleepForSeconds/SleepForMillis/internal_sleep a wrapper
around internal_usleep that is implemented in sanitizer_common.cpp once.
Also remove return values for internal_sleep, it's not used anywhere.
Eventually it would be nice to remove SleepForSeconds/SleepForMillis/internal_sleep.
There is no point in having that many different names for the same thing.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D105718
LibIgnore is checked in every interceptor.
Currently it has all logic in the single function
in the header, which makes it uninlinable.
Split it into fast path (no libraries ignored)
and slow path (have ignored libraries).
It makes the fast path inlinable (single load).
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D105719
This reverts commit 9a9bc76c0e.
That commit broke "ninja install" when building compiler-rt for mingw
targets, building standalone (pointing cmake at the compiler-rt
directory) with cmake 3.16.3 (the one shipped in ubuntu 20.04), with
errors like this:
-- Install configuration: "Release"
CMake Error at cmake_install.cmake:44 (file):
file cannot create directory: /include/sanitizer. Maybe need
administrative privileges.
Call Stack (most recent call first):
/home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/cmake_install.cmake:37 (include)
FAILED: include/CMakeFiles/install-compiler-rt-headers
cd /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/include && /usr/bin/cmake -DCMAKE_INSTALL_COMPONENT="compiler-rt-headers" -P /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/cmake_install.cmake
ninja: build stopped: subcommand failed.
This makes sure we have support for MTE instructions.
Later the check can be extended to support MTE on other compilers.
Reviewed By: pcc
Differential Revision: https://reviews.llvm.org/D105722
Instead of using `COMPILER_RT_INSTALL_PATH` through the CMake for
complier-rt, just use it to define variables for the subdirs which
themselves are used.
This preserves compatibility, but later on we might consider getting rid
of `COMPILER_RT_INSTALL_PATH` and just changing the defaults for the
subdir variables directly.
---
There was a seaming bug where the (non-Apple) per-target libdir was
`${target}` not `lib/${target}`. I suspect that has to do with the docs
on `COMPILER_RT_INSTALL_PATH` saying was the library dir when that's no
longer true, so I just went ahead and fixed it, allowing me to define
fewer and more sensible variables.
That last part should be the only behavior changes; everything else
should be a pure refactoring.
---
D99484 is the main thrust of the `GnuInstallDirs` work. Once this lands,
it should be feasible to follow both of these up with a simple patch for
compiler-rt analogous to the one for libcxx.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D101497
The existing one actually failed on the int* p, not on int z (as can be
seen by the fault being 8 bytes rather than 4).
This is also needed to make sure the stack safety analysis does not
classify the alloca as safe.
Reviewed By: hctim
Differential Revision: https://reviews.llvm.org/D105705
Based on comments in D91466, we can make the current implementation
linux-speciic. The fuchsia implementation will just be a call to
`__sanitizer_fill_shadow`.
Differential Revision: https://reviews.llvm.org/D105663
The new name is something less linux-y and more platform generic. Once we
finalize the tagged pointer ABI in zircon, we will do the appropriate check
for fuchsia to see that TBI is enabled.
Differential Revision: https://reviews.llvm.org/D105667
This reverts commit 52aeacfbf5.
There isn't full agreement on a path forward yet, but there is agreement that
this shouldn't land as-is. See discussion on https://reviews.llvm.org/D105338
Also reverts unreviewed "[clang] Improve `-Wnull-dereference` diag to be more in-line with reality"
This reverts commit f4877c78c0.
And all the related changes to tests:
This reverts commit 9a0152799f.
This reverts commit 3f7c9cc274.
This reverts commit 329f8197ef.
This reverts commit aa9f58cc2c.
This reverts commit 2df37d5ddd.
This reverts commit a72a441812.
Store to null is deleted, so the test no longer did what it was expecting to do.
Conceal that by creating null pointer in a more elaborate way,
thus retaining original test coverage.
trapOnAddress is designed to SEGV on a specific address. Unfortunately,
with an IR change, __builtin_unreachable() ends up doing DCE on things
that have side effects, like the load that causes the trap.
Change to __builtin_trap() to avoid the optimisation.
Root cause is still an LLVM bug, and tracked in
https://bugs.llvm.org/show_bug.cgi?id=47480.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D105654
This contains all the definitions required by hwasan for the fuchsia
implementation and can be landed independently from the remaining parts of D91466.
Differential Revision: https://reviews.llvm.org/D103936
This disables use of hwasan interceptors which we do not use on Fuchsia. This
explicitly sets the macro for defining the hwasan versions of new/delete.
Differential Revision: https://reviews.llvm.org/D103544
This patch splits up hwasan thread creation between `__sanitizer_before_thread_create_hook`,
`__sanitizer_thread_create_hook`, and `__sanitizer_thread_start_hook`.
The linux implementation creates the hwasan thread object inside the
new thread. On Fuchsia, we know the stack bounds before thread creation,
so we can initialize part of the thread object in `__sanitizer_before_thread_create_hook`,
then initialize the stack ring buffer in `__sanitizer_thread_start_hook`
once we enter the thread.
Differential Revision: https://reviews.llvm.org/D104085
Update the asan_symbolize_script for changes in argparse output
in Python 3.10. The parser output 'options' instead of 'optional
arguments'.
Differential Revision: https://reviews.llvm.org/D105489
The __llvm_prf_names section uses SHF_GNU_RETAIN. However, GNU ld before 2015-10
(https://sourceware.org/bugzilla/show_bug.cgi?id=19161) neither supports it nor
retains __llvm_prf_names according to __start___llvm_prf_names. So --gc-sections
does not work on such old GNU ld.
This is not a problem for gold and sufficiently new lld.
We would find an address with matching tag, only to discover in
ShowCandidate that it's very far away from [stack].
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D105197
If the fault address is at the boundary of memory regions, this could
cause us to segfault otherwise.
Ran test with old compiler_rt to make sure it fails.
Reviewed By: eugenis
Differential Revision: https://reviews.llvm.org/D105032
Compiling compiler-rt with Clang modules and libc++ revealed that the
global `operator new` is being called without including `<new>`.
Differential Revision: https://reviews.llvm.org/D105401
This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.
Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:
* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
* UseCmp
* UseMemmem
* SetCustomMutator
* SetCustomCrossOver
* LateInitialize (similar to the MutationDispatcher's original constructor)
* Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
* StartMutationSequence
* MutationSequence
* DictionaryEntrySequence
* RecommendDictionary
* RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)
A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D102447
Rainer Orth