It's not undefined behavior for an unsigned left shift to overflow (i.e. to
shift bits out), but it has been the source of bugs and exploits in certain
codebases in the past. As we do in other parts of UBSan, this patch adds a
dynamic checker which acts beyond UBSan and checks other sources of errors. The
option is enabled as part of -fsanitize=integer.
The flag is named: -fsanitize=unsigned-shift-base
This matches shift-base and shift-exponent flags.
<rdar://problem/46129047>
Differential Revision: https://reviews.llvm.org/D86000
Add functions exposed via the MSAN interface to enable MSAN within
binaries that perform manual stack switching (e.g. through using fibers
or coroutines).
This functionality is analogous to the fiber APIs available for ASAN and TSAN.
Fixesgoogle/sanitizers#1232
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D86471
The introduction of find_library for ncurses caused more issues than it solved problems. The current open issue is it makes the static build of LLVM fail. It is better to revert for now, and get back to it later.
Revert "[CMake] Fix an issue where get_system_libname creates an empty regex capture on windows"
This reverts commit 1ed1e16ab8.
Revert "Fix msan build"
This reverts commit 34fe9613dd.
Revert "[CMake] Always mark terminfo as unavailable on Windows"
This reverts commit 76bf26236f.
Revert "[CMake] Fix OCaml build failure because of absolute path in system libs"
This reverts commit 8e4acb82f7.
Revert "[CMake] Don't look for terminfo libs when LLVM_ENABLE_TERMINFO=OFF"
This reverts commit 495f91fd33.
Revert "Use find_library for ncurses"
This reverts commit a52173a3e5.
Differential revision: https://reviews.llvm.org/D86521
The CrossOver mutator is meant to cross over two given buffers (referred to as
the first/second buffer henceforth). Previously InsertPartOf/CopyPartOf calls
used in the CrossOver mutator incorrectly inserted/copied part of the second
buffer into a "scratch buffer" (MutateInPlaceHere of the size
CurrentMaxMutationLen), rather than the first buffer. This is not intended
behavior, because the scratch buffer does not always (i) contain the content of
the first buffer, and (ii) have the same size as the first buffer;
CurrentMaxMutationLen is typically a lot larger than the size of the first
buffer. This patch fixes the issue by using the first buffer instead of the
scratch buffer in InsertPartOf/CopyPartOf calls.
A FuzzBench experiment was run to make sure that this change does not
inadvertently degrade the performance. The performance is largely the same; more
details can be found at:
https://storage.googleapis.com/fuzzer-test-suite-public/fixcrossover-report/index.html
This patch also adds two new tests, namely "cross_over_insert" and
"cross_over_copy", which specifically target InsertPartOf and CopyPartOf,
respectively.
- cross_over_insert.test checks if the fuzzer can use InsertPartOf to trigger
the crash.
- cross_over_copy.test checks if the fuzzer can use CopyPartOf to trigger the
crash.
These newly added tests were designed to pass with the current patch, but not
without the it (with 790878f291 these tests do not
pass). To achieve this, -max_len was intentionally given a high value. Without
this patch, InsertPartOf/CopyPartOf will generate larger inputs, possibly with
unpredictable data in it, thereby failing to trigger the crash.
The test pass condition for these new tests is narrowed down by (i) limiting
mutation depth to 1 (i.e., a single CrossOver mutation should be able to trigger
the crash) and (ii) checking whether the mutation sequence of "CrossOver-" leads
to the crash.
Also note that these newly added tests and an existing test (cross_over.test)
all use "-reduce_inputs=0" flags to prevent reducing inputs; it's easier to
force the fuzzer to keep original input string this way than tweaking
cov-instrumented basic blocks in the source code of the fuzzer executable.
Differential Revision: https://reviews.llvm.org/D85554
value-profile-load.test needs adjustment with a mutator change in
bb54bcf849, which reverted as of now, but will be
recommitted after landing this patch.
This patch makes value-profile-load.test more friendly to (and aware of) the
current value profiling strategy, which is based on the hamming as well as the
absolute distance. To this end, this patch adjusts the set of input values that
trigger an expected crash. More specifically, this patch now uses a single value
0x01effffe as a crashing input, because this value is close to values like
{0x1ffffff, 0xffffff, ...}, which are very likely to be added to the corpus per
the current hamming- and absolute-distance-based value profiling strategy. Note
that previously the crashing input values were {1234567 * {1, 2, ...}, s.t. <
INT_MAX}.
Every byte in the chosen value 0x01effeef is intentionally different; this was
to make it harder to find the value without the intermediate inputs added to the
corpus by the value profiling strategy.
Also note that LoadTest.cpp now uses a narrower condition (Size != 8) for
initial pruning of inputs, effectively preventing libFuzzer from generating
inputs longer than necessary and spending time on mutating such long inputs in
the corpus - a functionality not meant to be tested by this specific test.
Differential Revision: https://reviews.llvm.org/D86247
It isn't very wise to pass an assembly file to the compiler and tell it to compile as a C file and hope that the compiler recognizes it as assembly instead.
Simply don't mark the file as C and CMake will recognize the rest.
This was attempted earlier in https://reviews.llvm.org/D85706, but reverted due to architecture issues on Apple.
Subsequent digging revealed a similar change was done earlier for libunwind in https://reviews.llvm.org/rGb780df052dd2b246a760d00e00f7de9ebdab9d09.
Afterwards workarounds were added for MinGW and Apple:
* https://reviews.llvm.org/rGb780df052dd2b246a760d00e00f7de9ebdab9d09
* https://reviews.llvm.org/rGd4ded05ba851304b26a437896bc3962ef56f62cb
The workarounds in libunwind and compiler-rt are unified and comments added pointing to each other.
The workaround is updated to only be used for MinGW for CMake versions before 3.17, which fixed the issue (https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4287).
Additionally fixed Clang not being passed as the assembly compiler for compiler-rt runtime build.
Example error:
[525/634] Building C object lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
FAILED: lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
/opt/tooling/drive/host/bin/clang --target=aarch64-linux-gnu -I/opt/tooling/drive/llvm/compiler-rt/lib/tsan/.. -isystem /opt/tooling/drive/toolchain/opt/drive/toolchain/include -x c -Wall -Wno-unused-parameter -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fPIE -fno-rtti -Wframe-larger-than=530 -Wglobal-constructors --sysroot=. -MD -MT lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -MF lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o.d -o lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -c /opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S
/opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S:29:1: error: expected identifier or '('
.section .text
^
1 error generated.
Differential Revision: https://reviews.llvm.org/D86308
The existing implementations are almost identical except for width of the
integer type.
Factor them out to int_mulo_impl.inc for better maintainability.
This patch is almost identical to D86277.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D86289
The existing implementations are almost identical except for width of the
integer type.
Factor them out to int_mulv_impl.inc for better maintainability.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D86277
Fixes a build failure in the Darwin kernel. Tested with:
% nm -mU lib/libclang_rt.cc_kext_x86_64h_osx.a | grep __llvm_profile_raw_version
rdar://67809173
Currently, libFuzzer will exit with an error message if a non-existent
corpus directory is provided. However, if a user provides a non-existent
directory for the `artifact_prefix`, `exact_artifact_path`, or
`features_dir`, libFuzzer will continue execution but silently fail to
write artifacts/features.
To improve the user experience, this PR adds validation for the existence of
all user supplied directories before executing the main fuzzing loop. If they
don't exist, libFuzzer will exit with an error message.
Patch By: dgg5503
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D84808
With the 'new' way of releasing on 32-bit, we iterate through all the
regions in between `First` and `Last`, which covers regions that do not
belong to the class size we are working with. This is effectively wasted
cycles.
With this change, we add a `SkipRegion` lambda to `releaseFreeMemoryToOS`
that will allow the release function to know when to skip a region.
For the 64-bit primary, since we are only working with 1 region, we never
skip.
Reviewed By: hctim
Differential Revision: https://reviews.llvm.org/D86399
* Make the three tests look more uniformly
* Explicitly specify types of integer and floating point literals
* Add more test cases (mostly inspired by divtf3_test.c)
- tests are added for obviously special cases such as +/-Inf, +/-0.0 and some
more implementation-specific cases such as divisor being almost 1.0
* Make NaN in the second test case of `divtf3` to be `sNaN` instead of
testing for `qNaN` again
Reviewed By: sepavloff
Differential Revision: https://reviews.llvm.org/D84932
FreeBSD doesn't provide a crypt.h header but instead defines the functions
in unistd.h. Use __has_include() to handle that case.
Reviewed By: #sanitizers, vitalybuka
Differential Revision: https://reviews.llvm.org/D85406
FreeBSD delivers a SIGBUS signal for bad addresses rather than SIGSEGV.
Reviewed By: #sanitizers, vitalybuka, yln
Differential Revision: https://reviews.llvm.org/D85409
The dynamically linked ASan tests rely on `LD_LIBRARY_PATH` to find
`libclang_rt.asan-*.so` at runtime.
However, the Solaris runtime linker `ld.so.1` also supports more specific
variables: `LD_LIBRARY_PATH_32` and `LD_LIBRARY_PATH_64` respectively. If
those happen to be set, `LD_LIBRARY_PATH` is ignored. In such a case, all
dynamically linked ASan tests `FAIL`. For i386 alone, this affects about
200 tests.
The following patch fixes that by also setting `LD_LIBRARY_PATH_{32,64}` on
Solaris.
Tested on `amd64-pc-solaris2.11` both with only `LD_LIBRARY_PATH` set and
with `LD_LIBRARY_PATH_{32,64}` set too.
Differential Revision: https://reviews.llvm.org/D86333
Support -march=sapphirerapids for x86.
Compare with Icelake Server, it includes 14 more new features. They are
amxtile, amxint8, amxbf16, avx512bf16, avx512vp2intersect, cldemote,
enqcmd, movdir64b, movdiri, ptwrite, serialize, shstk, tsxldtrk, waitpkg.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D86503
It's full featured now and we can use it for the runtimes build instead
of relying on an external libtool, which means the CMAKE_HOST_APPLE
restriction serves no purpose either now. Restrict llvm-lipo to Darwin
targets while I'm here, since it's only needed there.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D86367
libtool already produces a table of contents, and ranlib just gives
spurious errors because it doesn't understand universal binaries.
Reviewed By: phosek
Differential Revision: https://reviews.llvm.org/D86365
Handle NULL address argument in the `mach_vm_[de]allocate()`
interceptors and fix test: `Assignment 2` is not valid if we weren't
able to re-allocate memory.
rdar://67680613
Currently SimpleCmpTest passes after 9,831,994 trials on x86_64/Linux
when the number of given trials is 10,000,000, just a little bigger than
that. This patch modifies SimpleCmpTest.cpp so that the test passes with less
trials, reducing its chances of future failures as libFuzzer evolves. More
specifically, this patch changes a 32-bit equality check to a 8-bit equality
check, making this test pass at 4,635,303 trials.
Differential Revision: https://reviews.llvm.org/D86382
D80465 added an assembly implementation of muldi3 for RISC-V but it didn't
add it to the cmake `*_SOURCES` list, so the C implementation was being used
instead. This patch fixes that.
Differential Revision: https://reviews.llvm.org/D86036
Instead of ANDing with a one hot mask representing the bit to
be tested, we were ANDing with just the bit number. This tests
multiple bits none of them the correct one.
This caused skylake-avx512, cascadelake and cooperlake to all
be misdetected. Based on experiments with the Intel SDE, it seems
that all of these CPUs are being detected as being cooperlake.
This is bad since its the newest CPU of the 3.
We are now using a properly-substituted minimal deployment target
compiler flag (`%min_macos_deployment_target=10.11`). Enable test on
iOS and watchOS plus simulators. We are also not testing on very old
platforms anymore, so we can remove some obsolete lit infrastructure.
* Support macOS 11+ version scheme
* Standardize substitution name `%min_deployment_target=x.y`
* Remove unneeded error cases (the input version is hard-coded)
* Specify version as tuple instead of string; no need to parse it
These changes should also facilitate a future addition of a substitution
that expands to "set deployment target to current target version"
(https://reviews.llvm.org/D70151).
Reviewed By: delcypher
Differential Revision: https://reviews.llvm.org/D85925
We don't test on very old versions of Apple platforms anymore. The
following lit substitution concerning the minimum deployment target for
ARC support can be removed.
```
%darwin_min_target_with_full_runtime_arc_support -> 10.11
```
Differential Revision: https://reviews.llvm.org/D85803
After removing the unnecessary `-mmacosx-version-min=10.12` compiler
flag this test can run on all platforms. I confirmed that this test is
green for iOS, iOS simulator, and watchOS simulator.
Differential Revision: https://reviews.llvm.org/D85952
This reverts commit d58fd4e521. This broke
compiler-rt compilation on macOS:
codesign --sign - /Users/buildslave/jenkins/workspace/lldb-cmake/lldb-build/lib/clang/12.0.99/lib/darwin/libclang_rt.tsan_ios_dynamic.dylib
ld: warning: ignoring file projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_ios_dynamic.dir/rtl/tsan_rtl_amd64.S.o, building for iOS-arm64 but attempting to link with file built for iOS Simulator-x86_64
ld: warning: ignoring file projects/compiler-rt/lib/tsan/CMakeFiles/clang_rt.tsan_ios_dynamic.dir/rtl/tsan_rtl_aarch64.S.o, building for iOS-arm64 but attempting to link with file built for iOS Simulator-x86_64
Undefined symbols for architecture arm64:
"_wrap__setjmp", referenced from:
substitution__setjmp in tsan_interceptors_posix.cpp.o
"_wrap_setjmp", referenced from:
substitution_setjmp in tsan_interceptors_posix.cpp.o
"_wrap_sigsetjmp", referenced from:
substitution_sigsetjmp in tsan_interceptors_posix.cpp.o
ld: symbol(s) not found for architecture arm64
It isn't very wise to pass an assembly file to the compiler and tell it to compile as a C file and hope that the compiler recognizes it as assembly instead.
Instead enable the ASM language and mark the files as being ASM.
[525/634] Building C object lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
FAILED: lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o
/opt/tooling/drive/host/bin/clang --target=aarch64-linux-gnu -I/opt/tooling/drive/llvm/compiler-rt/lib/tsan/.. -isystem /opt/tooling/drive/toolchain/opt/drive/toolchain/include -x c -Wall -Wno-unused-parameter -fno-lto -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -fPIE -fno-rtti -Wframe-larger-than=530 -Wglobal-constructors --sysroot=. -MD -MT lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -MF lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o.d -o lib/tsan/CMakeFiles/clang_rt.tsan-aarch64.dir/rtl/tsan_rtl_aarch64.S.o -c /opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S
/opt/tooling/drive/llvm/compiler-rt/lib/tsan/rtl/tsan_rtl_aarch64.S:29:1: error: expected identifier or '('
.section .text
^
1 error generated.
Fixed Clang not being passed as the assembly compiler for compiler-rt runtime build.
Patch By: tambre
Differential Revision: https://reviews.llvm.org/D85706
The behavior of the CrossOver mutator has changed with
bb54bcf849. This seems to affect the
value-profile-load test on Darwin. This patch provides a wider margin for
determining success of the value-profile-load test, by testing the targeted
functionality (i.e., GEP index value profile) more directly and faster. To this
end, LoadTest.cpp now uses a narrower condition (Size != 8) for initial pruning
of inputs, effectively preventing libFuzzer from generating inputs longer than
necessary and spending time on mutating such long inputs in the corpus - a
functionality not meant to be tested by this specific test.
Previously, on x86/Linux, it required 6,597,751 execs with -use_value_profile=1
and 19,605,575 execs with -use_value_profile=0 to hit the crash. With this
patch, the test passes with 174,493 execs, providing a wider margin from the
given trials of 10,000,000. Note that, without the value profile (i.e.,
-use_value_profile=0), the test wouldn't pass as it still requires 19,605,575
execs to hit the crash.
Differential Revision: https://reviews.llvm.org/D86247
InitializeInterceptors() calls dlsym(), which calls calloc(). Depending
on the allocator implementation, calloc() may invoke mmap(), which
results in a segfault since REAL(mmap) is still being resolved.
We fix this by doing a direct syscall if interceptors haven't been fully
resolved yet.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D86168
D85820 introduced a bug where LLVM_ENABLE_TERMINFO was set to true when
the library was found, even when the user had set
-DLLVM_ENABLE_TERMINFO=OFF.
Patch By: haampie
Differential Revision: https://reviews.llvm.org/D86173
`dispatch_async_and_wait()` was introduced in macOS 10.14. Let's
forward declare it to ensure we can compile the test with older SDKs and
guard execution by checking if the symbol is available. (We can't use
`__builtin_available()`, because that itself requires a higher minimum
deployment target.) We also need to specify the `-undefined
dynamic_lookup` compiler flag.
Differential Revision: https://reviews.llvm.org/D85995
`dispatch_async_and_wait()` was introduced in macOS 10.14, which is
greater than our minimal deployment target. We need to forward declare
it as a "weak import" to ensure we generate a weak reference so the TSan
dylib continues to work on older systems. We cannot simply `#include
<dispatch.h>` or use the Darwin availability macros since this file is
multi-platform.
In addition, we want to prevent building these interceptors at all when
building with older SDKs because linking always fails.
Before:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA __interpose 0x000F5E68 pointer 0 libSystem _dispatch_async_and_wait_f
```
After:
```
➤ dyldinfo -bind ./lib/clang/12.0.0/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib | grep dispatch_async_and_wait
__DATA __got 0x000EC0A8 pointer 0 libSystem _dispatch_async_and_wait (weak import)
__DATA __interpose 0x000F5E78 pointer 0 libSystem _dispatch_async_and_wait (weak import)
```
This is a follow-up to D85854 and should fix:
https://reviews.llvm.org/D85854#2221529
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D86103
The linker errors caused by this revision have been addressed.
Add interceptors for `dispatch_async_and_wait[_f]()` which was added in
macOS 10.14. This pair of functions is similar to `dispatch_sync()`,
but does not force a context switch of the queue onto the caller thread
when the queue is active (and hence is more efficient). For TSan, we
can apply the same semantics as for `dispatch_sync()`.
From the header docs:
> Differences with dispatch_sync()
>
> When the runtime has brought up a thread to invoke the asynchronous
> workitems already submitted to the specified queue, that servicing
> thread will also be used to execute synchronous work submitted to the
> queue with dispatch_async_and_wait().
>
> However, if the runtime has not brought up a thread to service the
> specified queue (because it has no workitems enqueued, or only
> synchronous workitems), then dispatch_async_and_wait() will invoke the
> workitem on the calling thread, similar to the behaviour of functions
> in the dispatch_sync family.
Additional context:
> The guidance is to use `dispatch_async_and_wait()` instead of
> `dispatch_sync()` when it is necessary to mix async and sync calls on
> the same queue. `dispatch_async_and_wait()` does not guarantee
> execution on the caller thread which allows to reduce context switches
> when the target queue is active.
> https://gist.github.com/tclementdev/6af616354912b0347cdf6db159c37057
rdar://35757961
Reviewed By: kubamracek
Differential Revision: https://reviews.llvm.org/D85854
Marco Vanotti