Breaks TSan on Mac, which does
return REAL(func)(...
in COMMON_INTERCEPTOR_ENTER, which is not OK when REAL(func) has return
type of void.
llvm-svn: 355256
macOS has implementation of LogFullErrorReport and
INLINE void LogFullErrorReport(const char *buffer) {}
was causing
> compiler-rt/lib/sanitizer_common/sanitizer_mac.cc:658:6: error: redefinition of 'LogFullErrorReport'
Fixup for r355236.
rdar://problem/48526020
llvm-svn: 355244
Replace remaining uses of old Unwind API in unit tests.
Allows us to remove the old API and WillUseFastUnwind can be made
private.
Reviewed By: vitalybuka
Differential Revision: https://reviews.llvm.org/D58754
llvm-svn: 355242
mingw-w64 currently is lacking the headers for this feature.
Make the include lowercase at the same time. We consistently
use lowercase for windows header includes, as windows itself is
case insensitive, the SDK headers (in general, not necessarily
considering this particular header) aren't consistent among themselves
about what the proper canonical capitalization for headers are,
and MinGW uses all lowercase names for the headers (as it is often
used on case sensitive filesystems).
In case mingw-w64 later gets this header, we can revert this
(but keep the include lowercased).
Differential Revision: https://reviews.llvm.org/D58765
llvm-svn: 355236
GetStackTrace is a implementation detail of BufferedStackTrace. Make it
a private method.
Reviewed By: vitalybuka
Differential-Revision: https://reviews.llvm.org/D58753
llvm-svn: 355232
Summary:
A missing `STATIC` entailed some annoying to debug failures wrt 32 vs 64
binaries. Additionally I noticed I was using the wrong variable (the Scudo
one as opposed to the Scudo Standalone one).
See https://reviews.llvm.org/D58184#1412417 and below for discussion.
Reviewers: vitalybuka, eugenis, brzycki
Reviewed By: vitalybuka, brzycki
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58794
llvm-svn: 355203
Retrying without replacing call sites in sanitizer_common (which might
not have a symbol definition).
Add new Unwind API. This is the final envisioned API with the correct
abstraction level. It hides/slow fast unwinder selection from the caller
and doesn't take any arguments that would leak that abstraction (i.e.,
arguments like stack_top/stack_bottom).
GetStackTrace will become an implementation detail (private method) of
the BufferedStackTrace class.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58741
> llvm-svn: 355168
llvm-svn: 355172
Add new Unwind API. This is the final envisioned API with the correct
abstraction level. It hides/slow fast unwinder selection from the caller
and doesn't take any arguments that would leak that abstraction (i.e.,
arguments like stack_top/stack_bottom).
GetStackTrace will become an implementation detail (private method) of
the BufferedStackTrace class.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58741
llvm-svn: 355168
Summary:
This fixes inconsistent symbol visibility. This shows up as a linker
warning if r336238 (43f633564e338a6dde83d49a48e5bfcbfdce292c) is
reverted.
```
ld: warning: direct access in function 'fuzzer::CleanseCrashInput(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, fuzzer::fuzzer_allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, fuzzer::FuzzingOptions const&)' from file '/Volumes/data/dev/llvm/upstream/master/builds/projects/compiler-rt/lib/fuzzer/tests/libRTFuzzerTest.x86_64.a(FuzzerDriver.cpp.o)' to global weak symbol 'fuzzer::Command::ignoreRemainingArgs()::kIgnoreRemaining' from file 'FuzzerTestObjects.FuzzerUnittest.cpp.x86_64.o' means the weak symbol cannot be overridden
at runtime. This was likely caused by different translation units being compiled with different visibility settings.
```
r336238 just hid the issue rather than fixing the real issue. On macOS
and other platforms we usually compile with `-fvisibility=hidden` but
the unit tests were compiled without this flag.
Reviewers: george.karpenkov, kubamracek, kcc, yln
Subscribers: mgorny, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58055
llvm-svn: 355143
Remove the maximum stack cleanup size check. With ulimit -s unlimited
main thread stack can be very large, but we don't really have a choice
other than cleaning all of it. It should be reasonably fast - hwasan
cleans large shadow ranges with a single madvise call.
This change fixes check-hwasan after ulimit -s unlimited.
llvm-svn: 355137
This reverts revision 354601 and disables ReadBinaryNameCached check on
Windows since Windows has no working ReadBinaryName.
Differential Revision: https://reviews.llvm.org/D58788
llvm-svn: 355129
PLT calls on i386 expect GOT base address in %ebx.
This call does not need to go through PLT strictly speaking, but I'd
rather avoid future surprises and implement the most general case.
llvm-svn: 355125
The concept of weak linkage is not available on Windows. The available
workarounds in LLVM/sanitizer runtimes have their own problems. Define a
separte symbol ubsan_GetStackTrace to work around the issue now. At lest
this way it is painfully obvious that we still have to do more cleanup.
Follow-up to revision: https://reviews.llvm.org/D58651
llvm-svn: 355113
Adds option for collecting sanitixer dumps via trace logging.
- Set log_to_syslog=1 to enable this output.
- Consult https://aka.ms/windowstracelogging for details on use.
llvm-svn: 355045
Disable hwasan interceptor on non-linux, non-x86-or-arm platforms.
Add @plt to the asm call that clang intergrated-as infers but gcc does
not.
llvm-svn: 355041
We already independently declare GetStackTrace in all (except TSan)
sanitizer runtime headers. Lets move it to sanitizer_stacktrace.h to
have one canonical way to fill in a BufferedStackFrame. Also enables us
to use it in sanitizer_common itself.
This patch defines GetStackTrace for TSan and moves the function from
ubsan_diag.cc to ubsan_diag_standalone.cc to avoid duplicate symbols
for the UBSan-ASan runtime.
Other than that this patch just moves the code out of headers and into
the correct namespace.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58651
llvm-svn: 355039
Also assert that the caller always gets what it requested.
This purely mechanical change simplifies future refactorings and
eventual removal of BufferedStackTrace::Unwind.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58557
llvm-svn: 355022
Summary:
Port libFuzzer's fork mode to Windows.
Implement Windows versions of MkDir, RmDir, and IterateDirRecursive to do this.
Don't print error messages under new normal uses of FileSize (on a non-existent file).
Implement portable way of piping output to /dev/null.
Fix test for Windows and comment fork-sigusr.test on why it won't be ported to Win.
Reviewers: zturner
Reviewed By: zturner
Subscribers: kcc, zturner, jdoerfert, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58513
llvm-svn: 355019
Summary:
This CL adds a standalone vector class that will be used by the scoped
strings when they land. We reimplement our own vector class because we
can't use the std library one.
It's mostly borrowed from the current sanitizer_common one, with LLVM
code style changes.
Additionnally a casing change in a function name that slipped through
the previous review (the function isn't used yet).
Reviewers: vitalybuka, eugenis, flowerhack, dmmoore415, mcgrathr, morehouse
Reviewed By: vitalybuka
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58689
llvm-svn: 354999
As discussed elsewhere: LLVM uses cpp as its C++ source extension; the
sanitizers should too. This updates files in hwasan.
Patch generated by
for f in lib/hwasan/*.cc ; do svn mv $f ${f%.cc}.cpp; done
followed by
for f in lib/hwasan/*.cpp ; do sed -i '' -e '1s/\.cc -/.cpp /' $f; done
CMakeLists.txt updated manually.
Differential Revision: https://reviews.llvm.org/D58620
llvm-svn: 354989
For 'cascadelake' this is adding a 'avx512vnni' feature check to the 0x55 skylake-avx512 model check. These CPUs use the same model number and only differ in the stepping number. But the feature flag is simpler than collecting all the stepping numbers.
For 'znver2' this is just syncing with LLVM's Host.cpp.
llvm-svn: 354927
Summary:
When running the standalone main on a large corpus, I eventually get a
EMFILE error ("Too many open files").
Patch by Paul Chaignon
Reviewers: kcc, vitalybuka
Reviewed By: vitalybuka
Subscribers: lebedev.ri, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58622
llvm-svn: 354918
Summary:
This CL adds the platform specific code for Fuchsia, Linux & Android,
as well as some tests related to those (more tests to come later).
While some of it is pretty much a straight port of the existing scudo &
sanitizer_common code, the memory mapping functions have been reworked
a bit to fit the limited usage scenario that Scudo has for them.
For Fuchsia, I can now track the Vmar/Vmo pair for memory mappings if
there is an intent to grow or decommit some mapping (that will be
useful for the Primary).
Reviewers: eugenis, vitalybuka, mcgrathr, phosek, flowerhack, morehouse, dmmoore415
Reviewed By: vitalybuka, morehouse
Subscribers: kcc, dvyukov, srhines, mgorny, delcypher, jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58184
llvm-svn: 354895
Summary:
pthread_sigmask is just like sigprocmask, except that its behavior in
multithreaded programs is explicitly specified. Sanitizers were lacking
a common interceptor for pthread_sigmask (although some specific
sanitizers defined custom version), which lead to false positives
(at least in msan) when using this function.
The interceptor implementation, and its test are based on the equivalent
code for sigprocmask.
Reviewers: eugenis, vitalybuka
Subscribers: kubamracek, delcypher, jfb, jdoerfert, llvm-commits, #sanitizers
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58382
llvm-svn: 354874
Summary:
I have not introduced a separate hook for `bcmp()` as I don't think there
should be any reason for a sanitizer to treat it differently from `memcmp()`.
This is only enabled when building on POSIX with GNU extensions.
Context: this is to avoid losing coverage when emitting `bcmp() == 0` instead
of `memcmp() == 0` in llvm, see https://reviews.llvm.org/D56593.
Reviewers: mgorny, krytarowski, vitalybuka, dvyukov
Subscribers: kubamracek, dberris, delcypher, jdoerfert, #sanitizers, llvm-commits, jyknight
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58379
llvm-svn: 354851
Summary:
The llvm-cov tool needs to be able to find coverage names in the
executable, so the .lprfn and .lcovmap sections cannot be merged into
.rdata.
Also, the linker merges .lprfn$M into .lprfn, so llvm-cov needs to
handle that when looking up sections. It has to support running on both
relocatable object files and linked PE files.
Lastly, when loading .lprfn from a PE file, llvm-cov needs to skip the
leading zero byte added by the profile runtime.
Reviewers: vsk
Subscribers: hiraditya, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58661
llvm-svn: 354840
This is a second attempt at r342652 using a TLS callback instead of an
interceptor.
In long-running builds we've seen some ASan complaints during thread creation
that we suspect are due to leftover poisoning from previous threads whose
stacks occupied that memory. This patch adds a callback that unpoisons the
stack memory when a thread exits.
Differential Revision: https://reviews.llvm.org/D58641
llvm-svn: 354836
FastUnwindStack -> UnwindFast
SlowUnwindStack -> UnwindSlow
Stack is redundant, verb should come first.
SlowUnwindStackWithContext(uptr pc, void *context, u32 max_depth) ->
SlowUnwindStack
WithContext is redundant, since it is a required parameter.
Reviewers: vitalybuka
Differential Revision: https://reviews.llvm.org/D58551
llvm-svn: 354696
__asan_handle_vfork was unpoisoning the wrong part of the stack.
Adjust the test to catch this reliably (current failure is
non-deterministic).
llvm-svn: 354627
In compiler-rt we have the notion of a `fast` and a `slow` stack
unwinder. Darwin currently only supports the fast unwinder.
From reading the code, my understanding is that
`BufferedStackTrace::Unwind` can be called with `bp=0, stack_top=0,
stack_bottom=0, request_fast_unwind=false`. If
`request_fast_unwind=true`, then we alos need to supply bp, stack_top,
and stack_bottom.
However, `BufferedStackTrace::Unwind` uses
`StackTrace::WillUseFastUnwind` which will adapt `request_fast_unwind`
if the requested unwinder is not supported. On Darwin, the result is
that we don't pass actual values for bp, stack_top, and stack_bottom,
but end up using the fast unwinder. The tests then fail because we only
print the topmost stack frame.
This patch adds a check to `WillUseFastUnwind` at the point of usage to
avoid the mismatch between `request_fast_unwind` and what `Unwind`
actually does. I am also interested in cleaning up the
`request_fast_unwind` machinery so this patch just the simplest thing
possible so I can enable the tests.
Reviewers: vitalybuka, vsk
Differential Revision: https://reviews.llvm.org/D58156
llvm-svn: 354282
According to the logs and local debugging there were two issues:
1) tsan tests listed libc++.a before the source file. That's usually
ok for shared libraries, but the linker will not add symbols from
a static library unless needed at that time. As a result the tests
that rely upon symbols from the library (and not only include the
headers) had undefined references.
To solve this I'm adding a new substitution %link_libcxx_tsan which
expands to libc++.a if available.
2) The target Fuzzer-x86_64-Test linked in SANITIZER_TEST_CXX_LIBRARIES
which defaults to -lstdc++. This resulted in error messages like
hidden symbol '_ZdlPv' is not defined locally
hidden symbol '_Znwm' is not defined locally
when using GNU gold (ld.bfd and lld are fine). Removing the linkage
is fine because we build a custom libc++ for that purpose.
llvm-svn: 354231
This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:
1) The resulting libFuzzer doesn't expose C++ internals and looks
like a plain C library.
2) We don't have to manually link in libstdc++ to provide cxxabi.
3) The sanitizer tests cannot interfere with an installed version
of libc++.so in LD_LIBRARY_PATH.
Differential Revision: https://reviews.llvm.org/D58013
llvm-svn: 354212
Add missed value "libcxxabi" and introduce SANITIZER_TEST_CXX for linking
unit tests. This needs to be a full C++ library and cannot be libcxxabi.
Recommit r354132 which I reverted in r354153 because it broke a sanitizer
bot. This was because of the "fixes" for pthread linking, so I've removed
these changes.
Differential Revision: https://reviews.llvm.org/D58012
llvm-svn: 354198
There is an ambiguity between ::SizeClassMap (the typedef declared near
the start of this file) and __sanitizer::SizeClassMap (found by the
'using namespace __sanitizer;' near the start of this file).
Historically a Clang bug has meant that the error was not diagnosed, but
soon Clang will start diagnosing it. Explicitly qualify this use of
SizeClassMap so that it finds __sanitizer::SizeClassMap rather than
being ill-formed due to ambiguity.
llvm-svn: 354174
Summary:
When symbols are unavailable, the current code prints
sp: ... pc: ... (null) (null)
instead of module name + offset.
Change the output to include module name and offset, and also to match
the regular sanitizer stack trace format so that it is recognized by
symbolize.py out of the box.
Reviewers: kcc, pcc
Subscribers: kubamracek, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58267
llvm-svn: 354157
Summary:
With tag_in_free=1, malloc() can not assume that the memory is untagged,
and needs to retag is to 0.
Reviewers: pcc, kcc
Subscribers: kubamracek, jfb, jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D58158
llvm-svn: 354155
Add missed value "libcxxabi" and introduce SANITIZER_TEST_CXX for linking
unit tests. This needs to be a full C++ library and cannot be libcxxabi.
Differential Revision: https://reviews.llvm.org/D58012
llvm-svn: 354132
Summary:
The changes in https://reviews.llvm.org/D44847 cause load time failure
due to lprofMergeValueProfData in Android libs enabled with profile generation:
"dlopen failed: cannot locate symbol "lprofMergeValueProfData" referenced by..."
Marking lprofMergeValueProfData as hidden so the correct in-module definition
is picked by the linker.
Reviewers: davidxl
Reviewed By: davidxl
Subscribers: efriedma, xur, davidxl, llvm-commits
Differential Revision: https://reviews.llvm.org/D55893
llvm-svn: 354064
Summary:
Pre 2.27 libc can run same atexit handler twice
We will keep MSanAtExitRecord and reset fun to mark it as executed.
Fix PR40162
Reviewers: eugenis
Subscribers: jfb, jdoerfert, #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D58221
llvm-svn: 354005
This patch adds functions for managing fibers:
__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()
See the added tests for use examples.
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889
[The previous commit of this change was reverted,
this is a resubmit with a squashed fix for check_analyze.sh
and COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED]
llvm-svn: 353947
This function doesn't use anything MSVC specific but works fine
for any _WIN32 target.
Differential Revision: https://reviews.llvm.org/D58106
llvm-svn: 353918
Summary:
A link error was encountered when using the Red Hat Developer Toolset.
In the RHDTS, `libstdc++.so` is a linker script that may resolve symbols
to a static library. This patch places `-lstdc++` later in the ordering.
Reviewers: sfertile, nemanjai, tstellar, dberris
Reviewed By: dberris
Subscribers: dberris, mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D58144
llvm-svn: 353905
buildgo.sh puts most of sanitizer_common together in a single source
file. These single-letter macros end up affecting a lot of unrelated
code; #undef them as early as possible.
llvm-svn: 353902
This patch adds functions for managing fibers:
__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()
See the added tests for use examples.
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889
llvm-svn: 353817
The __sanitizer_acquire_crash_state function has int return type, but the
fuzzer's external function definitions give it bool.
Places where __sanitizer_acquire_crash_state is declared:
include/sanitizer_common/sanitizer_interface_defs.h
lib/sanitizer_common/sanitizer_interface_internal.h
lib/sanitizer_common/sanitizer_common.cc
lib/fuzzer/FuzzerExtFunctions.def (this is the only bool)
llvm-svn: 353596
Summary:
Scoped interceptor should not be used when calling real pthread_exit().
On macOS C++ destructors are not called by pthread_exit(), and later check for empty thread ignore set fails.
Patch by Yuri Per.
Reviewers: dvyukov, vitalybuka
Reviewed By: vitalybuka
Subscribers: vitalybuka, thegameg, kubamracek, jfb, llvm-commits, #sanitizers
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57963
llvm-svn: 353561
Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.
Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.
Now, Windows uses static registration and is in line with all the other
platforms.
Reviewers: davidxl, wmi, inglorion, void, calixte
Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57929
llvm-svn: 353547
Current interception code does not cover all of the required registers
on Windows for a specific flavor of MOV, so this patch adds cases to
identify the following 5-byte instructions on 64-bit Windows:
mov QWORD PTR [rsp + XX], rdx <- second integer argument
mov QWORD PTR [rsp + XX], r9 <- third integer argument
mov QWORD PTR [rsp + XX], r8 <- fourth integer argument
The instruction for MOV [...] RCX is already covered in the previous
version.
Patch by Matthew McGovern!
Reviewers: rnk
Differential Revision: https://reviews.llvm.org/D57339
llvm-svn: 353483
Summary:
3rd party sysconf interceptor may crash if it's called before unsafe_stack_setup
However pageSize is not useful here. mmap should round up on it's own, SFS_CHECK can be removed.
Reviewers: eugenis, vlad.tsyrklevich
Subscribers: #sanitizers, llvm-commits
Tags: #sanitizers, #llvm
Differential Revision: https://reviews.llvm.org/D57924
llvm-svn: 353481
Summary:
From runtime side looks it's OK to RoundUpTo to needed alignment as buffer is
going to be RoundUpTo to page size anyway.
Reviewers: eugenis, pcc
Subscribers: #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D57866
llvm-svn: 353475
Summary:
The standalone Scudo version is being built with `-Werror` which can be
tripped by extraneous command line arguments. We have little control over
those as they can be passed down to us by `CMAKE_C(XX)_FLAGS`, the reported
scenario involving `-stdlib=libc++` (see https://reviews.llvm.org/D57412#1384504).
To work around this, disable `-Wunused-command-line-argument`.
Reviewers: eugenis, vitalybuka, Eugene.Zelenko
Reviewed By: eugenis
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D57757
llvm-svn: 353418
func_entry_exit.cc is for __tsan_func_entry/exit (spends ~75% there),
we don't yet have any.
mop.cc is for memory access functions, as compared to mini_bench_local/shared.cc
this benchmark passes through deduplication logic (ContainsSameAccess).
llvm-svn: 353407
- Manually unwind code in MemoryAccessImpl1() because clang do not optimize it
- Check for .rodata section only in read operations
- Place LIKELY/UNLIKELY on fast paths
This speeds up synthetic memory access benchmarks by 10-20%.
[dvyukov: fixed up consts in check_analyze.sh]
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57882
Context: https://reviews.llvm.org/D54889
llvm-svn: 353401
Replace bool workerthread flag with ThreadType enum.
This change is preparation for fiber support.
[dvyukov: fixed build of sanitizer_thread_registry_test.cc]
Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57839
Context: https://reviews.llvm.org/D54889
llvm-svn: 353390
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.
This fixes TSAN performance issue made us to disable this interceptors.
Reviewers: eugenis, dvyukov
Subscribers: srhines, kubamracek, krytarowski, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D57731
llvm-svn: 353351
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.
Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.
Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.
The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.
Reviewers: pcc, vitalybuka
Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc
Differential Revision: https://reviews.llvm.org/D57720
llvm-svn: 353255
There is no reason for these sections to remain separate in the final
DLL or EXE.
I have not yet added a InstrProfilingPlatformWindows.c for these, since
avoiding dynamic profile data registration is a larger project for
later.
llvm-svn: 353221
Summary:
Use `_write` instead of the deprecated alias `write` on Windows.
Also, remove comment saying RawPrint is untested on Windows.
Reviewers: vitalybuka
Reviewed By: vitalybuka
Subscribers: vitalybuka
Differential Revision: https://reviews.llvm.org/D57589
llvm-svn: 353108
Summary:
This is the initial check-in for the Standalone version of Scudo.
The project is initially going to live in scudo/standalone then will
replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html
for details.
This initial CL is meant to lay out the project structure, of both
code & tests, providing a minimal amount of functionalities, namely
various definitions, some atomic helpers and an intrusive list.
(empty.cc is just here to have a compilation unit, but will go away
in the upcoming CLs).
Initial support is restricted to Linux i386 & x86_64 in make files
and will be extended once things land & work.
We will grow organically from here, adding functionalities in limited
amounts.
Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack
Reviewed By: morehouse, vitalybuka
Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits
Tags: #llvm, #sanitizers
Differential Revision: https://reviews.llvm.org/D57412
llvm-svn: 353055
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.
While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.
llvm-svn: 352921
This function initializes enough of the runtime to be able to run
instrumented code in a statically linked executable. It replaces
__hwasan_shadow_init() which wasn't doing enough initialization for
instrumented code that uses either TLS or IFUNC to work.
Differential Revision: https://reviews.llvm.org/D57490
llvm-svn: 352816
Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).
Reviewers: kcc
Reviewed By: kcc
Differential Revision: https://reviews.llvm.org/D57531
llvm-svn: 352789
Summary:
Set default `ASAN_OPTIONS` when running libFuzzer tests. This allows us
to remove special casing in code for Darwin where we usually pass
`abort_on_error=0` to override platform defaults for tests.
A previous commit changed the code to make the tests pass:
7764a04af0
Adapted a few tests to use `%env_asan_opts=` instead of directly setting
the environment variable.
rdar://problem/47515276
Reviewers: kcc, george.karpenkov
Differential Revision: https://reviews.llvm.org/D57465
llvm-svn: 352711
This fixes most references to the paths:
llvm.org/svn/
llvm.org/git/
llvm.org/viewvc/
github.com/llvm-mirror/
github.com/llvm-project/
reviews.llvm.org/diffusion/
to instead point to https://github.com/llvm/llvm-project.
This is *not* a trivial substitution, because additionally, all the
checkout instructions had to be migrated to instruct users on how to
use the monorepo layout, setting LLVM_ENABLE_PROJECTS instead of
checking out various projects into various subdirectories.
I've attempted to not change any scripts here, only documentation. The
scripts will have to be addressed separately.
Additionally, I've deleted one document which appeared to be outdated
and unneeded:
lldb/docs/building-with-debug-llvm.txt
Differential Revision: https://reviews.llvm.org/D57330
llvm-svn: 352514
Summary:
Use dllexport for all declarations in FuzzerInterface.h Use it for clang
even though clang supports default visibility attribute to prevent a
warning from being thrown when LLVMFuzzerMutate is defined with dllexport.
This makes `FUZZER_INTERFACE_VISIBILITY` (FuzzerInterface.h) consistent with
`ATTRIBUTE_INTERFACE` (FuzzerDefs.h) when using clang on Windows.
Reviewers: vitalybuka, morehouse
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D57305
llvm-svn: 352395
Summary:
Remove this feature as it is unused, buggy, and not worth correcting
since the forkserver makes it difficult.
Reviewers: morehouse, jfb
Reviewed By: morehouse
Differential Revision: https://reviews.llvm.org/D57308
llvm-svn: 352392
Summary:
This makes `GetBlockBegin()` and `GetBlockBeginFastLocked()` work correctly with `RemoteAddressSpaceView`.
This has a knock on effect of also making the `PointerIsMine()` and
`GetMetaData()` methods behave correctly when `RemoteAddressSpaceView`
is used to instantiate the allocators.
This will be used by future out-of-process allocator enumeration
patches.
rdar://problem/45284065
Reviewers: kcc, vitalybuka, dvyukov, cryptoad, eugenis, george.karpenkov, yln
Subscribers: #sanitizers, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D56964
llvm-svn: 352335
Summary:
As reported on llvm-testers, during 8.0.0-rc1 testing I got errors while
building of `XRayTest`, during `check-all`:
```
[100%] Generating XRayTest-x86_64-Test
/home/dim/llvm/8.0.0/rc1/Phase3/Release/llvmCore-8.0.0-rc1.obj/./lib/libLLVMSupport.a(Signals.cpp.o): In function `llvm::sys::PrintStackTrace(llvm::raw_ostream&)':
Signals.cpp:(.text._ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x24): undefined reference to `backtrace'
Signals.cpp:(.text._ZN4llvm3sys15PrintStackTraceERNS_11raw_ostreamE+0x254): undefined reference to `llvm::itaniumDemangle(char const*, char*, unsigned long*, int*)'
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[3]: *** [projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/build.make:73: projects/compiler-rt/lib/xray/tests/unit/XRayTest-x86_64-Test] Error 1
gmake[3]: Target 'projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/build' not remade because of errors.
gmake[2]: *** [CMakeFiles/Makefile2:33513: projects/compiler-rt/lib/xray/tests/unit/CMakeFiles/TXRayTest-x86_64-Test.dir/all] Error 2
gmake[2]: Target 'CMakeFiles/check-all.dir/all' not remade because of errors.
gmake[1]: *** [CMakeFiles/Makefile2:737: CMakeFiles/check-all.dir/rule] Error 2
gmake[1]: Target 'check-all' not remade because of errors.
gmake: *** [Makefile:277: check-all] Error 2
[Release Phase3] check-all failed
```
This is because the `backtrace` function requires `-lexecinfo` on BSD
platforms. To fix this, detect the `execinfo` library in
`cmake/config-ix.cmake`, and add it to the unit test link flags.
Additionally, since the code in `sys::PrintStackTrace` makes use of
`itaniumDemangle`, also add `-lLLVMDemangle`. (Note that this is more
of a general problem with libLLVMSupport, but I'm looking for a quick
fix now so it can be merged to the 8.0 branch.)
Reviewers: dberris, hans, mgorny, samsonov
Reviewed By: dberris
Subscribers: krytarowski, delcypher, erik.pilkington, #sanitizers, emaste, llvm-commits
Differential Revision: https://reviews.llvm.org/D57181
llvm-svn: 352234
Summary:
Release memory pages for thread data (allocator cache, stack allocations
ring buffer, etc) when a thread exits. We can not simply munmap them
because this memory is custom allocated within a limited address range,
and it needs to stay "reserved".
This change alters thread storage layout by putting the ring buffer
before Thread instead of after it. This makes it possible to find the
start of the thread aux allocation given only the Thread pointer.
Reviewers: kcc, pcc
Subscribers: kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56621
llvm-svn: 352151
Summary:
This tunes several of the default parameters used within the allocator:
- disable the deallocation type mismatch on Android by default; this
was causing too many issues with third party libraries;
- change the default `SizeClassMap` to `Dense`, it caches less entries
and is way more memory efficient overall;
- relax the timing of the RSS checks, 10 times per second was too much,
lower it to 4 times (every 250ms), and update the test so that it
passes with the new default.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: srhines, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D57116
llvm-svn: 352057
Each hwasan check requires emitting a small piece of code like this:
https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#memory-accesses
The problem with this is that these code blocks typically bloat code
size significantly.
An obvious solution is to outline these blocks of code. In fact, this
has already been implemented under the -hwasan-instrument-with-calls
flag. However, as currently implemented this has a number of problems:
- The functions use the same calling convention as regular C functions.
This means that the backend must spill all temporary registers as
required by the platform's C calling convention, even though the
check only needs two registers on the hot path.
- The functions take the address to be checked in a fixed register,
which increases register pressure.
Both of these factors can diminish the code size effect and increase
the performance hit of -hwasan-instrument-with-calls.
The solution that this patch implements is to involve the aarch64
backend in outlining the checks. An intrinsic and pseudo-instruction
are created to represent a hwasan check. The pseudo-instruction
is register allocated like any other instruction, and we allow the
register allocator to select almost any register for the address to
check. A particular combination of (register selection, type of check)
triggers the creation in the backend of a function to handle the check
for specifically that pair. The resulting functions are deduplicated by
the linker. The pseudo-instruction (really the function) is specified
to preserve all registers except for the registers that the AAPCS
specifies may be clobbered by a call.
To measure the code size and performance effect of this change, I
took a number of measurements using Chromium for Android on aarch64,
comparing a browser with inlined checks (the baseline) against a
browser with outlined checks.
Code size: Size of .text decreases from 243897420 to 171619972 bytes,
or a 30% decrease.
Performance: Using Chromium's blink_perf.layout microbenchmarks I
measured a median performance regression of 6.24%.
The fact that a perf/size tradeoff is evident here suggests that
we might want to make the new behaviour conditional on -Os/-Oz.
But for now I've enabled it unconditionally, my reasoning being that
hwasan users typically expect a relatively large perf hit, and ~6%
isn't really adding much. We may want to revisit this decision in
the future, though.
I also tried experimenting with varying the number of registers
selectable by the hwasan check pseudo-instruction (which would result
in fewer variants being created), on the hypothesis that creating
fewer variants of the function would expose another perf/size tradeoff
by reducing icache pressure from the check functions at the cost of
register pressure. Although I did observe a code size increase with
fewer registers, I did not observe a strong correlation between the
number of registers and the performance of the resulting browser on the
microbenchmarks, so I conclude that we might as well use ~all registers
to get the maximum code size improvement. My results are below:
Regs | .text size | Perf hit
-----+------------+---------
~all | 171619972 | 6.24%
16 | 171765192 | 7.03%
8 | 172917788 | 5.82%
4 | 177054016 | 6.89%
Differential Revision: https://reviews.llvm.org/D56954
llvm-svn: 351920
Bionic libc relies on an old libgcc behaviour which does not set hidden
visibility attribute. Keep exporting these symbols on Android for
compatibility.
Differential Revision: https://reviews.llvm.org/D56977
llvm-svn: 351915
Summary:
Enable building libFuzzer with MSVC.
* Don't try to include <endian.h> in FuzzerSHA1.cpp. MSVC
doesn't have this header, and WINDOWS is always little
endian (even on ARM)
Subscribers: srhines, mgorny, javed.absar, kristof.beyls
Differential Revision: https://reviews.llvm.org/D56510
llvm-svn: 351855
Reports correct size and tags when either size is not power of two
or offset to bad granule is not zero.
Differential revision: https://reviews.llvm.org/D56603
llvm-svn: 351730
Summary:
Make Sanitizer Coverage work when compiled work when compiler-rt
is compiled with MSVC.
The previous solution did not work for MSVC because MSVC tried to
align the .SCOV$CZ section even though we used
__declspec(align(1)) on its only symbol:
__stop___sancov_cntrs.
Because the counter array is composed
of 1 byte elements, it does not always end on an 8 or 4 byte
boundary. This means that padding was sometimes added to
added to align the next section, .SCOV$CZ.
Use a different strategy now: instead of only instructing
the compiler not to align the symbol, make the section
one byte long by making its only symbol a uint8_t, so that
the linker won't try to align it.
Reviewers: morehouse, rnk
Reviewed By: rnk
Subscribers: kubamracek
Differential Revision: https://reviews.llvm.org/D56866
llvm-svn: 351714
This implements `mi_extra_init(...)` for the ASan allocator on
Darwin and uses the `__lsan::GetAllocatorGlobalRange(...)` function
to retrieve the allocator pointer and size.
rdar://problem/45284065
llvm-svn: 351713
`sanitizer_malloc_introspection_t` and initialize them to zero.
We allow sanitizer implementations to perform different initialization
by defining `COMMON_MALLOC_HAS_EXTRA_INTROSPECTION_INIT` to be `1`
and providing an implementation of `mi_extra_init(...)`.
We use these changes in future patches to implement malloc zone enumeration.
rdar://problem/45284065
llvm-svn: 351712
enumerator.
This is done by defining `COMMON_MALLOC_HAS_ZONE_ENUMERATOR` to `1` and
then by providing an implementation of the `mi_enumerator(...)` function.
If a custom implementation isn't desired the macro is set to `0` which
causes a stub version (that fails) to be used.
Currently all Darwin sanitizers that have malloc implementations define
this to be `0` so there is no functionality change.
rdar://problem/45284065
llvm-svn: 351711
We forgot to pass `AddressSpaceView` to the `CombinedAllocator`
which meant we would always use `LocalAddressSpaceView` for the
`CombinedAllocator` leading to a static_assert failing when we
tried to do `AsanAllocatorASVT<RemoteAddressSpaceView>` or
`AllocatorASVT<RemoteAddressSpaceView>`.
rdar://problem/45284065
llvm-svn: 351689
to reflect the new license. These used slightly different spellings that
defeated my regular expressions.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351648
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary:
Whenever a large shadow region is tagged to zero, madvise(DONT_NEED)
as much of it as possible.
This reduces shadow RSS on Android by 45% or so, and total memory use
by 2-4%, probably even more on long running multithreaded programs.
CPU time seems to be in the noise.
Reviewers: kcc, pcc
Subscribers: srhines, kubamracek, llvm-commits
Differential Revision: https://reviews.llvm.org/D56757
llvm-svn: 351620
Summary:
SafeStack needs just few functions from there, but sanitizer_common
introduces conflicts with other runtimes, e.g. SCUDO.
Reviewers: eugenis, kcc, cryptoad
Subscribers: mgorny, krytarowski, fedor.sergeev, jfb, llvm-commits
Differential Revision: https://reviews.llvm.org/D56886
llvm-svn: 351506
Summary:
Small refactoring: replace some if-else cascades with switches so that the compiler warns us about missing cases.
Maybe found a small bug?
Reviewers: dcoughlin, kubamracek, dvyukov, delcypher, jfb
Reviewed By: dvyukov
Subscribers: llvm-commits, #sanitizers
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D56295
llvm-svn: 351288
Summary:
Remove code for handling unstable edges from libFuzzer since
it has not been found useful.
Differential Revision: https://reviews.llvm.org/D56730
llvm-svn: 351262
Summary:
This is the compiler-rt part.
The clang part is D54589.
This is a second commit, the original one was r351106,
which was mass-reverted in r351159 because 2 compiler-rt tests were failing.
Now, i have fundamentally changed the testing approach:
i malloc a few bytes, intentionally mis-align the pointer
(increment it by one), and check that. Also, i have decreased
the expected alignment. This hopefully should be enough to pacify
all the bots. If not, i guess i might just drop the two 'bad' tests.
Reviewers: filcab, vsk, #sanitizers, vitalybuka, rsmith, morehouse
Reviewed By: morehouse
Subscribers: rjmccall, krytarowski, rsmith, kcc, srhines, kubamracek, dberris, llvm-commits
Tags: #sanitizers
Differential Revision: https://reviews.llvm.org/D54590
llvm-svn: 351178
Summary:
Use alternatename for external functions only when using
MSVC since Clang doesn't support it and MSVC doesn't support
Clang's method (weak aliases).
Reviewers: morehouse
Reviewed By: morehouse
Subscribers: rnk, thakis, mgorny
Differential Revision: https://reviews.llvm.org/D56514
llvm-svn: 351152
Disable tests requiring sunrpc when the relevant headers are missing.
In order to accommodate that, move the header check
from sanitizer_common to base-config-ix, and define the check result
as a global variable there. Use it afterwards both for definition
needed by sanitizer_common, and to control 'sunrpc' test feature.
While at it, remove the append_have_file_definition macro that was used
only once, and no longer fits the split check-definition.
Bug report: https://github.com/google/sanitizers/issues/974
Differential Revision: https://reviews.llvm.org/D47819
llvm-svn: 351109
Evgeniy Stepanov