Commit Graph

9571 Commits

Author SHA1 Message Date
Vitaly Buka f2827e133b [sanitizer] Remove references and hide CommonSanitizerReportMutex
llvm-svn: 317865
2017-11-10 02:07:11 +00:00
Vitaly Buka bec32e9ac4 [msan] Deadly signal handler for msan
Summary: Part of https://github.com/google/sanitizers/issues/637

Reviewers: eugenis, alekseyshl

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39826

llvm-svn: 317864
2017-11-10 02:06:59 +00:00
Vitaly Buka a452f9cc37 [msan] Move sigaction_impl and signal_impl
llvm-svn: 317863
2017-11-10 02:06:50 +00:00
Kostya Serebryany c183e2c33b [libFuzzer] trying to make sigusr tests more reliable
llvm-svn: 317852
2017-11-10 00:26:23 +00:00
Kostya Serebryany e8637f45ea [libFuzzer] split a test into two
llvm-svn: 317851
2017-11-10 00:18:13 +00:00
Vitaly Buka bc55ab6397 [sanitizer] Remove unneeded "struct"
llvm-svn: 317847
2017-11-09 23:06:59 +00:00
Vitaly Buka 893bf354eb [sanitizer] Fix signal interceptors build on Windows
llvm-svn: 317846
2017-11-09 23:05:01 +00:00
Vitaly Buka 8e92025718 [msan] Extract signal_impl and sigaction_impl
Summary: Preparation for using interceptor from sanitizer_common.

Reviewers: eugenis

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39824

llvm-svn: 317844
2017-11-09 22:52:15 +00:00
Vitaly Buka 43c3e6ae1c [sanitizer] Allow sanitizers to redefine implementation of signal interceptors
Reviewers: eugenis

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39870

llvm-svn: 317843
2017-11-09 22:48:54 +00:00
Vitaly Buka 771e3995a4 [msan] Replace CommonSanitizerReportMutex with ScopedErrorReportLock
llvm-svn: 317842
2017-11-09 22:48:45 +00:00
Kostya Serebryany 533cbf749c [libFuzzer] use pkill instead of killall in a test since the bots don't have killall
llvm-svn: 317840
2017-11-09 22:01:06 +00:00
Kostya Serebryany 54a8456005 [libFuzzer] make sure to flush IO when done merging one file
llvm-svn: 317835
2017-11-09 21:30:33 +00:00
Kostya Kortchinsky 5604ad1c9b [sanitizer] Revert rL317822
Summary:
This reverts D39490.

For toolchains generated with older NDKs (<=r13b as far as we tested),
`cpu_set_t` doesn't exist in `sched.h`.
We have to figure out another way to get the number of CPUs without this.

Reviewers: rnk

Reviewed By: rnk

Subscribers: kubamracek, llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D39867

llvm-svn: 317834
2017-11-09 21:26:07 +00:00
Matt Morehouse 947838c9c5 [libFuzzer] Don't add leaking inputs to corpus.
Reviewers: kcc

Reviewed By: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39850

llvm-svn: 317831
2017-11-09 20:44:08 +00:00
Kostya Serebryany a2ca2dcc46 [libFuzzer] handle SIGUSR1/SIGUSR2 and try to exit grafully on these signals
llvm-svn: 317829
2017-11-09 20:30:19 +00:00
Teresa Johnson 179b8fbba6 Revert new ThinLTO ASAN test until lit support added
This reverts commit r317723 and r317728. Will be re-added when
support for LTO/ThinLTO added to test/asan/lit.cfg.

llvm-svn: 317823
2017-11-09 19:26:54 +00:00
Kostya Kortchinsky 6458216b28 [scudo] Make getNumberOfCPUs Fuchsia compliant
Summary: This change allows Fuchsia to boot properly using the Scudo allocator.

Reviewers: cryptoad, alekseyshl, krytarowski

Reviewed By: cryptoad, krytarowski

Subscribers: rnk, krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39490

llvm-svn: 317822
2017-11-09 19:18:55 +00:00
Weiming Zhao e58bb5dc4e [Builtins] Do not use tailcall for Thumb1
Summary:
The `b` instruction in Thumb1 has limited range, which may cause link-time errors if the jump target is far away.
This patch guards the tailcalls for non-Thumb1

Reviewers: peter.smith, compnerd, rengolin, eli.friedman

Reviewed By: rengolin

Subscribers: joerg, dalias, javed.absar, llvm-commits

Differential Revision: https://reviews.llvm.org/D39700

llvm-svn: 317814
2017-11-09 17:32:57 +00:00
Bill Seurer 823338d1fe [PowerPC][msan] Update msan to handle changed memory layouts in newer kernels
In more recent Linux kernels (including those with 47 bit VMAs) the layout of
virtual memory for powerpc64 changed causing the memory sanitizer to not
work properly.  This patch adjusts the memory ranges in the tables for the
memory sanitizer to work on the newer kernels while continuing to work on the
older ones as well.

Tested on several 4.x and 3.x kernel releases.

llvm-svn: 317802
2017-11-09 16:14:57 +00:00
Vitaly Buka 66f32fc431 [sanitizers] Rename GetStackTraceWithPcBpAndContext
Name does not need to enumerate arguments.

llvm-svn: 317774
2017-11-09 07:53:06 +00:00
Vitaly Buka 2b2d3aaa04 [msan] Add context argument into GetStackTrace
llvm-svn: 317773
2017-11-09 07:48:53 +00:00
Vitaly Buka cb4b2c0ffc [lsan] Add "static" to internal function
llvm-svn: 317772
2017-11-09 07:46:30 +00:00
Kostya Serebryany 3ac31d4453 [libFuzzer] allow merge to resume after being preempted
llvm-svn: 317767
2017-11-09 05:49:28 +00:00
Peter Collingbourne c6721f580d ubsan: Allow programs to use setenv to configure ubsan_standalone.
Previously ubsan_standalone used the GetEnv function to read the
environment variables UBSAN_OPTIONS and UBSAN_SYMBOLIZER_PATH. The
problem with GetEnv is that it does not respect changes to the
environment variables made using the libc setenv function, which
prevents clients from setting environment variables to configure
ubsan before loading ubsan-instrumented libraries.

The reason why we have GetEnv is that some runtimes need to read
environment variables while they initialize using .preinit_array,
and getenv does not work while .preinit_array functions are being
called. However, it is unnecessary for ubsan_standalone to initialize
that early. So this change switches ubsan_standalone to using getenv
and removes the .preinit_array entry. The static version of the runtime
still ends up being initialized using a C++ constructor that exists
to support the shared runtime.

Differential Revision: https://reviews.llvm.org/D39827

llvm-svn: 317757
2017-11-09 02:22:07 +00:00
Kostya Serebryany e1d121d715 [libFuzzer] mechanically simplify a test, NFC
llvm-svn: 317756
2017-11-09 02:13:43 +00:00
Kostya Serebryany 8a94f220aa [libFuzzer] fix a test (and hopefully, the bot)
llvm-svn: 317749
2017-11-09 01:45:59 +00:00
Kostya Serebryany 68fdef1f7a [libFuzzer] allow user to specify the merge control file
llvm-svn: 317747
2017-11-09 01:05:29 +00:00
Evgeniy Stepanov 75a35179de [Sanitizers, CMake] Also use version script for libclang_rt.asan-i386.so
When building LLVM on x86_64-pc-linux-gnu (Fedora 25) with the bundled gcc 6.4.1
which uses gld 2.26.1-1.fc25, the dynamic/Asan-i386-calls-Dynamic-Test and
dynamic/Asan-i386-inline-Dynamic-Test tests failed to link with

/usr/bin/ld: /var/scratch/gcc/llvm/dist/lib/clang/6.0.0/lib/linux/libclang_rt.asan-i386.so: fork: invalid version 21 (max 0)
/var/scratch/gcc/llvm/dist/lib/clang/6.0.0/lib/linux/libclang_rt.asan-i386.so: error adding symbols: Bad value

I tried building with a self-compiled gcc 7.1.0 using gld 2.28, but the error remained.

It seems the error has been hit before (cf. https://reviews.llvm.org/rL314085), but
no real explanation has been found.

However, the problem goes away when linking the i386 libclang_rt.asan with a version
script just like every other variant is. Not using the version script in this single case
dates back to the initial introduction of the version script in r236551, but this change
was just checked in without any explanation AFAICT.

Since I've not found any other workaround and no reason for not always using the
version script, I propose to do so.

Tested on x86_64-pc-linux-gnu.

Patch by Rainer Orth.

Differential Revision: https://reviews.llvm.org/D39795

llvm-svn: 317738
2017-11-08 22:51:09 +00:00
Kamil Rytarowski 2fd314e2e2 Correct atexit(3) support in TSan/NetBSD
Summary:
The NetBSD specific implementation of cxa_atexit() does not
preserve the 2nd argument if dso is equal to NULL.

Changes:

 - Split paths of handling intercepted __cxa_atexit() and atexit(3).
   This affects all supported Operating Systems.
 - Add a local stack-like structure to hold the __cxa_atexit() context.
   atexit(3) is documented in the C standard as calling callback from the
   earliest to the oldest entry. This path also fixes potential ABI
   problem of passing an argument to a function from the atexit(3)
   callback mechanism.
 - Add new test to ensure LIFO style of atexit(3) callbacks: atexit3.cc

Proposal to change the behavior of __cxa_atexit() in NetBSD has been rejected.

With the above changes TSan/NetBSD with the current tsan_interceptors.cc
can bootstrap into operation.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, dvyukov, joerg, kcc, eugenis

Reviewed By: dvyukov

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39619

llvm-svn: 317735
2017-11-08 22:34:17 +00:00
Teresa Johnson c707c6f3a7 [ThinLTO] New test needs to require LTO
Fix buildbot failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/5262/steps/annotate/logs/stdio

llvm-svn: 317728
2017-11-08 21:48:27 +00:00
Teresa Johnson edf782f0d0 [ThinLTO] Ensure sanitizer passes are run
Recommit new test as linux-only.

llvm-svn: 317723
2017-11-08 20:27:28 +00:00
Teresa Johnson 67a781c4fc Revert "[ThinLTO] Ensure sanitizer passes are run"
This reverts commit r317715. It failed a Windows buildbot since
ThinLTO is presumably not supported, leading to a corrupt file error
on the object file:
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/19395/steps/run%20check-asan/logs/stdio

Will re-commit the new ThinLTO part of the test to a linux only test
file.

llvm-svn: 317717
2017-11-08 20:08:15 +00:00
Teresa Johnson 0cfd530b2b [ThinLTO] Ensure sanitizer passes are run
Summary:
Test fix to pass manager for ThinLTO.

Depends on D39565.

Reviewers: pcc

Subscribers: kubamracek, mehdi_amini, llvm-commits, inglorion

Differential Revision: https://reviews.llvm.org/D39566

llvm-svn: 317715
2017-11-08 19:46:25 +00:00
Marco Castelluccio ff4da375ae Implement flock for Windows in compiler-rt
Summary:
This patch implements flock for Windows, needed to make gcda writing work in a multiprocessing scenario.

Fixes https://bugs.llvm.org/show_bug.cgi?id=34923.

Reviewers: zturner

Reviewed By: zturner

Subscribers: rnk, zturner, llvm-commits

Differential Revision: https://reviews.llvm.org/D38891

llvm-svn: 317705
2017-11-08 19:11:54 +00:00
Kostya Kortchinsky 6edadae34a [sanitizer] Add Scudo to the sanitizer lint checks.
Summary:
Scudo abides by the coding style enforced by the sanitizer_common
linter, but as of right now, it's not linter-enforced.

Add Scudo to the list of directories checked by check_lint.sh.

Also: fixes some linter errors found after getting this running.

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39757

llvm-svn: 317699
2017-11-08 16:42:29 +00:00
Sylvestre Ledru a8c5460c8a Always use prctl(PR_SET_PTRACER)
Summary:
Sufficiently old Linux kernel headers don't provide the PR_SET_PTRACER, but we can still call prctl with it if the runtime kernel is newer. Even if it's not, prctl will only return EINVAL.

Patch by Mike Hommey <mh-llvm@glandium.org>

Reviewers: eugenis

Reviewed By: eugenis

Subscribers: sylvestre.ledru, cfe-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39717

llvm-svn: 317668
2017-11-08 07:25:19 +00:00
Reid Kleckner f7fdac4508 Revert "[scudo] Make getNumberOfCPUs Fuchsia compliant"
This reverts commit r317604.

Android doesn't have cpu_set_t.

llvm-svn: 317655
2017-11-08 01:33:15 +00:00
Evgeniy Stepanov 98fe207986 [sanitizer] Asm implementation of syscall() for arm32.
Summary:
These will be used in an ifunc resolver, when the binary may not be
completely relocated, and syscall() function from libc could not be
used.

Reviewers: dvyukov, vitalybuka

Subscribers: aemerson, kubamracek, javed.absar, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D39701

llvm-svn: 317640
2017-11-08 00:15:12 +00:00
Evgeniy Stepanov 0379d3f844 (NFC) Rename GetMax{,User}VirtualAddress.
Future change will introduce GetMaxVirtualAddress that will not take
the kernel area into account.

llvm-svn: 317638
2017-11-07 23:51:22 +00:00
Vitaly Buka fdc0795098 [sanitizer] Fix sanitizer_common compile errors on gcc trunk
Summary:
When testing a merge of compiler-rt r304709 into gcc trunk on x86-64-pc-linux-gnu,
I ran into two compile errors:
sanitizer_common/sanitizer_symbolizer_libbacktrace.cc:96:73: error: no matching function for call to '__sanitizer::AddressInfo::FillModuleInfo(char*&, __sanitizer::uptr&)'

All other files in sanitizer_common (with the exception of sanitizer_malloc_mac.inc
which is special) include sanitizer_platform.h without directory name.

Patch by Mike Jongen

Reviewers: kcc, vitalybuka

Reviewed By: kcc

Subscribers: kubamracek, fedor.sergeev, llvm-commits

Differential Revision: https://reviews.llvm.org/D33933

llvm-svn: 317608
2017-11-07 20:10:17 +00:00
Kostya Kortchinsky 4e8ce0225f [scudo] Make getNumberOfCPUs Fuchsia compliant
Summary: This change allows Fuchsia to boot properly using the Scudo allocator.

Reviewers: cryptoad, alekseyshl, krytarowski

Reviewed By: cryptoad, krytarowski

Subscribers: krytarowski, kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39490

llvm-svn: 317604
2017-11-07 19:30:08 +00:00
Sam Clegg 12166ea1f2 [WebAssembly] Include GENERIC_TF_SOURCES in wasm builds
Differential Revision: https://reviews.llvm.org/D39748

llvm-svn: 317601
2017-11-07 19:03:11 +00:00
Dmitry Vyukov 3cd4022c39 tsan: allow usage of global vars with ctors in interceptors
We allow usage of global/per-thread data with non-trivial ctors/dtors
throughout tsan code base by placing all global/per-thread data into
Context/ThreadState and then explicitly constructing them with
placement new. This greatly simplifies code by restricting the
"linker initialized plague" to only these 2 objects.

Do the same for interceptors data.

This allows to use Vector instead of bunch of hand-written code in:
https://reviews.llvm.org/D39619

Reviewed in: https://reviews.llvm.org/D39721

llvm-svn: 317587
2017-11-07 16:31:08 +00:00
Kostya Kortchinsky c6f0ade6c3 Update sanitizer_allocator to use new API.
Summary:
Update sanitizer_allocator to use new API.
    
Second patch in a series.  First patch https://reviews.llvm.org/D39072
    
Updates MmapNoAccess / MmapFixed call sites in the saniziter_allocator
to use the new Init/Map APIs instead.


Reviewers: alekseyshl, cryptoad, phosek, mcgrathr, dvyukov

Reviewed By: alekseyshl, cryptoad

Subscribers: dvyukov, mcgrathr, kubamracek

Differential Revision: https://reviews.llvm.org/D38592

llvm-svn: 317586
2017-11-07 16:19:24 +00:00
Alex Shlyapnikov 2093b94e70 [LSan] Detect dynamic loader by its base address.
Summary:
Relanding D38600, which was reverted due to various PPC bot failures.

If it breaks something again, please provide some pointers to broken
bots, not just revert it, otherwise it's very hard to reason what's
wrong with this commit.

Whenever possible (Linux + glibc 2.16+), detect dynamic loader module by
its base address, not by the module name matching. The current name
matching approach fails on some configurations.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39275

llvm-svn: 317512
2017-11-06 21:27:06 +00:00
Alex Shlyapnikov adbf476a88 [Sanitizers] Check pthread_setcancel{state|type} interceptor arguments for != nullptr.
Summary:
According to man, pthread_setcancelstate's oldstate and
pthread_setcanceltype's oldtype parameters can be nullptr.
Check these parameters for != nullptr before attempting to
access their shadow memory.

Reviewers: dvyukov

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39626

llvm-svn: 317494
2017-11-06 17:43:28 +00:00
Mehdi Amini a2a539e4c5 Fix CMake definitions of tsan runtime to make it installed by "install-compiler-rt"
Summary: The PARENT_TARGET was correctly set under APPLE but not under linux.

Reviewers: kubamracek, samsonov

Subscribers: dberris, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39621

llvm-svn: 317391
2017-11-04 00:07:20 +00:00
Kostya Kortchinsky 4a0ebbfe97 [scudo] Rearrange #include order
Summary:
To be compliant with https://llvm.org/docs/CodingStandards.html#include-style,
system headers have to come after local headers.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39623

llvm-svn: 317390
2017-11-03 23:48:25 +00:00
Alex Shlyapnikov 32224fe842 [Sanitizers] Call NanoTime() conditionally.
Summary:
Call NanoTime() in primary 64 bit allocator only when necessary,
otherwise the unwarranted syscall causes problems in sandbox environments.
ReleaseToOSIntervalMs() conditional allows them to turn the feature off
with allocator_release_to_os_interval_ms=-1 flag.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39624

llvm-svn: 317386
2017-11-03 23:31:00 +00:00
Kamil Rytarowski e6b53dad42 Late fixup in _lwp_exit on TSan/NetBSD
Call DestroyThreadState() before REAL(_lwp_exit)();

This variation is less racy.

Sponsored by <The NetBSD Foundation>

llvm-svn: 317369
2017-11-03 21:08:52 +00:00
Kamil Rytarowski 55435b7427 Correct detection of a thread termination
Summary:
Stop using the Linux solution with pthread_key_create(3).
This approach does not work on NetBSD, because calling
the thread destructor is not the latest operation on a POSIX
thread entity. NetBSD's libpthread still calls at least
pthread_mutex_lock and pthread_mutex_unlock.

Detect _lwp_exit(2) call as it is really the latest operation
called from a detaching POSIX thread.

This resolves one set of crashes observed in
the Thread Sanitizer execution.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, kcc, vitalybuka, dvyukov, eugenis

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39618

llvm-svn: 317363
2017-11-03 20:48:19 +00:00
Kamil Rytarowski 99a91f6744 Disable detection of on_exit()/TSan on NetBSD
Summary:
NetBSD does not ship with on_exit() function.

Introduce TSAN_MAYBE_INTERCEPT_ON_EXIT.
It looks like this addition fixes build for Darwin.

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg, eugenis, dvyukov, kcc

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39617

llvm-svn: 317361
2017-11-03 20:39:51 +00:00
Vitaly Buka 7d22324b87 [fuzzer] Fix nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317186
2017-11-02 04:12:10 +00:00
Vitaly Buka ca9075272b [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317119
2017-11-01 20:27:06 +00:00
Kostya Kortchinsky a2cdb2d541 [scudo] Fix standlone build -lrt requirement
Summary:
The split in D39461 introduced separate C++ flags, but `cxx_flags` needs `-lrt` as well for the standalone build.


Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39497

llvm-svn: 317103
2017-11-01 17:00:26 +00:00
Kostya Kortchinsky d937b0a10c [scudo] Implement stricter separation of C vs C++
Summary:
Initially, Scudo had a monolithic design where both C and C++ functions were
living in the same library. This was not necessarily ideal, and with the work
on -fsanitize=scudo, it became more apparent that this needed to change.

We are splitting the new/delete interceptor in their own C++ library. This
allows more flexibility, notably with regard to std::bad_alloc when the work is
done. This also allows us to not link new & delete when using pure C.

Additionally, we add the UBSan runtimes with Scudo, in order to be able to have
a -fsanitize=scudo,undefined in Clang (see work in D39334).

The changes in this patch:
- split the cxx specific code in the scudo cmake file into a new library;
  (remove the spurious foreach loop, that was not necessary)
- add the UBSan runtimes (both C and C++);
- change the test cmake file to allow for specific C & C++ tests;
- make C tests pure C, rename their extension accordingly.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39461

llvm-svn: 317097
2017-11-01 15:28:20 +00:00
Vitaly Buka 7dbc1d8433 [fuzzer] Fix threaded stack printing
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317071
2017-11-01 03:02:59 +00:00
Vitaly Buka df00e89cdc Revert "[fuzzer] Fix threaded stack printing and nested mallocs"
Fails on darwin

Revert "[fuzzer] Script to detect unbalanced allocation in -trace_malloc output"
Needs previous one.

This reverts commit r317034, r317036.

llvm-svn: 317061
2017-11-01 00:55:52 +00:00
Vitaly Buka 0747700ce1 [fuzzer] Script to detect unbalanced allocation in -trace_malloc output
Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39466

llvm-svn: 317036
2017-10-31 20:50:07 +00:00
Vitaly Buka 58da33e35c [fuzzer] Fix threaded stack printing and nested mallocs
Summary: Nested mallocs are possible with internal symbolizer.

Reviewers: kcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39397

llvm-svn: 317034
2017-10-31 20:49:48 +00:00
Petr Hosek 29476f759b [sanitizer][Fuchsia] Add a missing semicolon
Differential Revision: https://reviews.llvm.org/D39433

llvm-svn: 316959
2017-10-30 21:29:26 +00:00
Kostya Kortchinsky 264e1b73eb Fix warning + death test + failing test on Windows (D39072).
Summary: Fixes https://reviews.llvm.org/D39072

Reviewers: cryptoad

Reviewed By: cryptoad

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D39427

llvm-svn: 316943
2017-10-30 19:06:59 +00:00
Kostya Kortchinsky 8c39ccf10c [sanitizer] Fixing an error introduced in D39072
Summary: This should fix the Windows bots after D39072.

Reviewers: alekseyshl, flowerhack

Reviewed By: flowerhack

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39426

llvm-svn: 316937
2017-10-30 18:16:05 +00:00
Kostya Kortchinsky 96da9fa4ca Introduce ReservedAddressRange to sanitizer_common.
Summary:
Fixed version of https://reviews.llvm.org/D38437 (fixes Win/Fuchsia failures).

Creating a new revision, since the old one was getting a bit old/crowded.

    In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
    VMAR, which means that MmapNoAccess can only be called once. This works
    for the sanitizer allocator but *not* for the Scudo allocator.
    
    Hence, this changeset introduces a new ReservedAddressRange object to
    serve as the new API for these calls. In this changeset, the object
    still calls into the old Mmap implementations.
    
    The next changeset two changesets will convert the sanitizer and scudo
    allocators to use the new APIs, respectively. (ReservedAddressRange will
    replace the SecondaryHeader in Scudo.)
    
    Finally, a last changeset will update the Fuchsia implementation.


Reviewers: alekseyshl, cryptoad, phosek

Reviewed By: alekseyshl, cryptoad

Subscribers: kubamracek

Differential Revision: https://reviews.llvm.org/D39072

llvm-svn: 316934
2017-10-30 17:56:24 +00:00
Reid Kleckner 86fca5d058 Fix clang warnings in winasan code
There are two instances of -Wcast-qual and one of -Wsign-compare.

llvm-svn: 316930
2017-10-30 17:26:57 +00:00
Reid Kleckner af86cba0cf [asan] Intercept heap routines in VS2010 CRT
Users have requested that we add it to the list:
https://github.com/google/sanitizers/issues/864

llvm-svn: 316929
2017-10-30 17:26:13 +00:00
Martin Pelikan f5f4c009fe [XRay] [compiler-rt] fix build by including errno.h into FDR mode
The build got broken after D39277 (and rL316816) deleted <cerrno>.

llvm-svn: 316821
2017-10-28 01:35:07 +00:00
Dean Michael Berris e8fec1955a [XRay][compiler-rt] Remove more STL dependenices from FDR mode
Summary:
This change removes dependencies on STL types:

  - std::aligned_storage -- we're using manually-aligned character
    buffers instead for metadata and function records.

  - std::tuple -- use a plain old struct instead.

This is an incremental step in removing all STL references from the
compiler-rt implementation of XRay (llvm.org/PR32274).

Reviewers: dblaikie, pelikan, kpw

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39277

llvm-svn: 316816
2017-10-27 23:59:41 +00:00
Kostya Kortchinsky 91b7558ca8 [scudo] Allow to specify the maximum number of TSDs at compile time
Summary:
This introduces `SCUDO_MAX_CACHES` allowing to define an upper bound to the
number of `ScudoTSD` created in the Shared TSD model (by default 32U).
This name felt clearer than `SCUDO_MAX_TSDS` which is technically what it really
is. I am opened to suggestions if that doesn't feel right.

Additionally change `getNumberOfCPUs` to return a `u32` to be more consistent.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39338

llvm-svn: 316788
2017-10-27 20:10:14 +00:00
Alex Shlyapnikov f280eff080 [LSan] Disable a couple of failing tests on PPC64 (pending investigation).
llvm-svn: 316720
2017-10-27 00:29:40 +00:00
Alex Shlyapnikov e4b9ae666e [LSan] Enable LSan tests on PPC64 Linux.
Summary:
LSan is functional on PPC64 Linux now, let's enable all tests.

One test required ppc specific changes: use_registers.cc.

Reviewers: eugenis

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D39316

llvm-svn: 316698
2017-10-26 20:59:04 +00:00
Alex Shlyapnikov 9b4e32785a [Sanitizers] Set default allocator_release_to_os_interval_ms to 5 seconds
Summary:
With new release to OS approach (see D38245) it's reasonable to enable
it by default. Setting allocator_release_to_os_interval_ms to 5000 seems
to be a reasonable default (might be tuned later, based on the
feedback).

Also delaying the first release to OS in each bucket for at least
allocator_release_to_os_interval_ms after the first allocation to
prevent just allocated memory to be madvised back to OS and let short
lived processes to avoid release to OS overhead altogether.

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D39318

llvm-svn: 316683
2017-10-26 17:59:24 +00:00
Vitaly Buka f3e0f76196 [sanitizer] Fix internal symbolized build on Debian 9
llvm-svn: 316637
2017-10-26 06:08:01 +00:00
Alex Shlyapnikov 02bda37492 [LSan] Adjust LSan allocator limits for PPC64.
Summary: Now the limits are the same as for ASan allocator.

Reviewers: cryptoad

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39309

llvm-svn: 316633
2017-10-26 01:22:48 +00:00
Kostya Serebryany 3c9e59d0f6 [libFuzzer] trying to make a test more stable on Mac
llvm-svn: 316627
2017-10-25 23:24:45 +00:00
Kostya Kortchinsky f9008a3a06 [scudo] Remove comment about security of the 32-bit allocator
Summary:
The 32-bit allocator is now on par with the 64-bit in terms of security (chunks
randomization is done, batches separation is done).

Unless objection, the comment can go away.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39303

llvm-svn: 316620
2017-10-25 22:00:26 +00:00
Evgeniy Stepanov 0b8602791b [msan] Intercept __strxfrm_l.
llvm-svn: 316613
2017-10-25 21:40:17 +00:00
Kamil Rytarowski 175e775e0f Try to unbreak Linux sanitizers
Add fallback definition of internal_syscall_ptr and internal_syscall64
for Linux/x86_64 and Linux/aarch64.

llvm-svn: 316598
2017-10-25 17:56:36 +00:00
Kostya Kortchinsky c484912b06 [sanitizer] Random shuffling of chunks for the 32-bit Primary Allocator
Summary:
The 64-bit primary has had random shuffling of chunks for a while, this
implements it for the 32-bit primary. Scudo is currently the only user of
`kRandomShuffleChunks`.

This change consists of a few modifications:
- move the random shuffling functions out of the 64-bit primary to
  `sanitizer_common.h`. Alternatively I could move them to
  `sanitizer_allocator.h` as they are only used in the allocator, I don't feel
  strongly either way;
- small change in the 64-bit primary to make the `rand_state` initialization
  `UNLIKELY`;
- addition of a `rand_state` in the 32-bit primary's `SizeClassInfo` and
  shuffling of chunks when populating the free list.
- enabling the `random_shuffle.cpp` test on platforms using the 32-bit primary
  for Scudo.

Some comments on why the shuffling is done that way. Initially I just
implemented a `Shuffle` function in the `TransferBatch` which was simpler but I
came to realize this wasn't good enough: for chunks of 10000 bytes for example,
with a `CompactSizeClassMap`, a batch holds only 1 chunk, meaning shuffling the
batch has no effect, while a region is usually 1MB, eg: 104 chunks of that size.
So I decided to "stage" the newly gathered chunks in a temporary array that
would be shuffled prior to placing the chunks in batches.
The result is looping twice through n_chunks even if shuffling is not enabled,
but I didn't notice any significant significant performance impact.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39244

llvm-svn: 316596
2017-10-25 17:24:56 +00:00
Alex Shlyapnikov a53b55f66c [Sanitizers] ASan: detect new/delete calls with mismatched alignment.
ASan allocator stores the requested alignment for new and new[] calls
and on delete and delete[] verifies that alignments do match.

The representable alignments are: default alignment, 8, 16, 32, 64, 128,
256 and 512 bytes. Alignments > 512 are stored as 512, hence two
different alignments > 512 will pass the check (possibly masking the bug),
but limited memory requirements deemed to be a resonable tradeoff for
relaxed conditions.

The feature is controlled by new_delete_type_mismatch flag, the same one
protecting new/delete matching size check.

Differential revision: https://reviews.llvm.org/D38574

Issue: https://github.com/google/sanitizers/issues/799
llvm-svn: 316595
2017-10-25 17:21:37 +00:00
Kamil Rytarowski 9c1eeaca80 Add NetBSD improvements in sanitizers
Summary:
Changes:

 * Add initial msan stub support.
 * Handle NetBSD specific pthread_setname_np(3).
 * NetBSD supports __attribute__((tls_model("initial-exec"))),
   define it in SANITIZER_TLS_INITIAL_EXEC_ATTRIBUTE.
 * Add ReExec() specific bits for NetBSD.
 * Simplify code and add syscall64 and syscall_ptr for !NetBSD.
 * Correct bunch of syscall wrappers for NetBSD.
 * Disable test/tsan/map32bit on NetBSD as not applicable.
 * Port test/tsan/strerror_r to a POSIX-compliant OSes.
 * Disable __libc_stack_end on NetBSD.
 * Disable ReadNullSepFileToArray() on NetBSD.
 * Define struct_ElfW_Phdr_sz, detected missing symbol by msan.
 * Change type of __sanitizer_FILE from void to char. This helps
   to reuse this type as an array. Long term it will be properly
   implemented along with SANITIZER_HAS_STRUCT_FILE setting to 1.
 * Add initial NetBSD support in lib/tsan/go/buildgo.sh.
 * Correct referencing stdout and stderr in tsan_interceptors.cc
   on NetBSD.
 * Document NetBSD x86_64 specific virtual memory layout in
   tsan_platform.h.
 * Port tests/rtl/tsan_test_util_posix.cc to NetBSD.
 * Enable NetBSD tests in test/msan/lit.cfg.
 * Enable NetBSD tests in test/tsan/lit.cfg.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis, kcc, dvyukov

Reviewed By: dvyukov

Subscribers: #sanitizers, llvm-commits, kubamracek

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D39124

llvm-svn: 316591
2017-10-25 17:09:05 +00:00
Reid Kleckner 03d02a0e1d [asan] Don't print rows of shadow bytes outside shadow memory
Summary:
They might not be mapped on some platforms such as Win64.  In
particular, this happens if the user address is null. There will not be
any shadow memory 5*16 bytes before the user address. This happens on
Win64 in the error_report_callback.cc test case. It's not clear why this
isn't a problem on Linux as well.

Fixes PR35058

Reviewers: vitalybuka

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39260

llvm-svn: 316589
2017-10-25 16:54:12 +00:00
Dmitry Vyukov 60d8ad2849 [tsan] Fix warnings in tsan_interceptors.cc from expansion of variadic macros
C99 technically requires the rest arguments to be used in C variadic macros.
This presents a problem with the macro SCOPED_TSAN_INTERCEPTOR when func
takes no arguments. This happens with the function pause. Like other void
argument functions, we pass in a fake argument to avoid this warning.

Author: Alex Langford (xiaobai)
Reviewed in: https://reviews.llvm.org/D39151

llvm-svn: 316558
2017-10-25 08:05:13 +00:00
Dean Michael Berris 389ebc0c4f [XRay][compiler-rt][NFC] Clean up xray log files before running test
Improves the test behaviour in the face of failure. Without this change
the fdr-single-thread.cc test may leave around artefacts of a previous
failing run since the cleanup doesn't happen if any of the intermediary
steps fail.

Non-functional change.

Subscribers: llvm-commits
llvm-svn: 316548
2017-10-25 05:19:20 +00:00
Dimitry Andric 95af9e654f On FreeBSD, skip the first entry in the dl_iterate_phdr list.
Summary:
Similar to NetBSD, in FreeBSD, the first returned entry when callbacks
are done via dl_iterate_phdr will return the main program.  Ignore that
entry when checking that the dynamic ASan lib is loaded first.

Reviewers: eugenis, krytarowski, emaste, joerg

Reviewed By: eugenis, krytarowski

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39253

llvm-svn: 316487
2017-10-24 19:45:59 +00:00
Reid Kleckner 80db37ec25 [asan] Add more x86 prologue decodings to handle x64 VC 2017 CRT
Fixes atoi and strtol interception.

llvm-svn: 316482
2017-10-24 18:22:07 +00:00
Dean Michael Berris 23e54d85be [XRay][compiler-rt] More fixups.
Follow-up to D39175.

llvm-svn: 316410
2017-10-24 02:43:49 +00:00
Dean Michael Berris 2e592ea7ed [XRay][compiler-rt] Fixup shadowing
Follow-up to D39175.

llvm-svn: 316409
2017-10-24 02:36:32 +00:00
Dean Michael Berris 0b1cce2036 [XRay][compiler-rt] Remove C++ STL from the buffer queue implementation
Summary:
This change removes the dependency on C++ standard library
types/functions in the implementation of the buffer queue. This is an
incremental step in resolving llvm.org/PR32274.

Reviewers: dblaikie, pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39175

llvm-svn: 316406
2017-10-24 01:39:59 +00:00
Alex Shlyapnikov 5ded0701a9 [Sanitizers-libFuzzer] Addressing coding style issues.
Summary: The result of clang-format and few manual changes (as prompted on D39155).

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39211

llvm-svn: 316395
2017-10-23 23:24:33 +00:00
Alex Shlyapnikov 6f1c26f222 [libFuzzer] Periodically purge allocator's quarantine to prolong fuzzing sessions.
Summary:
Fuzzing targets that allocate/deallocate a lot of memory tend to consume
a lot of RSS when ASan quarantine is enabled. Purging quarantine between
iterations and returning memory to OS keeps RSS down and should not
reduce the quarantine effectiveness provided the fuzz target does not
preserve state between iterations (in this case this feature can be turned off).

Based on D39153.

Reviewers: vitalybuka

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D39155

llvm-svn: 316382
2017-10-23 22:04:30 +00:00
Alex Shlyapnikov 1108d072f4 [Sanitizers] Add total primary allocator RSS to allocator report.
Summary: .

Reviewers: cryptoad

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D39131

llvm-svn: 316356
2017-10-23 17:58:16 +00:00
Evgeniy Stepanov aedda3bafc Revert "[Compiler-rt][MIPS] Fix cross build for XRAY."
Breaks build:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux/builds/4677/steps/build%20with%20ninja/logs/stdio

In file included from compiler-rt/lib/xray/xray_fdr_logging.cc:34:
In file included from compiler-rt/lib/xray/xray_fdr_logging_impl.h:36:
In file included from compiler-rt/lib/xray/xray_flags.h:18:
compiler-rt/lib/xray/../sanitizer_common/sanitizer_flag_parser.h:23:7: error: '__sanitizer::FlagHandlerBase' has virtual functions but non-virtual destructor [-Werror,-Wnon-virtual-dtor]
class FlagHandlerBase {

llvm-svn: 316348
2017-10-23 17:13:24 +00:00
Alex Shlyapnikov 028c4cddad [Sanitizers] New sanitizer API to purge allocator quarantine.
Summary:
Purging allocator quarantine and returning memory to OS might be desired
between fuzzer iterations since, most likely, the quarantine is not
going to catch bugs in the code under fuzz, but reducing RSS might
significantly prolong the fuzzing session.

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D39153

llvm-svn: 316347
2017-10-23 17:12:07 +00:00
Kostya Kortchinsky 9e917a13fb [scudo] Add a shared runtime
Summary:
Up to now, the Scudo cmake target only provided a static library that had to be
linked to an executable to benefit from the hardened allocator.
This introduces a shared library as well, that can be LD_PRELOAD'ed.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38980

llvm-svn: 316342
2017-10-23 16:27:47 +00:00
Nitesh Jain cf8a5c26f9 [Compiler-rt][MIPS] Fix cross build for XRAY.
Reviewers: dberris, sdardis

Subscribers: jaydeep, bhushan, llvm-commits

Differential Revision: https://reviews.llvm.org/D38021

llvm-svn: 316286
2017-10-22 09:37:50 +00:00
Dmitry Vyukov 3d34d877cb tsan: add tests missed in r316209
llvm-svn: 316210
2017-10-20 12:10:21 +00:00
Dmitry Vyukov 9e2cd1c125 [tsan] Add Mutex annotation flag for constant-initialized __tsan_mutex_linker_init behavior
Add a new flag, _⁠_tsan_mutex_not_static, which has the opposite sense
of _⁠_tsan_mutex_linker_init. When the new _⁠_tsan_mutex_not_static flag
is passed to _⁠_tsan_mutex_destroy, tsan ignores the destruction unless
the mutex was also created with the _⁠_tsan_mutex_not_static flag.

This is useful for constructors that otherwise woud set
_⁠_tsan_mutex_linker_init but cannot, because they are declared constexpr.

Google has a custom mutex with two constructors, a "linker initialized"
constructor that relies on zero-initialization and sets
⁠_⁠_tsan_mutex_linker_init, and a normal one which sets no tsan flags.
The "linker initialized" constructor is morally constexpr, but we can't
declare it constexpr because of the need to call into tsan as a side effect.

With this new flag, the normal c'tor can set _⁠_tsan_mutex_not_static,
the "linker initialized" constructor can rely on tsan's lazy initialization,
and _⁠_tsan_mutex_destroy can still handle both cases correctly.

Author: Greg Falcon (gfalcon)
Reviewed in: https://reviews.llvm.org/D39095

llvm-svn: 316209
2017-10-20 12:08:53 +00:00
Marco Castelluccio 17103fc12f Use O_BINARY when opening GCDA file on Windows
Summary:
Fixes https://bugs.llvm.org/show_bug.cgi?id=34922.

Apparently, the mode in **fdopen** gets simply ignored and Windows only cares about the mode of the original **open**.

I have verified this both with the simple case from bug 34922 and with a full Firefox build.

Reviewers: zturner

Reviewed By: zturner

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38984

llvm-svn: 316048
2017-10-18 00:22:01 +00:00
Evgeniy Stepanov db8d53cdad Revert "[asan] Deflake one test by running it 3 times."
Disable this test on Android/x86 only.

llvm-svn: 316023
2017-10-17 19:57:48 +00:00
Dean Michael Berris 1d2f9e2bdd [XRay][compiler-rt] Always place the CPU record first for every buffer
Summary:
In FDR Mode, when we set up a new buffer for a thread that's just
overflowed, we must place the CPU identifier with the TSC record as the
first record. This is so that we can reconstruct all the function
entry/exit with deltas rooted on a TSC record for the CPU at the
beginning of the buffer.

Without doing this, the tools are rejecting the log for cases when we've
overflown and have different buffers that don't have the CPU and TSC
records as the first entry in the buffers.

Reviewers: pelikan

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38995

llvm-svn: 315987
2017-10-17 10:33:24 +00:00
Evgeniy Stepanov 1993de54ee [cfi] Test cross-dso CFI on Android.
Reviewers: vitalybuka, pcc

Subscribers: llvm-commits, srhines

Differential Revision: https://reviews.llvm.org/D38911

llvm-svn: 315922
2017-10-16 18:03:11 +00:00
Kostya Kortchinsky 73a80c5493 [scudo] Do not include sanitizer_posix.h if not on a Posix platform
Summary:
Move the `sanitizer_posix.h` include within the `SANITIZER_ANDROID` `#if`,
otherwise this errors when built on non-Posix platforms (eg: Fuchsia).

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D38956

llvm-svn: 315917
2017-10-16 17:06:13 +00:00
Vitaly Buka ac03fb616f [asan] Increase kHandlerStackSize for TracerThreadSignalHandler
4096 is not enough on some platform, e.g. Debian 4.9.0-3-amd64

llvm-svn: 315844
2017-10-15 04:18:29 +00:00
George Karpenkov 1b11460610 [xray] Fix CMake for X-RAY tests
Correctly depend on llvm-xray, make sure unit tests are being run.

Differential Revision: https://reviews.llvm.org/D38917

llvm-svn: 315827
2017-10-14 21:38:13 +00:00
Kostya Serebryany fc3079963f [libFuzzer] print a better warning if we hit the ld bug
llvm-svn: 315778
2017-10-14 00:07:11 +00:00
Evgeniy Stepanov 4fb209ede2 [asan] Deflake one test by running it 3 times.
The test seems to trigger an android platform bug under load.

llvm-svn: 315777
2017-10-13 23:57:08 +00:00
Evgeniy Stepanov f156c932ae [cfi] Use %ld_flags_rpath_* substitutions in tests (NFC)
llvm-svn: 315775
2017-10-13 23:33:12 +00:00
Alex Shlyapnikov 29d3ba7576 [Sanitizers] Always build libFuzzer with frame pointers.
Lacking frame pointers, sanitized fuzzers collect bogus stack
traces and the set of stack traces grows indefinitely, leading to OOMs.

llvm-svn: 315770
2017-10-13 21:57:43 +00:00
Peter Collingbourne 868783e855 LowerTypeTests: Give imported symbols a type with size 0 so that they are not assumed not to alias.
It is possible for both a base and a derived class to be satisfied
with a unique vtable. If a program contains casts of the same pointer
to both of those types, the CFI checks will be lowered to this
(with ThinLTO):

if (p != &__typeid_base_global_addr)
  trap();
if (p != &__typeid_derived_global_addr)
  trap();

The optimizer may then use the first condition combined
with the assumption that __typeid_base_global_addr and
__typeid_derived_global_addr may not alias to optimize away the second
comparison, resulting in an unconditional trap.

This patch fixes the bug by giving imported globals the type [0 x i8]*,
which prevents the optimizer from assuming that they do not alias.

Differential Revision: https://reviews.llvm.org/D38873

llvm-svn: 315753
2017-10-13 21:02:16 +00:00
Kostya Kortchinsky f4c11e353a [scudo] Allow for non-Android Shared TSD platforms, part 2
Summary:
Follow up to D38826.

We introduce `pthread_{get,set}specific` versions of `{get,set}CurrentTSD` to
allow for non Android platforms to use the Shared TSD model.
We now allow `SCUDO_TSD_EXCLUSIVE` to be defined at compile time.

A couple of things:
- I know that `#if SANITIZER_ANDROID` is not ideal within a function, but in
  the end I feel it looks more compact and clean than going the .inc route; I
  am open to an alternative if anyone has one;
- `SCUDO_TSD_EXCLUSIVE=1` requires ELF TLS support (and not emutls as this uses
  malloc). I haven't found anything to enforce that, so it's currently not
  checked.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, llvm-commits

Differential Revision: https://reviews.llvm.org/D38854

llvm-svn: 315751
2017-10-13 20:55:31 +00:00
Alex Shlyapnikov 8a8d945df7 [Sanitizers] Add more details to ASan allocator stats report.
Summary: .

Reviewers: cryptoad

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D38834

llvm-svn: 315730
2017-10-13 18:38:10 +00:00
Kostya Serebryany 51823d3aae [libFuzzer] tweam use_feature_frequency to be less aggressive; run a dummy input before the seed corpus
llvm-svn: 315657
2017-10-13 01:12:23 +00:00
Evgeniy Stepanov 150b7d6f55 [sanitizer] Workaround a Linux kernel bug in hard_rss_limit_mb_test.
llvm-svn: 315632
2017-10-12 22:24:58 +00:00
Michal Gorny 4e9e939b4f [cmake] [asan] Remove unnecessary gtest dep from dynamic tests
Remove the redundant dependency on 'gtest' target from the dynamic tests
in non-MSVC environment. The tests reuse compiled objects
from ASAN_INST_TEST_OBJECTS, and therefore they have been built against
gtest already.

This both fixes the spurious dependency on 'gtest' target that breaks
stand-alone builds, and brings the dynamic tests more in line with
regular tests which do not pass this dependency
to add_compiler_rt_test() through generate_compiler_rt_tests().

Differential Revision: https://reviews.llvm.org/D38840

llvm-svn: 315620
2017-10-12 21:07:43 +00:00
Michal Gorny 007b540feb [cmake] [interception] Remove duplicate gtest from test COMPILE_DEPS
Fix the gtest dependency to be included in DEPS only, rather than
in COMPILE_DEPS + DEPS. The former variable is apparently used to
provide unconditional dependencies, while the latter are only used
for non-standalone builds. Since they are concatenated, specifying gtest
in both is redundant. Furthermore, including it in COMPILE_DEPS causes
build failure for standalone builds where 'gtest' target is not present.

Differential Revision: https://reviews.llvm.org/D38839

llvm-svn: 315605
2017-10-12 18:51:41 +00:00
Michal Gorny 858509cfb3 [cmake] Fix skipping DEPS (typo) in sanitizer_test_compile()
Fix typo in variable assignment inside sanitizer_test_compile() that
resulted in TEST_DEPS parameter not being included in the clang_compile()
call. Spotted by George Karpenkov in D38444.

Differential Revision: https://reviews.llvm.org/D38838

llvm-svn: 315604
2017-10-12 18:51:37 +00:00
Evgeniy Stepanov fe35898b42 [asan] Disable a flaky test on android.
llvm-svn: 315602
2017-10-12 18:34:20 +00:00
Kostya Kortchinsky 8d4ba5fd23 [scudo] Allow for non-Android Shared TSD platforms, part 1
Summary:
This first part just prepares the grounds for part 2 and doesn't add any new
functionality. It mostly consists of small refactors:
- move the `pthread.h` include higher as it will be used in the headers;
- use `errno.h` in `scudo_allocator.cpp` instead of the sanitizer one, update
  the `errno` assignments accordingly (otherwise it creates conflicts on some
  platforms due to `pthread.h` including `errno.h`);
- introduce and use `getCurrentTSD` and `setCurrentTSD` for the shared TSD
  model code;

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: llvm-commits, srhines

Differential Revision: https://reviews.llvm.org/D38826

llvm-svn: 315583
2017-10-12 15:01:09 +00:00
Hans Wennborg eb2218ceae Revert r315533 "Reland "[sanitizer] Introduce ReservedAddressRange to sanitizer_common""
The SanitizerCommon.ReservedAddressRangeUnmap test fails on Windows:

FAIL: SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap (34003 of 35554)
******************** TEST 'SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap' FAILED ********************
Note: Google Test filter = SanitizerCommon.ReservedAddressRangeUnmap
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from SanitizerCommon
[ RUN      ] SanitizerCommon.ReservedAddressRangeUnmap
==3780==ERROR: SanitizerTool failed to deallocate 0x1000 (4096) bytes at address 0x0000000c3000 (error code: 487)
==3780==Sanitizer CHECK failed: E:\b\build\slave\win_upload_clang\build\src\third_party\llvm\projects\compiler-rt\lib\sanitizer_common\sanitizer_win.cc:129 (("unable to unmap" && 0)) != (0) (0, 0)
********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
Testing Time: 299.76s
********************
Failing Tests (1):
    SanitizerCommon-Unit :: ./Sanitizer-x86_64-Test.exe/SanitizerCommon.ReservedAddressRangeUnmap

> In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
> VMAR, which means that MmapNoAccess can only be called once. This works
> for the sanitizer allocator but *not* for the Scudo allocator.
>
> Hence, this changeset introduces a new ReservedAddressRange object to
> serve as the new API for these calls. In this changeset, the object
> still calls into the old Mmap implementations.
>
> The next changeset two changesets will convert the sanitizer and scudo
> allocators to use the new APIs, respectively. (ReservedAddressRange will
> replace the SecondaryHeader in Scudo.)
>
> Finally, a last changeset will update the Fuchsia implementation.
>
> Patch by Julia Hansbrough
>
> Differential Revision: https://reviews.llvm.org/D38437

llvm-svn: 315553
2017-10-12 03:23:31 +00:00
Petr Hosek 4f213ae3a7 Reland "[sanitizer] Introduce ReservedAddressRange to sanitizer_common"
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.

Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.

The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)

Finally, a last changeset will update the Fuchsia implementation.

Patch by Julia Hansbrough

Differential Revision: https://reviews.llvm.org/D38437

llvm-svn: 315533
2017-10-11 23:41:32 +00:00
Evgeniy Stepanov 1961c6cb2c Disable TSan tests on Android.
They never passed. This change excludes them from 'check-all'.

llvm-svn: 315512
2017-10-11 21:22:45 +00:00
Evgeniy Stepanov 3747b826c4 Disable profile tests on Android.
They never actually worked, but this way they are not included in "check-all".

llvm-svn: 315511
2017-10-11 21:22:32 +00:00
Kostya Kortchinsky 6a45f9ce82 [sanitizer] Move the errno/ENOMEM allocator checks logic to separate .cc
Summary:
This is a new attempt at D38706, which had 2 issues.

The first one was that it broke TSan, because `sanitizer_errno.h` was not
directly included in `tsan_mman.cc`. This fixes the include.

The second one was that it broke the nolibc build, because `__errno_location`
couldn't be found. This adds the new .cc to the libcdep list instead of the
base one.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38743

llvm-svn: 315509
2017-10-11 21:20:04 +00:00
Evgeniy Stepanov 450ae592b4 [ubsan] Support ubsan-minimal tests on Android.
llvm-svn: 315507
2017-10-11 20:46:13 +00:00
Craig Topper 705b969f80 [X86] Add Knights Mill CPU to cpu_indicator support to match libgcc.
llvm-svn: 315505
2017-10-11 20:35:43 +00:00
Kostya Serebryany 75480e3871 [libFuzzer] disable use_feature_frequency as it degrades some of the benchmarks too much :(
llvm-svn: 315503
2017-10-11 20:31:01 +00:00
Petr Hosek c530f497b8 Revert "[sanitizer] Introduce ReservedAddressRange to sanitizer_common"
This reverts commit r315493 which is failing to build on sanitizer-windows.

llvm-svn: 315494
2017-10-11 19:29:14 +00:00
Petr Hosek 2bd0123afc [sanitizer] Introduce ReservedAddressRange to sanitizer_common
In Fuchsia, MmapNoAccess/MmapFixedOrDie are implemented using a global
VMAR, which means that MmapNoAccess can only be called once. This works
for the sanitizer allocator but *not* for the Scudo allocator.

Hence, this changeset introduces a new ReservedAddressRange object to
serve as the new API for these calls. In this changeset, the object
still calls into the old Mmap implementations.

The next changeset two changesets will convert the sanitizer and scudo
allocators to use the new APIs, respectively. (ReservedAddressRange will
replace the SecondaryHeader in Scudo.)

Finally, a last changeset will update the Fuchsia implementation.

Patch by Julia Hansbrough

Differential Revision: https://reviews.llvm.org/D38759

llvm-svn: 315493
2017-10-11 19:17:35 +00:00
Kostya Serebryany 198d385a77 [libFuzzer] make -use_feature_frequency less aggressive and enable by default
llvm-svn: 315490
2017-10-11 19:01:35 +00:00
Evgeniy Stepanov b751082d56 [asan] Tweak test output to diagnose buildbot failures.
llvm-svn: 315479
2017-10-11 17:32:38 +00:00
Kostya Serebryany 4083d54675 [libFuzzer] experimental flag to tweak the corpus distribution. Seems to improve the situation dramatically on the png benchmark and make things worse on a number of micro-puzzles. Needs more A/B testing
llvm-svn: 315407
2017-10-11 01:44:26 +00:00
Evgeniy Stepanov 922973a466 [sanitizer] Re-disable several tests on Android.
The tests have been enabled by accident in r315389.

llvm-svn: 315396
2017-10-11 00:32:03 +00:00
Evgeniy Stepanov 0bf7717a02 Factor out "stable-runtime" feature and enable it on all android.
This is a very poorly named feature. I think originally it meant to cover linux only, but the use of it in msan
seems to be about any aarch64 platform. Anyway, this change should be NFC on everything except Android.

llvm-svn: 315389
2017-10-10 23:37:26 +00:00
Hans Wennborg 4a6ee430bb XFAIL ubsan/TestCases/TypeCheck/Function/function.cpp on Windows
I think it got accidentally enabled in r315105 or thereabouts.

llvm-svn: 315374
2017-10-10 22:11:25 +00:00
Kostya Kortchinsky ea2f800bbc [sanitizer] Revert D38706
Summary:
D38706 breaks tsan and the nolibc build.
Reverting while working on a fix.

Reviewers: alekseyshl

Subscribers: kubamracek, mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D38739

llvm-svn: 315320
2017-10-10 15:35:11 +00:00
Kostya Kortchinsky 03f9e76790 [sanitizer] Move the errno/ENOMEM allocator checks logic to separate .cc
Summary:
The fact that `sanitizer_allocator_checks.h` is including `sanitizer_errno.h`
creates complications for future changes, where it would conflict with `errno.h`
definitions on Android and Fuchsia (macro redefinition).

By moving the portion that sets errno in the checks to a separate compilation
unit, we avoid the inclusion of the header there, which solves the issue.

Not that it is not vital to have that function in a header as it is called as a
result of an unlikely event, and doesn't need to be inlined.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38706

llvm-svn: 315319
2017-10-10 14:58:09 +00:00
Dean Michael Berris 9ba7401a7b [XRay][compiler-rt] Fix rdtscp support check for x86_64
Follow-up to D29438.

llvm-svn: 315306
2017-10-10 12:44:20 +00:00
Evgeniy Stepanov cb993304ac [ubsan] Fix Asan internal alloc corruption in PR33221 test.
MAP_FIXED discards the existing mapping at the given address.

llvm-svn: 315247
2017-10-09 22:52:13 +00:00
Petr Hosek 1c0c35d034 [sanitizer] Don't intercept signal and sigaction on Fuchsia
Fuchsia doesn't support signals, so don't use interceptors for signal or
sigaction.

Differential Revision: https://reviews.llvm.org/D38669

llvm-svn: 315227
2017-10-09 18:29:52 +00:00
Evgeniy Stepanov 8804483b05 [ubsan] Disable one test on Android.
llvm-svn: 315220
2017-10-09 17:45:03 +00:00
Vlad Tsyrklevich a0b02833bc Limit CFI blacklist entries to specific CFI mode
Summary:
Since D37924 and D37925 were merged, it's now possible to specify
individual sanitizers or CFI modes in sanitizer blacklists. Update the
CFI blacklist entries to only apply to cfi-unrelated-cast checks.

Reviewers: eugenis, pcc

Reviewed By: eugenis

Subscribers: kcc

Differential Revision: https://reviews.llvm.org/D38385

llvm-svn: 315216
2017-10-09 17:11:44 +00:00
Peter Collingbourne 107a9f624d Make the cfi target available on more platforms.
On non-Linux targets it just installs the blacklist.

Differential Revision: https://reviews.llvm.org/D38661

llvm-svn: 315215
2017-10-09 17:07:47 +00:00
Vedant Kumar 2465e64846 cmake: Fix one more usage of append()
append() isn't available with some cmake versions, so I need to use a
different construct. I missed this case in r315144.

http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA/39355

llvm-svn: 315157
2017-10-07 20:20:42 +00:00
Vedant Kumar 16d41609ab Use list(APPEND) instead of append()
append() seems to be available with the version of cmake I'm using, but
not on the bots:

http://green.lab.llvm.org/green//job/clang-stage1-configure-RA/39354

llvm-svn: 315144
2017-10-07 03:44:46 +00:00
Vedant Kumar 074a997b13 [ubsan] Add a static runtime on Darwin
As a follow-up to r315142, this makes it possible to use ubsan with a
static runtime on Darwin. I've also added a new StandaloneStatic testing
configuration so the new setup can be tested.

llvm-svn: 315143
2017-10-07 01:46:36 +00:00
Evgeniy Stepanov 729bcd1f6e [asan] Disable wcslen test on 32-bit Android.
llvm-svn: 315132
2017-10-07 00:04:24 +00:00
Evgeniy Stepanov 0a4217c14b Factor out default_(a|ub)sanitizer_opts in lit.
Reviewers: vitalybuka

Subscribers: srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D38644

llvm-svn: 315106
2017-10-06 20:53:40 +00:00
Evgeniy Stepanov f29ee9a050 [sanitizer] Test ubsan and cfi on android.
Summary:
Enable check-cfi and check-ubsan on Android.
Check-ubsan includes standalone and ubsan+asan, but not tsan or msan.
Cross-dso cfi tests are disabled for now.

Reviewers: vitalybuka, pcc

Subscribers: srhines, kubamracek, llvm-commits, mgorny

Differential Revision: https://reviews.llvm.org/D38608

llvm-svn: 315105
2017-10-06 20:51:51 +00:00
Evgeniy Stepanov 505260449e [asan] Use gold linker in android tests.
Replace a partial workaround for ld.bfd strangeness with the ultimate one: -fuse-ld=gold.
Reason: ld.bfd problem gets worse with libc++-based NDK toolchain.

llvm-svn: 315039
2017-10-06 00:00:53 +00:00