Commit Graph

8687 Commits

Author SHA1 Message Date
Kostya Serebryany d085748484 [libFuzzer] a bit of refactoring of the fork mode
llvm-svn: 353910
2019-02-13 04:04:45 +00:00
Hubert Tong a9f91c810c [xray][tests][RHDTS] Add -lstdc++ after LLVM libs, resolving link error with RHDTS
Summary:
A link error was encountered when using the Red Hat Developer Toolset.
In the RHDTS, `libstdc++.so` is a linker script that may resolve symbols
to a static library. This patch places `-lstdc++` later in the ordering.

Reviewers: sfertile, nemanjai, tstellar, dberris

Reviewed By: dberris

Subscribers: dberris, mgorny, delcypher, jdoerfert, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D58144

llvm-svn: 353905
2019-02-13 00:55:30 +00:00
Evgeniy Stepanov 8bb5d7e76a [tsan] #undef one-letter macros
buildgo.sh puts most of sanitizer_common together in a single source
file. These single-letter macros end up affecting a lot of unrelated
code; #undef them as early as possible.

llvm-svn: 353902
2019-02-12 23:49:02 +00:00
Kostya Serebryany 5c08e811de [libFuzzer] move the implementation of the fork mode into a separate file
llvm-svn: 353891
2019-02-12 22:48:55 +00:00
Dmitry Vyukov 19e41fb0ca tsan: update check_analyze.sh
Update numbers after 353817.
This was extensively benchmarked in https://reviews.llvm.org/D54889

llvm-svn: 353820
2019-02-12 10:18:27 +00:00
Dmitry Vyukov 6e7089ad40 tsan: add fiber support
This patch adds functions for managing fibers:

__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()

See the added tests for use examples.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889

llvm-svn: 353817
2019-02-12 10:11:58 +00:00
Dmitry Vyukov ca524b19c1 tsan: Introduce in_symbolizer() function for Thread sanitizer
This change is preparation for fiber support.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D58104

llvm-svn: 353805
2019-02-12 08:11:07 +00:00
Kostya Serebryany 2b9a8f37a4 [libFuzzer] make the fork mode less verbose
llvm-svn: 353794
2019-02-12 03:12:40 +00:00
Kostya Serebryany cdbb9dc962 [libFuzzer] teach the fork mode to ignore OOMs and timeouts
llvm-svn: 353792
2019-02-12 02:18:53 +00:00
Petr Hosek 26eec703f7 [sanitizer] Don't compile GetPathAssumingFileIsRelativeToExec on Fuchsia
This avoids the unused function warning during compilation.

Differential Revision: https://reviews.llvm.org/D58082

llvm-svn: 353787
2019-02-12 01:36:08 +00:00
Kostya Serebryany 73bf6d5b35 [libFuzzer] replace slow std::mt19937 with a much faster std::minstd_rand; second attempt after failed r352732, this time with a fix for cmake
llvm-svn: 353782
2019-02-12 01:00:08 +00:00
Kostya Serebryany 56b2d57cbf [libFuzzer] simplify the code for print_coverage=1 so that it doesn't fail on broken debug info
llvm-svn: 353781
2019-02-12 00:52:11 +00:00
Kostya Serebryany 63f48717b5 [libFuzzer] extend the -fork=1 functionality. Still not fully usable, but good enough for the first unit test
llvm-svn: 353775
2019-02-12 00:12:33 +00:00
Vitaly Buka 86a8d50965 tsan: Implement pthread_exit() interceptor for Thread sanitizer
This change is preparation for fiber support.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57876
Context: https://reviews.llvm.org/D54889

> llvm-svn: 353385

llvm-svn: 353627
2019-02-09 22:08:31 +00:00
Vitaly Buka 2abcb30f8c [tsan] Remove pthread_exit interceptor
Crashes PPC bot

llvm-svn: 353604
2019-02-09 09:58:57 +00:00
David L. Jones 6218b8e783 [Fuzzer] Fix function prototype in fuzzer::ExternalFunctions. [NFC]
The __sanitizer_acquire_crash_state function has int return type, but the
fuzzer's external function definitions give it bool.

Places where __sanitizer_acquire_crash_state is declared:

  include/sanitizer_common/sanitizer_interface_defs.h
  lib/sanitizer_common/sanitizer_interface_internal.h
  lib/sanitizer_common/sanitizer_common.cc
  lib/fuzzer/FuzzerExtFunctions.def (this is the only bool)

llvm-svn: 353596
2019-02-09 01:45:29 +00:00
Kostya Serebryany 0fda9dcbe9 [libFuzzer] more refactoring; change some of the exit codes (timeout, OOM, interrupt) so that the parent process can distinguish those
llvm-svn: 353584
2019-02-09 00:16:21 +00:00
Kostya Serebryany 114cfafe05 [libFuzzer] refactor the merging code, NFC
llvm-svn: 353576
2019-02-08 22:59:03 +00:00
Kostya Serebryany b1e8b8149b [libFuzzer] remove two unused experimental flags
llvm-svn: 353573
2019-02-08 22:02:37 +00:00
Kostya Serebryany f762a11544 [libFuzzer] introduce an experimental mode -fork=1, where fuzzing happens in a subprocess (still running multiple inputs per process), thus making the fuzzing more resilient to timeouts and OOMs. This is just a skeleton of the code, and some associated refactoring, not a fully working feature yet.
llvm-svn: 353570
2019-02-08 21:27:23 +00:00
Vitaly Buka f333118ee0 Fix Die() after pthread_exit call on macOS
Summary:
Scoped interceptor should not be used when calling real pthread_exit().
On macOS C++ destructors are not called by pthread_exit(), and later check for empty thread ignore set fails.

Patch by Yuri Per.

Reviewers: dvyukov, vitalybuka

Reviewed By: vitalybuka

Subscribers: vitalybuka, thegameg, kubamracek, jfb, llvm-commits, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57963

llvm-svn: 353561
2019-02-08 20:43:26 +00:00
Francis Visoiu Mistrih c716bee053 [InstrProf] Fix darwin errors after r353547
The macros were changed but not updated in the Darwin version.

Fail here: http://green.lab.llvm.org/green/job/clang-stage1-configure-RA/53739/

rdar://47925820

llvm-svn: 353560
2019-02-08 20:37:40 +00:00
Vitaly Buka a1cec5b264 [tsan] Remove SCOPED_TSAN_INTERCEPTOR to try to fix ppc bot
llvm-svn: 353552
2019-02-08 19:37:32 +00:00
Reid Kleckner 987d331fab [InstrProf] Implement static profdata registration
Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.

Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.

Now, Windows uses static registration and is in line with all the other
platforms.

Reviewers: davidxl, wmi, inglorion, void, calixte

Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57929

llvm-svn: 353547
2019-02-08 19:03:50 +00:00
Kostya Serebryany 0719b3527f [libFuzzer] refactor the way we choose the element to cross-over with, NFC (expected1); add a flag -seed_inputs= to pass extra seed inputs as file paths, not dirs
llvm-svn: 353494
2019-02-08 01:20:54 +00:00
Reid Kleckner c20ff80845 Silence -Wformat warnings about GetLastError returning ULONG
llvm-svn: 353485
2019-02-07 23:58:48 +00:00
Reid Kleckner c795e62ece Commit macro intended to be included in r353483.
llvm-svn: 353484
2019-02-07 23:57:21 +00:00
Reid Kleckner 2428224ffe [compiler rt] Win64 GetInstructionSize additional register MOV + stack alignment AND
Current interception code does not cover all of the required registers
on Windows for a specific flavor of MOV, so this patch adds cases to
identify the following 5-byte instructions on 64-bit Windows:

mov QWORD PTR [rsp + XX], rdx  <- second integer argument
mov QWORD PTR [rsp + XX], r9    <- third integer argument
mov QWORD PTR [rsp + XX], r8    <- fourth integer argument

The instruction for MOV [...] RCX is already covered in the previous
version.

Patch by Matthew McGovern!

Reviewers: rnk

Differential Revision: https://reviews.llvm.org/D57339

llvm-svn: 353483
2019-02-07 23:56:37 +00:00
Vitaly Buka 871b2bd980 [safestack] Explain why tinfo at the end of the buffer
Reviewers: pcc, eugenis, vlad.tsyrklevich

Reviewed By: vlad.tsyrklevich

Subscribers: llvm-commits, jfb, #sanitizers

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57863

llvm-svn: 353482
2019-02-07 23:20:47 +00:00
Vitaly Buka 539e6aa421 [safestack] Remove pageSize
Summary:
3rd party sysconf interceptor may crash if it's called before unsafe_stack_setup

However pageSize is not useful here. mmap should round up on it's own, SFS_CHECK can be removed.

Reviewers: eugenis, vlad.tsyrklevich

Subscribers: #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57924

llvm-svn: 353481
2019-02-07 23:20:39 +00:00
Vitaly Buka 6569120b81 [safestack] Don't crash if stack size is not aligned as expected
Summary:
From runtime side looks it's OK to RoundUpTo to needed alignment as buffer is
going to be RoundUpTo to page size anyway.

Reviewers: eugenis, pcc

Subscribers: #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57866

llvm-svn: 353475
2019-02-07 22:26:04 +00:00
Kostya Kortchinsky 7547170ca4 [scudo][standalone] Do not error out on spurious C(XX) flags
Summary:
The standalone Scudo version is being built with `-Werror` which can be
tripped by extraneous command line arguments. We have little control over
those as they can be passed down to us by `CMAKE_C(XX)_FLAGS`, the reported
scenario involving `-stdlib=libc++` (see https://reviews.llvm.org/D57412#1384504).

To work around this, disable `-Wunused-command-line-argument`.

Reviewers: eugenis, vitalybuka, Eugene.Zelenko

Reviewed By: eugenis

Subscribers: mgorny, delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D57757

llvm-svn: 353418
2019-02-07 15:44:36 +00:00
Dmitry Vyukov bdfba86047 tsan: add more benchmarks
func_entry_exit.cc is for __tsan_func_entry/exit (spends ~75% there),
we don't yet have any.
mop.cc is for memory access functions, as compared to mini_bench_local/shared.cc
this benchmark passes through deduplication logic (ContainsSameAccess).

llvm-svn: 353407
2019-02-07 14:04:04 +00:00
Dmitry Vyukov fddaf1f369 tsan: Optimize performance of Thread sanitizer memory access functions
- Manually unwind code in MemoryAccessImpl1() because clang do not optimize it
- Check for .rodata section only in read operations
- Place LIKELY/UNLIKELY on fast paths

This speeds up synthetic memory access benchmarks by 10-20%.

[dvyukov: fixed up consts in check_analyze.sh]

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57882
Context: https://reviews.llvm.org/D54889

llvm-svn: 353401
2019-02-07 12:43:09 +00:00
Dmitry Vyukov baf2f35ec4 sanitizers: Introduce ThreadType enum
Replace bool workerthread flag with ThreadType enum.
This change is preparation for fiber support.

[dvyukov: fixed build of sanitizer_thread_registry_test.cc]

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57839
Context: https://reviews.llvm.org/D54889

llvm-svn: 353390
2019-02-07 11:01:22 +00:00
Dmitry Vyukov 17132b62e0 tsan: Implement pthread_exit() interceptor for Thread sanitizer
This change is preparation for fiber support.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57876
Context: https://reviews.llvm.org/D54889

llvm-svn: 353385
2019-02-07 10:46:02 +00:00
Vitaly Buka 1665ea19e9 [sanitizer] Remove unneeded pointer check
Summary: unpoison_passwd and unpoison_group support nullptrs

Reviewers: eugenis

Subscribers: kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57784

llvm-svn: 353365
2019-02-07 01:54:59 +00:00
Vitaly Buka eb7031ecf6 [sanitizer] Don't unpoison buffer in getpw/getgr functions
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.

This fixes TSAN performance issue made us to disable this interceptors.

Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57731

llvm-svn: 353351
2019-02-07 00:08:14 +00:00
Evgeniy Stepanov 54c82886cb [sanitizer] Fix fuchsia and windows build.
llvm-svn: 353261
2019-02-06 01:58:23 +00:00
Evgeniy Stepanov 443c034391 [sanitizer] Decorate /proc/self/maps better.
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.

Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.

Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.

The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D57720

llvm-svn: 353255
2019-02-06 01:14:50 +00:00
Kamil Rytarowski 3349bd662a Update the ioctl(2) list in sanitizers with NetBSD 8.99.34
llvm-svn: 353224
2019-02-05 22:20:25 +00:00
Reid Kleckner 0a142346b3 [InstrProf] Merge COFF .lprof* and .lcovmap sections into .data/.rdata
There is no reason for these sections to remain separate in the final
DLL or EXE.

I have not yet added a InstrProfilingPlatformWindows.c for these, since
avoiding dynamic profile data registration is a larger project for
later.

llvm-svn: 353221
2019-02-05 21:26:31 +00:00
Vitaly Buka 0ca702f8d5 [sanitizer] Fix unused function 'unpoison_passwd'
llvm-svn: 353205
2019-02-05 19:46:44 +00:00
Vitaly Buka d2a42c3815 [tsan] Disable fgetpwent_r to work around performance issues
This was missed from D54041 when SANITIZER_INTERCEPT_FGETPWENT_R was branched from SANITIZER_INTERCEPT_GETPWENT_R

llvm-svn: 353110
2019-02-04 23:25:55 +00:00
Jonathan Metzman 7c907bac6f [libFuzzer][Windows] Clean up RawPrint
Summary:
Use `_write` instead of the deprecated alias `write` on Windows.
Also, remove comment saying RawPrint is untested on Windows.

Reviewers: vitalybuka

Reviewed By: vitalybuka

Subscribers: vitalybuka

Differential Revision: https://reviews.llvm.org/D57589

llvm-svn: 353108
2019-02-04 23:01:06 +00:00
Kostya Kortchinsky 47f0d136f1 [scudo] Initial standalone skeleton check-in
Summary:
This is the initial check-in for the Standalone version of Scudo.

The project is initially going to live in scudo/standalone then will
replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html
for details.

This initial CL is meant to lay out the project structure, of both
code & tests, providing a minimal amount of functionalities, namely
various definitions, some atomic helpers and an intrusive list.
(empty.cc is just here to have a compilation unit, but will go away
in the upcoming CLs).

Initial support is restricted to Linux i386 & x86_64 in make files
and will be extended once things land & work.

We will grow organically from here, adding functionalities in limited
amounts.

Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack

Reviewed By: morehouse, vitalybuka

Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D57412

llvm-svn: 353055
2019-02-04 16:25:40 +00:00
Reid Kleckner 33706e3cbf [ubsan] Make suppressions.cpp test pass for me on Windows
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.

While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.

llvm-svn: 352921
2019-02-01 21:35:17 +00:00
Peter Collingbourne 6fa43f8b9c Mark __rela_iplt_{start,end} as weak.
Should fix non-lld links.

llvm-svn: 352823
2019-02-01 00:43:31 +00:00
Peter Collingbourne 886b7cc107 hwasan: Add __hwasan_init_static() function.
This function initializes enough of the runtime to be able to run
instrumented code in a statically linked executable. It replaces
__hwasan_shadow_init() which wasn't doing enough initialization for
instrumented code that uses either TLS or IFUNC to work.

Differential Revision: https://reviews.llvm.org/D57490

llvm-svn: 352816
2019-01-31 23:37:12 +00:00
Jonathan Metzman fc7faecb40 [fuzzer] Use RawPrint instead of Printf for instrumentation warning
Summary:
Use RawPrint instead of Printf for instrumentation warning because
Printf doesn't work on Win when instrumentation is being
initialized (since OutputFile is not yet initialized).

Reviewers: kcc

Reviewed By: kcc

Differential Revision: https://reviews.llvm.org/D57531

llvm-svn: 352789
2019-01-31 20:32:20 +00:00