Commit Graph

7159 Commits

Author SHA1 Message Date
Derek Bruening c45e51c891 [esan|wset] Reduce flakiness in samples test
Generalizes the workingset-samples test to pass when a sample has a
size of 0, which can happen on a loaded machine.

llvm-svn: 272175
2016-06-08 17:35:52 +00:00
Vedant Kumar 17af892d6f [profile] Hide some external symbols in InstrProfData.inc
Differential Revision: http://reviews.llvm.org/D21116

llvm-svn: 272167
2016-06-08 16:39:43 +00:00
Xinliang David Li f88d3b9afb [profile] Pass extra build flags (feature enabling macros) to Darwin build
Differential Revision: http://reviews.llvm.org/D21119

llvm-svn: 272162
2016-06-08 16:21:35 +00:00
Vedant Kumar 33b8b64aef [profile] Un-hide two symbols
- lprofCurFilename was intended to have external visibility. This is
  pending further discussion.
- The raw version number doesn't need to be hidden: hiding it may make
  it easier to accidentally combine FE/IR profiles.

See the mailing list discussion on r272081.

llvm-svn: 272089
2016-06-08 01:33:15 +00:00
Vedant Kumar b202ad6518 [profile] Hide a few external symbols (NFCI)
There are still a few external symbols visible from InstrProfData.inc.
The plan for dealing with those isn't as straightforward, so I'll try it
in a separate commit.

llvm-svn: 272081
2016-06-08 00:44:38 +00:00
Derek Bruening ff5cafa2ec [esan] Intercept calloc to avoid deadlocks with tcmalloc
Summary:
When tcmalloc initializes before esan, esan's initialization ends up
calling back into tcmalloc due to the calloc done by dlsym.  This results
in a deadlock.  We avoid this by special-casing this single allocation.

Intercepting calloc also gives us the opportunity to act on its zeroing as
stores by the application.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D21086

llvm-svn: 272076
2016-06-08 00:00:27 +00:00
Kostya Serebryany 86020ba32b [scudo] trying to fix the bot: aligned_alloc is not known there; attempt 3
llvm-svn: 272074
2016-06-07 23:49:11 +00:00
Kostya Serebryany 8aa0d7332a [scudo] trying to fix the bot: aligned_alloc is not known there; attempt 2
llvm-svn: 272051
2016-06-07 20:09:49 +00:00
Kuba Brecka 688c3d3bf1 [asan] Turn LSan-related #if’s into regular if’s in ASan initializer
Removing some preprocessor #if’s in favor of regular if’s. However, we need to declare empty stub functions to avoid linker errors.

Differential Revision: http://reviews.llvm.org/D20911

llvm-svn: 272047
2016-06-07 19:13:38 +00:00
Kostya Serebryany 86f8d33442 [scudo] trying to fix the bot: aligned_alloc is not known there
llvm-svn: 272037
2016-06-07 18:29:10 +00:00
Kostya Serebryany 422ea24d45 [asan] make print_memory_profile_test more reliable by using large allocation sizes. Hoping to fix the llvm-clang-lld-x86_64-debian-fast bot
llvm-svn: 272020
2016-06-07 16:08:59 +00:00
Adhemerval Zanella 2c8fc6f8c9 [sanitizer] Fix build for new GLIBC msghdr/cmsghdr definition
GLIBC now follows POSIX [1] for both msghdr and cmsghdr definitions,
which means that msg_iovlen, msg_controllen, and cmsg_len are no
longer size_t but sockelen_t for 64-bits architectures. The final struct
size does not change, since paddings were added.

This patch fixes the build issue against GLIBC 2.24 socket.h header by
using the same definition for internal __sanitizer_msghdr and
__sanitizer_cmsghdr.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/

llvm-svn: 272008
2016-06-07 13:19:38 +00:00
Kostya Serebryany 712fc9803a [sanitizer] Initial implementation of a Hardened Allocator
Summary:
This is an initial implementation of a Hardened Allocator based on Sanitizer Common's CombinedAllocator.
It aims at mitigating heap based vulnerabilities by adding several features to the base allocator, while staying relatively fast.
The following were implemented:
- additional consistency checks on the allocation function parameters and on the heap chunks;
- use of checksum protected chunk header, to detect corruption;
- randomness to the allocator base;
- delayed freelist (quarantine), to mitigate use after free and overall determinism.
Additional mitigations are in the works.

Reviewers: eugenis, aizatsky, pcc, krasin, vitalybuka, glider, dvyukov, kcc

Subscribers: kubabrecka, filcab, llvm-commits

Differential Revision: http://reviews.llvm.org/D20084

llvm-svn: 271968
2016-06-07 01:20:26 +00:00
David Blaikie 7d492c4ff8 add -f to rm so the test passes the first time it's run the file file is not present
(otherwise it just keeps failing at the rm step and never passes... )

llvm-svn: 271959
2016-06-06 23:31:58 +00:00
Sean Silva 7e4e69abe2 [libprofile] Also use `env` for this test.
(necessary on Windows)

llvm-svn: 271955
2016-06-06 23:19:23 +00:00
Sean Silva d7caf5889b [libprofile] Add %run to a couple tests.
llvm-svn: 271947
2016-06-06 22:19:49 +00:00
Xinliang David Li 0891b6d59b [profile] code cleanup /NFC
Address review feedback for better
readability.

llvm-svn: 271922
2016-06-06 18:31:29 +00:00
Kuba Brecka 304b2c2a45 [tsan] Switch to InternalAlloc everywhere __libc_malloc is currently used
This patch replaces all uses of __libc_malloc and friends with the internal allocator.

It seems that the only reason why we have calls to __libc_malloc in the first place was the lack of the internal allocator at the time. Using the internal allocator will also make sure that the system allocator is never used (this is the same behavior as ASan), and we don’t have to worry about working with unknown pointers coming from the system allocator.

Differential Revision: http://reviews.llvm.org/D21025

llvm-svn: 271916
2016-06-06 18:18:47 +00:00
Etienne Bergeron 7a1bafd6ae This patch attempts to primitive support for Win64 asan
Some known issues are:

When "head" include instructions that involve branching, the "cut and paste" approach may break down in a way that function interception still work but calling back the original function does not work.
The jmp [rip -8] saves some bytes in the "head" but finding the safe zone of 0xCC is not implemented yet. So it may stomp on preceding codes.
The shadow offset is not working yet on Win64. More complexity maybe involved since there are some differences regarding virtual address space between Window 8 and Windows 8.1/10.

Patch by: Wang Wei

Differential Revision: http://reviews.llvm.org/D20884

llvm-svn: 271915
2016-06-06 18:09:54 +00:00
Kuba Brecka cdf3492191 [tsan] On OS X, optimize main thread’s ThreadState accesses
This is a very simple optimization that gets about 10% speedup for certain programs. We’re currently storing the pointer to the main thread’s ThreadState, but we can store the state directly in a static variable, which avoid the load acquire.

Differential Revision: http://reviews.llvm.org/D20910

llvm-svn: 271906
2016-06-06 16:27:38 +00:00
Kuba Brecka 2e26d9faac [asan] Reenable ASan tests in "check-all"
Somehow, in r271049, ASan lit tests and unit tests were removed from “check-all”. Doesn’t seem intentional, let’s fix it.

Differential Revision: http://reviews.llvm.org/D21017

llvm-svn: 271905
2016-06-06 16:24:47 +00:00
Xinliang David Li 5cd1f94d4f [profile] in-process mergeing support (part-2)
(Part-1 merging API is in profile runtime)

This patch implements a portable file opening API
with exclusive access for the process. In-process
profile merge requires profile file update to be
atomic/fully sychronized.

llvm-svn: 271864
2016-06-06 03:17:58 +00:00
Kuba Brecka 312071943c [tsan] On OS X, build Go runtime with -mmacosx-version-min
We're not building the Go runtime with -mmacosx-version-min, which means it'll have a minimum deployment target set to the system you're building on. Let's make the code compile (and link) with -mmacosx-version-min=10.7.

Differential Revision: http://reviews.llvm.org/D20670

llvm-svn: 271833
2016-06-05 15:15:36 +00:00
Chandler Carruth 7573cfe2b3 It really helps to not randomly delete one of the closing brackets. =/
llvm-svn: 271787
2016-06-04 09:06:59 +00:00
Chandler Carruth b330dcab19 Ok, now I know why I kept getting confused and not making progress
fixing tis test. There are different configurations running, and they
have subtly different backtraces. I didn't notice that the configs
changed and so I kept occilating between the two.

Now I've looked at two different configs at the same time, and so this
should be much more likely to work.

llvm-svn: 271786
2016-06-04 08:54:57 +00:00
Chandler Carruth 9ef4243824 Fix the test for printing the memory profile. This fuctionality is only
available along side the leak checking, so use the REQUIRES for that.

Also, use %run as other tests do when launching the built binary.

This fixes check-asan for me on Linux and looks like it should fix the
linux sanitizer bots as well.

llvm-svn: 271785
2016-06-04 08:45:32 +00:00
Chandler Carruth ece6d7bf2e Update this test yet again, this time based on a nice consecutive pair
of lines provided with the filecheck output from the previous run. I'll
probably give up after this and get someone with a Windows build to help
me out.

llvm-svn: 271784
2016-06-04 08:24:56 +00:00
Chandler Carruth 4f785dc1f6 I didn't actually get this right, but now I have frame #0 and frame #2.
This should fail on frame #1 and show what that should actually be.

llvm-svn: 271783
2016-06-04 08:09:24 +00:00
Chandler Carruth d26defd57e Amazingly, my guess was correct for the top two frames here. Hopefully
with this, the Windows sanitizer bot will go green!

llvm-svn: 271782
2016-06-04 07:29:10 +00:00
Chandler Carruth 79abce39bb Temporary hack to this test to try and get the sanitizer-windows bot to
show enough information for me to fix the patterns used here.

llvm-svn: 271779
2016-06-04 06:16:49 +00:00
Xinliang David Li c012d086fa reapply r271577 after OSX bug is fixed (hopefully)
llvm-svn: 271759
2016-06-04 01:51:25 +00:00
Chris Bieneman c0d89826a7 [CMake] Support constructing output paths from LLVM variables
This tweak to constructing output paths allows compiler-rt to use LLVM
output variables if they are set regardless of whether or not the build
is in-tree.

llvm-svn: 271749
2016-06-03 23:15:04 +00:00
Derek Bruening c1c67d651d [esan] Initialize runtime during early interceptors
Summary:
Adds initialization of esan's runtime library during any early interceptors
that are sometimes called prior to the official __esan_init() invocation
(we see this with apps using tcmalloc).

Adds handling of interceptors called during interceptor initialization.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20976

llvm-svn: 271744
2016-06-03 22:30:10 +00:00
Qin Zhao bc929e4765 [esan|cfrag] Compute the struct field access difference ratio
Summary:
Computes the struct field access variation based on each field access
count.

Adds a flag to control the report thresholds.

Updates struct-simple.cpp with variance report output.

Reviewers: aizatsky

Subscribers: kubabrecka, zhaoqin, llvm-commits, eugenis, vitalybuka, kcc, bruening

Differential Revision: http://reviews.llvm.org/D20914

llvm-svn: 271734
2016-06-03 20:48:17 +00:00
Derek Bruening 328e899301 [esan] Specify which tool via a global variable
Summary:
Adds a global variable to specify the tool, to support handling early
interceptors that invoke instrumented code, thus requiring shadow memory to
be initialized prior to __esan_init() being invoked.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20974

llvm-svn: 271714
2016-06-03 19:40:08 +00:00
Chris Bieneman 648d3bc8c7 [CMake] detect_target_arch needs to be moved to Utils
This macro is called from the base config, so it can't live in config-ix, it needs to be in the utils. I suspect the only reason this hasn't caused problems is that nobody is building the Android builtins from the builtins subdirectory.

llvm-svn: 271693
2016-06-03 17:34:02 +00:00
Derek Bruening 45afd7e8fc [esan] Eliminate flakiness in large-stack test
Turns off sampling for the large-stack test to remove nondeterminism in the
output and fix bot failures.

llvm-svn: 271691
2016-06-03 17:18:09 +00:00
Derek Bruening e78e4a6e94 [esan|wset] Add 8-level working set snapshot accumulation
Summary:
Adds a new option -snapshot_step controlling the frequency distribution for
an 8-level series of samples using each bit of each shadow byte.
Implements accumulation from each level to the next higher level at the
specified frequency.

Adds storage of the 8 series of samples using CircularBuffer instances.
Fixes an error in the circular buffer data structure where a static
object's destructor will be called too early.

Prints the results out at the end in a simple manner to give us something
to start with.

Updates the workingset-samples test to test the new feature.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20833

llvm-svn: 271683
2016-06-03 16:27:50 +00:00
Derek Bruening 07814769a8 [esan] Add sideline itimer support
Summary:
Adds support for creating a separate thread for performing "sideline"
actions on a periodic basis via an itimer.  A new class SidelineThread
implements this feature, exposing a sampling callback to the caller.

Adds initial usage of sideline sampling to the working set tool.  For now
it simply prints the usage at each snapshot at verbosity level 1.  Adds a
test of this behavior.  Adds a new option -record_snapshots to control
whether we sample and a new option -sample_freq to control the periodicity
of the sampling.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20751

llvm-svn: 271682
2016-06-03 16:14:07 +00:00
Dmitry Vyukov 0f7e949a42 tsan: rely on AnnotateRWLockCreateStatic to detect linker-initialized mutexes
The new annotation was added a while ago, but was not actually used.
Use the annotation to detect linker-initialized mutexes instead
of the broken IsGlobalVar which has both false positives and false
negatives. Remove IsGlobalVar mess.

llvm-svn: 271663
2016-06-03 11:48:27 +00:00
Derek Bruening a601c2ada2 [esan] Ensure internal_sigaction() bypasses interceptors
Summary:
Implements real_sigaction() which it turns out is required for
internal_sigaction() to bypass the libc interceptors.

Without real_sigaction(), our internal_sigaction() calls during init happen
to work due to the EsanDuringInit check in COMMON_INTERCEPTOR_ENTER (though
even here it does not feel right for an "internal_" call to go through the
interceptor).  The real problem is when we call internal_sigaction() after
we're initialized, which only happens on an unhandled SIGSEGV for which the
app has no handler: then we'll spin in an infinite loop as our attempts to
remove our own handler repeatedly fail.  It's not easy to add a test for
that, unfortunately.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20832

llvm-svn: 271626
2016-06-03 04:30:47 +00:00
Mehdi Amini e1db5a0400 Revert "[profile] enable a subtest for vcall inline virtual func"
This reverts commit r271577.

The test does not pass on OSX.

llvm-svn: 271617
2016-06-03 00:45:26 +00:00
Derek Bruening 0b04950116 [esan] Add support for log_exe_name
Summary:
Adds the call needed to cache the binary name to support the sanitizer
option log_exe_name for usable log file results when running multiple
applications.  Adds a test.

Reviewers: aizatsky

Subscribers: vitalybuka, zhaoqin, kcc, eugenis, llvm-commits, kubabrecka

Differential Revision: http://reviews.llvm.org/D20747

llvm-svn: 271586
2016-06-02 20:50:30 +00:00
Xinliang David Li 2baff11d05 Fix build failure with cmake version bump
llvm-svn: 271582
2016-06-02 20:33:33 +00:00
Xinliang David Li d4c23fc76f [profile] enable a subtest for vcall inline virtual func
llvm-svn: 271577
2016-06-02 20:12:06 +00:00
Dmitry Vyukov 1ae44e6a2b tsan: clean up dynamic TLS memory between reuse
Currently the added test produces false race reports with glibc 2.19,
because DLTS memory is reused by pthread under the hood.

Use the DTLS machinery to intercept new DTLS ranges.
__tls_get_addr known to cause issues for tsan in the past,
so write the interceptor more carefully.

Reviewed in http://reviews.llvm.org/D20927

llvm-svn: 271568
2016-06-02 19:18:22 +00:00
Qin Zhao 4175a6d580 [esan|cfrag] Add struct info registration
Summary:
Adds StructInfo to CacheFragInfo to match the LLVM's EfficiencySanitizer
structs.

Uses StructHashMap to keep track of the struct info used by the app.

Adds registerStructInfo/unregisterStructInfo to add/remove struct infos
to/from StructHashMap.

updates test struct-simple.cpp with more C structs.

Reviewers: aizatsky, filcab

Subscribers: filcab, zhaoqin, llvm-commits, eugenis, vitalybuka, kcc, bruening, kubabrecka

Differential Revision: http://reviews.llvm.org/D20590

llvm-svn: 271564
2016-06-02 18:45:25 +00:00
Ivan Krasin 048155c399 UBSan: crash less often on corrupted Vtables.
Summary:
This CL adds a weak check for a Vtable prefix: for a well-formed
Vtable, we require the prefix to be within [-1<<20; 1<<20].

Practically, this solves most of the known cases when UBSan segfaults
without providing any useful diagnostics.

Reviewers: pcc

Subscribers: kubabrecka

Differential Revision: http://reviews.llvm.org/D19750

llvm-svn: 271560
2016-06-02 18:36:12 +00:00
Xinliang David Li 66703ebfe5 [profile] Add end-end test of vp for inline virtual functions
Make sure inline virtual function's address is properly recorded and
dumped in raw profile so that value profiling does not loss tracking.
(Second part of the test will be enabled after the lowering bug is fixed
in LLVM)

llvm-svn: 271528
2016-06-02 16:24:21 +00:00
Kostya Serebryany 3e7bf586f8 [asan] fix arm build
llvm-svn: 271474
2016-06-02 04:01:58 +00:00