Commit Graph

14437 Commits

Author SHA1 Message Date
Dmitry Vyukov 6ca7247866 sanitizer_common: support printing __m128i type
__m128i is vector SSE type used in tsan.
It's handy to be able to print it for debugging.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D102167
2021-07-12 16:12:46 +02:00
Dmitry Vyukov b3bd885017 sanitizer_common: allow COMMON_INTERCEPTOR_ENTER to use labels
The memcpy interceptor is the only one that uses COMMON_INTERCEPTOR_ENTER
more than once in a single function. This does not allow COMMON_INTERCEPTOR_ENTER
to use labels, because they are global for the whole function (not block scoped).
Don't include COMMON_INTERCEPTOR_ENTER code twice.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105774
2021-07-12 15:54:37 +02:00
Dmitry Vyukov 10158b52dc sanitizer_common: fix 32-bit build
https://reviews.llvm.org/D105716 enabled thread safety annotations,
and that broke 32-bit build:
https://green.lab.llvm.org/green/job/lldb-cmake/33604/consoleFull#-77815080549ba4694-19c4-4d7e-bec5-911270d8a58c

1. Enable thread-safety analysis in unit tests
(this catches the breakage even in 64-bit mode).
2. Add NO_THREAD_SAFETY_ANALYSIS to sanitizer_allocator_primary32.h
to unbreak the build.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105808
2021-07-12 14:03:13 +02:00
Dmitry Vyukov 8bbf2f7a44 sanitizer_common: make sem_trywait as non-blocking
sem_trywait never blocks.
Use REAL instead of COMMON_INTERCEPTOR_BLOCK_REAL.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105775
2021-07-12 13:10:04 +02:00
Dmitry Vyukov fde34d9f89 sanitizer_common: remove debugging logic from the internal allocator
The internal allocator adds 8-byte header for debugging purposes.
The problem with it is that it's not possible to allocate nicely-sized
objects without a significant overhead. For example, if we allocate
512-byte objects, that will be rounded up to 768 or something.
This logic migrated from tsan where it was added during initial development,
I don't remember that it ever caught anything (we don't do bugs!).
Remove it so that it's possible to allocate nicely-sized objects
without overheads.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105777
2021-07-12 13:02:38 +02:00
Dmitry Vyukov 326b0054fd sanitizer_common: use 0 for empty stack id
We use 0 for empty stack id from stack depot.
Deadlock detector 1 is the only place that uses -1
as a special case. Use 0 because there is a number
of checks of the form "if (stack id) ...".

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105776
2021-07-12 12:24:27 +02:00
Dmitry Vyukov 0da172b176 sanitizer_common: add thread safety annotations
Enable clang Thread Safety Analysis for sanitizers:
https://clang.llvm.org/docs/ThreadSafetyAnalysis.html

Thread Safety Analysis can detect inconsistent locking,
deadlocks and data races. Without GUARDED_BY annotations
it has limited value. But this does all the heavy lifting
to enable analysis and allows to add GUARDED_BY incrementally.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105716
2021-07-12 11:46:49 +02:00
Dmitry Vyukov 41ba96f531 sanitizer_common: rename Mutex to MutexState
We have 3 different mutexes (RWMutex, BlockingMutex __tsan::Mutex),
each with own set of downsides. I want to unify them under a name Mutex.
But it will conflict with Mutex in the deadlock detector,
which is a way too generic name. Rename it to MutexState.

Reviewed By: melver

Differential Revision: https://reviews.llvm.org/D105773
2021-07-12 11:23:46 +02:00
Dmitry Vyukov 6062c672bc sanitizer_common: unbreak ThreadRegistry tests
https://reviews.llvm.org/D105713
forgot to update tests for the new ctor.

Differential Revision: https://reviews.llvm.org/D105772
2021-07-11 12:58:20 +02:00
Dmitry Vyukov 8e489b4b96 sanitizer_common: add simpler ThreadRegistry ctor
Currently ThreadRegistry is overcomplicated because of tsan,
it needs tid quarantine and reuse counters. Other sanitizers
don't need that. It also seems that no other sanitizer now
needs max number of threads. Asan used to need 2^24 limit,
but it does not seem to be needed now. Other sanitizers blindly
copy-pasted that without reasons. Lsan also uses quarantine,
but I don't see why that may be potentially needed.

Add a ThreadRegistry ctor that does not require any sizes
and use it in all sanitizers except for tsan.
In preparation for new tsan runtime, which won't need
any of these parameters as well.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D105713
2021-07-11 12:36:28 +02:00
Dmitry Vyukov 8df3c7ded2 sanitizer_common: sanitize time functions
We have SleepForSeconds, SleepForMillis and internal_sleep.
Some are implemented in terms of libc functions, some -- in terms
of syscalls. Some are implemented in per OS files,
some -- in libc/nolibc files. That's unnecessary complex
and libc functions cause crashes in some contexts because
we intercept them. There is no single reason to have calls to libc
when we have syscalls (and we have them anyway).

Add internal_usleep that is implemented in terms of syscalls per OS.
Make SleepForSeconds/SleepForMillis/internal_sleep a wrapper
around internal_usleep that is implemented in sanitizer_common.cpp once.

Also remove return values for internal_sleep, it's not used anywhere.

Eventually it would be nice to remove SleepForSeconds/SleepForMillis/internal_sleep.
There is no point in having that many different names for the same thing.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D105718
2021-07-11 12:09:47 +02:00
Dmitry Vyukov 1e1f752027 sanitizer_common: split LibIgnore into fast/slow paths
LibIgnore is checked in every interceptor.
Currently it has all logic in the single function
in the header, which makes it uninlinable.
Split it into fast path (no libraries ignored)
and slow path (have ignored libraries).
It makes the fast path inlinable (single load).

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D105719
2021-07-11 11:34:15 +02:00
Martin Storsjö 8cf7ddbdd4 Revert "Prepare Compiler-RT for GnuInstallDirs, matching libcxx"
This reverts commit 9a9bc76c0e.

That commit broke "ninja install" when building compiler-rt for mingw
targets, building standalone (pointing cmake at the compiler-rt
directory) with cmake 3.16.3 (the one shipped in ubuntu 20.04), with
errors like this:

-- Install configuration: "Release"
CMake Error at cmake_install.cmake:44 (file):
  file cannot create directory: /include/sanitizer.  Maybe need
  administrative privileges.
Call Stack (most recent call first):
  /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/cmake_install.cmake:37 (include)

FAILED: include/CMakeFiles/install-compiler-rt-headers
cd /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/include && /usr/bin/cmake -DCMAKE_INSTALL_COMPONENT="compiler-rt-headers" -P /home/martin/code/llvm-mingw/src/llvm-project/compiler-rt/build-i686-sanitizers/cmake_install.cmake
ninja: build stopped: subcommand failed.
2021-07-10 10:45:54 +03:00
Vitaly Buka db4c25822a [scudo] Check if we use __clang_major__ >= 12
This makes sure we have support for MTE instructions.
Later the check can be extended to support MTE on other compilers.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D105722
2021-07-09 17:48:44 -07:00
John Ericson 9a9bc76c0e Prepare Compiler-RT for GnuInstallDirs, matching libcxx
Instead of using `COMPILER_RT_INSTALL_PATH` through the CMake for
complier-rt, just use it to define variables for the subdirs which
themselves are used.

This preserves compatibility, but later on we might consider getting rid
of `COMPILER_RT_INSTALL_PATH` and just changing the defaults for the
subdir variables directly.

---

There was a seaming bug where the (non-Apple) per-target libdir was
`${target}` not `lib/${target}`. I suspect that has to do with the docs
on `COMPILER_RT_INSTALL_PATH` saying was the library dir when that's no
longer true, so I just went ahead and fixed it, allowing me to define
fewer and more sensible variables.

That last part should be the only behavior changes; everything else
should be a pure refactoring.

---

D99484 is the main thrust of the `GnuInstallDirs` work. Once this lands,
it should be feasible to follow both of these up with a simple patch for
compiler-rt analogous to the one for libcxx.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D101497
2021-07-09 20:41:53 +00:00
Florian Mayer 5511bfdb67 [hwasan] More realistic setjmp test.
The existing one actually failed on the int* p, not on int z (as can be
seen by the fault being 8 bytes rather than 4).

This is also needed to make sure the stack safety analysis does not
classify the alloca as safe.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D105705
2021-07-09 20:27:32 +01:00
Leonard Chan 8b68aeebd3 [NFC][compiler-rt][hwasan] Move TagMemoryAligned into hwasan_linux.cpp
Based on comments in D91466, we can make the current implementation
linux-speciic. The fuchsia implementation will just be a call to
`__sanitizer_fill_shadow`.

Differential Revision: https://reviews.llvm.org/D105663
2021-07-09 11:02:44 -07:00
Leonard Chan 89f778ceea [NFC][compiler-rt][hwasan] Rename InitPrctl to InitializeOsSupport
The new name is something less linux-y and more platform generic. Once we
finalize the tagged pointer ABI in zircon, we will do the appropriate check
for fuchsia to see that TBI is enabled.

Differential Revision: https://reviews.llvm.org/D105667
2021-07-09 11:00:51 -07:00
Leonard Chan d564cfb53c [NFC][compiler-rt][hwasan] Move __hwasan_tag_mismatch4 to hwasan.cpp
This can be moved out of hwasan_linux.cpp because it can be re-used in the fuchsia port.

Differential Revision: https://reviews.llvm.org/D105669
2021-07-09 10:51:22 -07:00
Nico Weber 97c675d3d4 Revert "Revert "Temporarily do not drop volatile stores before unreachable""
This reverts commit 52aeacfbf5.
There isn't full agreement on a path forward yet, but there is agreement that
this shouldn't land as-is.  See discussion on https://reviews.llvm.org/D105338

Also reverts unreviewed "[clang] Improve `-Wnull-dereference` diag to be more in-line with reality"
This reverts commit f4877c78c0.

And all the related changes to tests:
This reverts commit 9a0152799f.
This reverts commit 3f7c9cc274.
This reverts commit 329f8197ef.
This reverts commit aa9f58cc2c.
This reverts commit 2df37d5ddd.
This reverts commit a72a441812.
2021-07-09 11:44:34 -04:00
Roman Lebedev 9a0152799f
[compiler-rt] fuzzer: adjust tests to not expect that store to null traps and is not erase
Instead, mostly just trap directly.
2021-07-09 15:03:11 +03:00
Roman Lebedev 3f7c9cc274
[compiler-rt] Conceal UB in sanitizer_common/TestCases/Linux/signal_line.cpp test
Store to null is deleted, so the test no longer did what it was expecting to do.
Conceal that by creating null pointer in a more elaborate way,
thus retaining original test coverage.
2021-07-09 14:39:47 +03:00
Roman Lebedev a72a441812
[compiler-rt][ASAN] Speculatively fix Windows/dll_control_c.cpp test - use trap
Please refer to https://reviews.llvm.org/D105338,
such store will not trap, it will be removed.
2021-07-09 13:11:27 +03:00
Leonard Chan a7992b43a5 [NFC][compiler-rt][fuchsia] Add InitShadowBounds declaration to header
Forgot to include this as a part of
a11aea68a4.
2021-07-08 13:19:17 -07:00
Mitch Phillips d458f37932 [GWP-ASan] Change unreachable -> trap to work around DCE bug.
trapOnAddress is designed to SEGV on a specific address. Unfortunately,
with an IR change, __builtin_unreachable() ends up doing DCE on things
that have side effects, like the load that causes the trap.

Change to __builtin_trap() to avoid the optimisation.

Root cause is still an LLVM bug, and tracked in
https://bugs.llvm.org/show_bug.cgi?id=47480.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105654
2021-07-08 13:06:42 -07:00
Leonard Chan a11aea68a4 [compiler-rt][hwasan] Define fuchsia implementations of required hwasan functions
This contains all the definitions required by hwasan for the fuchsia
implementation and can be landed independently from the remaining parts of D91466.

Differential Revision: https://reviews.llvm.org/D103936
2021-07-08 10:24:27 -07:00
Leonard Chan 398bfa2ead [compiler-rt][Fuchsia] Disable interceptors while enabling new/delete replacements
This disables use of hwasan interceptors which we do not use on Fuchsia. This
explicitly sets the macro for defining the hwasan versions of new/delete.

Differential Revision: https://reviews.llvm.org/D103544
2021-07-07 15:14:34 -07:00
Leonard Chan 966386514b [compiler-rt][hwasan] Setup hwasan thread handling on Fuchsia
This patch splits up hwasan thread creation between `__sanitizer_before_thread_create_hook`,
`__sanitizer_thread_create_hook`, and `__sanitizer_thread_start_hook`.
The linux implementation creates the hwasan thread object inside the
new thread. On Fuchsia, we know the stack bounds before thread creation,
so we can initialize part of the thread object in `__sanitizer_before_thread_create_hook`,
then initialize the stack ring buffer in `__sanitizer_thread_start_hook`
once we enter the thread.

Differential Revision: https://reviews.llvm.org/D104085
2021-07-07 15:05:28 -07:00
Michał Górny 2d68bb1765 [compiler-rt] [test] Fix asan symbolize tests on py3.10
Update the asan_symbolize_script for changes in argparse output
in Python 3.10.  The parser output 'options' instead of 'optional
arguments'.

Differential Revision: https://reviews.llvm.org/D105489
2021-07-06 20:41:35 +02:00
Fangrui Song 7b6b15e010 [profile][test] Improve coverage-linkage.cpp with ld.lld --gc-sections
The __llvm_prf_names section uses SHF_GNU_RETAIN.  However, GNU ld before 2015-10
(https://sourceware.org/bugzilla/show_bug.cgi?id=19161) neither supports it nor
retains __llvm_prf_names according to __start___llvm_prf_names. So --gc-sections
does not work on such old GNU ld.

This is not a problem for gold and sufficiently new lld.
2021-07-06 11:08:47 -07:00
Nico Weber f814cd7406 Revert "[profile][test] Improve coverage-linkage.cpp"
This reverts commit 36ba86fe8a.
Fails on some bots, see comments on
https://reviews.llvm.org/rG36ba86fe8a29cdf3251b786db7f342efde666cb2
2021-07-06 08:49:43 -04:00
Florian Mayer 745758acf3 [hwasan] Fix incorrect candidate matching for stack OOB.
We would find an address with matching tag, only to discover in
ShowCandidate that it's very far away from [stack].

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105197
2021-07-06 12:24:07 +01:00
Florian Mayer a0b1f3aac5 [hwasan] Check for overflow when searching candidates.
If the fault address is at the boundary of memory regions, this could
cause us to segfault otherwise.

Ran test with old compiler_rt to make sure it fails.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105032
2021-07-06 11:22:13 +01:00
Christopher Di Bella 873e8b96b1 [compiler-rt][iwyu] explicitly includes `<new>` in xray_utils.cpp
Compiling compiler-rt with Clang modules and libc++ revealed that the
global `operator new` is being called without including `<new>`.

Differential Revision: https://reviews.llvm.org/D105401
2021-07-05 21:46:42 +00:00
Fangrui Song 36ba86fe8a [profile][test] Improve coverage-linkage.cpp 2021-07-05 12:46:06 -07:00
Mitch Phillips dd1c4bd09d Name all DEATH tests using 'DeathTest' suffix.
gtest highly recommends this prefix, and runs death tests first
(https://github.com/google/googletest/blob/master/docs/advanced.md#death-test-naming).
This may help with some spurious bot failures like
https://lab.llvm.org/buildbot/#/builders/169/builds/1290/steps/25/logs/stdio.

Reviewed By: cryptoad, vitalybuka

Differential Revision: https://reviews.llvm.org/D105371
2021-07-02 13:51:16 -07:00
Marco Vanotti c5d725172d Revert "Refactor mutation strategies into a standalone library"
This reverts commit 361f742f16.
2021-07-02 09:45:11 -07:00
Aaron Green 361f742f16 Refactor mutation strategies into a standalone library
This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
  * UseCmp
  * UseMemmem
  * SetCustomMutator
  * SetCustomCrossOver
  * LateInitialize (similar to the MutationDispatcher's original constructor)
  * Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
  * StartMutationSequence
  * MutationSequence
  * DictionaryEntrySequence
  * RecommendDictionary
  * RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447
2021-07-02 09:20:23 -07:00
Nico Weber a92964779c Revert "[InstrProfiling] Use external weak reference for bias variable"
This reverts commit 33a7b4d9d8.
Breaks check-profile on macOS, see comments on https://reviews.llvm.org/D105176
2021-07-02 09:05:12 -04:00
Vitaly Buka 07a1f3513e [scudo] Fix test on aarch64 without MTE 2021-07-01 21:40:04 -07:00
Petr Hosek 33a7b4d9d8 [InstrProfiling] Use external weak reference for bias variable
We need the compiler generated variable to override the weak symbol of
the same name inside the profile runtime, but using LinkOnceODRLinkage
results in weak symbol being emitted which leads to an issue where the
linker might choose either of the weak symbols potentially disabling the
runtime counter relocation.

This change replaces the use of weak definition inside the runtime with
an external weak reference to address the issue. We also place the
compiler generated symbol inside a COMDAT group so dead definition can
be garbage collected by the linker.

Differential Revision: https://reviews.llvm.org/D105176
2021-07-01 15:25:31 -07:00
Vitaly Buka fe30963600 [scudo] Untag BlockEnd in reallocate
If we get here from reallocate, BlockEnd is tagged. Then we
will storeTag(UntaggedEnd) into the header of the next chunk.

Luckily header tag is 0 so unpatched code still works.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D105261
2021-07-01 13:02:58 -07:00
Vitaly Buka 78e70cee0d [scudo] Remove false DCHECK
MTE Cache.store passes MAP_NOACCESS here.

Reviewed By: pcc, cryptoad

Differential Revision: https://reviews.llvm.org/D105266
2021-07-01 12:06:33 -07:00
Vitaly Buka 33b579c8a5 [NFC][scudo] Exctract getOptionsForConfig in test 2021-07-01 11:59:30 -07:00
Vitaly Buka 7d20747203 [gwp_asan] Improve symbolizer script
Show inlined functions.
Hide unhelpful DWARF related warnings.

Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D105230
2021-07-01 10:37:07 -07:00
Vitaly Buka 39a15b5ae0 [NFC][scudo] Extract MapAllocatorTest for TEST_F 2021-07-01 10:22:55 -07:00
Emily Shi f03d29601e [NFC][compiler-rt] add back solaris xfail for unpoison-alternate-stack.cpp 2021-07-01 10:13:00 -07:00
theidexisted ae7c3959e0 [Compiler-rt] Add delete for noncopyable
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D105077
2021-06-30 20:53:47 -07:00
Vitaly Buka 00f474e9b1 [GWP_ASAN] Enable ARM tests
Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D105233
2021-06-30 20:43:33 -07:00
Vitaly Buka b1fd009aab [scudo] GWP_ASAN runs on untagged pointers
It's already covered by multiple tests, but to trigger
this path we need MTE+GWP which disabled.

Reviewed By: hctim, pcc

Differential Revision: https://reviews.llvm.org/D105232
2021-06-30 20:41:30 -07:00
Florian Mayer 90f244c843 [hwasan] Add missing newline in report.
Reviewed By: glider

Differential Revision: https://reviews.llvm.org/D105190
2021-06-30 15:48:45 +01:00
Florian Mayer ad8494c021 [hwasan] Make sure we retag with a new tag on free.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105021
2021-06-30 11:13:38 +01:00
Jianzhou Zhao ae6648cee0 [dfsan] Expose dfsan_get_track_origins to get origin tracking status
This allows application code checks if origin tracking is on before
printing out traces.

-dfsan-track-origins can be 0,1,2.
The current code only distinguishes 1 and 2 in compile time, but not at runtime.
Made runtime distinguish 1 and 2 too.

Reviewed By: browneee

Differential Revision: https://reviews.llvm.org/D105128
2021-06-29 20:32:39 +00:00
Leonard Chan 8e74668e96 [NFC][compiler-rt][hwasan] Re-use ring buffer size calculation
Users can call HwasanThreadList::GetRingBufferSize rather than RingBufferSize
to prevent having to do the calculation in RingBufferSize. This will be useful
for Fuchsia where we plan to initialize the stack ring buffer separately from
the rest of thread initialization.

Differential Revision: https://reviews.llvm.org/D104823
2021-06-29 11:57:33 -07:00
Leonard Chan c4a00ed851 [NFC][compiler-rt][hwasan] Move GetCurrentThread to hwasan.cpp
We can reuse the same implementation for getting the current thread on fuchsia.

Differential Revision: https://reviews.llvm.org/D104824
2021-06-29 11:55:30 -07:00
Florian Mayer b458bb8c04 [hwasan] Display causes in order of probability.
A heap or global buffer that is far away from the faulting address is
unlikely to be the cause, especially if there is a potential
use-after-free as well, so we want to show it after the other
causes.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104781
2021-06-29 13:00:05 +01:00
Florian Mayer 400509238a Revert "[hwasan] print exact mismatch offset for short granules."
Broke x86 LAM bot.

This reverts commit 2a60ab76a7.
2021-06-29 11:57:04 +01:00
Kai Luo ae79854e1c [AIX][compiler-rt] Deliver libatomic.a at top level library directory
Install libatomic.a in top level library directory so that compiler can find it in search directories.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D104908
2021-06-29 03:14:20 +00:00
Florian Mayer 2a60ab76a7 [hwasan] print exact mismatch offset for short granules.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104463
2021-06-28 19:01:31 +01:00
David Spickett ad81dea9f6 [compiler-rt][asan] Disable two tests on Arm Thumb
I can't be sure of the cause but I believe these fail
due to to fast unwinding not working on Thumb.

Whatever the case, they have been failing on our bots
for a long time:
https://lab.llvm.org/buildbot/#/builders/170/builds/46

Require fast-unwinder-works for both.
2021-06-28 10:34:37 +00:00
Florian Mayer 8f9db0aeeb [hwasan] Show sp in register dump.
Reviewed By: hctim, eugenis

Differential Revision: https://reviews.llvm.org/D104787
2021-06-28 10:28:59 +01:00
Andrew Browne 45f6d5522f [DFSan] Change shadow and origin memory layouts to match MSan.
Previously on x86_64:

  +--------------------+ 0x800000000000 (top of memory)
  | application memory |
  +--------------------+ 0x700000008000 (kAppAddr)
  |                    |
  |       unused       |
  |                    |
  +--------------------+ 0x300000000000 (kUnusedAddr)
  |       origin       |
  +--------------------+ 0x200000008000 (kOriginAddr)
  |       unused       |
  +--------------------+ 0x200000000000
  |   shadow memory    |
  +--------------------+ 0x100000008000 (kShadowAddr)
  |       unused       |
  +--------------------+ 0x000000010000
  | reserved by kernel |
  +--------------------+ 0x000000000000

  MEM_TO_SHADOW(mem) = mem & ~0x600000000000
  SHADOW_TO_ORIGIN(shadow) = kOriginAddr - kShadowAddr + shadow

Now for x86_64:

  +--------------------+ 0x800000000000 (top of memory)
  |    application 3   |
  +--------------------+ 0x700000000000
  |      invalid       |
  +--------------------+ 0x610000000000
  |      origin 1      |
  +--------------------+ 0x600000000000
  |    application 2   |
  +--------------------+ 0x510000000000
  |      shadow 1      |
  +--------------------+ 0x500000000000
  |      invalid       |
  +--------------------+ 0x400000000000
  |      origin 3      |
  +--------------------+ 0x300000000000
  |      shadow 3      |
  +--------------------+ 0x200000000000
  |      origin 2      |
  +--------------------+ 0x110000000000
  |      invalid       |
  +--------------------+ 0x100000000000
  |      shadow 2      |
  +--------------------+ 0x010000000000
  |    application 1   |
  +--------------------+ 0x000000000000

  MEM_TO_SHADOW(mem) = mem ^ 0x500000000000
  SHADOW_TO_ORIGIN(shadow) = shadow + 0x100000000000

Reviewed By: stephan.yichao.zhao, gbalats

Differential Revision: https://reviews.llvm.org/D104896
2021-06-25 17:00:38 -07:00
Matheus Izvekov ad14b5b008 [clang] Stop providing builtin overload candidate for relational function pointer comparisons
Word on the grapevine was that the committee had some discussion that
ended with unanimous agreement on eliminating relational function pointer comparisons.

We wanted to be bold and just ban all of them cold turkey.
But then we chickened out at the last second and are going for
eliminating just the spaceship overload candidate instead, for now.

See D104680 for reference.

This should be fine and "safe", because the only possible semantic change this
would cause is that overload resolution could possibly be ambiguous if
there was another viable candidate equally as good.

But to save face a little we are going to:
* Issue an "error" for three-way comparisons on function pointers.
  But all this is doing really is changing one vague error message,
  from an "invalid operands to binary expression" into an
  "ordered comparison of function pointers", which sounds more like we mean business.
* Otherwise "warn" that comparing function pointers like that is totally
  not cool (unless we are told to keep quiet about this).

Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>

Reviewed By: rsmith

Differential Revision: https://reviews.llvm.org/D104892
2021-06-26 00:08:02 +02:00
Vitaly Buka a9f3ac9e3d Revert "[hwasan] print exact mismatch offset for short granules."
Breaks sanitizer-x86_64-linux-android and sanitizer-x86_64-linux-qemu bots.

This reverts commit 7e3f8b8aff.
2021-06-24 17:33:24 -07:00
Vitaly Buka a435741110 [scudo] Clang-format tests 2021-06-24 17:33:24 -07:00
Emily Shi f26adaa28d [compiler-rt][test] fix zero_page_pc on arm64e
on arm64e, pointer auth would catch this access violation before asan.
sign the function pointer so pointer auth will ignore this violation and let asan catch it in this test case.

rdar://79652167

Reviewed By: delcypher

Differential Revision: https://reviews.llvm.org/D104828
2021-06-24 13:40:12 -07:00
Florian Mayer 7e3f8b8aff [hwasan] print exact mismatch offset for short granules.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104463
2021-06-24 09:58:02 +01:00
Dmitry Vyukov 2f6f24f010 tsan: re-enable mmap_stress.cpp test
The comment says it was flaky in 2016,
but it wasn't possible to debug it back then.
Re-enable the test at least on linux/x86_64.
It will either work, or at least we should
see failure output from lit today.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D104592
2021-06-24 09:48:34 +02:00
Dmitry Vyukov 1db68fcd8e tsan: fix mmap atomicity
Mmap interceptor is not atomic in the sense that it
exposes unmapped shadow for a brief period of time.
This breaks programs that mmap over another mmap
and access the region concurrently.
Don't unmap shadow in the mmap interceptor to fix this.
Just mapping new shadow on top should be enough to zero it.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D104593
2021-06-24 09:47:44 +02:00
Vitaly Buka 35e1dbd189 [scudo] Fix test on arch without MTE 2021-06-24 00:07:24 -07:00
Vitaly Buka 6fd963ab64 [scudo] Avoid ifdef in test 2021-06-23 23:58:09 -07:00
Vitaly Buka 99ec78c0fe [scudo] Fix use of ScopedDisableMemoryTagChecks in test 2021-06-23 23:57:17 -07:00
Vitaly Buka 533abb7ecf [scudo] Enabled MTE before the first allocator
Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D103726
2021-06-23 23:01:26 -07:00
Emily Shi 05e48eccdd [NFC][compiler-rt] Remove iOS xfail for unpoison-alternate-stack
This test was originally xfailed because of a bug on iOS. This has since been fixed, so reenabling the test.
2021-06-23 13:47:56 -07:00
Leonard Chan b06fbdeae7 [compiler-rt][hwasan] Add InitState options to thread initialization
Similar to InitOptions in asan, we can use this optional struct for
initializing some members thread objects before they are created. On
linux, this is unused and can remain undefined. On fuchsia, this will
just be the stack bounds.

Differential Revision: https://reviews.llvm.org/D104553
2021-06-23 11:42:14 -07:00
Evgenii Stepanov d693957e58 [scudo] Handle predefined M_MEMTAG_TUNING_* constants (NFC).
Bionic <malloc.h> may provide the definitions of M_MEMTAG_TUNING_* constants.
Do not redefine them in that case.

Differential Revision: https://reviews.llvm.org/D104758
2021-06-22 17:39:55 -07:00
Petr Hosek 959dbd1761 Revert "Revert "[cmake] [compiler-rt] Call llvm_setup_rpath() when adding shared libraries.""
This reverts commit 21c008d5a5 since
it broke the build on macOS and Windows with the following error:

  The install of the clang_rt.<na,e> target requires changing an
  RPATH from the build tree, but this is not supported with the Ninja
  generator unless on an ELF-based platform.  The
  CMAKE_BUILD_WITH_INSTALL_RPATH variable may be set to avoid this relinking
  step.
2021-06-22 16:11:29 -07:00
Petr Hosek cadfaf2df4 [compiler-rt] Make use of undefined symbols configurable
We want to disable the use of undefined symbols on Fuchsia, but there
are cases where it might be desirable so may it configurable.

Differential Revision: https://reviews.llvm.org/D104728
2021-06-22 14:25:36 -07:00
Petr Hosek 21c008d5a5 Revert "[cmake] [compiler-rt] Call llvm_setup_rpath() when adding shared libraries."
This reverts commit 78fd93e039 as
a follow up to D91099.
2021-06-22 12:42:39 -07:00
Nikita Popov ae1093921f Revert "[compiler-rt] Make use of undefined symbols configurable"
This reverts commit ed7086ad46.
This reverts commit b9792638b0.

This breaks cmake with message:

    CMake Error at llvm-project/compiler-rt/CMakeLists.txt:449:
      Parse error.  Expected "(", got newline with text "
2021-06-22 21:20:20 +02:00
Petr Hosek ed7086ad46 [CMake] Fix the option declaration
This addresses build issue introduced in
b9792638b0.
2021-06-22 11:58:26 -07:00
Petr Hosek b9792638b0 [compiler-rt] Make use of undefined symbols configurable
We want to disable the use of undefined symbols on Fuchsia, but there
are cases where it might be desirable so may it configurable.

Differential Revision: https://reviews.llvm.org/D104728
2021-06-22 11:49:31 -07:00
Petr Hosek fa5f425209 [compiler-rt][CMake] Drop flags that are set by default for Fuchsia
-Wl,-z,now is set by the Fuchsia driver, -Wl,-z,relro is the default
in LLD.
2021-06-22 11:49:30 -07:00
Andrew Browne 759e797767 [DFSan][NFC] Refactor Origin Address Alignment code.
Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D104565
2021-06-21 14:52:02 -07:00
Petr Hosek d4c2b973ed [profile] Fix variable name
This fixes a bug introduced in d85c258fd1.
2021-06-19 14:55:32 -07:00
Fangrui Song 8ea2a58a2e [llvm-profdata] Make diagnostics consistent with the (no capitalization, no period) style
The format is currently inconsistent. Use the https://llvm.org/docs/CodingStandards.html#error-and-warning-messages style.

And add `error:` or `warning:` to CHECK lines wherever appropriate.
2021-06-19 14:54:25 -07:00
Petr Hosek d85c258fd1 [profile] Don't publish VMO if there are no counters
If there are no counters, there's no need to publish the VMO.

Differential Revision: https://reviews.llvm.org/D102786
2021-06-19 14:47:57 -07:00
George Balatsouras 070556237e [libfuzzer] Disable failing DFSan-related tests
These have been broken by https://reviews.llvm.org/D104494.
However, `lib/fuzzer/dataflow/` is unused (?) so addressing this is not a priority.

Added TODOs to re-enable them in the future.

Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D104568
2021-06-19 01:09:19 +00:00
Fangrui Song 5540470f64 [profile][test] Delete profraw directory so that tests are immune to format version upgrade 2021-06-18 16:44:03 -07:00
Leonard Chan f7999e73ca [NFC][compiler-rt][hwasan] Move hwasanThreadList().CreateCurrentThread() into InitThreads
Once D104553 lands, CreateCurrentThread will be able to accept optional
parameters for initializing the hwasan thread object. On fuchsia, we can get
stack info in the platform-specific InitThreads and pass it through
CreateCurrentThread. On linux, this is a no-op.

Differential Revision: https://reviews.llvm.org/D104561
2021-06-18 13:40:55 -07:00
Andrew Browne 14407332de [DFSan] Cleanup code for platforms other than Linux x86_64.
These other platforms are unsupported and untested.
They could be re-added later based on MSan code.

Reviewed By: gbalats, stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D104481
2021-06-18 11:21:46 -07:00
Florian Mayer e0b68f7149 [hwasan] Clarify report for allocation-tail-overwritten.
Explain what the given stack trace means before showing it, rather than
only in the paragraph at the end.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104523
2021-06-18 18:52:41 +01:00
Leonard Chan 88d93923e6 [compiler-rt][hwasan] Move Thread::Init into hwasan_linux.cpp
This allows for other implementations to define their own version of `Thread::Init`.
This will be the case for Fuchsia where much of the thread initialization can be
broken up between different thread hooks (`__sanitizer_before_thread_create_hook`,
`__sanitizer_thread_create_hook`, `__sanitizer_thread_start_hook`). Namely, setting
up the heap ring buffer and stack info and can be setup before thread creation.
The stack ring buffer can also be setup before thread creation, but storing it into
`__hwasan_tls` can only be done on the thread start hook since it's only then we
can access `__hwasan_tls` for that thread correctly.

Differential Revision: https://reviews.llvm.org/D104248
2021-06-18 10:32:41 -07:00
Matt Morehouse 493565a4aa [HWASan] Run LAM tests with -hwasan-generate-tags-with-calls.
The default callback instrumentation in x86 LAM mode uses ASLR bits
to randomly choose a tag, and thus has a 1/64 chance of choosing a
stack tag of 0, causing stack tests to fail intermittently.  By using
__hwasan_generate_tag to pick tags, we guarantee non-zero tags and
eliminate the test flakiness.

aarch64 doesn't seem to have this problem using thread-local addresses
to pick tags, so perhaps we can remove this workaround once we implement
a similar mechanism for LAM.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104470
2021-06-18 08:10:51 -07:00
George Balatsouras c6b5a25eeb [dfsan] Replace dfs$ prefix with .dfsan suffix
The current naming scheme adds the `dfs$` prefix to all
DFSan-instrumented functions.  This breaks mangling and prevents stack
trace printers and other tools from automatically demangling function
names.

This new naming scheme is mangling-compatible, with the `.dfsan`
suffix being a vendor-specific suffix:
https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-structure

With this fix, demangling utils would work out-of-the-box.

Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D104494
2021-06-17 22:42:47 -07:00
Andrew Browne 39295e92f7 Revert "[DFSan] Cleanup code for platforms other than Linux x86_64."
This reverts commit 8441b993bd.

Buildbot failures.
2021-06-17 14:19:18 -07:00
Andrew Browne 8441b993bd [DFSan] Cleanup code for platforms other than Linux x86_64.
These other platforms are unsupported and untested.
They could be re-added later based on MSan code.

Reviewed By: gbalats, stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D104481
2021-06-17 14:08:40 -07:00
Yabin Cui ac35ed5d34 [compiler-rt][hwasan]: undefine new/delete operators with alignment on Android.
This is to fix build on Android. And we don't want to intercept more new/delete operators on Android.

Differential Revision: https://reviews.llvm.org/D104313
2021-06-17 09:28:12 -07:00
Kevin Athey 249d9b051f Revert "Remove obsolete call to AsyncSignalSafeLazyInitiFakeStack."
This reverts commit 07481b3796.

Differential Revision: https://reviews.llvm.org/D104469
2021-06-17 09:27:02 -07:00
Florian Mayer ccc0f777f6 [hwasan] Improve report for addresses within regions.
Before: ADDR is located -320 bytes to the right of 1072-byte region
After: ADDR is located 752 bytes inside 1072-byte region

Reviewed By: eugenis, walli99

Differential Revision: https://reviews.llvm.org/D104412
2021-06-17 12:01:30 +01:00
Florian Mayer 18070723ef [hwasan] Do not use short granule tags as poison tags.
Short granule tags as poison cause a UaF to read the referenced
memory to retrieve the tag, and means we do not detect the UaF
if the last granule's tag is still around.

This only increases the change of not catching a UaF from
0.39 % (1 / 256) to 0.42 % (1 / (256 - 17)).

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D104304
2021-06-17 11:59:37 +01:00
Florian Mayer b18f30fb2d [NFC] test commit, fix namespace ending comment. 2021-06-17 11:18:36 +01:00
Kristof Beyls 6f0e74cd58 Avoid unnecessary AArch64 DSB in __clear_cache in some situations.
The dsb after instruction cache invalidation only needs to be executed
if any instruction cache invalidation did happen.
Without this change, if the CTR_EL0.DIC bit indicates that instruction
cache invalidation is not needed, __clear_cache would execute two dsb
instructions in a row; with the second one being unnecessary.

Differential Revision: https://reviews.llvm.org/D104371
2021-06-17 07:45:06 +01:00
Kevin Athey 07481b3796 Remove obsolete call to AsyncSignalSafeLazyInitiFakeStack.
Code was originally added for Myriad D46626 which was removed
with D104279.

related to: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka, morehouse

Differential Revision: https://reviews.llvm.org/D104419
2021-06-16 18:26:33 -07:00
Kostya Kortchinsky 8b062b6160 [scudo] Ensure proper allocator alignment in TSD test
The `MockAllocator` used in `ScudoTSDTest` wasn't allocated
properly aligned, which resulted in the `TSDs` of the shared
registry not being aligned either. This lead to some failures
like: https://reviews.llvm.org/D103119#2822008

This changes how the `MockAllocator` is allocated, same as
Vitaly did in the combined tests, properly aligning it, which
results in the `TSDs` being aligned as well.

Add a `DCHECK` in the shared registry to check that it is.

Differential Revision: https://reviews.llvm.org/D104402
2021-06-16 14:21:58 -07:00
Kevin Athey c4992bf593 [NFC][sanitizer] Remove calls to __asan_get_current_fake_stack
Unnecessary with -fsanitize-address-use-after-return=never.

for issue: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D104154
2021-06-15 18:52:22 -07:00
Vitaly Buka 6478ef61b1 [asan] Remove Asan, Ubsan support of RTEMS and Myriad
Differential Revision: https://reviews.llvm.org/D104279
2021-06-15 12:59:05 -07:00
Leonard Chan fc006b3e5d [compiler-rt][hwasan] Add GetShadowOffset function
Similar to SHADOW_OFFSET on asan, we can use this for hwasan so platforms that
use a constant value for the start of shadow memory can just use the constant
rather than access a global.

Differential Revision: https://reviews.llvm.org/D104275
2021-06-15 10:46:11 -07:00
Alex Richardson 13f0b85212 Fix Windows builders after 244601f472
Apparently __builtin_abort() is not supported when targetting Windows.
This should fix the following builder errors:
clang_rt.builtins-x86_64.lib(int_util.c.obj) : error LNK2019: unresolved
external symbol __builtin_abort referenced in function __compilerrt_abort_impl
2021-06-15 09:47:29 +01:00
Alex Richardson 244601f472 [builtins] Allow compiling the builtins without libc headers
When compiled with -ffreestanding, we should not assume that headers
declaring functions such as abort() are available. While the compiler may
still emit calls to those functions [1], we should not require the headers
to build compiler-rt since that can result in a cyclic dependency graph:
The compiler-rt functions might be required to build libc.so, but the libc
headers such as stdlib.h might only be available once libc has been built.

[1] From https://gcc.gnu.org/onlinedocs/gcc/Standards.html:
GCC requires the freestanding environment provide memcpy, memmove,
memset and memcmp. Finally, if __builtin_trap is used, and the target
does not implement the trap pattern, then GCC emits a call to abort.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D103876
2021-06-15 09:08:59 +01:00
Lang Hames 68c161090e [ORC-RT] Fix void function handling in the WrapperFunction utility.
Handlers returning void previously caused compile errors. Fix that by
substituting SPSEmpty placeholder values.
2021-06-15 15:49:40 +10:00
Vitaly Buka 51ed1c6ccc [NFC][hwasan] Fix "implicitly declaring library function" 2021-06-14 21:58:59 -07:00
Vitaly Buka b8919fb0ea [NFC][sanitizer] clang-format some code 2021-06-14 18:05:22 -07:00
Leonard Chan 312011899a [compiler-rt][hwasan] Add newline between record_addr lines on frame record dumps
If SymbolizePC failes, it's possible for the newline to not be emitted.

Differential Revision: https://reviews.llvm.org/D103845
2021-06-14 17:13:13 -07:00
Matt Morehouse b87894a1d2 [HWASan] Enable globals support for LAM.
Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D104265
2021-06-14 14:20:44 -07:00
George Balatsouras 98504959a6 [dfsan] Add stack-trace printing functions to dfsan interface
Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D104165
2021-06-14 14:09:00 -07:00
Lang Hames ba9e28064b [ORC-RT] Fix an error check. 2021-06-14 11:59:20 +10:00
Khem Raj 60e32a1f34 [fuzzer] Fix build on musl
cstddef is needed for size_t definition.
(Multiple headers can provide size_t but none of them exists.)

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D96213
2021-06-13 12:32:07 -07:00
Lang Hames a7c3105adb [ORC-RT] Remove unused header in unit test. 2021-06-13 20:45:20 +10:00
Lang Hames e405db075b [ORC-RT] Fix a comment. 2021-06-13 20:26:51 +10:00
Lang Hames 49f4a58d53 [ORC-RT] Split Simple-Packed-Serialization code into its own header.
This will simplify integration of this code into LLVM -- The
Simple-Packed-Serialization code can be copied near-verbatim, but
WrapperFunctionResult will require more adaptation.
2021-06-13 10:17:13 +10:00
Kai Luo 6393164cf2 [AIX][compiler-rt] Fix cmake build of libatomic for cmake-3.16+
cmake-3.16+ for AIX changes the default behavior of building a `SHARED` library which breaks AIX's build of libatomic, i.e., cmake-3.16+ builds `SHARED` as an archive of dynamic libraries. To fix it, we have to build `libatomic.so.1` as `MODULE` which keeps `libatomic.so.1` as an normal dynamic library.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D103786
2021-06-12 00:12:29 +00:00
Reid Kleckner b9af157fd1 [ASan/Win] Hide index from compiler to avoid new clang warning 2021-06-11 16:12:28 -07:00
Kevin Athey 60084d4900 [sanitizer] Replace -mllvm -asan-use-after-return in compile-rt tests with -fsanitize-address-use-after-return (NFC)
for issue: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D104146
2021-06-11 15:13:22 -07:00
John Paul Adrian Glaubitz 27ef43d279 [compiler-rt] Add platform detection support for x32
Currently, the compiler-rt build system checks only whether __X86_64
is defined to determine whether the default compiler-rt target arch
is x86_64. Since x32 defines __X86_64 as well, we must also check that
the default pointer size is eight bytes and not four bytes to properly
detect a 64-bit x86_64 compiler-rt default target arch.

Reviewed By: hvdijk, vitalybuka

Differential Revision: https://reviews.llvm.org/D99988
2021-06-11 19:44:04 +01:00
Matt Morehouse 0867edfc64 [HWASan] Add basic stack tagging support for LAM.
Adds the basic instrumentation needed for stack tagging.

Currently does not support stack short granules or TLS stack histories,
since a different code path is followed for the callback instrumentation
we use.

We may simply wait to support these two features until we switch to
a custom calling convention.

Patch By: xiangzhangllvm, morehouse

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D102901
2021-06-11 08:21:17 -07:00
Stephen Hines 6455418d3d [compiler-rt] [builtins] [AArch64] Add missing AArch64 data synchronization barrier (dsb) to __clear_cache
https://developer.arm.com/documentation/den0024/a/Caches/Cache-maintenance
covers how to properly clear caches on AArch64, and the builtin
implementation was missing a `dsb ish` after clearing the icache for the
selected range.

Reviewed By: kristof.beyls

Differential Revision: https://reviews.llvm.org/D104094
2021-06-11 02:13:48 -07:00
Arthur Eubanks b73742bc8d [Profile] Remove redundant check
This is already checked outside the loop.

Followup to D104050.
2021-06-10 16:24:53 -07:00
Arthur Eubanks 189428c8fc [Profile] Handle invalid profile data
This mostly follows LLVM's InstrProfReader.cpp error handling.
Previously, attempting to merge corrupted profile data would result in
crashes. See https://crbug.com/1216811#c4.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D104050
2021-06-10 16:10:13 -07:00
Kevin Athey ddac31c593 Cleanup for final comment on https://reviews.llvm.org/D103304
Consolidate DEFINE_STACK_MALLOC to simplify.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103965
2021-06-09 13:47:12 -07:00
David Spickett 11ad9e31eb [compiler-rt] Mark symbolize_stack_fp test unsupported on Arm Thumb
The new test `symbolize_stack_fp.cpp` added in
https://reviews.llvm.org/D102046 assumes that
we can fall back to the fast unwinder.

This is not the case for Thumb and the test is currently
failing on our v7 thumb bot:
https://lab.llvm.org/buildbot/#/builders/26/builds/2096

Skip the test if we're building for a Thumb target.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D103512
2021-06-09 08:50:21 +00:00
serge-sans-paille 4144827514 Prevent generation of dependency on _cxa_guard for static initialization
This fixes an issue introduced by https://reviews.llvm.org/D70662

Function-scope static initialization are guarded in C++, so we should probably
not use it because it introduces a dependency on __cxa_guard* symbols.
In the context of clang, libasan is linked statically, and it currently needs to
the odd situation where compiling C code with clang and asan requires -lstdc++

Differential Revision: https://reviews.llvm.org/D102475
2021-06-09 09:38:59 +02:00
David Blaikie 8051a48e65 ORTRT: Add tests for string_view equality and inequality operators 2021-06-08 17:53:59 -07:00
David Blaikie 4d9cc7c244 Add a couple of missing includes 2021-06-08 17:53:58 -07:00
David Blaikie cb09f2b10c Rename compiler-rt/lib/orc/endian.h to endianness.h to avoid conflict with system headers 2021-06-08 17:53:58 -07:00
Kevin Athey af8c59e06d Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always).
In addition:
  - optionally add global flag to capture compile intent for UAR:
    __asan_detect_use_after_return_always.
    The global is a SANITIZER_WEAK_ATTRIBUTE.

for issue: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103304
2021-06-08 14:39:06 -07:00
Daniel Michael 2551053e8d [scudo] Add Scudo support for Trusty OS
trusty.cpp and trusty.h define Trusty implementations of map and other
platform-specific functions. In addition to adding Trusty configurations
in allocator_config.h and size_class_map.h, MapSizeIncrement and
PrimaryEnableRandomOffset are added as configurable options in
allocator_config.h.
Background on Trusty: https://source.android.com/security/trusty

Differential Revision: https://reviews.llvm.org/D103578
2021-06-08 14:02:10 -07:00
Leonard Chan a9ea0a6a77 Fix for failing test mentioned in https://reviews.llvm.org/D103564.
This updates the path shown in the stack trace.
2021-06-08 12:38:01 -07:00
Leonard Chan 944b3c53ae [NFC][compiler-rt][hwasan] Move allocation functions into their own file
This removes the `__sanitizer_*` allocation function definitions from
`hwasan_interceptors.cpp` and moves them into their own file. This way
implementations that do not use interceptors at all can just ignore
(almost) everything in `hwasan_interceptors.cpp`.

Also remove some unused headers in `hwasan_interceptors.cpp` after the move.

Differential Revision: https://reviews.llvm.org/D103564
2021-06-08 12:08:23 -07:00
David Blaikie c5d56fec50 NFC: .clang-tidy: Inherit configs from parents to improve maintainability
In the interests of disabling misc-no-recursion across LLVM (this seems
like a stylistic choice that is not consistent with LLVM's
style/development approach) this NFC preliminary change adjusts all the
.clang-tidy files to inherit from their parents as much as possible.

This change specifically preserves all the quirks of the current configs
in order to make it easier to review as NFC.

I validatad the change is NFC as follows:

for X in `cat ../files.txt`;
do
  mkdir -p ../tmp/$(dirname $X)
  touch $(dirname $X)/blaikie.cpp
  clang-tidy -dump-config $(dirname $X)/blaikie.cpp > ../tmp/$(dirname $X)/after
  rm $(dirname $X)/blaikie.cpp
done

(similarly for the "before" state, without this patch applied)

for X in `cat ../files.txt`;
do
  echo $X
  diff \
    ../tmp/$(dirname $X)/before \
    <(cat ../tmp/$(dirname $X)/after \
      | sed -e "s/,readability-identifier-naming\(.*\),-readability-identifier-naming/\1/" \
      | sed -e "s/,-llvm-include-order\(.*\),llvm-include-order/\1/" \
      | sed -e "s/,-misc-no-recursion\(.*\),misc-no-recursion/\1/" \
      | sed -e "s/,-clang-diagnostic-\*\(.*\),clang-diagnostic-\*/\1/")
done

(using sed to strip some add/remove pairs to reduce the diff and make it easier to read)

The resulting report is:
  .clang-tidy
  clang/.clang-tidy
  2c2
  < Checks:          'clang-diagnostic-*,clang-analyzer-*,-*,clang-diagnostic-*,llvm-*,misc-*,-misc-unused-parameters,-misc-non-private-member-variables-in-classes,-readability-identifier-naming,-misc-no-recursion'
  ---
  > Checks:          'clang-diagnostic-*,clang-analyzer-*,-*,clang-diagnostic-*,llvm-*,misc-*,-misc-unused-parameters,-misc-non-private-member-variables-in-classes,-misc-no-recursion'
  compiler-rt/.clang-tidy
  2c2
  < Checks:          'clang-diagnostic-*,clang-analyzer-*,-*,clang-diagnostic-*,llvm-*,-llvm-header-guard,misc-*,-misc-unused-parameters,-misc-non-private-member-variables-in-classes'
  ---
  > Checks:          'clang-diagnostic-*,clang-analyzer-*,-*,clang-diagnostic-*,llvm-*,misc-*,-misc-unused-parameters,-misc-non-private-member-variables-in-classes,-llvm-header-guard'
  flang/.clang-tidy
  2c2
  < Checks:          'clang-diagnostic-*,clang-analyzer-*,-*,llvm-*,-llvm-include-order,misc-*,-misc-no-recursion,-misc-unused-parameters,-misc-non-private-member-variables-in-classes'
  ---
  > Checks:          'clang-diagnostic-*,clang-analyzer-*,-*,llvm-*,misc-*,-misc-unused-parameters,-misc-non-private-member-variables-in-classes,-llvm-include-order,-misc-no-recursion'
  flang/include/flang/Lower/.clang-tidy
  flang/include/flang/Optimizer/.clang-tidy
  flang/lib/Lower/.clang-tidy
  flang/lib/Optimizer/.clang-tidy
  lld/.clang-tidy
  lldb/.clang-tidy
  llvm/tools/split-file/.clang-tidy
  mlir/.clang-tidy

The `clang/.clang-tidy` change is a no-op, disabling an option that was never enabled.
The compiler-rt and flang changes are no-op reorderings of the same flags.

(side note, the .clang-tidy file in parallel-libs is broken and crashes
clang-tidy because it uses "lowerCase" as the style instead of "lower_case" -
so I'll deal with that separately)

Differential Revision: https://reviews.llvm.org/D103842
2021-06-08 08:25:59 -07:00
Vitaly Buka b41b76b303 [NFC][scudo] Print errno of fork failure
This fork fails sometime on sanitizer-x86_64-linux-qemu bot.
2021-06-07 18:59:35 -07:00
Vitaly Buka 11539edf52 [NFC][LSAN] Limit the number of concurrent threads is the test
Test still fails with D88184 reverted.

The test was flaky on https://bugs.chromium.org/p/chromium/issues/detail?id=1206745 and
https://lab.llvm.org/buildbot/#/builders/sanitizer-x86_64-linux

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102218
2021-06-07 17:38:12 -07:00
George Balatsouras 5b4dda550e [dfsan] Add full fast8 support
Complete support for fast8:
- amend shadow size and mapping in runtime
- remove fast16 mode and -dfsan-fast-16-labels flag
- remove legacy mode and make fast8 mode the default
- remove dfsan-fast-8-labels flag
- remove functions in dfsan interface only applicable to legacy
- remove legacy-related instrumentation code and tests
- update documentation.

Reviewed By: stephan.yichao.zhao, browneee

Differential Revision: https://reviews.llvm.org/D103745
2021-06-07 17:20:54 -07:00
Jianzhou Zhao a82747fafe [dfsan] Fix internal build errors because of more strict warning checks 2021-06-07 16:55:56 +00:00
Jianzhou Zhao 2c82588dac [dfsan] Use the sanitizer allocator to reduce memory cost
dfsan does not use sanitizer allocator as others. In practice,
we let it use glibc's allocator since tcmalloc needs more work
to be working with dfsan well. With glibc, we observe large
memory leakage. This could relate to two things:

1) glibc allocator has limitation: for example, tcmalloc can reduce memory footprint 2x easily

2) glibc may call unmmap directly as an internal system call by using system call number. so DFSan has no way to release shadow spaces for those unmmap.

Using sanitizer allocator addresses the above issues
1) its memory management is close to tcmalloc

2) we can register callback when sanitizer allocator calls unmmap, so dfsan can release shadow spaces correctly.

Our experiment with internal server-based application proved that with the change, in a-few-day run, memory usage leakage is close to what tcmalloc does w/o dfsan.

This change mainly follows MSan's code.

1) define allocator callbacks at dfsan_allocator.h|cpp

2) mark allocator APIs to be discard

3) intercept allocator APIs

4) make dfsan_set_label consistent with MSan's SetShadow when setting 0 labels, define dfsan_release_meta_memory when unmap is called

5) add flags about whether zeroing memory after malloc/free. dfsan works at byte-level, so bit-level oparations can cause reading undefined shadow. See D96842. zeroing memory after malloc helps this. About zeroing after free, reading after free is definitely UB, but if user code does so, it is hard to debug an overtainting caused by this w/o running MSan. So we add the flag to help debugging.

This change will be split to small changes for review. Before that, a question is
"this code shares a lot of with MSan, for example, dfsan_allocator.* and dfsan_new_delete.*.
Does it make sense to unify the code at sanitizer_common? will that introduce some
maintenance issue?"

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D101204
2021-06-06 22:09:31 +00:00
Vitaly Buka e3258b0894 Revert "Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always)."
Windows is still broken.

This reverts commit 927688a4cd.
2021-06-05 00:39:50 -07:00
Kevin Athey 927688a4cd Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always).
In addition:
  - optionally add global flag to capture compile intent for UAR:
    __asan_detect_use_after_return_always.
    The global is a SANITIZER_WEAK_ATTRIBUTE.

for issue: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103304
2021-06-05 00:26:10 -07:00
Vitaly Buka 5b0e50550c [NFC][asan] Split fake_stack() into two funcions 2021-06-04 23:29:04 -07:00
Vitaly Buka 64b1cbca19 [NFC][asan] Fix include order 2021-06-04 23:25:03 -07:00
Vitaly Buka 6f122d96f7 [NFC][memprof] FIx delete[] usage in test 2021-06-04 23:17:23 -07:00
Vitaly Buka 585b9679ad [NFC][asan] format ThreadStackContainsAddress 2021-06-04 23:16:43 -07:00
Vitaly Buka 4a91118793 [NFC][memprof] Compile *.c tests as C 2021-06-04 23:16:42 -07:00
Vitaly Buka d8a4a2cb93 Revert "Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always)."
Reverts commits of D103304, it breaks Darwin.

This reverts commit 60e5243e59.
This reverts commit 26b3ea224e.
This reverts commit 17600ec32a.
2021-06-04 20:20:11 -07:00
Fangrui Song 35fb4b9a7c [lsan] Bump the thread limit from 1<<13 to 1<<22
This matches asan. Some applications need more than 1<<13 threads.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103731
2021-06-04 20:15:56 -07:00
Vitaly Buka 9ff982dbbd [NFC][scudo] Fix sign-compare warning in test 2021-06-04 19:33:42 -07:00
Vitaly Buka 57ba226296 [NFC][scudo] Convert to TYPED more ScudoCombinedTest 2021-06-04 19:26:03 -07:00
Vitaly Buka 70b29213eb [scudo] Remove ScopedString::Length
Differential Revision: https://reviews.llvm.org/D103725
2021-06-04 18:28:51 -07:00
Vitaly Buka df87aeb826 [Scudo] Improve ScopedString constructor
Avoid referencing elements beyond internal vector size.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D103718
2021-06-04 18:28:29 -07:00
Vitaly Buka b850798f11 [NFC][scudo] Fix sign-compare warning in test 2021-06-04 18:24:00 -07:00
Kevin Athey 17600ec32a remove windows tests for -asan_use-after-return=always (as this is currently disabled for Windows) 2021-06-04 16:59:24 -07:00
Kevin Athey 26b3ea224e fix SANITIZE_WINDOWS pragma for https://reviews.llvm.org/rG60e5243e593ec7e0feca42783edb735e9ac2c4a7 2021-06-04 16:46:54 -07:00
Kevin Athey 60e5243e59 Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always).
In addition:
  - optionally add global flag to capture compile intent for UAR:
    __asan_detect_use_after_return_always.
    The global is a SANITIZER_WEAK_ATTRIBUTE.

for issue: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103304
2021-06-04 16:30:47 -07:00
Kostya Kortchinsky 5019b0a565 [scudo] Fix String DCHECK
This resolves an issue tripping a `DCHECK`, as I was checking for the
capacity and not the size. We don't need to 0-init the Vector as it's
done already, and make sure we only 0-out the string on clear if it's
not empty.

Differential Revision: https://reviews.llvm.org/D103716
2021-06-04 13:41:59 -07:00
Vitaly Buka 1e6d135325 [scudo] Untag pointer in iterateOverChunks
Pointer comparison in Lambda will not work on tagged pointers.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D103496
2021-06-04 12:45:07 -07:00
Vitaly Buka 07c92b2e95 [scudo] Add memtag_test
Differential Revision: https://reviews.llvm.org/D103074
2021-06-04 12:38:29 -07:00
Vitaly Buka 39f928ed01 [scudo] Remove disableMemoryTagChecksTestOnly
And replace with ScopedDisableMemoryTagChecks.

Differential Revision: https://reviews.llvm.org/D103708
2021-06-04 12:38:29 -07:00
Fangrui Song b19c0ac7dd [profile] Add -fprofile-instr-generate tests for weak definition and various linkages 2021-06-04 10:26:55 -07:00
Vitaly Buka ba04c7c128 [scudo] Always exclude Tag 0
prepareTaggedChunk uses Tag 0 for header.

Android already PR_MTE_TAG_MASK to 0xfffe,
but with the patch we will not need to deppend
on the system configuration.

Reviewed By: pcc

Differential Revision: https://reviews.llvm.org/D103134
2021-06-04 10:20:32 -07:00
Nico Weber 50c0aaed47 Temporarily remove another test added in one of the tests added in effb87d
This test reads the test file removed in db3e4faa4d.
2021-06-04 10:42:37 -04:00
Nico Weber db3e4faa4d Temporarily remove one of the tests added in effb87dfa8
It fails on some Linux systems. Remove the test until we've figured
out what's going on. See https://crbug.com/1216005 for details.
2021-06-04 10:03:21 -04:00
Kostya Kortchinsky 868317b3fd [scudo] Rework Vector/String
Some platforms (eg: Trusty) are extremelly memory constrained, which
doesn't necessarily work well with some of Scudo's current assumptions.

`Vector` by default (and as such `String` and `ScopedString`) maps a
page, which is a bit of a waste. This CL changes `Vector` to use a
buffer local to the class first, then potentially map more memory if
needed (`ScopedString` currently are all stack based so it would be
stack data). We also want to allow a platform to prevent any dynamic
resizing, so I added a `CanGrow` templated parameter that for now is
always `true` but would be set to `false` on Trusty.

Differential Revision: https://reviews.llvm.org/D103641
2021-06-03 18:12:24 -07:00
Nico Weber 5c600dc6d4 Revert "Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always)."
This reverts commit 41b3088c3f.
Doesn't build on macOS, see comments on https://reviews.llvm.org/D103304
2021-06-03 21:01:11 -04:00
Leonard Chan d4e4799998 [NFC][compiler-rt][hwasan] Refactor hwasan functions
This moves the implementations for HandleTagMismatch, __hwasan_tag_mismatch4,
and HwasanAtExit from hwasan_linux.cpp to hwasan.cpp and declares them in hwasan.h.
This way, calls to those functions can be shared with the fuchsia implementation
without duplicating code.

Differential Revision: https://reviews.llvm.org/D103562
2021-06-03 14:26:35 -07:00
Kevin Athey 41b3088c3f Update and improve compiler-rt tests for -mllvm -asan_use_after_return=(never|[runtime]|always).
In addition:
  - optionally add global flag to capture compile intent for UAR:
    __asan_detect_use_after_return_always.
    The global is a SANITIZER_WEAK_ATTRIBUTE.

for issue: https://github.com/google/sanitizers/issues/1394

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103304
2021-06-03 13:13:51 -07:00
Vitaly Buka b40908e639 [NFC][scudo] Avoid integer overflow in test
releasePagesToOS may shrink RSS below the value stored on start.
2021-06-03 00:08:41 -07:00
Fangrui Song 87c43f3aa9 [InstrProfiling] Delete linkage/visibility toggling for Windows
The linkage/visibility of `__profn_*` variables are derived
from the profiled functions.

    extern_weak => linkonce
    available_externally => linkonce_odr
    internal => private
    extern => private
    _ => unchanged

The linkage/visibility of `__profc_*`/`__profd_*` variables are derived from
`__profn_*` with linkage/visibility wrestling for Windows.

The changes can be folded to the following without changing semantics.

```
if (TT.isOSBinFormatCOFF() && !NeedComdat) {
  Linkage = GlobalValue::InternalLinkage;
  Visibility = GlobalValue::DefaultVisibility;
}
```

That said, I think we can just delete the code block.

An extern/internal function will now use private `__profc_*`/`__profd_*`
variables, instead of internal ones. This saves some symbol table entries.

A non-comdat {linkonce,weak}_odr function will now use hidden external
`__profc_*`/`__profd_*` variables instead of internal ones.  There is potential
object file size increase because such symbols need `/INCLUDE:` directives.
However such non-comdat functions are rare (note that non-comdat weak
definitions don't prevent duplicate definition error).

The behavior changes match ELF.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D103355
2021-06-02 16:49:54 -07:00
Fangrui Song effb87dfa8 [profile] Add -fprofile-instr-generate tests for weak definition and various linkages 2021-06-02 16:12:08 -07:00
David Spickett 8c363efecc [compiler-rt][lsan] Increase libdl_deadlock test timeout
We have been seeing this test fail intermittently on our
2 stage AArch64 bot.

As far back as https://lab.llvm.org/buildbot/#/builders/53/builds/2694

Likely due to a lack of resources at certain times on the
shared machine. Up the time limit to give us some more room.

(this limit only applies to the watchdog thread, so if the
test passes then it won't take 20s)
2021-06-02 13:37:41 +00:00
David Spickett ba99359796 [compiler-rt][asan] Enable unwind-tables for Arm Linux
Since https://reviews.llvm.org/D102046 some tests have
been falling back to fast unwinding on our Thumb bot.

This fails because fast unwinding does not work on Thumb.
By adding the extra information we ensure this does not happen
during testing, but the built library can still fast unwind
as a last resort.

Since there are some situations it can work in, like if
eveything is built with clang. During testing we've got gcc
built system libs and clang built tests.

The same change was made for sanitizer-common in
https://reviews.llvm.org/D96337.

Reviewed By: zatrazz

Differential Revision: https://reviews.llvm.org/D103463
2021-06-02 12:59:46 +00:00
Vitaly Buka 60c0256ef1 [NFC][msan] Fix assigned-unused warning 2021-06-02 00:09:02 -07:00
Vitaly Buka 2445838f74 Revert "[NFC][msan] Fix warning on sanitizer-ppc64le-linux bot"
This fix breaks the test.

This reverts commit 6a2807bc81.
2021-06-02 00:09:02 -07:00
Vitaly Buka 6a2807bc81 [NFC][msan] Fix warning on sanitizer-ppc64le-linux bot 2021-06-01 21:03:41 -07:00
Vitaly Buka 4124bca309 [scudo] Enabled MTE in tests
Reviewed By: pcc, hctim

Differential Revision: https://reviews.llvm.org/D103305
2021-06-01 19:56:57 -07:00
Lang Hames da59241327 [ORC-RT] Add WrapperFunctionResult, Simple Packed Serialization (SPS) system.
WrapperFunctionResult is a C++ wrapper for __orc_rt_CWrapperFunctionResult
that automatically manages the underlying struct.

The Simple Packed Serialization (SPS) utilities support a simple serialization
scheme for wrapper function argument and result buffers:

Primitive typess (bool, char, int8_t, and uint8_t, int16_t, uint16_t, int32_t,
uint32_t, int64_t, uint64_t) are serialized in little-endian form.

SPSTuples are serialized by serializing each of the tuple members in order
without padding.

SPSSequences are serialized by serializing a sequence length (as a uint64_t)
followed by each of the elements of the sequence in order without padding.

Serialization/deserialization always involves a pair of SPS type tag (a tag
representing the serialized format to use, e.g. uint32_t, or
SPSTuple<bool, SPSString>) and a concrete type to be serialized from or
deserialized to (uint32_t, std::pair<bool, std::string>). Serialization for new
types can be implemented by specializing the SPSSerializationTraits type.
2021-06-01 11:24:19 -07:00
Lang Hames 28c3e9c0d1 [ORC-RT] Rename C-API functions to use __orc_rt_ prefix (instead of OrcRT).
This matches the C++ namespace name, and is consistent with other C linkage
functions (e.g. __orc_rt_jit_dispatch).
2021-06-01 11:24:19 -07:00
Lang Hames d791f0c219 [ORC-RT] Add equality/inequality comparison to string_view. 2021-06-01 11:24:19 -07:00
Evgeny Leviant 7879fa6884 [compiler-rt] Fix compilation when multiple architectures can be targeted
When toolchain can supports all of arm, armhf and armv6m architectures compiler-rt
libraries won't compile because architecture specific flags are appended to single
BUILTIN_CFLAGS variable.

Differential revision: https://reviews.llvm.org/D103363
2021-06-01 15:26:51 +03:00
Lang Hames 1ed29f8784 [ORC-RT] Add common.h -- Logging, casting and remote dispatch utilities. 2021-05-30 11:21:17 -07:00
Lang Hames 442f2d7bc0 [ORC-RT] Add OrcRTCWrapperFunctionResult.
OrcRTCWrapperFunctionResult is a C struct that can be used to return serialized
results from "wrapper functions" -- functions that deserialize an argument
buffer, call through to an actual implementation function, then serialize and
return the result of that function. Wrapper functions allow calls between ORC
and the ORC Runtime to be written using a single signature,
WrapperFunctionResult(const char *ArgData, size_t ArgSize), and without coupling
either side to a particular transport mechanism (in-memory, TCP, IPC, ... the
actual mechanism will be determined by the TargetProcessControl implementation).

OrcRTCWrapperFunctionResult is designed to allow small serialized buffers to
be returned by value, with larger serialized results stored on the heap. They
also provide an error state to report failures in serialization/deserialization.
2021-05-30 11:20:57 -07:00
Vitaly Buka bd04d78d64 [cmake][scudo] Add missing headers 2021-05-28 02:06:14 -07:00
Vitaly Buka a6e5a4b464 [NFC][scudo] Re-enable check in the test
It should pass with patched QEMU.
2021-05-28 01:57:55 -07:00
Vitaly Buka 4458e8c4b4 Revert "[scudo] Check if MADV_DONTNEED zeroes memory"
This reverts commit d423509b80.

We are going to use patched QEMU.
2021-05-28 01:53:42 -07:00
Jianzhou Zhao fc1d39849e [dfsan] Add a flag about whether to propagate offset labels at gep
DFSan has flags to control flows between pointers and objects referred
by pointers. For example,

a = *p;
L(a) = L(*p)        when -dfsan-combine-pointer-labels-on-load = false
L(a) = L(*p) + L(p) when -dfsan-combine-pointer-labels-on-load = true

*p = b;
L(*p) = L(b)        when -dfsan-combine-pointer-labels-on-store = false
L(*p) = L(b) + L(p) when -dfsan-combine-pointer-labels-on-store = true
The question is what to do with p += c.

In practice we found many confusing flows if we propagate labels from c
to p. So a new flag works like this

p += c;
L(p) = L(p)        when -dfsan-propagate-via-pointer-arithmetic = false
L(p) = L(p) + L(c) when -dfsan-propagate-via-pointer-arithmetic = true

Reviewed-by: gbalats

Differential Revision: https://reviews.llvm.org/D103176
2021-05-28 00:06:19 +00:00
Ryan Prichard b834d63094 [sanitizer] Android ELF TLS is supported from Q (API 29)
Reviewed By: oontvoo, MaskRay

Differential Revision: https://reviews.llvm.org/D103214
2021-05-27 14:53:49 -07:00
Vitaly Buka c261edb277 [NFC][scudo] Check zeros on smaller allocations
1Tb counting was the slowest test under the QEMU with MTE.
2021-05-27 11:14:26 -07:00
Vitaly Buka eb69763ad8 [NFC][scudo] Rename internal function 2021-05-27 10:41:07 -07:00
Lang Hames 95ce0def7c [ORC-RT] Add endianness support to the ORC runtime.
endian.h is a cut-down version of llvm/Support/SwapByteOrder.h. It will be used
in upcoming serialization utilities for the ORC runtime.
2021-05-26 17:23:35 -07:00
Mitch Phillips 2dba40561a Revert "[scudo] Build scudo_standalone on Android and Fuchsia."
This reverts commit 2fe987e6ba.

Broke the Android buildbots. Turns out a couple more tweaks are
necessary to turn them back on.
2021-05-26 16:52:13 -07:00
Mitch Phillips 2fe987e6ba [scudo] Build scudo_standalone on Android and Fuchsia.
This should be fine now, and is necessary for D102543.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D103200
2021-05-26 15:29:28 -07:00
Matt Morehouse fd0a2f75ff Revert "Refactor mutation strategies into a standalone library"
This reverts commit c4a41cd77c due to
buildbot failure.
2021-05-26 15:16:43 -07:00
Matt Morehouse 1032886a41 [libFuzzer] Add missing FuzzerBuiltinsMsvc.h include.
Should fix the Windows build.
2021-05-26 14:38:27 -07:00
Aaron Green c4a41cd77c Refactor mutation strategies into a standalone library
This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
  * UseCmp
  * UseMemmem
  * SetCustomMutator
  * SetCustomCrossOver
  * LateInitialize (similar to the MutationDispatcher's original constructor)
  * Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
  * StartMutationSequence
  * MutationSequence
  * DictionaryEntrySequence
  * RecommendDictionary
  * RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447
2021-05-26 13:27:49 -07:00
Mitch Phillips f7c5c0d87b Revert "[Scudo] Make -fsanitize=scudo use standalone. Migrate tests."
This reverts commit 6911114d8c.

Broke the QEMU sanitizer bots due to a missing header dependency. This
actually needs to be fixed on the bot-side, but for now reverting this
patch until I can fix up the bot.
2021-05-26 10:50:26 -07:00
Mitch Phillips 6911114d8c [Scudo] Make -fsanitize=scudo use standalone. Migrate tests.
This patch moves -fsanitize=scudo to link the standalone scudo library,
rather than the original compiler-rt based library. This is one of the
major remaining roadblocks to deleting the compiler-rt based scudo,
which should not be used any more. The standalone Scudo is better in
pretty much every way and is much more suitable for production usage.

As well as patching the litmus tests for checking that the
scudo_standalone lib is linked instead of the scudo lib, this patch also
ports all the scudo lit tests to run under scudo standalone.

This patch also adds a feature to scudo standalone that was under test
in the original scudo - that arguments passed to an aligned operator new
were checked that the alignment was a power of two.

Some lit tests could not be migrated, due to the following issues:
 1. Features that aren't supported in scudo standalone, like the rss
 limit.
 2. Different quarantine implementation where the test needs some more
 thought.
 3. Small bugs in scudo standalone that should probably be fixed, like
 the Secondary allocator having a full page on the LHS of an allocation
 that only contains the chunk header, so underflows by <= a page aren't
 caught.
 4. Slight differences in behaviour that's technically correct, like
 'realloc(malloc(1), 0)' returns nullptr in standalone, but a real
 pointer in old scudo.
 5. Some tests that might be migratable, but not easily.

Tests that are obviously not applicable to scudo standalone (like
testing that no sanitizer symbols made it into the DSO) have been
deleted.

After this patch, the remaining work is:
 1. Update the Scudo documentation. The flags have changed, etc.
 2. Delete the old version of scudo.
 3. Patch up the tests in lit-unmigrated, or fix Scudo standalone.

Reviewed By: cryptoad, vitalybuka

Differential Revision: https://reviews.llvm.org/D102543
2021-05-26 10:03:17 -07:00
Kostya Kortchinsky a45877eea8 [scudo] Get rid of initLinkerInitialized
Now that everything is forcibly linker initialized, it feels like a
good time to get rid of the `init`/`initLinkerInitialized` split.

This allows to get rid of various `memset` construct in `init` that
gcc complains about (this fixes a Fuchsia open issue).

I added various `DCHECK`s to ensure that we would get a zero-inited
object when entering `init`, which required ensuring that
`unmapTestOnly` leaves the object in a good state (tests are currently
the only location where an allocator can be "de-initialized").

Running the tests with `--gtest_repeat=` showed no issue.

Differential Revision: https://reviews.llvm.org/D103119
2021-05-26 09:53:40 -07:00
Fangrui Song e67259531d [sanitizer] Let glibc aarch64 use O(1) GetTls
The generic approach can still be used by musl and FreeBSD. Note: on glibc
2.31, TLS_PRE_TCB_SIZE is 0x700, larger than ThreadDescriptorSize() by 16, but
this is benign: as long as the range includes pthread::{specific_1stblock,specific}
pthread_setspecific will not cause false positives.

Note: the state before afec953857 underestimated
the TLS size a lot (nearly ThreadDescriptorSize() = 1776).
That may explain why afec953857 actually made some
tests pass.
2021-05-25 16:28:17 -07:00
Alexandre Ganea dd2be15ff9 [gcov] Silence warning: comparison of integers of different signs
When building with Clang 11 on Windows, silence the following:

[432/5643] Building C object projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-x86_64.dir\GCDAProfiling.c.obj
F:\aganea\llvm-project\compiler-rt\lib\profile\GCDAProfiling.c(464,13): warning: comparison of integers of different signs: 'uint32_t' (aka 'unsigned int') and 'int' [-Wsign-compare]
    if (val != (gcov_version >= 90 ? GCOV_TAG_OBJECT_SUMMARY
        ~~~ ^   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
2021-05-25 18:46:37 -04:00
Vitaly Buka e14696bfd7 [NFC][SCUDO] Fix unittest for -gtest_repeat=10
Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D103122
2021-05-25 15:32:42 -07:00
Fangrui Song 3a678fe3e2 [sanitizer][test] s/A<10>/A<7>/ to fix "WARNING: Symbolizer buffer too small" which is somehow a hard error on s390x
https://reviews.llvm.org/D102046#2766553
2021-05-25 12:41:07 -07:00
Bruno Cardoso Lopes 6c35991ca0 [TSAN][CMake] Add support to run lit on individual tests
Handy when testing specific files, already supported in other components.

Example:
cd build; ./bin/llvm-lit ../compiler-rt/test/tsan/ignore_free.cpp

Differential Revision: https://reviews.llvm.org/D103054
2021-05-25 12:33:02 -07:00
Vitaly Buka d1e5f046cc Revert "[NFC][scudo] Let disableMemoryTagChecksTestOnly to fail"
This reverts commit 2c212db4ea.

It's not needed.
2021-05-25 11:53:42 -07:00
Vitaly Buka 8e30b55c82 [scudo] Fix CHECK implementation
Cast of signed types to u64 breaks comparison.
Also remove double () around operands.

Reviewed By: cryptoad, hctim

Differential Revision: https://reviews.llvm.org/D103060
2021-05-25 10:55:52 -07:00
Vitaly Buka 6a84d374d7 [scudo] Consistent setting of SCUDO_DEBUG
Make sure that if SCUDO_DEBUG=1 in tests
then we had the same in the scudo
library itself.

Reviewed By: cryptoad, hctim

Differential Revision: https://reviews.llvm.org/D103061
2021-05-25 10:49:01 -07:00
Kostya Kortchinsky 1872283457 [scudo] Rework dieOnMapUnmapError
Said function had a few shortfalls:
- didn't set an abort message on Android
- was logged on several lines
- didn't provide extra information like the size requested if OOM'ing

This improves the function to address those points.

Differential Revision: https://reviews.llvm.org/D103034
2021-05-25 08:27:37 -07:00
Vitaly Buka a0169b2ed1 [NFC][scudo] Add paramenters DCHECKs
Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D103042
2021-05-24 18:06:21 -07:00
Vitaly Buka f5bde3d476 [NFC][scudo] Avoid cast in test 2021-05-24 17:14:38 -07:00
Mitch Phillips 1fb6a03072 [scudo] Add unmapTestOnly() to secondary.
When trying to track down a vaddr-poisoning bug, I found that that the
secondary cache isn't emptied on test teardown. We should probably do
that to make the tests hermetic. Otherwise, repeating the tests lots of
times using --gtest_repeat fails after the mmap vaddr space is
exhausted.

To repro:
$ ninja check-scudo_standalone # build
$ ./projects/compiler-rt/lib/scudo/standalone/tests/ScudoUnitTest-x86_64-Test \
--gtest_filter=ScudoSecondaryTest.*:-ScudoSecondaryTest.SecondaryCombinations \
--gtest_repeat=10000

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D102874
2021-05-24 16:09:02 -07:00
Jinsong Ji bec6b02252 [compiler-rt][scudo] Fix sign-compare warnings
Fix buildbot failure
https://lab.llvm.org/buildbot/#/builders/57/builds/6542/steps/6/logs/stdio

/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:1629:28:
error: comparison of integers of different signs: 'const unsigned long'
and 'const int' [-Werror,-Wsign-compare]
GTEST_IMPL_CMP_HELPER_(GT, >);
~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:1609:12:
note: expanded from macro 'GTEST_IMPL_CMP_HELPER_'
  if (val1 op val2) {\
      ~~~~ ^  ~~~~
/llvm-project/compiler-rt/lib/scudo/standalone/tests/common_test.cpp:30:3:
note: in instantiation of function template specialization
'testing::internal::CmpHelperGT<unsigned long, int>' requested here
  EXPECT_GT(OnStart, 0);
  ^

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103029
2021-05-24 21:33:02 +00:00
George Balatsouras a11cb10a36 [dfsan] Add function that prints origin stack trace to buffer
Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D102451
2021-05-24 11:09:03 -07:00
Vitaly Buka 6435ca4e2b [NFC][scudo] Small test cleanup
Fixing issues raised on D102979 review.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D102994
2021-05-24 10:16:44 -07:00
Kostya Kortchinsky 20c1f94220 [scudo] Separate Fuchsia & Default SizeClassMap
The Fuchsia allocator config was using the default size class map.

This CL gives Fuchsia its own size class map and changes a couple of
things in the default one:
- make `SizeDelta` configurable in `Config` for a fixed size class map
  as it currently is for a table size class map;
- switch `SizeDelta` to 0 for the default config, it allows for size
  classes that allow for power of 2s, and overall better wrt pages
  filling;
- increase the max number of caches pointers to 14 in the default,
  this makes the transfer batch 64/128 bytes on 32/64-bit platforms,
  which is cache-line friendly (previous size was 48/96 bytes).

The Fuchsia size class map remains untouched for now, this doesn't
impact Android which uses the table size class map.

Differential Revision: https://reviews.llvm.org/D102783
2021-05-24 08:54:08 -07:00
Vitaly Buka 93d1e5822e [NFC][scudo] Enforce header size alignment
As-is it should not change struct size, but it will
help to keep correct size if more fields added.
2021-05-23 14:21:35 -07:00
Vitaly Buka 887dda5dcd [NFC][scudo] Replace size_t with uptr 2021-05-22 22:55:53 -07:00
Vitaly Buka 0bccdf82f7 [NFC][scudo] Add releasePagesToOS test 2021-05-22 22:42:59 -07:00
Vitaly Buka 6994bf7dad [NFC][scudo] Move SKIP_ON_FUCHSIA to common header 2021-05-22 22:42:59 -07:00
Vitaly Buka 5992823008 [NFC][sanitizer] Remove unused variable 2021-05-21 16:11:51 -07:00
Jon Roelofs cc9c895d88 [compiler-rt][profile] Explicitly specify PROFILE_SOURCES extensions. NFC 2021-05-21 14:46:08 -07:00
Vitaly Buka f50b87e9ef [NFC][sanitizer] Fix android bot after D102815
https://lab.llvm.org/buildbot/#/builders/77/builds/6519
2021-05-21 14:08:04 -07:00
Vitaly Buka 1c6272a481 [scudo] Try to re-enabled the test on arm
It's probably fixed by D102886.

Builder to watch https://lab.llvm.org/buildbot/#/builders/clang-cmake-armv7-full

Reviewed By: hctim, cryptoad

Differential Revision: https://reviews.llvm.org/D102887
2021-05-21 10:57:49 -07:00
Lang Hames 16b284e99e [ORC-RT] Add missing headers to CMakeLists.txt. 2021-05-21 10:17:47 -07:00
Vitaly Buka 53ec41a49c [scudo][NFC] Split BasicTest further
It's still the slowest test under our emulator.
2021-05-20 23:28:08 -07:00
Vitaly Buka 384a460e59 [scudo] Fix compilation after D102886 2021-05-20 22:10:12 -07:00
George Balatsouras 7c35bf85bf [sanitizer_common] Remove unused var in test
Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D102898
2021-05-20 20:42:30 -07:00
George Balatsouras d10c1ba487 [sanitizer_common] Add StackTrace::PrintTo method
This method is like StackTrace::Print but instead of printing to stderr
it copies its output to a user-provided buffer.

Part of https://reviews.llvm.org/D102451.

Reviewed By: vitalybuka, stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D102815
2021-05-20 19:39:38 -07:00
Vitaly Buka 51fe7ddce2 [NFC][scudo] Fix typo in comment 2021-05-20 19:37:33 -07:00
Vitaly Buka 2c212db4ea [NFC][scudo] Let disableMemoryTagChecksTestOnly to fail
If this happend we can't run corresponding test.
2021-05-20 19:25:20 -07:00
Vitaly Buka 96b760607f [scudo] Fix EXPECT_DEATH tests
Put allocate/deallocate next to memory
access inside EXPECT_DEATH block.
This way we reduce probability that memory is not mapped
by unrelated code.

It's still not absolutely guaranty that mmap does not
happen so we repeat it few times to be sure.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D102886
2021-05-20 17:08:15 -07:00
Mitch Phillips f4ccbaf310 [scudo] Add supported architectures.
Adds extra supported architectures that were available for vanilla
scudo, in preparation for D102543. Hopefully the dust has settled and
7d0a81ca38 is no longer an issue.

Reviewed By: cryptoad, vitalybuka

Differential Revision: https://reviews.llvm.org/D102648
2021-05-20 11:22:51 -07:00
Reid Kleckner 8f20ac9595 [PGO] Don't reference functions unless value profiling is enabled
This reduces the size of chrome.dll.pdb built with optimizations,
coverage, and line table info from 4,690,210,816 to 2,181,128,192, which
makes it possible to fit under the 4GB limit.

This change can greatly reduce binary size in coverage builds, which do
not need value profiling. IR PGO builds are unaffected. There is a minor
behavior change for frontend PGO.

PGO and coverage both use InstrProfiling to create profile data with
counters. PGO records the address of each function in the __profd_
global. It is used later to map runtime function pointer values back to
source-level function names. Coverage does not appear to use this
information.

Recording the address of every function with code coverage drastically
increases code size. Consider this program:

  void foo();
  void bar();
  inline void inlineMe(int x) {
    if (x > 0)
      foo();
    else
      bar();
  }
  int getVal();
  int main() { inlineMe(getVal()); }

With code coverage, the InstrProfiling pass runs before inlining, and it
captures the address of inlineMe in the __profd_ global. This greatly
increases code size, because now the compiler can no longer delete
trivial code.

One downside to this approach is that users of frontend PGO must apply
the -mllvm -enable-value-profiling flag globally in TUs that enable PGO.
Otherwise, some inline virtual method addresses may not be recorded and
will not be able to be promoted. My assumption is that this mllvm flag
is not popular, and most frontend PGO users don't enable it.

Differential Revision: https://reviews.llvm.org/D102818
2021-05-20 11:09:24 -07:00
Mitch Phillips 577a80bff8 [scudo] Disable secondary cache-unmap tests on arm32.
Looks like secondary pointers don't get unmapped on one of the arm32
bots. In the interests of landing some dependent patches, disable this
test on arm32 so that it can be tested in isolation later.

Reviewed By: cryptoad, vitalybuka

Split from differential patchset (1/2): https://reviews.llvm.org/D102648
2021-05-20 11:07:45 -07:00
Lang Hames d22b27cfde [ORC-RT] Add string_view and span utilities for use by the ORC runtime.
These are substitutes for std::string_view (and llvm::StringRef) and std::span
(and llvm::ArrayRef) for use by the ORC runtime.
2021-05-20 11:02:44 -07:00
Tamar Christina 68d5235cb5 libsanitizer: Remove cyclades inclusion in sanitizer
The Linux kernel has removed the interface to cyclades from
the latest kernel headers[1] due to them being orphaned for the
past 13 years.

libsanitizer uses this header when compiling against glibc, but
glibcs itself doesn't seem to have any references to cyclades.

Further more it seems that the driver is broken in the kernel and
the firmware doesn't seem to be available anymore.

As such since this is breaking the build of libsanitizer (and so the
GCC bootstrap[2]) I propose to remove this.

[1] https://lkml.org/lkml/2021/3/2/153
[2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100379

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D102059
2021-05-20 18:55:26 +01:00
Tamar Christina 0d3619864c Revert "libsanitizer: Guard cyclades inclusion in sanitizer"
This reverts commit f7c5351552.

To investigate a test failure.
2021-05-20 14:43:57 +01:00
Tamar Christina f7c5351552 libsanitizer: Guard cyclades inclusion in sanitizer
The Linux kernel has removed the interface to cyclades from
the latest kernel headers[1] due to them being orphaned for the
past 13 years.

libsanitizer uses this header when compiling against glibc, but
glibcs itself doesn't seem to have any references to cyclades.

Further more it seems that the driver is broken in the kernel and
the firmware doesn't seem to be available anymore.

As such since this is breaking the build of libsanitizer (and so the
GCC bootstrap[2]) I propose to remove this.

[1] https://lkml.org/lkml/2021/3/2/153
[2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100379

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D102059
2021-05-20 11:06:56 +01:00
Vitaly Buka 5faeefd4fa [tsan] Deflake pthread_atfork_deadlock3
sleep(1) does not guaranty afterfork order.
Also relative child/parent afterfork order is not important for this test so we
can just avoid checking that.

Reviewed By: dvyukov

Differential Revision: https://reviews.llvm.org/D102810
2021-05-19 22:59:37 -07:00
Vitaly Buka 09a8372726 [NFC][tsan] clang-format the test 2021-05-19 14:03:50 -07:00
Lang Hames 1dfa47910a [ORC-RT] Add ORC runtime error and expected types.
These will be used for error propagation and handling in the ORC runtime.

The implementations of these types are cut-down versions of the error
support in llvm/Support/Error.h. Most advice on llvm::Error and llvm::Expected
(e.g. from the LLVM Programmer's manual) applies equally to __orc_rt::Error
and __orc_rt::Expected. The primary difference is the mechanism for testing
and handling error types: The ORC runtime uses a new 'error_cast' operation
to replace the handleErrors family of functions. See error_cast comments in
error.h.
2021-05-19 13:31:25 -07:00
Vedant Kumar 7014a10161 [profile] Skip mmap() if there are no counters
If there are no counters, an mmap() of the counters section would fail
due to the size argument being too small (EINVAL).

rdar://78175925

Differential Revision: https://reviews.llvm.org/D102735
2021-05-19 09:31:40 -07:00
Dmitry Vyukov c1eaa1168a tsan: mark sigwait as blocking
Add a test case reported in:
https://github.com/google/sanitizers/issues/1401
and fix it.
The code assumes sigwait will process other signals.

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D102057
2021-05-19 13:03:20 +02:00
Peter Collingbourne 8e93d10633 scudo: Test realloc on increasing size buffers.
While developing a change to the allocator I ended up breaking
realloc on secondary allocations with increasing sizes. That didn't
cause any of the unit tests to fail, which indicated that we're
missing some test coverage here. Add a unit test for that case.

Differential Revision: https://reviews.llvm.org/D102716
2021-05-18 14:59:30 -07:00