Commit Graph

14060 Commits

Author SHA1 Message Date
David Spickett 8c363efecc [compiler-rt][lsan] Increase libdl_deadlock test timeout
We have been seeing this test fail intermittently on our
2 stage AArch64 bot.

As far back as https://lab.llvm.org/buildbot/#/builders/53/builds/2694

Likely due to a lack of resources at certain times on the
shared machine. Up the time limit to give us some more room.

(this limit only applies to the watchdog thread, so if the
test passes then it won't take 20s)
2021-06-02 13:37:41 +00:00
David Spickett ba99359796 [compiler-rt][asan] Enable unwind-tables for Arm Linux
Since https://reviews.llvm.org/D102046 some tests have
been falling back to fast unwinding on our Thumb bot.

This fails because fast unwinding does not work on Thumb.
By adding the extra information we ensure this does not happen
during testing, but the built library can still fast unwind
as a last resort.

Since there are some situations it can work in, like if
eveything is built with clang. During testing we've got gcc
built system libs and clang built tests.

The same change was made for sanitizer-common in
https://reviews.llvm.org/D96337.

Reviewed By: zatrazz

Differential Revision: https://reviews.llvm.org/D103463
2021-06-02 12:59:46 +00:00
Vitaly Buka 60c0256ef1 [NFC][msan] Fix assigned-unused warning 2021-06-02 00:09:02 -07:00
Vitaly Buka 2445838f74 Revert "[NFC][msan] Fix warning on sanitizer-ppc64le-linux bot"
This fix breaks the test.

This reverts commit 6a2807bc81.
2021-06-02 00:09:02 -07:00
Vitaly Buka 6a2807bc81 [NFC][msan] Fix warning on sanitizer-ppc64le-linux bot 2021-06-01 21:03:41 -07:00
Vitaly Buka 4124bca309 [scudo] Enabled MTE in tests
Reviewed By: pcc, hctim

Differential Revision: https://reviews.llvm.org/D103305
2021-06-01 19:56:57 -07:00
Lang Hames da59241327 [ORC-RT] Add WrapperFunctionResult, Simple Packed Serialization (SPS) system.
WrapperFunctionResult is a C++ wrapper for __orc_rt_CWrapperFunctionResult
that automatically manages the underlying struct.

The Simple Packed Serialization (SPS) utilities support a simple serialization
scheme for wrapper function argument and result buffers:

Primitive typess (bool, char, int8_t, and uint8_t, int16_t, uint16_t, int32_t,
uint32_t, int64_t, uint64_t) are serialized in little-endian form.

SPSTuples are serialized by serializing each of the tuple members in order
without padding.

SPSSequences are serialized by serializing a sequence length (as a uint64_t)
followed by each of the elements of the sequence in order without padding.

Serialization/deserialization always involves a pair of SPS type tag (a tag
representing the serialized format to use, e.g. uint32_t, or
SPSTuple<bool, SPSString>) and a concrete type to be serialized from or
deserialized to (uint32_t, std::pair<bool, std::string>). Serialization for new
types can be implemented by specializing the SPSSerializationTraits type.
2021-06-01 11:24:19 -07:00
Lang Hames 28c3e9c0d1 [ORC-RT] Rename C-API functions to use __orc_rt_ prefix (instead of OrcRT).
This matches the C++ namespace name, and is consistent with other C linkage
functions (e.g. __orc_rt_jit_dispatch).
2021-06-01 11:24:19 -07:00
Lang Hames d791f0c219 [ORC-RT] Add equality/inequality comparison to string_view. 2021-06-01 11:24:19 -07:00
Evgeny Leviant 7879fa6884 [compiler-rt] Fix compilation when multiple architectures can be targeted
When toolchain can supports all of arm, armhf and armv6m architectures compiler-rt
libraries won't compile because architecture specific flags are appended to single
BUILTIN_CFLAGS variable.

Differential revision: https://reviews.llvm.org/D103363
2021-06-01 15:26:51 +03:00
Lang Hames 1ed29f8784 [ORC-RT] Add common.h -- Logging, casting and remote dispatch utilities. 2021-05-30 11:21:17 -07:00
Lang Hames 442f2d7bc0 [ORC-RT] Add OrcRTCWrapperFunctionResult.
OrcRTCWrapperFunctionResult is a C struct that can be used to return serialized
results from "wrapper functions" -- functions that deserialize an argument
buffer, call through to an actual implementation function, then serialize and
return the result of that function. Wrapper functions allow calls between ORC
and the ORC Runtime to be written using a single signature,
WrapperFunctionResult(const char *ArgData, size_t ArgSize), and without coupling
either side to a particular transport mechanism (in-memory, TCP, IPC, ... the
actual mechanism will be determined by the TargetProcessControl implementation).

OrcRTCWrapperFunctionResult is designed to allow small serialized buffers to
be returned by value, with larger serialized results stored on the heap. They
also provide an error state to report failures in serialization/deserialization.
2021-05-30 11:20:57 -07:00
Vitaly Buka bd04d78d64 [cmake][scudo] Add missing headers 2021-05-28 02:06:14 -07:00
Vitaly Buka a6e5a4b464 [NFC][scudo] Re-enable check in the test
It should pass with patched QEMU.
2021-05-28 01:57:55 -07:00
Vitaly Buka 4458e8c4b4 Revert "[scudo] Check if MADV_DONTNEED zeroes memory"
This reverts commit d423509b80.

We are going to use patched QEMU.
2021-05-28 01:53:42 -07:00
Jianzhou Zhao fc1d39849e [dfsan] Add a flag about whether to propagate offset labels at gep
DFSan has flags to control flows between pointers and objects referred
by pointers. For example,

a = *p;
L(a) = L(*p)        when -dfsan-combine-pointer-labels-on-load = false
L(a) = L(*p) + L(p) when -dfsan-combine-pointer-labels-on-load = true

*p = b;
L(*p) = L(b)        when -dfsan-combine-pointer-labels-on-store = false
L(*p) = L(b) + L(p) when -dfsan-combine-pointer-labels-on-store = true
The question is what to do with p += c.

In practice we found many confusing flows if we propagate labels from c
to p. So a new flag works like this

p += c;
L(p) = L(p)        when -dfsan-propagate-via-pointer-arithmetic = false
L(p) = L(p) + L(c) when -dfsan-propagate-via-pointer-arithmetic = true

Reviewed-by: gbalats

Differential Revision: https://reviews.llvm.org/D103176
2021-05-28 00:06:19 +00:00
Ryan Prichard b834d63094 [sanitizer] Android ELF TLS is supported from Q (API 29)
Reviewed By: oontvoo, MaskRay

Differential Revision: https://reviews.llvm.org/D103214
2021-05-27 14:53:49 -07:00
Vitaly Buka c261edb277 [NFC][scudo] Check zeros on smaller allocations
1Tb counting was the slowest test under the QEMU with MTE.
2021-05-27 11:14:26 -07:00
Vitaly Buka eb69763ad8 [NFC][scudo] Rename internal function 2021-05-27 10:41:07 -07:00
Lang Hames 95ce0def7c [ORC-RT] Add endianness support to the ORC runtime.
endian.h is a cut-down version of llvm/Support/SwapByteOrder.h. It will be used
in upcoming serialization utilities for the ORC runtime.
2021-05-26 17:23:35 -07:00
Mitch Phillips 2dba40561a Revert "[scudo] Build scudo_standalone on Android and Fuchsia."
This reverts commit 2fe987e6ba.

Broke the Android buildbots. Turns out a couple more tweaks are
necessary to turn them back on.
2021-05-26 16:52:13 -07:00
Mitch Phillips 2fe987e6ba [scudo] Build scudo_standalone on Android and Fuchsia.
This should be fine now, and is necessary for D102543.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D103200
2021-05-26 15:29:28 -07:00
Matt Morehouse fd0a2f75ff Revert "Refactor mutation strategies into a standalone library"
This reverts commit c4a41cd77c due to
buildbot failure.
2021-05-26 15:16:43 -07:00
Matt Morehouse 1032886a41 [libFuzzer] Add missing FuzzerBuiltinsMsvc.h include.
Should fix the Windows build.
2021-05-26 14:38:27 -07:00
Aaron Green c4a41cd77c Refactor mutation strategies into a standalone library
This change introduces libMutagen/libclang_rt.mutagen.a as a subset of libFuzzer/libclang_rt.fuzzer.a. This library contains only the fuzzing strategies used by libFuzzer to produce new test inputs from provided inputs, dictionaries, and SanitizerCoverage feedback.

Most of this change is simply moving sections of code to one side or the other of the library boundary. The only meaningful new code is:

* The Mutagen.h interface and its implementation in Mutagen.cpp.
* The following methods in MutagenDispatcher.cpp:
  * UseCmp
  * UseMemmem
  * SetCustomMutator
  * SetCustomCrossOver
  * LateInitialize (similar to the MutationDispatcher's original constructor)
  * Mutate_AddWordFromTORC (uses callbacks instead of accessing TPC directly)
  * StartMutationSequence
  * MutationSequence
  * DictionaryEntrySequence
  * RecommendDictionary
  * RecommendDictionaryEntry
* FuzzerMutate.cpp (which now justs sets callbacks and handles printing)
* MutagenUnittest.cpp (which adds tests of Mutagen.h)

A note on performance: This change was tested with a 100 passes of test/fuzzer/LargeTest.cpp with 1000 runs per pass, both with and without the change. The running time distribution was qualitatively similar both with and without the change, and the average difference was within 30 microseconds (2.240 ms/run vs 2.212 ms/run, respectively). Both times were much higher than observed with the fully optimized system clang (~0.38 ms/run), most likely due to the combination of CMake "dev mode" settings (e.g. CMAKE_BUILD_TYPE="Debug", LLVM_ENABLE_LTO=OFF, etc.). The difference between the two versions built similarly seems to be "in the noise" and suggests no meaningful performance degradation.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D102447
2021-05-26 13:27:49 -07:00
Mitch Phillips f7c5c0d87b Revert "[Scudo] Make -fsanitize=scudo use standalone. Migrate tests."
This reverts commit 6911114d8c.

Broke the QEMU sanitizer bots due to a missing header dependency. This
actually needs to be fixed on the bot-side, but for now reverting this
patch until I can fix up the bot.
2021-05-26 10:50:26 -07:00
Mitch Phillips 6911114d8c [Scudo] Make -fsanitize=scudo use standalone. Migrate tests.
This patch moves -fsanitize=scudo to link the standalone scudo library,
rather than the original compiler-rt based library. This is one of the
major remaining roadblocks to deleting the compiler-rt based scudo,
which should not be used any more. The standalone Scudo is better in
pretty much every way and is much more suitable for production usage.

As well as patching the litmus tests for checking that the
scudo_standalone lib is linked instead of the scudo lib, this patch also
ports all the scudo lit tests to run under scudo standalone.

This patch also adds a feature to scudo standalone that was under test
in the original scudo - that arguments passed to an aligned operator new
were checked that the alignment was a power of two.

Some lit tests could not be migrated, due to the following issues:
 1. Features that aren't supported in scudo standalone, like the rss
 limit.
 2. Different quarantine implementation where the test needs some more
 thought.
 3. Small bugs in scudo standalone that should probably be fixed, like
 the Secondary allocator having a full page on the LHS of an allocation
 that only contains the chunk header, so underflows by <= a page aren't
 caught.
 4. Slight differences in behaviour that's technically correct, like
 'realloc(malloc(1), 0)' returns nullptr in standalone, but a real
 pointer in old scudo.
 5. Some tests that might be migratable, but not easily.

Tests that are obviously not applicable to scudo standalone (like
testing that no sanitizer symbols made it into the DSO) have been
deleted.

After this patch, the remaining work is:
 1. Update the Scudo documentation. The flags have changed, etc.
 2. Delete the old version of scudo.
 3. Patch up the tests in lit-unmigrated, or fix Scudo standalone.

Reviewed By: cryptoad, vitalybuka

Differential Revision: https://reviews.llvm.org/D102543
2021-05-26 10:03:17 -07:00
Kostya Kortchinsky a45877eea8 [scudo] Get rid of initLinkerInitialized
Now that everything is forcibly linker initialized, it feels like a
good time to get rid of the `init`/`initLinkerInitialized` split.

This allows to get rid of various `memset` construct in `init` that
gcc complains about (this fixes a Fuchsia open issue).

I added various `DCHECK`s to ensure that we would get a zero-inited
object when entering `init`, which required ensuring that
`unmapTestOnly` leaves the object in a good state (tests are currently
the only location where an allocator can be "de-initialized").

Running the tests with `--gtest_repeat=` showed no issue.

Differential Revision: https://reviews.llvm.org/D103119
2021-05-26 09:53:40 -07:00
Fangrui Song e67259531d [sanitizer] Let glibc aarch64 use O(1) GetTls
The generic approach can still be used by musl and FreeBSD. Note: on glibc
2.31, TLS_PRE_TCB_SIZE is 0x700, larger than ThreadDescriptorSize() by 16, but
this is benign: as long as the range includes pthread::{specific_1stblock,specific}
pthread_setspecific will not cause false positives.

Note: the state before afec953857 underestimated
the TLS size a lot (nearly ThreadDescriptorSize() = 1776).
That may explain why afec953857 actually made some
tests pass.
2021-05-25 16:28:17 -07:00
Alexandre Ganea dd2be15ff9 [gcov] Silence warning: comparison of integers of different signs
When building with Clang 11 on Windows, silence the following:

[432/5643] Building C object projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-x86_64.dir\GCDAProfiling.c.obj
F:\aganea\llvm-project\compiler-rt\lib\profile\GCDAProfiling.c(464,13): warning: comparison of integers of different signs: 'uint32_t' (aka 'unsigned int') and 'int' [-Wsign-compare]
    if (val != (gcov_version >= 90 ? GCOV_TAG_OBJECT_SUMMARY
        ~~~ ^   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
2021-05-25 18:46:37 -04:00
Vitaly Buka e14696bfd7 [NFC][SCUDO] Fix unittest for -gtest_repeat=10
Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D103122
2021-05-25 15:32:42 -07:00
Fangrui Song 3a678fe3e2 [sanitizer][test] s/A<10>/A<7>/ to fix "WARNING: Symbolizer buffer too small" which is somehow a hard error on s390x
https://reviews.llvm.org/D102046#2766553
2021-05-25 12:41:07 -07:00
Bruno Cardoso Lopes 6c35991ca0 [TSAN][CMake] Add support to run lit on individual tests
Handy when testing specific files, already supported in other components.

Example:
cd build; ./bin/llvm-lit ../compiler-rt/test/tsan/ignore_free.cpp

Differential Revision: https://reviews.llvm.org/D103054
2021-05-25 12:33:02 -07:00
Vitaly Buka d1e5f046cc Revert "[NFC][scudo] Let disableMemoryTagChecksTestOnly to fail"
This reverts commit 2c212db4ea.

It's not needed.
2021-05-25 11:53:42 -07:00
Vitaly Buka 8e30b55c82 [scudo] Fix CHECK implementation
Cast of signed types to u64 breaks comparison.
Also remove double () around operands.

Reviewed By: cryptoad, hctim

Differential Revision: https://reviews.llvm.org/D103060
2021-05-25 10:55:52 -07:00
Vitaly Buka 6a84d374d7 [scudo] Consistent setting of SCUDO_DEBUG
Make sure that if SCUDO_DEBUG=1 in tests
then we had the same in the scudo
library itself.

Reviewed By: cryptoad, hctim

Differential Revision: https://reviews.llvm.org/D103061
2021-05-25 10:49:01 -07:00
Kostya Kortchinsky 1872283457 [scudo] Rework dieOnMapUnmapError
Said function had a few shortfalls:
- didn't set an abort message on Android
- was logged on several lines
- didn't provide extra information like the size requested if OOM'ing

This improves the function to address those points.

Differential Revision: https://reviews.llvm.org/D103034
2021-05-25 08:27:37 -07:00
Vitaly Buka a0169b2ed1 [NFC][scudo] Add paramenters DCHECKs
Reviewed By: hctim

Differential Revision: https://reviews.llvm.org/D103042
2021-05-24 18:06:21 -07:00
Vitaly Buka f5bde3d476 [NFC][scudo] Avoid cast in test 2021-05-24 17:14:38 -07:00
Mitch Phillips 1fb6a03072 [scudo] Add unmapTestOnly() to secondary.
When trying to track down a vaddr-poisoning bug, I found that that the
secondary cache isn't emptied on test teardown. We should probably do
that to make the tests hermetic. Otherwise, repeating the tests lots of
times using --gtest_repeat fails after the mmap vaddr space is
exhausted.

To repro:
$ ninja check-scudo_standalone # build
$ ./projects/compiler-rt/lib/scudo/standalone/tests/ScudoUnitTest-x86_64-Test \
--gtest_filter=ScudoSecondaryTest.*:-ScudoSecondaryTest.SecondaryCombinations \
--gtest_repeat=10000

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D102874
2021-05-24 16:09:02 -07:00
Jinsong Ji bec6b02252 [compiler-rt][scudo] Fix sign-compare warnings
Fix buildbot failure
https://lab.llvm.org/buildbot/#/builders/57/builds/6542/steps/6/logs/stdio

/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:1629:28:
error: comparison of integers of different signs: 'const unsigned long'
and 'const int' [-Werror,-Wsign-compare]
GTEST_IMPL_CMP_HELPER_(GT, >);
~~~~~~~~~~~~~~~~~~~~~~~~~~^~
/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:1609:12:
note: expanded from macro 'GTEST_IMPL_CMP_HELPER_'
  if (val1 op val2) {\
      ~~~~ ^  ~~~~
/llvm-project/compiler-rt/lib/scudo/standalone/tests/common_test.cpp:30:3:
note: in instantiation of function template specialization
'testing::internal::CmpHelperGT<unsigned long, int>' requested here
  EXPECT_GT(OnStart, 0);
  ^

Reviewed By: vitalybuka

Differential Revision: https://reviews.llvm.org/D103029
2021-05-24 21:33:02 +00:00
George Balatsouras a11cb10a36 [dfsan] Add function that prints origin stack trace to buffer
Reviewed By: stephan.yichao.zhao

Differential Revision: https://reviews.llvm.org/D102451
2021-05-24 11:09:03 -07:00
Vitaly Buka 6435ca4e2b [NFC][scudo] Small test cleanup
Fixing issues raised on D102979 review.

Reviewed By: cryptoad

Differential Revision: https://reviews.llvm.org/D102994
2021-05-24 10:16:44 -07:00
Kostya Kortchinsky 20c1f94220 [scudo] Separate Fuchsia & Default SizeClassMap
The Fuchsia allocator config was using the default size class map.

This CL gives Fuchsia its own size class map and changes a couple of
things in the default one:
- make `SizeDelta` configurable in `Config` for a fixed size class map
  as it currently is for a table size class map;
- switch `SizeDelta` to 0 for the default config, it allows for size
  classes that allow for power of 2s, and overall better wrt pages
  filling;
- increase the max number of caches pointers to 14 in the default,
  this makes the transfer batch 64/128 bytes on 32/64-bit platforms,
  which is cache-line friendly (previous size was 48/96 bytes).

The Fuchsia size class map remains untouched for now, this doesn't
impact Android which uses the table size class map.

Differential Revision: https://reviews.llvm.org/D102783
2021-05-24 08:54:08 -07:00
Vitaly Buka 93d1e5822e [NFC][scudo] Enforce header size alignment
As-is it should not change struct size, but it will
help to keep correct size if more fields added.
2021-05-23 14:21:35 -07:00
Vitaly Buka 887dda5dcd [NFC][scudo] Replace size_t with uptr 2021-05-22 22:55:53 -07:00
Vitaly Buka 0bccdf82f7 [NFC][scudo] Add releasePagesToOS test 2021-05-22 22:42:59 -07:00
Vitaly Buka 6994bf7dad [NFC][scudo] Move SKIP_ON_FUCHSIA to common header 2021-05-22 22:42:59 -07:00
Vitaly Buka 5992823008 [NFC][sanitizer] Remove unused variable 2021-05-21 16:11:51 -07:00
Jon Roelofs cc9c895d88 [compiler-rt][profile] Explicitly specify PROFILE_SOURCES extensions. NFC 2021-05-21 14:46:08 -07:00