Commit Graph

13109 Commits

Author SHA1 Message Date
Kamil Rytarowski bba736e503 [compiler-rt] [netbsd] Update generate_netbsd_syscalls.awk
Sync with NetBSD 9.99.72.
2020-09-10 20:16:17 +02:00
Kamil Rytarowski b85c085c84 [compiler-rt] [netbsd] Improve code formatting
No functional change.
2020-09-10 19:38:57 +02:00
Kamil Rytarowski cb8cb28ed9 [compiler-rt] [netbsd] Add fallback definitions for MKISCSI=no
Add dev/iscsi/iscsi_ioctl.h fallback ioctl(2) operations.
2020-09-10 19:38:57 +02:00
Fangrui Song ab1de1fcfb [gcov] Delete flush_fn_list (unused since D83149) 2020-09-10 10:15:27 -07:00
Azharuddin Mohammed 626209cac0 Revert "[gcov] Delete flush_fn_list (unused since D83149)"
This reverts commit 01cdab0b33.

It was causing the instrprof-darwin-exports.c test to fail.
```
Undefined symbols for architecture x86_64:
  "_flush_fn_list", referenced from:
     -exported_symbol[s_list] command line option
```
2020-09-10 09:53:34 -07:00
Craig Topper f5ad9c2e0e [builtins] Write __divmoddi4/__divmodsi4 in terms __udivmod instead of __div and multiply.
Previously we calculating the remainder by multiplying the
quotient and divisor and subtracting from the dividend.

__udivmod can calculate the remainder while calculating the
quotient. We just need to correct the sign afterward.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D87433
2020-09-10 08:08:55 -07:00
Kamil Rytarowski f51e55e09e [compiler-rt] [netbsd] Reintroduce __sanitizer_protoent
Partial revert of https://reviews.llvm.org/D82424
2020-09-10 11:46:41 +02:00
Fangrui Song 01cdab0b33 [gcov] Delete flush_fn_list (unused since D83149) 2020-09-09 17:27:30 -07:00
Fangrui Song f559bf31ad [gcov] Delete unused llvm_gcda_increment_indirect_counter
It has been unused since r157564 (2012).
2020-09-09 16:58:03 -07:00
Vitaly Buka 91c28bbe74 [Asan] Return nullptr for invalid chunks
CHUNK_ALLOCATED. CHUNK_QUARANTINE are only states
which make AsanChunk useful for GetAsanChunk callers.
In either case member of AsanChunk are not useful.

Fix few cases which didn't expect nullptr. Most of the callers are already
expects nullptr.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D87135
2020-09-09 16:32:47 -07:00
Vitaly Buka b5bc56da8a [NFC][Asan] Fit ChunkHeader into redzone
In code as-is min redzone and ChunkHeader are 16 byte.
This patch just makes sure that redzone is calculated correctly if we
extend ChunkHeader.
2020-09-09 03:29:51 -07:00
Dokyung Song 1bb1eac6b1 [libFuzzer] Add a command-line option for tracing mutation of corpus inputs in the dot graph format.
This patch adds a new command-line option -mutation_graph_file=FILE for
debugging purposes, which traces how corpus inputs evolve during a fuzzing
run. For each new input that is added to the corpus, a new vertex corresponding
to the added input, as well as a new edge that connects its base input to itself
are written to the given file. Each vertex is labeled with the filename of the
input, and each edge is labeled with the mutation sequence that led to the input
w.r.t. its base input.

The format of the mutation graph file is the dot file format. Once prepended and
appended with "graph {" and "}", respectively, the graph becomes a valid dot
file and can be visualized.

Differential Revision: https://reviews.llvm.org/D86560
2020-09-09 03:28:53 +00:00
Fangrui Song b9d086693b [llvm-cov gcov] Compute unmeasured arc counts by Kirchhoff's circuit law
For a CFG G=(V,E), Knuth describes that by Kirchoff's circuit law, the minimum
number of counters necessary is |E|-(|V|-1). The emitted edges form a spanning
tree. libgcov emitted .gcda files leverages this optimization while clang
--coverage's doesn't.

Propagate counts by Kirchhoff's circuit law so that llvm-cov gcov can
correctly print line counts of gcc --coverage emitted files and enable
the future improvement of clang --coverage.
2020-09-08 18:45:11 -07:00
Vitaly Buka 27650a5fed [NFC][Asan] Remove Debug code
Used for https://github.com/google/sanitizers/issues/1193

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86933
2020-09-08 13:59:59 -07:00
Vitaly Buka c05095cd68 [Asan] Don't crash if metadata is not initialized
Fixes https://github.com/google/sanitizers/issues/1193.

AsanChunk can be uninitialized yet just after return from the secondary
allocator. If lsan starts scan just before metadata assignment it can
fail to find corresponding AsanChunk.

It should be safe to ignore this and let lsan to assume that
AsanChunk is in the beginning of the block. This block is from the
secondary allocator and created with mmap, so it should not contain
any pointers and will make lsan to miss some leaks.

Similar already happens for primary allocator. If it can't find real
AsanChunk it falls back and assume that block starts with AsanChunk.
Then if the block is already returned to allocator we have  garbage in
AsanChunk and may scan dead memory hiding some leaks.
I'll fix this in D87135.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86931
2020-09-08 13:58:34 -07:00
Fangrui Song 2d7fd38cf7 [sanitizers] Remove unneeded MaybeCall*DefaultOptions() and nullptr checks
D28596 added SANITIZER_INTERFACE_WEAK_DEF which can guarantee `*_default_options` are always defined.
The weak attributes on the `__{asan,lsan,msan,ubsan}_default_options` declarations can thus be removed.

`MaybeCall*DefaultOptions` no longer need nullptr checks, so their call sites can just be replaced by `__*_default_options`.

Reviewed By: #sanitizers, vitalybuka

Differential Revision: https://reviews.llvm.org/D87175
2020-09-08 10:12:05 -07:00
Raul Tambre 86bd8f82cc [CMake] Remove dead FindPythonInterp code
LLVM has bumped the minimum required CMake version to 3.13.4, so this has become dead code.

Reviewed By: #libc, ldionne

Differential Revision: https://reviews.llvm.org/D87189
2020-09-08 15:23:23 +03:00
Craig Topper 35f708a3c9 [builtins] Inline __paritysi2 into __paritydi2 and inline __paritydi2 into __parityti2.
No point in making __parityti2 go through 2 calls to get to
__paritysi2.

Reviewed By: MaskRay, efriedma

Differential Revision: https://reviews.llvm.org/D87218
2020-09-07 17:57:39 -07:00
Fangrui Song 5f5a0bb087 [asan][test] Use --image-base for Linux/asan_prelink_test.cpp if ld is LLD
LLD supports -Ttext but with the option there is still a PT_LOAD at address zero
and thus the Linux kernel will map it to a different address and the test will fail.

Use --image-base instead.
2020-09-07 14:45:21 -07:00
Fangrui Song ab68517e6b [asan_symbolize] Pass --demangle/--no-demangle instead of --demangle={True,False}
`--demangle={True,False}` were accepted but disallowed after llvm-symbolizer's switch to OptTable.
(`--demangle={true,false}` were temporarily supported but they are case sensitive.)
2020-09-06 14:44:37 -07:00
Brad Smith 8542dab909 [compiler-rt] Implement __clear_cache() on OpenBSD/arm 2020-09-06 15:54:24 -04:00
Vitaly Buka b11db3606c [Asan] Cleanup atomic usage in allocator
There are no know bugs related to this, still it may fix some latent ones.
Main concerns with preexisting code:
1. Inconsistent atomic/non-atomic access to the same field.
2. Assumption that bitfield chunk_state is always the first byte without
    even taking into account endianness.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86917
2020-09-06 00:14:33 -07:00
Vitaly Buka b2e27a86c1 Revert "[Asan] Cleanup atomic usage in allocator"
Crashes on PPC

This reverts commit eb87e1dbcf.
2020-09-05 23:41:25 -07:00
Vitaly Buka eb87e1dbcf [Asan] Cleanup atomic usage in allocator
There are no know bugs related to this, still it may fix some latent ones.
Main concerns with preexisting code:
1. Inconsistent atomic/non-atomic access to the same field.
2. Assumption that bitfield chunk_state is always the first byte without
    even taking into account endianness.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86917
2020-09-05 19:55:38 -07:00
Vitaly Buka 289c08e4bc [NFC][Asan] Fix clang-tidy warning 2020-09-05 19:09:59 -07:00
Vitaly Buka 9128f5c8a7 [NFC][Asan] Move free_tid from ChunkHeader
The goal to make chunk_state atomic, but we can't pack it with
free_tid on Windows.
2020-09-05 18:42:32 -07:00
Vitaly Buka d5d09d93ee [NFC][Asan] Reformat some allocator code 2020-09-05 15:56:02 -07:00
Vitaly Buka 398c44ba84 [NFC][Asan] Inline enum doc strings 2020-09-05 15:54:18 -07:00
Vitaly Buka 51597cc37a [NFC][Asan] Rename internal enum value.
New name better represents the state of chunk.
2020-09-05 15:53:52 -07:00
Vitaly Buka 298c9fae93 [NFC][compiler-rt] Refine .clang-tidy checks
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D87182
2020-09-05 15:42:15 -07:00
Fangrui Song a5d6af421d [compiler-rt] Add .clang-tidy with customization to disable readability-identifier-naming
Copied from lldb/.clang-tidy (D75810).

Most compiler-rt code actually uses variableName or variable_name but not VariableName.
Lots of functions use `__function_name` and FunctionName instead of functionName.
Just exclude readability-identifier-naming.
2020-09-04 16:05:20 -07:00
Daniel Sanders 5e04b539c8 [compiler-rt] Try again to correct test after 3f1a9b7eca added segment names to objdump output
One check was missed on the previous attempt
2020-09-04 15:49:11 -07:00
Daniel Sanders 1eae19a87f [compiler-rt] Try to correct test after 3f1a9b7eca added segment names to objdump output 2020-09-04 12:24:46 -07:00
Hubert Tong dde16ef031 [tests][libFuzzer] Fix `-Wmissing-field-initializers` after D86092
Speculatively fix `-Werror,-Wmissing-field-initializers` failures
relating to the `ScalePerExecTime` field added by D86092.
2020-09-04 13:07:44 -04:00
Nico Weber fab60665eb sanitizer_common: Implement COMPILER_CHECK using static_assert
Since this is an internal header, we can just assume static_assert
exists.

If this doesn't upset any bots, I'll replace all uses of
COMPILER_CHECK in a follow-up.
2020-09-04 11:05:54 -04:00
Nico Weber dbf04aaade Revert "[Asan] Cleanup atomic usage in allocator"
This reverts commit 8b8be6f38a
and follow-ups 99a93c3a22,
a9c0bf0404,
48ac5b4833.

It breaks building on Windows, see https://reviews.llvm.org/D86917#2255872
2020-09-04 10:51:08 -04:00
Vitaly Buka 4770f80a7c [NFC][Asan] Fix typo in comment 2020-09-04 02:04:25 -07:00
Vitaly Buka 48ac5b4833 [NFC][Asan] Reformat some allocator code 2020-09-04 01:22:21 -07:00
Vitaly Buka a9c0bf0404 [NFC][Asan] Inline enum doc strings 2020-09-04 00:18:59 -07:00
Vitaly Buka 99a93c3a22 [NFC][Asan] Rename internal enum value.
New name better represents the state of chunk.
2020-09-03 18:54:52 -07:00
Vitaly Buka a803ddc522 [NFC][Asan] Add FIXME into GetAsanChunk 2020-09-03 18:50:09 -07:00
Vitaly Buka 2b71b7f791 [NFC][Asan] Set AP32::kMetadataSize to 0
Asan does not use metadata with primary allocators.
It should match AP64::kMetadataSize whic is 0.

Depends on D86917.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86919
2020-09-03 14:23:51 -07:00
Vitaly Buka 8b8be6f38a [Asan] Cleanup atomic usage in allocator
There are no know bugs related to this, still it may fix some latent ones.
Main concerns with preexisting code:
1. Inconsistent atomic/non-atomic access to the same field.
2. Assumption that bitfield chunk_state is always the first byte without
    even taking into account endianness.

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86917
2020-09-03 14:22:10 -07:00
Dokyung Song 5cda4dc7b4 [libFuzzer] Scale energy assigned to each input based on input execution time.
This patch scales the energy computed by the Entropic schedule based on the
execution time of each input. The input execution time is compared with the
average execution time of inputs in the corpus, and, based on the amount by
which they differ, the energy is scaled from 0.1x (for inputs executing slow) to
3x (for inputs executing fast). Note that the exact scaling criteria and formula
is borrowed from AFL.

On FuzzBench, this gives a sizeable throughput increase, which in turn leads to
more coverage on several benchmarks. For details, see the following report.

https://storage.googleapis.com/fuzzer-test-suite-public/exectime-report/index.html

Differential Revision: https://reviews.llvm.org/D86092
2020-09-03 20:38:20 +00:00
Dokyung Song b53243e194 [libFuzzer] Evenly select inputs to cross over with from the corpus regardless of the input's coverage.
This patch adds an option "cross_over_uniform_dist", which, if 1, considers all
inputs in the corpus for the crossover input selection. More specifically, this
patch uses a uniform distribution of all inputs in the corpus for the CrossOver
input selection. Note that input selection for mutation is still fully
determined by the scheduling policy (i.e., vanilla or Entropic); the uniform
distribution only applies to the secondary input selection, only for the
crossover mutation of the base input chosen by the scheduling policy. This way
the corpus inputs that have useful fragments in them, even though they are
deprioritized by the scheduling policy, have chances of getting mixed with other
inputs that are prioritized and selected as base input for mutation.

Differential Revision: https://reviews.llvm.org/D86954
2020-09-03 19:47:00 +00:00
Dokyung Song 62673c430d [libFuzzer] Add an option to keep initial seed inputs around.
This patch adds an option "keep_seed" to keep all initial seed inputs in the
corpus. Previously, only the initial seed inputs that find new coverage were
added to the corpus, and all the other initial inputs were discarded. We
observed in some circumstances that useful initial seed inputs are discarded as
they find no new coverage, even though they contain useful fragments in them
(e.g., SQLITE3 FuzzBench benchmark). This newly added option provides a way to
keeping seed inputs in the corpus for those circumstances. With this patch, and
with -keep_seed=1, all initial seed inputs are kept in the corpus regardless of
whether they find new coverage or not. Further, these seed inputs are not
replaced with smaller inputs even if -reduce_inputs=1.

Differential Revision: https://reviews.llvm.org/D86577
2020-09-03 15:54:39 +00:00
Matt Morehouse 711b980654 [fuzzer] Create user provided fuzzer writeable directories when requested if they dont exist
Currently, libFuzzer will exit with an error message if a non-existent
directory is provided for any of the appropriate arguments. For cases
where libFuzzer is used in a specialized embedded environment, it would
be much easier to have libFuzzer create the directories for the user.

This patch accommodates for this scenario by allowing the user to provide
the argument `-create_missing_dirs=1` which makes libFuzzer attempt to
create the `artifact_prefix`, `exact_artifact_path`,
`features_dir` and/or corpus directory if they don't already exist rather
than throw an error and exit.

Split off from D84808 as requested [here](https://reviews.llvm.org/D84808#2208546).

Reviewed By: morehouse

Differential Revision: https://reviews.llvm.org/D86733
2020-09-03 08:31:59 -07:00
Jinsong Ji 0cefaec2e8 [compiler-rt] Don't build llvm-lit in RUNTIMES-BUILD
With runtimes-build, we lost map config for llvm-lit, so we can NOT run
lit from source dir anymore.
All the subdir target like: ninja check-llvm-codegen-powerpc will fail
too.

We will get some cfg error like:
  File "../lvm-project/llvm/test/lit.cfg.py", line 21, in <module>
    config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
AttributeError: 'NoneType' object has no attribute 'use_lit_shell'

This is reset of map config in llvm-lit rebuild.
We already have llvm-lit in runtimes-build, so don't build it.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D87002
2020-09-03 01:08:16 +00:00
Fangrui Song 2324482383 [asan_symbolize] Remove --use-symbol-table=true which is the default 2020-09-02 17:34:35 -07:00
Kostya Kortchinsky 00d9907a7a [scudo][standalone] Enable secondary cache release on Fuchsia
I had left this as a TODO, but it turns out it wasn't complicated.
By specifying `MAP_RESIZABLE`, it allows us to keep the VMO which we
can then use for release purposes.
`releasePagesToOS` also had to be called the "proper" way, as Fuchsia
requires the `Offset` field to be correct. This has no impact on
non-Fuchsia platforms.

Differential Revision: https://reviews.llvm.org/D86800
2020-09-02 14:28:17 -07:00