This reverts commit ccdb56ac10.
Still seeing windows failures on GN bots: http://45.33.8.238/win/58316/step_9.txt
Unfortunately I can't debug these at all - it's a bare unsymbolized
stacktrace, and I can't reproduce the failure.
- add missing benchmark for lex/preprocess steps
- name benchmarks after the function they're benchmarking, when appropriate
- remove unergonomic "run" prefixes from benchmark names
- give a useful error message if --grammar or --source are missing
- Use realistic example of how to run, run all benchmarks by default.
(for someone who doesn't know the commands, this is the most useful action)
- Improve typos/wording in comment
- clean up unused vars
- avoid "parseable stream" name, which isn't a great name & not one I expected
to escape from ClangPseudoMain
Differential Revision: https://reviews.llvm.org/D125312
This is a followup to D124715, which changed the default, and it anticipates
future patches raising the priority of Low (which is currently equal to
Background on Windows & Linux).
The main point is to allow users to restore the old behavior, which e.g.
allows efficiency cores to remain idle.
I did consider making this a config setting, this is a more complicated change:
- needs to touch queue priorities as well as thread priorities
- we don't know the priority until evaluating the config inside the task
- users would want the ability to prioritize background indexing tasks relative
to each other without necessarily affecting thread priority, so using one
option for both may be confusing
I don't really have a use case, so I prefer the simpler thing.
Differential Revision: https://reviews.llvm.org/D125673
This provides a nice "warm start" with all headers indexed, not just
those included so far.
The standard library is indexed after a preamble is parsed, using that
file's configuration. The result is pushed into the dynamic index.
If we later see a higher language version, we reindex it.
It's configurable as Index.StandardLibrary, off by default for now.
Based on D105177 by @kuhnel
Fixes https://github.com/clangd/clangd/issues/618
Differential Revision: https://reviews.llvm.org/D115232
Previously the Expr returned by getOperand() was actually the
subexpression common to the "ready", "suspend", and "resume"
expressions, which often isn't just the operand but e.g.
await_transform() called on the operand.
It's important for the AST to expose the operand as written
in the source for traversals and tools like clangd to work
correctly.
Fixes https://github.com/clangd/clangd/issues/939
Differential Revision: https://reviews.llvm.org/D115187
Reimplement the matching logic using Visitors instead of matchers.
Benchmarks from running the check over SemaCodeComplete.cpp
Before 0.20s, After 0.04s
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D125026
I am working on support for forwarding parameter names in make_unique-like functions, first for inlay hints, later maybe for signature help.
For that to work generically, I'd like to parse all of these functions in the preamble. Not sure how this impacts performance on large codebases though.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D124688
With this patch, we're able to parse smaller chunks of C++ code (statement,
declaration), rather than translation-unit.
The start symbol is listed in the grammar in a form of `_ :=
statement`, each start symbol has a dedicated state (`_ := • statement`).
We create and track all these separate states in the LRTable. When we
start parsing, we lookup the corresponding state to start the parser.
LR pasing table changes with this patch:
- number of states: 1467 -> 1471
- number of actions: 82891 -> 83578
- size of the table (bytes): 334248 -> 336996
Differential Revision: https://reviews.llvm.org/D125006
Disable the warnings with `IWYU pragma: export` or `begin_exports` +
`end_exports` until we have support for these pragmas. There are too many
false-positive warnings for the headers that have the correct pragmas for now
and it makes the user experience very unpleasant.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D125468
On Apple Silicon Macs, using a Darwin thread priority of PRIO_DARWIN_BG seems to
map directly to the QoS class Background. With this priority, the thread is
confined to efficiency cores only, which makes background indexing take forever.
Introduce a new ThreadPriority "Low" that sits in the middle between Background
and Default, and maps to QoS class "Utility" on Mac. Make this new priority the
default for indexing. This makes the thread run on all cores, but still lowers
priority enough to keep the machine responsive, and not interfere with
user-initiated actions.
I didn't change the implementations for Windows and Linux; on these systems,
both ThreadPriority::Background and ThreadPriority::Low map to the same thread
priority. This could be changed as a followup (e.g. by using SCHED_BATCH for Low
on Linux).
See also https://github.com/clangd/clangd/issues/1119.
Reviewed By: sammccall, dgoldman
Differential Revision: https://reviews.llvm.org/D124715
Clang erroneously flagged the function as "unused", but it is most
definitely used by gtest to pretty print the parameter value when
a test fails.
Make the pretty printing function a friend function in the parameter
class similar to other clang unit tests.
Add a recursive descent parser to match macro expansion tokens against
fully formed valid expressions of integral literals. Partial
expressions will not be matched -- they can't be valid initializing
expressions for an enum.
Differential Revision: https://reviews.llvm.org/D124500Fixes#55055
The check should not report includes wrapped by `extern "C" { ... }` blocks,
such as:
```lang=C++
#ifdef __cplusplus
extern "C" {
#endif
#include "assert.h"
#ifdef __cplusplus
}
#endif
```
This pattern comes up sometimes in header files designed to be consumed
by both C and C++ source files.
The check now reports false reports when the header file is consumed by
a C++ translation unit.
In this change, I'm not emitting the reports immediately from the
`PPCallback`, rather aggregating them for further processing.
After all preprocessing is done, the matcher will be called on the
`TranslationUnitDecl`, ensuring that the check callback is called only
once.
Within that callback, I'm recursively visiting each decls, looking for
`LinkageSpecDecls` which represent the `extern "C"` specifier.
After this, I'm dropping all the reports coming from inside of it.
After the visitation is done, I'm emitting the reports I'm left with.
For performance reasons, I'm sorting the `IncludeMarkers` by their
corresponding locations.
This makes the scan `O(log(N)` when looking up the `IncludeMarkers`
affected by the given `extern "C"` block. For this, I'm using
`lower_bound()` and `upper_bound()`.
Reviewed By: whisperity
Differential Revision: https://reviews.llvm.org/D125209
Fixes the `FIXME:` related to adding `forEachTemplateArgument` to the
core AST Matchers library.
Reviewed By: aaron.ballman
Differential Revision: http://reviews.llvm.org/D125383
`FileNotFound` preprocessor callback is removed in D119708.
We should also remove it from the documentation.
Reviewed by: jansvoboda11
Differential Revision: https://reviews.llvm.org/D125258
- Make clangd's internal representation more aligned with the standard.
We keep range and extra inlayhint kinds around, but don't serialize
them on standard version.
- Have custom serialization for extension (ugly, but going to go away).
- Support both versions until clangd-17.
- Don't advertise extension if client has support for standard
implementation.
- Log a warning at startup about extension being deprecated, if client
doesn't have support.
Differential Revision: https://reviews.llvm.org/D125228
This is unneccesary work.
With this change, we skip generating and lexing ~10k of predefines twice.
A dumb benchmark of building a preamble for an empty file in a loop shows:
- before: 1.90ms/run
- after: 1.36ms/run
So this should be worth 0.5ms for each AST build and code completion.
There can be a functional difference, but it's very minor.
If the preamble contains e.g. `#ifndef __llvm__ ... #endif` then before we would
not take it. After this change we will take the branch (single-file mode takes
all branches with unknown conditions) and so gather different directives.
However I think this is negligible:
- this is already true of non-builtin macros (from included headers).
We've had no complaints.
- this affects the baseline and modified in the same way, so only makes a
difference transiently when code guarded by such an #ifdef is being edited
Differential Revision: https://reviews.llvm.org/D125179
Previously the EXPECT_AVAILABLE macros would rebuild the code at each marked
point, by expanding the cases textually.
There were often lots, and it's nice to have lots!
This reduces total unittest time by ~10% on my machine.
I did have to sacrifice a little apply() coverage in AddUsingTests (was calling
expandCases directly, which was otherwise unused), but we have
EXPECT_AVAILABLE tests covering that, I don't think there's real risk here.
Differential Revision: https://reviews.llvm.org/D125109
This is a clever cross-cutting sanity test for clang's arg parsing I suppose.
But clangd creates thousands of invocations, ~all with identical trivial
arguments, and problems with these would be caught by clang's tests.
This overhead accounts for 10% of total unittest time!
Differential Revision: https://reviews.llvm.org/D125169
These aren't needed. With them the generated predefines buffer is 13KB.
For every TestTU, we must:
- generate the buffer (3 times: parsing preamble, scanning preamble, main file)
- parse the buffer (again 3 times)
- serialize all the macros it defines in the PCH
- compress the buffer itself to write it into the PCH
- decompress it from the PCH
Avoiding this reduces unit test time by ~25%.
Differential Revision: https://reviews.llvm.org/D125172
There's many instances in clang tidy checks where owning strings are used when we already have a stable string from the options, so using a StringRef makes much more sense.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D124341
All llvm-project fuzzers use this library to parse command-line arguments.
Many of them don't deal with LLVM IR or modules in any way. Bundling those
functions in one library forces build dependencies that don't need to be there.
Among other things, this means check-clang-pseudo no longer depends on most of
LLVM.
Differential Revision: https://reviews.llvm.org/D125081
This change was intended to add the tests check-clang and check-clang-pseudo,
but afterwards it was *only* running those tests.
(This was because unlike add_lit_testsuite, add_lit_testsuite*s* does not
get included in umbrella suites).
This check verifies the safety of access to `std::optional` and related
types (including `absl::optional`). It is based on a corresponding Clang
Dataflow Analysis, which does most of the work. This check merely runs it and
converts its findings into diagnostics.
Differential Revision: https://reviews.llvm.org/D121120
This testcase runs slowly due to 3.2s of sleeps = 2 + 1 + 0.2s.
After this patch it has 0.55s only.
Reduced by:
- observed that the last test was bogus: we were sleeping until the queue was
idle, effectively just a second copy of the first test. This avoids 1s sleep.
- when waiting for debounce, sleep only until test passes, not for enough
time to be safe (in practice was 2x debounce time, now 1x debounce time)
- scaling delays down by a factor of 2 (note: factor of 10 caused bot failures)
Differential Revision: https://reviews.llvm.org/D125103
This method won't add a check if it isn't supported in the Contexts current LanguageOptions.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D124320
The mechanism behind "check-all" is recording params of add_lit_testsuite()
calls in global variables LLVM_LIT_*, and then creating an extra suite with
their union at the end.
This avoids composing the check-* targets directly, which doesn't work well.
We generalize this by allowing multiple families of variables LLVM_{name}_LIT_*:
umbrella_lit_testsuite_begin(check-foo)
... test suites here will be added to LLVM_FOO_LIT_* variables ...
umbrella_lit_testsuite_end(check-foo)
(This also moves some implementation muck out of {llvm,clang}/CMakeLists.txt
This patch also changes check-clang-tools to use be an umbrella test target,
which means the clangd and clang-pseudo tests are included in it, along with the
the other testsuites that already are (like check-clang-extra-clang-tidy).
Differential Revision: https://reviews.llvm.org/D121838
This includes only the taken branch of conditional sections.
The API allows for producing a stream for a particular PP branch, which
will be used later for the secondary GLR parses of not-taken branches.
Differential Revision: https://reviews.llvm.org/D123243
As confirmation, running this locally found 2 crashes:
- trivial: crashes on file with no tokens
- lexer: hits an assertion failure on bytes: 0x5c,0xa,0x5c,0x1,0x65,0x5c,0xa
Differential Revision: https://reviews.llvm.org/D125037
It turns out clang::expandUCNs only works on tokens that contain valid UCNs
and no other random escapes, and clang only uses it on raw_identifiers.
Currently we can hit an assertion by creating tokens with stray non-valid-UCN
backslashes in them.
Fortunately, expanding UCNs in raw_identifiers is actually all we need.
Most tokens (keywords, punctuation) can't have them. UCNs in literals can be
treated as escape sequences like \n even this isn't the standard's
interpretation. This more or less matches how clang works.
(See https://isocpp.org/files/papers/P2194R0.pdf which points out that the
standard's description of how UCNs work is misaligned with real implementations)
Differential Revision: https://reviews.llvm.org/D125049
Sam McCall