Before this change clangd would emit a diagnostic whenever remote-index
was configured but binary didn't have grpc support.
This can be annoying when projects are configuring remote-index through their
configs but developers have a clangd binary without the support.
Differential Revision: https://reviews.llvm.org/D100103
This will allow us to add code completion, which is too expensive at
every token, to --check too.
Differential Revision: https://reviews.llvm.org/D98970
When you pass in a payload with an invalid URI in a build with assertions enabled, it will crash.
Consuming the error from the failed URI parse prevents the error.
The crash is caused by the [llvm::expected](https://llvm.org/doxygen/classllvm_1_1Expected.html) having protection around trying to deconstruct without consuming the error first.
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D99872
As proposed in D97109, I tried to make target creation consistent in `clang` and `clangd` by replacing the original procedure with a single function introduced in D97493.
This also helps `clangd` works with CUDA, OpenMP, etc.
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D98128
Clangd drops symbols from static index whenever the dynamic index is
authoritative for the file. This results in regressions when static and
dynamic index contains different set of information, e.g.
IncludeHeaders.
After this patch, we'll choose to merge symbols from static index with
dynamic one rather than just dropping. This implies correctness problems
when the definition/documentation of the symbol is deleted. But seems
like it is worth having in more cases.
We still drop symbols if dynamic index owns the file and didn't report
the symbol, which means symbol is deleted.
Differential Revision: https://reviews.llvm.org/D98538
`expandedTokens(SourceRange)` used to do a binary search to get the
expanded tokens belonging to a source range. Each binary search uses
`isBeforeInTranslationUnit` to order two source locations. This is
inherently very slow.
By profiling clangd we found out that users like clangd::SelectionTree
spend 95% of time in `isBeforeInTranslationUnit`. Also it is worth
noting that users of `expandedTokens(SourceRange)` majorly use ranges
provided by AST to query this funciton. The ranges provided by AST are
token ranges (starting at the beginning of a token and ending at the
beginning of another token).
Therefore we can avoid the binary search in majority of the cases by
maintaining an index of ExpandedToken by their SourceLocations. We still
do binary search for ranges which are not token ranges but such
instances are quite low.
Performance:
`~/build/bin/clangd --check=clang/lib/Serialization/ASTReader.cpp`
Before: Took 2:10s to complete.
Now: Took 1:13s to complete.
Differential Revision: https://reviews.llvm.org/D99086
Dummy is a word with inappropriate associations. This patch updates the
references to it in clangd code base with more precise ones.
The only user-visible change is the default variable name used when extracting a
variable. It will be named as `placeholder` from now on.
Differential Revision: https://reviews.llvm.org/D99065
llvm supports specifying a non-standard layout where each project lies in its
own place. Do not assume a fixed layout and use the appropriate cmake variable
instead.
Differential Revision: https://reviews.llvm.org/D96787
The idiom:
```
DeclContext::lookup_result R = DeclContext::lookup(Name);
for (auto *D : R) {...}
```
is not safe when in the loop body we trigger deserialization from an AST file.
The deserialization can insert new declarations in the StoredDeclsList whose
underlying type is a vector. When the vector decides to reallocate its storage
the pointer we hold becomes invalid.
This patch replaces a SmallVector with an singly-linked list. The current
approach stores a SmallVector<NamedDecl*, 4> which is around 8 pointers.
The linked list is 3, 5, or 7. We do better in terms of memory usage for small
cases (and worse in terms of locality -- the linked list entries won't be near
each other, but will be near their corresponding declarations, and we were going
to fetch those memory pages anyway). For larger cases: the vector uses a
doubling strategy for reallocation, so will generally be between half-full and
full. Let's say it's 75% full on average, so there's N * 4/3 + 4 pointers' worth
of space allocated currently and will be 2N pointers with the linked list. So we
break even when there are N=6 entries and slightly lose in terms of memory usage
after that. We suspect that's still a win on average.
Thanks to @rsmith!
Differential revision: https://reviews.llvm.org/D91524
This allows requesting information about the server uptime and start time. This is the first patch in a series of monitoring changes, hence it's not immediately useful. Next step is propagating the index freshness information and then probably loading metadata into the index server.
The way to test new behaviour through command line:
```
$ grpc_cli call localhost:50051 Monitor/MonitoringInfo ''
connecting to localhost:50051
uptime_seconds: 42
index_age_seconds: 609568
Rpc succeeded with OK status
```
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D98246
Implement initial support for pull-based diagnostics in ClangdServer.
This is planned for LSP 3.17, and initial proposal is in
d15eb0671e/protocol/src/common/proposed.diagnostic.ts (L111).
We chose to serve the requests only when clangd has a fresh preamble
available. In case of a stale preamble we just drop the request on the
floor.
This patch doesn't plumb this to LSP layer yet, as pullDiags is still a
proposal with only an implementation in vscode.
Differential Revision: https://reviews.llvm.org/D98623
Somewhat surprisingly, signature help is emitted as a side-effect of
computing the expected type of a function argument.
The reason is that both actions require enumerating the possible
function signatures and running partial overload resolution, and doing
this twice would be wasteful and complicated.
Change #1: document this, it's subtle :-)
However, sometimes we need to compute the expected type without having
reached the code completion cursor yet - in particular to allow
completion of designators.
eb4ab3358c did this but introduced a
regression - it emits signature help in the wrong location as a side-effect.
Change #2: only emit signature help if the code completion cursor was reached.
Currently there is PP.isCodeCompletionReached(), but we can't use it
because it's set *after* running code completion.
It'd be nice to set this implicitly when the completion token is lexed,
but ConsumeCodeCompletionToken() makes this complicated.
Change #3: call cutOffParsing() *first* when seeing a completion token.
After this, the fact that the Sema::Produce*SignatureHelp() functions
are even more confusing, as they only sometimes do that.
I don't want to rename them in this patch as it's another large
mechanical change, but we should soon.
Change #4: prepare to rename ProduceSignatureHelp() to GuessArgumentType() etc.
Differential Revision: https://reviews.llvm.org/D98488
This enables unifying command line flags with config options in clangd
internals. This patch changes behaviour in 2 places:
- BackgroundIndex was previously disabled when -remote-index was
provided. After this patch, it will be enabled but all files will have
bkgindex policy set to Skip.
- -index-file was loaded at startup (at least load was initiated), now
the load will happen through ProjectAwareIndex with first index query.
Unfortunately this doesn't simplify any options initially, as
- CompileCommandsDir is also used by clangd --check workflow, which
doesn't use configs.
- EnableBackgroundIndex option controls whether the component will be
created at all, which implies creation of extra threads registering a
listener for compilation database discoveries.
Differential Revision: https://reviews.llvm.org/D98029
Explicit specifier can only be mentioned on the in-line declaration of a
constructor, so don't carry it over to the definition.
Differential Revision: https://reviews.llvm.org/D98164
Also give CanonicalIncludes a less powerful interface (canonicalizes
symbols vs headers separately) so we can cache its results better.
Prior to this:
- path->uri conversions were not consistently cached, this is
particularly cheap when we start from a FileEntry* (which we often can)
- only a small fraction of header-to-include calculation was cached
This is a significant speedup at least for dynamic indexing of preambles.
On my machine, opening XRefs.cpp:
```
PreambleCallback 1.208 -> 1.019 (-15.7%)
BuildPreamble 5.538 -> 5.214 (-5.8%)
```
Differential Revision: https://reviews.llvm.org/D98371
Refactor cross file rename to use a Filesystem instead of a function for getting buffer contents of open files.
Depends on D94554
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D95043
This allows sending requests through CLI and more debugging
opportunities. Example:
```bash
$ grpc_cli ls localhost:50051
clang.clangd.remote.v1.SymbolIndex
grpc.reflection.v1alpha.ServerReflection
grpc.health.v1.Health
```
https://reviews.llvm.org/D94554 introduced code which wont compile with some build flags due to a field having the same identifier as a type.
clang-tools-extra/clangd/DraftStore.h:55:11: error: declaration of ‘clang::clangd::DraftStore::Draft clang::clangd::DraftStore::DraftAndTime::Draft’ changes meaning of ‘Draft’ [-fpermissive]
55 | Draft Draft;
| ^~~~~
clang-tools-extra/clangd/DraftStore.h:30:10: note: ‘Draft’ declared here as ‘struct clang::clangd::DraftStore::Draft’
30 | struct Draft {
| ^~~~~
Create a `ThreadsafeFS` in the `DraftStore` that overlays the dirty file contents over another `ThreadsafeFS`.
This provides a nice thread-safe interface for using dirty file contents throughout the codebase, for example cross file refactoring.
Creating a Filesystem view will overlay a snapshot of the current contents, so if the draft store is updated while the view is being used, it will contain stale contents.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D94554
This was causing TSan failures due to a race on vptr during destruction,
see
https://lab.llvm.org/buildbot/#/builders/131/builds/6579/steps/6/logs/FAIL__Clangd_Unit_Tests__LSPTest_FeatureModulesThr.
The story is, during the execution of a destructor all the virtual
dispatches should resolve to implementations in the class being
destroyed, not the derived ones. And LSPTests will log some stuff during
destruction (we send shutdown/exit requests, which are logged), which is
a virtual dispatch when LSPTest is derived from clang::clangd::Logger.
It is a benign race in our case, as gtests that derive from LSPTest
doesn't override the `log` method. But still during destruction, we
might try to update vtable ptr (due to being done with destruction of
test and starting destruction of LSPTest) and read from it to dispatch a
log message at the same time.
This patch fixes that race by moving `log` out of the vtable of
`LSPTest`.
Differential Revision: https://reviews.llvm.org/D98241
Without this patch the file list of the preamble index contains URIs, but other indexes file lists contain file paths.
This makes `indexedFiles()` always returns `IndexContents::None` for the preamble index, because current implementation expects file paths inside the file list of the index.
This patch fixes this problem and also helps to avoid a lot of URI to path conversions during indexes merge.
Reviewed By: kadircet
Differential Revision: https://reviews.llvm.org/D97535
As pointed out in D96244, "Module" is already pretty overloaded to refer
to clang and llvm modules. (And clangd deals directly with the former).
FeatureModule is a bit of a mouthful but it's pretty self-descriptive.
I think it might be better than "Component" which doesn't really capture
the "common interface" aspect - it's IMO confusing to refer to
"components" but exclude CDB for example.
Differential Revision: https://reviews.llvm.org/D97950
GCC warning:
```
/llvm-project/clang-tools-extra/clangd/SemanticHighlighting.cpp: In function ‘bool clang::clangd::{anonymous}::canHighlightName(clang::DeclarationName)’:
/llvm-project/clang-tools-extra/clangd/SemanticHighlighting.cpp:64:1: warning: control reaches end of non-void function [-Wreturn-type]
64 | }
| ^
```
Clangd can invalidate client state of features like semantic higlighting
without client explicitly triggering, for example after a preamble build
caused by an onSave notification on a different file.
This patch introduces a mechanism to let client know of such actions,
and also calls the workspace/semanticTokens/refresh request to
demonstrate the situation after each preamble build.
Fixes https://github.com/clangd/clangd/issues/699.
Differential Revision: https://reviews.llvm.org/D97548
- highlight references to protocols in class/protocol/extension decls
- support multi-token selector highlights in semantic + xref highlights
(method calls and declarations only)
- In `@interface I(C)`, I now references the interface and C the category
- highlight uses of interfaces as types
- added semantic highlightings of protocol names (as "interface") and
category names (as "namespace").
These are both standard kinds, maybe "extension" will be standardized...
- highlight `auto` as "class" when it resolves to an ObjC pointer
- don't highlight `self` as a variable even though the AST models it as one
Not fixed: uses of protocols in type names (needs some refactoring of
unrelated code first)
Differential Revision: https://reviews.llvm.org/D97617
Clangd uses codecompletion limit as the limit for workspacesymbols, so
in theory this should only be an order of magnitude slower than a
codecompletion request with empty identifier (as code completion limits
the available symbols).
This is also what LSP suggests "Clients may send an empty string here to request all symbols.".
Clangd doesn't really fulfill the "all" part of that statement, but we
never do unless user set the index query limit to zero explicitly.
Differential Revision: https://reviews.llvm.org/D97773
ClangdServer already gets notified of every change, so it makes sense for it to
be the source of truth.
This is a step towards having ClangdServer expose a FS that includes dirty
buffers: D94554
Related changes:
- version is now optional for ClangdServer, to preserve our existing fuzziness
in this area (missing version ==> autoincrement)
- ClangdServer::format{File,Range} are now more regular ClangdServer functions
that don't need the code passed in. While here, combine into one function.
- incremental content update logic is moved from DraftStore to
ClangdLSPServer, with most of the implementation in SourceCode.cpp.
DraftStore is now fairly trivial, and will probably ultimately be
*replaced* by the dirty FS stuff.
Differential Revision: https://reviews.llvm.org/D97738
Browsing macro-generated symbols is confusing.
On the one hand, it seems very *useful* to be able to see the summary of
symbols that were generated.
On the other hand, some macros spew a lot of confusing symbols into the
namespace and when used repeatedly (ABSL_FLAG) can create a lot of spam
that's hard to navigate.
Design constraints:
- the macro expansion tree need not align with the AST, though it often
does in practice.
We address this by defining the nesting based on the *primary*
location of decls, rather than their ranges.
- DocumentSymbol.children[*].range should nest within DocumentSymbol.range
(This constraint is not in LSP "breadcrumbs" breaks without it)
We adjust macro ranges so they cover their "children", rather than
just the macro expansion
- LSP does not have a "macro expansion" symbolkind, nor does it allow a
symbol to have no kind. I've arbitrarily picked "null" as this is
unlikely to conflict with anything useful.
This patch makes all macros and children visible for simplicity+consistency,
though in some cases it may be better to elide the macro node.
We may consider adding heuristics for this in future (e.g. when it expands
to one decl only?) but it doesn't seem clear-cut to me.
Differential Revision: https://reviews.llvm.org/D97615
Don't show negative numbers
Don't show numbers <10 (hex is the same as decimal)
Show numeric enum values in hex too
Differential Revision: https://reviews.llvm.org/D97226
CodeCompletionContext::Kind has 36 Kinds. The completion model used to
support categorical features of 32 cardinality.
Due to this clangd tests were failing asan tests due to overflow.
This patch makes the completion model support 64 cardinality of
categorical features by storing ENUM Features as uint64_t instead of
uint32_t.
Verified that this fixes the asan failures.
Latency: 6.7ms (old) VS 6.8ms (new) per 1000 predictions.
Differential Revision: https://reviews.llvm.org/D97770
CodeCompletionContext::Kind has 36 Kinds. The completion model currently
only handles categorical features of 32 cardinality.
Changing the datatype to uint64_t will solve the problem.
This reverts commit 438b5bb05a.
clang-tools-extra/clangd/benchmarks/CompletionModel/DecisionForestBenchmark.cpp fails to compile since `"CompletionModel.h"` is auto-generated from clang-tools-extra/clangd/quality/model/features.json, which was changed in https://reviews.llvm.org/D94697 to remove `setFilterLength` and `setIsForbidden`, rename `setFileProximityDistance` and `setSymbolScopeDistance`, and add `setNumNameInContext` and `setFractionNameInContext`. This patch removes calls to the two removed functions, updates calls to the two renamed functions, and adds calls to the two new functions. (`20` is an arbitrary choice for the `setNumNameInContext` argument.) It also changes the `FlipCoin` argument from float to double to silence lossy conversion warnings.
Note: I don't use this tool but encountered the build errors and took a shot at fixing them. Please holler if there's another recommended solution. Thanks!
Reviewed By: usaxena95
Differential Revision: https://reviews.llvm.org/D97620
This makes code completion use a Decision Forest based ranking algorithm to rank
completion candidates. [Esitmated 6% accuracy boost]. This was
previously hidden behind the flag --ranking-model=decision_forest. This
patch makes it the default ranking algorithm.
Note: this is a generic model, not specialized for any particular
project. clangd does not collect or upload data to train code completion.
Also treat Keywords separately as they are not recorded by the training set generator.
Differential Revision: https://reviews.llvm.org/D96353
- Categories will now show up as `MyClass(Category)` instead of
`Category` and `MyCategory()` instead of `(anonymous)` in document
symbols
- Methods will now be shown as `-selector:` or `+selector:`
instead of `selector:` to differentiate between instance and class
methods in document symbols
Differential Revision: https://reviews.llvm.org/D96612
Kadir Cetinkaya