Commit Graph

591 Commits

Author SHA1 Message Date
Kadir Cetinkaya 4728aca9a8
[clangd] Drop TestTUs dependency on gtest
TestTU now prints errors to llvm::errs and aborts on failures via
llvm_unreachable, rather than executing ASSERT_FALSE.

We'd like to make use of these testing libraries in different test suits that
might be compiling with a different gtest version than LLVM has.

Differential Revision: https://reviews.llvm.org/D103685
2021-06-07 13:25:22 +02:00
Kadir Cetinkaya 9e9ac41388
[clangd] Drop optional on ExternalIndexSpec
Differential Revision: https://reviews.llvm.org/D100308
2021-06-02 23:26:37 +02:00
Kadir Cetinkaya 888307ee62
[clangd][remote-client] Set HasMore to true for failure
Currently client was setting the HasMore to true iff stream said so.
Hence if we had a broken stream for whatever reason (e.g. hitting deadline for a
huge response), HasMore would be false, which is semantically incorrect (e.g.
will throw rename off).

Differential Revision: https://reviews.llvm.org/D101915
2021-05-11 08:22:24 +02:00
Kadir Cetinkaya daf3cb3b8a
[clangd][index-sever] Limit results in repsonse
This is to prevent server from being DOS'd by possible malicious
parties issuing requests that can yield huge responses.

One possible drawback is on rename workflow. As it really requests all
occurences, but it has an internal limit on 50 files currently.
We are putting the limit on 10000 elements per response So for rename to regress
one should have 10k refs to a symbol in less than 50 files. This seems unlikely
and we fix it if there are complaints by giving up on the response based on the
number of files covered instead.

Differential Revision: https://reviews.llvm.org/D101914
2021-05-11 08:22:23 +02:00
Kirill Bobyrev 34593ae998 Introduce clangd-server-monitor tool
Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D101516
2021-05-04 12:48:21 +02:00
Kadir Cetinkaya 4581bf31bb
[clangd] Dont index deeply nested symbols
This is fix for some timeouts and OOM problems faced while indexing an
auto-generated file with thousands of nested lambdas.

Differential Revision: https://reviews.llvm.org/D101066
2021-04-27 12:34:56 +02:00
Kadir Cetinkaya 6d2fb3cefb
[clangd] Perform merging for stale symbols in MergeIndex
Clangd drops symbols from static index whenever the dynamic index is
authoritative for the file. This results in regressions when static and
dynamic index contains different set of information, e.g.
IncludeHeaders.

After this patch, we'll choose to merge symbols from static index with
dynamic one rather than just dropping. This implies correctness problems
when the definition/documentation of the symbol is deleted. But seems
like it is worth having in more cases.

We still drop symbols if dynamic index owns the file and didn't report
the symbol, which means symbol is deleted.

Differential Revision: https://reviews.llvm.org/D98538
2021-03-30 11:09:51 +02:00
Kadir Cetinkaya f71404c37c
[clangd] Replace usages of dummy with more descriptive words
Dummy is a word with inappropriate associations. This patch updates the
references to it in clangd code base with more precise ones.

The only user-visible change is the default variable name used when extracting a
variable. It will be named as `placeholder` from now on.

Differential Revision: https://reviews.llvm.org/D99065
2021-03-22 12:49:24 +01:00
Nathan Ridge 2e58226d8d [clangd] Fix linker error when linking clang-index-server with shared libraries
Fixes https://github.com/clangd/clangd/issues/723

Differential Revision: https://reviews.llvm.org/D99049
2021-03-22 02:38:58 -04:00
Kirill Bobyrev 524fe51509
[clangd] Add basic monitoring info request for remote index server
This allows requesting information about the server uptime and start time. This is the first patch in a series of monitoring changes, hence it's not immediately useful. Next step is propagating the index freshness information and then probably loading metadata into the index server.

The way to test new behaviour through command line:

```
$ grpc_cli call localhost:50051 Monitor/MonitoringInfo ''
connecting to localhost:50051
uptime_seconds: 42
index_age_seconds: 609568
Rpc succeeded with OK status
```

Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D98246
2021-03-16 13:37:58 +01:00
Kirill Bobyrev 9bcf0eff99
[clangd] Optionally add reflection for clangd-index-server
This was originally landed without the optional part and reverted later:

8080ea4c4b

Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D98404
2021-03-15 21:07:25 +01:00
Kadir Cetinkaya dc9c09632f
[clangd] Make ProjectAwareIndex optionally sync
Depends on D98029.

Differential Revision: https://reviews.llvm.org/D98165
2021-03-11 20:30:35 +01:00
Sam McCall b8c58374f6 [clangd] Group filename calculations in SymbolCollector, and cache mroe.
Also give CanonicalIncludes a less powerful interface (canonicalizes
symbols vs headers separately) so we can cache its results better.

Prior to this:
 - path->uri conversions were not consistently cached, this is
   particularly cheap when we start from a FileEntry* (which we often can)
 - only a small fraction of header-to-include calculation was cached

This is a significant speedup at least for dynamic indexing of preambles.
On my machine, opening XRefs.cpp:

```
PreambleCallback 1.208 -> 1.019 (-15.7%)
BuildPreamble    5.538 -> 5.214 (-5.8%)
```

Differential Revision: https://reviews.llvm.org/D98371
2021-03-11 12:59:26 +01:00
Kadir Cetinkaya 99b01cf28d
Revert "[clangd] Enable reflection for clangd-index-server"
This reverts commit 8080ea4c4b.

As discussed offline we should only do that for debug builds.
2021-03-10 14:12:37 +01:00
Kirill Bobyrev 8080ea4c4b [clangd] Enable reflection for clangd-index-server
This allows sending requests through CLI and more debugging
opportunities. Example:

```bash
$ grpc_cli ls localhost:50051
clang.clangd.remote.v1.SymbolIndex
grpc.reflection.v1alpha.ServerReflection
grpc.health.v1.Health
```
2021-03-10 09:07:39 +01:00
Fangrui Song cd6d1799ad [clangd] Treat __GCC_HAVE_DWARF2_CFI_ASM the same as isWrittenInBuiltinFile macros 2021-03-09 23:11:54 -08:00
Aleksandr Platonov c4efd04f18 [clangd] Use URIs instead of paths in the index file list
Without this patch the file list of the preamble index contains URIs, but other indexes file lists contain file paths.
This makes `indexedFiles()` always returns `IndexContents::None` for the preamble index, because current implementation expects file paths inside the file list of the index.

This patch fixes this problem and also helps to avoid a lot of URI to path conversions during indexes merge.

Reviewed By: kadircet

Differential Revision: https://reviews.llvm.org/D97535
2021-03-06 10:47:05 +03:00
Kadir Cetinkaya 1a5dfb7db2
[clangd][remote] Add flag to set idletimeout
By default gRPC has no idletimeout and some firewalls might drop idle
connections after a certain period. This results in idle clients
shouting into void until server resets the connection.

Differential Revision: https://reviews.llvm.org/D97536
2021-02-26 15:04:44 +01:00
Haojian Wu 7048cb5371 [clangd] IndexedFiles should include Fils from RefSlab and RelationSlab.
This looks like an oversight.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D96845
2021-02-17 15:41:09 +01:00
Haojian Wu e030de7e5a [clangd] Pass file when possible to resolve URI.
Some URI scheme needs the hint path to do a correct resolution, we pass
one of the open files as hint path.

This is not perfect, and it might not work for opening files across
project, but it would fix a bug with our internal scheme.

in the long run, removing URIs from all the index internals is a more proper fix.

Differential Revision: https://reviews.llvm.org/D96844
2021-02-17 15:33:50 +01:00
Aleksandr Platonov 91698fe45f [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge
Current indexes merge logic skip data from the static index if the file is in the dynamic index, but sometimes the dynamic index does not contain references (e.g. preamble (dynamic) index vs background (static) index).
This problem is masked with the fact, that the preamble index file list consists of file URI's and other indexes file lists consist of file paths.
This patch introduces the index contents (symbols, references, etc.), which makes indexes merge more flexible and makes it able to use URI's for the index file list.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D94952
2021-02-05 13:35:07 +03:00
Sam McCall 1eb7fd089e [clangd] Remove some obsolete options that are now always on
- always collect main-file refs when indexing
 - always build preambles asynchronously
 - always use dex for fast preamble index

Retire associated flags

Differential Revision: https://reviews.llvm.org/D95571
2021-02-01 17:24:03 +01:00
Sam McCall 8712df7a62 [clangd] references: decls of overrides of x are refs to x, not decls
This requires a second index query for refs to overrides, as the refs
call doesn't tell you which ref points at which symbol.

Differential Revision: https://reviews.llvm.org/D95451
2021-02-01 17:19:19 +01:00
Kadir Cetinkaya 9190f17a7c
[clangd] Add std::size_t to StdSymbol mapping
This is a common symbol that's missing from our mapping because
cppreference yields multiple headers.

Add it manually by picking cstddef to prevent insertion of some stdlib-internal
headers instead.

Fixes https://github.com/clangd/clangd/issues/666.

Differential Revision: https://reviews.llvm.org/D95423
2021-01-26 17:53:07 +01:00
Simon Pilgrim 110837fbf6 Fix signed/unsigned comparison warning. NFCI. 2021-01-26 15:04:49 +00:00
Aleksandr Platonov 7388c34685 [clangd][SwapIndex] ensure that the old index is alive while we are using it via the function returned by `SwapIndex::indexedFiles()` call
Without this patch the old index could be freed, but there still could be tries to access it via the function returned by `SwapIndex::indexedFiles()` call.
This leads to hard to reproduce clangd crashes at code completion.
This patch keeps the old index alive until the function returned by `SwapIndex::indexedFiles()` call is alive.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D95206
2021-01-22 16:26:39 +03:00
Haojian Wu facea4a2d4 [clangd] Fix a missing override keyword, NFC. 2021-01-21 11:06:43 +01:00
Hans Wennborg 8ba442bc21 Revert "Following up on PR48517, fix handling of template arguments that refer"
Combined with 'da98651 - Revert "DR2064:
decltype(E) is only a dependent', this change (5a391d3) caused verifier
errors when building Chromium. See https://crbug.com/1168494#c1 for a
reproducer.

Additionally it reverts changes that were dependent on this one, see
below.

> Following up on PR48517, fix handling of template arguments that refer
> to dependent declarations.
>
> Treat an id-expression that names a local variable in a templated
> function as being instantiation-dependent.
>
> This addresses a language defect whereby a reference to a dependent
> declaration can be formed without any construct being value-dependent.
> Fixing that through value-dependence turns out to be problematic, so
> instead this patch takes the approach (proposed on the core reflector)
> of allowing the use of pointers or references to (but not values of)
> dependent declarations inside value-dependent expressions, and instead
> treating template arguments as dependent if they evaluate to a constant
> involving such dependent declarations.
>
> This ends up affecting a bunch of OpenMP tests, due to OpenMP
> imprecisely handling instantiation-dependent constructs, bailing out
> early instead of processing dependent constructs to the extent possible
> when handling the template.
>
> Previously committed as 8c1f2d15b8, and
> reverted because a dependency commit was reverted.

This reverts commit 5a391d38ac.

It also restores clang/test/SemaCXX/coroutines.cpp to its state before
da986511fb.

Revert "[c++20] P1907R1: Support for generalized non-type template arguments of scalar type."

> Previously committed as 9e08e51a20, and
> reverted because a dependency commit was reverted. This incorporates the
> following follow-on commits that were also reverted:
>
> 7e84aa1b81 by Simon Pilgrim
> ed13d8c667 by me
> 95c7b6cadb by Sam McCall
> 430d5d8429 by Dave Zarzycki

This reverts commit 4b574008ae.

Revert "[msabi] Mangle a template argument referring to array-to-pointer decay"

> [msabi] Mangle a template argument referring to array-to-pointer decay
> applied to an array the same as the array itself.
>
> This follows MS ABI, and corrects a regression from the implementation
> of generalized non-type template parameters, where we "forgot" how to
> mangle this case.

This reverts commit 18e093faf7.
2021-01-20 15:55:35 +01:00
Utkarsh Saxena 8bf7116d50 [clangd] Index local classes, virtual and overriding methods.
Previously we did not record local class declarations. Now with features like
findImplementation and typeHierarchy, we have a need to index such local
classes to accurately report subclasses and implementations of methods.

Performance testing results:
- No changes in indexing timing.
- No significant change in memory usage.
- **1%** increase in #relations.
- **0.17%** increase in #refs.
- **0.22%** increase #symbols.

**New index stats**
Time to index: **4:13 min**
memory usage **543MB**
number of symbols: **521.5K**
number of refs: **8679K**
number of relations: **49K**

**Base Index stats**
Time to index: **4:15 min**
memory usage **542MB**
number of symbols: **520K**
number of refs: **8664K**
number of relations: **48.5K**

Fixes: https://github.com/clangd/clangd/issues/644

Differential Revision: https://reviews.llvm.org/D94785
2021-01-19 16:18:48 +01:00
Richard Smith 4b574008ae [c++20] P1907R1: Support for generalized non-type template arguments of scalar type.
Previously committed as 9e08e51a20, and
reverted because a dependency commit was reverted. This incorporates the
following follow-on commits that were also reverted:

7e84aa1b81 by Simon Pilgrim
ed13d8c667 by me
95c7b6cadb by Sam McCall
430d5d8429 by Dave Zarzycki
2021-01-18 21:05:01 -08:00
Aleksandr Platonov 2e25be0b61 [clangd] Add main file macros into the main-file index.
This patch is a try to fix `WorkspaceSymbols.Macros` test after D93796.
If a macro definition is in the preamble section, then it appears to be in the preamble (static) index and not in the main-file (dynamic) index.
Thus, a such macro could not be found at a symbol search according to the logic that we skip symbols from the static index if the location of these symbols is inside the dynamic index files.
To fix this behavior this patch adds main file macros into the main-file (dynamic) index.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D94477
2021-01-14 15:10:17 +03:00
Sam McCall 66d5994bd3 [clangd] Explicitly avoid background-indexing the same file twice.
This used to implicitly never happen due to only discovering each CDB
once.

We may want to carefully support reindexing one day, but we need to do
it carefully (tricky tradeoffs) and it would need further support in
background indexer.

Making this explicit here rather than just turning off rebroadcast in
background index for a few reasons:
- allows *new* files in the same CDB to be indexed
- relying on bugs-at-a-distance cancelling each other out is bound to bite us
- gets us closer to actually supporting reindexing, which requires similar tracking

Differential Revision: https://reviews.llvm.org/D94503
2021-01-13 17:29:30 +01:00
Haojian Wu b83b7d0873 [clangd] NFC, avoid potential ODR violation. 2021-01-08 13:29:11 +01:00
Aleksandr Platonov 979228f120 [clangd][fuzzyFind] Do not show stale symbols in the result.
This is follow up to D93393.
Without this patch `MergedIndex::fuzzyFind()` returns stale symbols from the static index even if these symbols were removed.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D93796
2021-01-06 11:17:12 +03:00
Aleksandr Platonov 2522fa053b [clangd] Do not take stale definition from the static index.
This is follow up to D93393.
Without this patch clangd takes the symbol definition from the static index if this definition was removed from the dynamic index.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D93683
2020-12-23 18:21:38 +03:00
Arthur Eubanks b2e734d5f4 Revert "[clangd] zap a few warnings"
This reverts commit 95c7b6cadb.

Depends on a reverted change.
2020-12-22 10:34:03 -08:00
Sam McCall 95c7b6cadb [clangd] zap a few warnings 2020-12-18 16:34:34 +01:00
Aleksandr Platonov e35f9229dc [clangd] Ignore the static index refs from the dynamic index files.
This patch fixes the following problem:
- open a file with references to the symbol `Foo`
- remove all references to `Foo` (from the dynamic index).
- `MergedIndex::refs()` result will contain positions of removed references (from the static index).

The idea of this patch is to keep a set of files which were used during index build inside the index.
Thus at processing the static index references we can check if the file of processing reference is a part of the dynamic index or not.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D93393
2020-12-18 15:36:30 +03:00
Kirill Bobyrev 0545680cb8 Reland dcdef5b5b3
Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D92198
2020-12-15 20:54:55 +01:00
Kirill Bobyrev 32e7a9cab6 Revert "[clangd] Log remote index connectivity status"
This reverts commit dcdef5b5b3.

This commit breaks clangd-ubuntu-tsan builds:
http://lab.llvm.org:8011/#builders/131/builds/1392
2020-12-15 18:42:42 +01:00
Kirill Bobyrev dcdef5b5b3 [clangd] Log remote index connectivity status
Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D92198
2020-12-15 18:29:48 +01:00
Haojian Wu 38d32e4fd7 [clangd] Reorder the class field to avoid -Wreorder-ctor warning, NFC. 2020-12-11 10:46:54 +01:00
Utkarsh Saxena 4ce242a163 [clangd] Find relations in Dex exploration tool.
Reviewed By: hokein

Differential Revision: https://reviews.llvm.org/D93029
2020-12-10 16:54:03 +01:00
Kirill Bobyrev ee02e20c08
[clangd] NFC: Use SmallVector<T> where possible
SmallVector<T> with default size is now the recommended version (D92522).

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D92788
2020-12-10 13:36:49 +01:00
Kirill Bobyrev 5a1bc69f81 [clangd] NFC: Add client-side logging for remote index requests
Figuring out whether the server is responding and debugging issues with remote
index setup is no easy task: add verbose logging for client side RPC requests
to relieve some pain.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D92181
2020-12-09 21:40:37 +01:00
Ilya Golovenko 2d539d7854 [clangd] Relation slabs should not be accounted when computing backing storage size
Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D92484
2020-12-03 16:56:53 +03:00
Kirill Bobyrev 4169c520f6
[clangd] Add symbol origin for remote index
Makes it easier to diagnose remote index issues with --debug-origins flag.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D92202
2020-11-28 15:38:11 +01:00
Kadir Cetinkaya f726101b62
[clangd] Fix shared-lib builds
Differential Revision: https://reviews.llvm.org/D91859
2020-11-24 13:05:20 +01:00
Kadir Cetinkaya cab3136807
[clangd] Use ProjectAwareIndex in ClangdMain
Put project-aware-index between command-line specified static index and
ClangdServer indexes.

This also moves remote-index dependency from clangDaemon to ClangdMain
in an attempt to prevent cyclic dependency between clangDaemon and
remote-index-marshalling.

Differential Revision: https://reviews.llvm.org/D91860
2020-11-22 20:59:38 +01:00
Kadir Cetinkaya 067ffbfe60
[clangd] Introduce ProjectAwareIndex
An index implementation that can dispatch to a variety of indexes
depending on the file path. Enables clangd to work with multiple indexes in the
same instance, configured via config files.

Depends on D90749, D90746

Differential Revision: https://reviews.llvm.org/D90750
2020-11-22 20:59:37 +01:00