Commit Graph

334 Commits

Author SHA1 Message Date
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Sam McCall 6ef1ccecf7 [clangd] Rename DiagnosticsConsumer -> ClangdServer::Callbacks, and make it optional
Summary:
This reflects its current function better and avoids confusion with clang::DiagnosticConsumer.

The old name/constructor is left around temporarily for compatibility.
(Metagame: merging with out-of-tree changes is harder than usual this month)

Reviewers: hokein

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D73346
2020-01-24 15:59:32 +01:00
Sam McCall 7d20e80225 [clangd] Show background index status using LSP 3.15 work-done progress notifications
Summary:
It simply shows the completed/total items on the background queue, e.g.
 indexing: 233/1000
The denominator is reset to zero every time the queue goes idle.

The protocol is fairly complicated here (requires creating a remote "progress"
resource before sending updates). We implement the full protocol, but I've added
an extension allowing it to be skipped to reduce the burden on clients - in
particular the lit test takes this shortcut.

The addition of background index progress to DiagnosticConsumer seems ridiculous
at first glance, but I believe that interface is trending in the direction of
"ClangdServer callbacks" anyway. It's due for a rename, but otherwise actually
fits.

Reviewers: kadircet, usaxena95

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits, llvm-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D73218
2020-01-24 12:21:08 +01:00
Sam McCall 8d7ecc1629 Revert "Revert "[clangd] Implement "textDocument/documentLink" protocol support""
This reverts commit 079ef783dd.

The revert describes a test failure without details, after offline
discussion this in in a private/unsupported build system and doesn't
seem to reflect a real upstream bug.
2020-01-02 16:36:21 +01:00
Sam McCall 6af1ad20d6 Revert "[clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit a0ff8cd631.
Buildbot failures I can't chase further tonight.
2019-12-16 19:07:49 +01:00
Sam McCall a0ff8cd631 [clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails.
This reverts commit 8f876d5105.
2019-12-16 15:58:51 +01:00
Sam McCall 8f876d5105 Revert "[clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit 2500a8d5d8.
2019-12-16 15:57:43 +01:00
Sam McCall 2500a8d5d8 [clangd] Reapply b60896fad9 Fall back to selecting token-before-cursor if token-after-cursor fails.
This reverts commit f0604e73a4
The issue with movability of Tweak::Selection was addressed in 7dc388bd95
2019-12-16 15:55:16 +01:00
Dmitri Gribenko 079ef783dd Revert "[clangd] Implement "textDocument/documentLink" protocol support"
This reverts commit d6417f5584. The tests
depend on builtin headers, which is not intentionally supported in
clangd tests; these tests are broken in some build environments.
2019-12-16 15:21:51 +01:00
Kirill Bobyrev 3b9715cb21 [NFC] Fix typos in Clangd and Clang
Reviewed by: Jim

Differential Revision: https://reviews.llvm.org/D71455
2019-12-16 10:54:40 +01:00
Nico Weber f0604e73a4 Revert "[clangd] Fall back to selecting token-before-cursor if token-after-cursor fails."
This reverts commit b60896fad9.
Breaks building with gcc:

/usr/include/c++/7/bits/stl_construct.h:75:7: error: use of deleted function ‘clang::clangd::Tweak::Selection::Selection(const clang::clangd::Tweak::Selection&)’
     { ::new(static_cast<void*>(__p)) _T1(std::forward<_Args>(__args)...); }
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.h:28:0,
                 from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.cpp:9:
/home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/refactor/Tweak.h:49:10: note: ‘clang::clangd::Tweak::Selection::Selection(const clang::clangd::Tweak::Selection&)’ is implicitly deleted because the default definition would be ill-formed:
   struct Selection {
          ^~~~~~~~~
/home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/refactor/Tweak.h:49:10: error: use of deleted function ‘clang::clangd::SelectionTree::SelectionTree(const clang::clangd::SelectionTree&)’
In file included from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/refactor/Tweak.h:25:0,
                 from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.h:28,
                 from /home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/ClangdServer.cpp:9:
/home/buildslave/buildslave/clang-cmake-armv7-selfhost-neon/llvm/clang-tools-extra/clangd/Selection.h:96:3: note: declared here
   SelectionTree(const SelectionTree &) = delete;
   ^~~~~~~~~~~~~

e.g. here:
http://lab.llvm.org:8011/builders/clang-cmake-armv7-selfhost-neon/builds/2714
http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/41866
2019-12-14 08:13:29 -05:00
Sam McCall b60896fad9 [clangd] Fall back to selecting token-before-cursor if token-after-cursor fails.
Summary:
The problem:

LSP specifies that Positions are between characters. Therefore when a position
(or an empty range) is used to target elements of the source code, there is an
ambiguity - should we look left or right of the cursor?

Until now, SelectionTree resolved this to the right except in trivial cases
(where there's whitespace, semicolon, or eof on the right).
This meant that it's unable to e.g. out-line `int foo^()` today.

Complicating this, LSP notwithstanding the cursor is *on* a character in many
editors (mostly terminal-based). In these cases there's no ambiguity - we must
"look right" - but there's also no way to tell in LSP.

(Several features currently resolve this by using getBeginningOfIdentifier,
which tries to rewind and supports end-of-identifier. But this relies on
raw lexing and is limited and buggy).

Precedent: well - most other languages aren't so full of densely packed symbols
that we might want to target. Bias-towards-identifier works well enough.
MS C++ for vscode seems to mostly use bias-toward-identifier too.
The problem with this solution is it doesn't provide any way to target some
things such as the constructor call in Foo^(bar());

Presented solution:

When an ambiguous selection is found, we generate *both* possible selection
trees. We try to run the feature on the rightward tree first, and then on the
leftward tree if it fails.

This is basically do-what-I-mean, the main downside is the need to do this on
a feature-by-feature basis (because each feature knows what "fail" means).
The most complicated instance of this is Tweaks, where the preferred selection
may vary tweak-by-tweak.

Wrinkles:

While production behavior is pretty consistent, this introduces some
inconsistency in testing, depending whether the interface we're testing is
inside or outside the "retry" wrapper.

In particular, for many features like Hover, the unit tests will show production
behavior, while for Tweaks the harness would have to run the loop itself if
we want this.

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71345
2019-12-13 16:57:03 +01:00
Michael Forster d6417f5584 [clangd] Implement "textDocument/documentLink" protocol support
Summary:
This adds an implementation for the "textDocument/documentLink" LSP request.

It returns links for all `#include` directives to the resolved target files.

Fixes https://github.com/clangd/clangd/issues/217.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70872
2019-12-12 14:55:20 +01:00
Alex Lorenz c0ee0224c4 [clangd] NFC, add getLangOpts helper to ParsedAST
The addition of the helper is split out from https://reviews.llvm.org/D69543
as suggested by Kadir. I also updated the existing uses to use the new API.
2019-12-04 16:55:25 -08:00
Haojian Wu 852bafae2b [clangd] Implement cross-file rename.
Summary:
This is the initial version. The cross-file rename is purely based on
the index.

It is hidden under a command-line flag, and only available for a small set
of symbols.

Reviewers: ilya-biryukov, sammccall

Subscribers: merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69263
2019-11-26 10:04:31 +01:00
Haojian Wu 5181adab61 [clangd] Expose the xref's incomplete flag to clangdServer API.
Summary: so that clangd C++ API users (via ClangdServer) can access it.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70380
2019-11-18 13:27:19 +01:00
Kadir Cetinkaya f4d7caf6df
[clangd] Reland Store Index in Tweak::Selection
Summary:
Incoming define out-of-line tweak requires access to index.

This patch simply propogates the index in ClangdServer to Tweak::Selection while
passing the AST. Also updates TweakTest to accommodate this change.

Reviewers: ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69165
2019-10-28 07:55:58 +01:00
Kadir Cetinkaya 8e567b0730
[clangd] Revert define-inline action changes to un-break windows build-bots 2019-10-25 18:40:01 +02:00
Kadir Cetinkaya 8aa84ad37d
[clangd] Store Index in Tweak::Selection
Summary:
Incoming define out-of-line tweak requires access to index.

This patch simply propogates the index in ClangdServer to Tweak::Selection while
passing the AST. Also updates TweakTest to accommodate this change.

Reviewers: ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69165
2019-10-25 12:15:20 +02:00
Haojian Wu 77c97002dc [clangd] Catch an unchecked "Expected<T>" in HeaderSourceSwitch.
Summary: Also fixes a potential user-after-scope issue of "Path".

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68564

llvm-svn: 373897
2019-10-07 11:37:25 +00:00
Haojian Wu d6d5eddcdb [clangd] Use the index-based API to do the header-source switch.
Summary:
If the file heuristic fails, we try to use the index&AST to do the
header/source inference.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68211

llvm-svn: 373320
2019-10-01 10:21:15 +00:00
Haojian Wu e259a5fd0d [clangd] Move the existing heder-source-switch implemenation out of clangdServer.
Summary: This is a NFC change.

Reviewers: kadircet

Reviewed By: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67960

llvm-svn: 372841
2019-09-25 10:30:22 +00:00
Utkarsh Saxena 79d19bdf81 Add SemanticRanges to Clangd server.
Summary:
Adds Semantic Ranges capabilities to Clangd server.
Also adds tests for running it via clangd server.

This differs from the LSP spec as the spec needs this to be evaluated on multiple 'pos' and the expected output is an list of list of semantic ranges.
This is majorly for multi cursor and assuming this is a rare thing, we don't want to optimize make things complicated just for this.
This should be done in the LSP level by queueing one request per 'pos' in the input.

LSP Spec:
https://github.com/microsoft/language-server-protocol/blob/dbaeumer/3.15/specification.md#textDocument_selectionRange

Reviewers: hokein

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67650

llvm-svn: 372102
2019-09-17 10:28:05 +00:00
Kadir Cetinkaya 5b270932cc [clangd] Support multifile edits as output of Tweaks
Summary:
First patch for propogating multifile changes from tweak outputs to LSP
WorkspaceEdits.

Uses SM to convert tooling::Replacements to TextEdits.
Errors out if there are any inconsistencies between the draft version and the
version generated the edits.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66637

llvm-svn: 371392
2019-09-09 12:28:44 +00:00
Sam McCall 915f9785e0 [clangd] Rename ClangdUnit.h -> ParsedAST.h. NFC
This much better reflects what is (now) in this header.
Maybe a rename to ParsedTU would be an improvement, but that's a much
more invasive change and life is too short.

ClangdUnit is dead, long live ClangdUnitTests!

llvm-svn: 370862
2019-09-04 09:46:06 +00:00
Sam McCall cf3a585fff [clangd] Split Preamble.h out of ClangdUnit.h. NFC
Summary:
Add comment describing use of preamble in clangd.
Remove deps on ClangdUnit.h where possible.

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67117

llvm-svn: 370843
2019-09-04 07:35:00 +00:00
Sam McCall 19cefc2878 [clangd] Move getBeginningOfIdentifier from ClangdUnit to SourceCode. Drop dependencies on ClangdUnit from some headers. NFC
llvm-svn: 370768
2019-09-03 15:34:47 +00:00
Ilya Biryukov d73ac96d80 [clangd] Surface errors from command-line parsing
Summary:
Those errors are exposed at the first character of a file,
for a lack of a better place.

Previously, all errors were stored inside the AST and report
accordingly. However, errors in command-line argument parsing could
result in failure to produce the AST, so we need an alternative ways to
report those errors.

We take the following approach in this patch:
  - buildCompilerInvocation() now requires an explicit DiagnosticConsumer.
  - TUScheduler and TestTU now collect the diagnostics produced when
    parsing command line arguments.
    If pasing of the AST failed, diagnostics are reported via a new
    ParsingCallbacks::onFailedAST method.
    If parsing of the AST succeeded, any errors produced during
    command-line parsing are stored alongside the AST inside the
    ParsedAST instance and reported as previously by calling the
    ParsingCallbacks::onMainAST method;
  - The client code that uses ClangdServer's DiagnosticConsumer
    does not need to change, it will receive new diagnostics in the
    onDiagnosticsReady() callback

Errors produced when parsing command-line arguments are collected using
the same StoreDiags class that is used to collect all other errors. They
are recognized by their location being invalid. IIUC, the location is
invalid as there is no source manager at this point, it is created at a
later stage.

Although technically we might also get diagnostics that mention the
command-line arguments FileID with after the source manager was created
(and they have valid source locations), we choose to not handle those
and they are dropped as not coming from the main file. AFAICT, those
diagnostics should always be notes, therefore it's safe to drop them
without loosing too much information.

Reviewers: kadircet

Reviewed By: kadircet

Subscribers: nridge, javed.absar, MaskRay, jkorous, arphaman, cfe-commits, gribozavr

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66759

llvm-svn: 370177
2019-08-28 09:24:55 +00:00
Haojian Wu 0a6000f2cb [clangd] Send highlighting diff beyond the end of the file.
Summary: This would make the client life (tracking the changes) easier.

Reviewers: jvikstrom

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66541

llvm-svn: 369884
2019-08-26 08:38:45 +00:00
Benjamin Kramer 15ee5ba6e7 [clangd] Fold string copy into lambda capture. NFC.
llvm-svn: 369679
2019-08-22 18:09:46 +00:00
Benjamin Kramer 9880b5dd21 [clangd] Don't use Bind() where C++14 move capture works
llvm-svn: 369005
2019-08-15 14:16:06 +00:00
Jonas Devlieghere 1c705d9c53 [clang-tools-extra] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368944
2019-08-14 23:52:23 +00:00
Sam McCall 4bd6ebb495 clangd: use -j for background index pool
Summary:
clangd supports a -j option to limit the amount of threads to use for parsing
TUs. However, when using -background-index (the default in later versions of
clangd), the parallelism used by clangd defaults to the hardware_parallelisn,
i.e. number of physical cores.

On shared hardware environments, with large projects, this can significantly
affect performance with no way to tune it down.

This change makes the -j parameter apply equally to parsing and background
index. It's not perfect, because the total number of threads is 2x the -j value,
which may still be unexpected. But at least this change allows users to prevent
clangd using all CPU cores.

Reviewers: kadircet, sammccall

Reviewed By: sammccall

Subscribers: javed.absar, jfb, sammccall, ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66031

llvm-svn: 368498
2019-08-09 23:03:32 +00:00
Johan Vikstrom c2653ef27c [clangd] Duplicate lines of semantic highlightings sent removed.
Summary: Added a class for diffing highlightings and removing duplicate lines. Integrated into the highlighting generation flow. Only works correctly if all tokens are on a single line. Also returns empty lines if the IDE should remove previous highlightings on a line.

Reviewers: hokein, sammccall, ilya-biryukov

Subscribers: MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64475

llvm-svn: 367521
2019-08-01 08:08:44 +00:00
Haojian Wu f429ab60e6 [clangd] Implement "prepareRename"
Summary:
- "prepareRename" request is added in LSP v3.12.0
- also update the vscode-client dependency to pick-up the rename bug fix[1]

[1]: https://github.com/microsoft/vscode-languageserver-node/issues/447

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63126

llvm-svn: 366873
2019-07-24 07:49:23 +00:00
Ilya Biryukov 8bb8915d43 [clangd] Provide a way to publish highlightings in non-racy manner
Summary:
By exposing a callback that can guard code publishing results of
'onMainAST' callback in the same manner we guard diagnostics.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, MaskRay, jkorous, arphaman, kadircet, hokein, jvikstrom, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64985

llvm-svn: 366577
2019-07-19 13:51:01 +00:00
Kadir Cetinkaya 91e5f4b46b Revert "Revert r366458, r366467 and r366468"
This reverts commit 9c377105da.

[clangd][BackgroundIndexLoader] Directly store DependentTU while loading shard

Summary:
We were deferring the population of DependentTU field in LoadedShard
until BackgroundIndexLoader was consumed. This actually triggers a use after
free since the shards FileToTU was pointing at could've been moved while
consuming the Loader.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64980

llvm-svn: 366559
2019-07-19 10:18:52 +00:00
Azharuddin Mohammed 9c377105da Revert r366458, r366467 and r366468
r366458 is causing test failures. r366467 and r366468 had to be reverted as
they were casuing conflict while reverting r366458.

r366468 [clangd] Remove dead code from BackgroundIndex
r366467 [clangd] BackgroundIndex stores shards to the closest project
r366458 [clangd] Refactor background-index shard loading

llvm-svn: 366551
2019-07-19 09:26:33 +00:00
Kadir Cetinkaya 006d1915e2 [clangd] BackgroundIndex stores shards to the closest project
Summary:
Changes persistance logic to store shards at the directory of closest
CDB. Previously we were storing all shards to directory of the CDB that
triggered indexing, it had its downsides.

For example, if you had two TUs coming from a different CDB but depending on the
same header foo.h, we will store the foo.h only for the first CDB, and it would
be missing for the second and we would never persist it since it was actually
present in the memory and persisted before.

This patch still stores only a single copy of a shard, but makes the directory a
function of the file name. So that the shard place will be unique even with
multiple CDBs accessing the file. This directory is determined as the first
directory containing a CDB in the file's parent directories, if no such
directory exists we make use of the home directory.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64745

llvm-svn: 366467
2019-07-18 17:20:41 +00:00
Nathan Ridge 087b044c49 [clangd] Implement typeHierarchy/resolve for subtypes
Summary:
This allows the client to resolve subtypes one level at a time.

For supertypes, this is not necessary, because we eagerly compute
supertypes and return all levels.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64308

llvm-svn: 365986
2019-07-13 03:24:48 +00:00
Russell Gallop de54e2c4df Revert "[clangd] Implement typeHierarchy/resolve for subtypes"
Causing test failure on Windows bot

This reverts commit 5b9484e559.

llvm-svn: 365899
2019-07-12 13:35:58 +00:00
Sam McCall 0f7146db9b [clangd] Prioritize indexing of files that share a basename with the open file.
Summary:
In practice, opening Foo.h will still often result in Foo.cpp making the
second index build instead of the first, as the rebuild policy doesn't
know to wait.

Reviewers: kadircet

Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64575

llvm-svn: 365888
2019-07-12 10:18:42 +00:00
Haojian Wu 7eeb82b585 [clangd] Don't run the prepare for tweaks that are disabled.
Summary: Previously, we ran the prepare, even for the tweaks that are disabled.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64565

llvm-svn: 365882
2019-07-12 08:50:20 +00:00
Nathan Ridge 5b9484e559 [clangd] Implement typeHierarchy/resolve for subtypes
Summary:
This allows the client to resolve subtypes one level at a time.

For supertypes, this is not necessary, because we eagerly compute
supertypes and return all levels.

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64308

llvm-svn: 365867
2019-07-12 03:26:32 +00:00
Haojian Wu 6b043f0518 [clangd] Add a flag to clangdServer rename function to control whether we want format the replacements.
Summary:
This would allow clangd embedders to use the ClangdServer::rename for other
purposes (highlighting all the occurrences of the symbol in prepare
stage).

Reviewers: sammccall, ilya-biryukov

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64481

llvm-svn: 365631
2019-07-10 13:44:22 +00:00
Sam McCall 2f760c44e6 [clangd] Rewrite of logic to rebuild the background index serving structures.
Summary:
Previously it was rebuilding every 5s by default, which was much too frequent
in the long run - the goal was to provide an early build. There were also some
bugs. There were also some bugs, and a dedicated thread was used in production
but not tested.

 - rebuilds are triggered by #TUs built, rather than time. This should scale
   more sensibly to fast vs slow machines.
 - there are two separate indexed-TU thresholds to trigger index build: 5 TUs
   for the first build, 100 for subsequent rebuilds.
 - rebuild is always done on the regular indexing threads, and is affected by
   blockUntilIdle. This means unit/lit tests run the production configuration.
 - fixed a bug where we'd rebuild after attempting to load shards, even if there
   were no shards.
 - the BackgroundIndexTests don't really test the subtleties of the rebuild
   policy (for determinism, we call blockUntilIdle, so rebuild-on-idle is enough
   to pass the tests). Instead, we expose the rebuilder as a separate class and
   have fine-grained tests for it.

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64291

llvm-svn: 365531
2019-07-09 18:30:49 +00:00
Sam McCall 5a458d6ada [clangd] Show documentation in hover, and fetch docs from index if needed.
Summary:
I assume showing docs is going to be part of structured hover rendering, but
it's unclear whether that's going to make clangd 9 so this is low-hanging fruit.

(Also fixes a bug uncovered in FormattedString's plain text output: need blank
lines when text follows codeblocks)

Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64296

llvm-svn: 365522
2019-07-09 17:59:50 +00:00
Shaurya Gupta 8cae7d79b5 Summary: [Clangd] Added hidden command line option -tweaks to specify which tweaks to enable
- Only for development purposes
- Disabled tweaks in fixits-duplications test
Reviewers: sammccall, kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63989

llvm-svn: 364809
2019-07-01 16:55:29 +00:00
Johan Vikstrom 6b8a2515ba [clangd] Emit semantic highlighting tokens when the main AST is built.
Differential Revision: https://reviews.llvm.org/D63821

llvm-svn: 364551
2019-06-27 15:13:03 +00:00
Haojian Wu 7276a446ce [clangd] Narrow rename to local symbols.
Summary:
Previously, we performed rename for all kinds of symbols (local, global).

This patch narrows the scope by only renaming symbols not being used
outside of the main file (with index asisitance). Renaming global
symbols is not supported at the moment (return an error).

Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63426

llvm-svn: 364283
2019-06-25 08:43:17 +00:00
Ilya Biryukov 482269b9fa [clangd] Consume error returned by cleanupAndFormat
When called by ClangdServer::applyTweak.
No idea how to actually trigger this in practice, so no tests.

llvm-svn: 363844
2019-06-19 17:30:02 +00:00
Ilya Biryukov a7acc7e855 [clangd] Format changes produced by rename
Reviewers: hokein, kadircet, sammccall

Reviewed By: kadircet

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63562

llvm-svn: 363843
2019-06-19 17:25:24 +00:00
Sam McCall 08372eb73b Revert "[clangd] Return vector<TextEdit> from applyTweak. NFC"
This reverts commit r363691.

llvm-svn: 363766
2019-06-19 07:29:10 +00:00
Sam McCall 38047dbca4 [clangd] Add ClangdServer accessor for buffer contents
llvm-svn: 363765
2019-06-19 07:29:05 +00:00
Ilya Biryukov df9ee08b64 [clangd] Return vector<TextEdit> from applyTweak. NFC
For the same reasons as r363150, which got overwritten by changes in
r363680.

Sending without review to unbreak our integrate.

llvm-svn: 363691
2019-06-18 15:15:41 +00:00
Sam McCall 395fde753c [clangd] Add hidden tweaks to dump AST/selection.
Summary:
This introduces a few new concepts:
 - tweaks have an Intent (they don't all advertise as refactorings)
 - tweaks may produce messages (for ShowMessage notification). Generalized
   Replacements -> Effect.
 - tweaks (and other features) may be hidden (clangd -hidden-features flag).
   We may choose to promote these one day. I'm not sure they're worth their own
   feature flags though.

Verified it in vim-clangd (not yet open source), curious if the UI is ok in VSCode.

Reviewers: ilya-biryukov

Subscribers: mgorny, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62538

llvm-svn: 363680
2019-06-18 13:37:54 +00:00
Nathan Ridge a552508841 [clangd] Type hierarchy subtypes
Summary:
This builds on the relations support added in D59407, D62459, D62471,
and D62839 to implement type hierarchy subtypes.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, mgrang, arphaman,
jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58880

llvm-svn: 363506
2019-06-16 02:31:37 +00:00
Ilya Biryukov 04112ecd41 [clangd] Return TextEdits from ClangdServer::applyTweak
Summary:
Instead of `tooling::Replacements`. So that embedders do not need to store
the contents of the file.

This also aligns better with `ClangdServer::rename`.

Reviewers: kadircet, hokein

Reviewed By: hokein

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63140

llvm-svn: 363150
2019-06-12 12:03:24 +00:00
Sam McCall 25c6257ba0 [clangd] Revamp textDocument/onTypeFormatting.
Summary:
The existing implementation (which triggers on }) is fairly simple and
has flaws:
 - doesn't trigger frequently/regularly enough (particularly in editors that type the }
 for you)
 - often reformats too much code around the edit
 - has jarring cases that I don't have clear ideas for fixing

This implementation is designed to trigger on newline, which feels to me more
intuitive than } or ;.
It does have allow for reformatting after other characters - it has a
basic behavior and a model for adding specialized behavior for
particular characters. But at least initially I'd stick to advertising
\n in the capabilities.

This also handles comment splitting: when you insert a line break inside
a line comment, it will make the new line into an aligned line comment.

Working on tests, but want people to patch it in and try it - it's hard to
see if "feel" is right purely by looking at a test.

Reviewers: ilya-biryukov, hokein

Subscribers: mgorny, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60605

llvm-svn: 362939
2019-06-10 14:26:21 +00:00
Ilya Biryukov f9169d0896 [clangd] Represent Hover result using FormattedString
Reviewers: sammccall, kadircet

Reviewed By: kadircet

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61601

llvm-svn: 361940
2019-05-29 10:01:00 +00:00
Kadir Cetinkaya c6578eefdd [clangd] Introduce a structured hover response
Summary:
Change ClangdServer layer to output a structured response for Hover,
which can be rendered by client according to their needs.

Reviewers: sammccall, ilya-biryukov

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61497

llvm-svn: 361803
2019-05-28 10:29:58 +00:00
Sam McCall 0321b370f2 [clangd] Turn no-parse-completion on by when preamble isn't ready. Add flag to force it.
Reviewers: kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62135

llvm-svn: 361258
2019-05-21 13:40:31 +00:00
Ilya Biryukov 7e6b6ea877 [clangd] Make it possible to use VFS from parsing for getting tidy options
Summary:
To give an option for clangd embedders with snapshotted filesystem to
read config files from exact snapshots, possibly loosing some
performance from caching capabilities of the current implementations.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62143

llvm-svn: 361178
2019-05-20 17:30:46 +00:00
Sam McCall c094912ae5 [clangd] Move Rename into its own file, and add unit test. NFC
Reviewers: kadircet

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, arphaman, jfb, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61596

llvm-svn: 360116
2019-05-07 07:11:56 +00:00
Sam McCall 949d69f5cf [clangd] Always call getFormatStyleForFile().
This means "format" will no longer return an error if the -fallback-style flag
is invalid, it will log and use LLVM style. This doesn't really matter.

Also document the dependence on global variables. (This patch is a
compromise - it's probably not worth actually avoiding the globals).

llvm-svn: 360020
2019-05-06 08:39:17 +00:00
Haojian Wu 0842daf5ba [clangd] Emit better error messages when rename fails.
Summary:
Currently we emit an unfriendly "clang diagnostic" message when rename fails. This
patch makes clangd to emit a detailed diagnostic message.

Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60821

llvm-svn: 358658
2019-04-18 11:35:22 +00:00
Eric Liu 9ef03dd20a [clangd] Wait for compile command in ASTWorker instead of ClangdServer
Summary:
This makes addDocument non-blocking and would also allow code completion
(in fallback mode) to run when worker waits for the compile command.

Reviewers: sammccall, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: javed.absar, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60607

llvm-svn: 358400
2019-04-15 12:32:28 +00:00
Eric Liu 00d99bd1c4 [clangd] Use identifiers in file as completion candidates when build is not ready.
Summary:
o Lex the code to get the identifiers and put them into a "symbol" index.
o Adds a new completion mode without compilation/sema into code completion workflow.
o Make IncludeInserter work even when no compile command is present, by avoiding
inserting non-verbatim headers.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60126

llvm-svn: 358159
2019-04-11 09:36:36 +00:00
Eric Liu dd02825937 [clangd] Add fallback mode for code completion when compile command or preamble is not ready.
Summary:
When calling TUScehduler::runWithPreamble (e.g. in code compleiton), allow
entering a fallback mode when compile command or preamble is not ready, instead of
waiting. This allows clangd to perform naive code completion e.g. using identifiers
in the current file or symbols in the index.

This patch simply returns empty result for code completion in fallback mode. Identifier-based
plus more advanced index-based completion will be added in followup patches.

Reviewers: ilya-biryukov, sammccall

Reviewed By: sammccall

Subscribers: sammccall, javed.absar, MaskRay, jkorous, arphaman, kadircet, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59811

llvm-svn: 357916
2019-04-08 14:53:16 +00:00
Sam McCall 4e56502be2 [clangd] Stop passing around PCHContainerOperations, just create it in place. NFC
llvm-svn: 357689
2019-04-04 12:56:03 +00:00
Ilya Biryukov d9c24dca73 [clangd] Return clangd::TextEdit in ClangdServer::rename. NFC
Summary:
Instead of tooling::Replacement. To avoid the need to have contents of
the file at the caller site. This also aligns better with other methods
in ClangdServer, majority of those already return LSP-specific data
types.

Reviewers: hokein, ioeric, sammccall

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60179

llvm-svn: 357561
2019-04-03 07:18:43 +00:00
Kadir Cetinkaya 8665802202 [clangd] Add support for type hierarchy (super types only for now)
Summary:
Patch by Nathan Ridge(@nridge)!

This is an LSP extension proposed here:
https://github.com/Microsoft/vscode-languageserver-node/pull/426

An example client implementation can be found here:
https://github.com/theia-ide/theia/pull/3802

Reviewers: kadircet, sammccall

Reviewed By: kadircet

Subscribers: jdoerfert, sammccall, cfe-commits, mgorny, dschaefer, simark, ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet

Tags: #clang

Differential Revision: https://reviews.llvm.org/D56370

llvm-svn: 356445
2019-03-19 09:27:04 +00:00
Francis Visoiu Mistrih 1608c12027 [NFC][clangd] Remove unused lambda capture
Avoid this warning:

llvm/clang-tools-extra/clangd/ClangdServer.cpp:365:23: warning: lambda
capture 'this' is not used [-Wunused-lambda-capture]

  auto Action = [Sel, this](decltype(CB) CB, std::string File,
                    ~~^~~~
1 warning generated.

llvm-svn: 353760
2019-02-11 22:36:47 +00:00
Haojian Wu e64ee7c645 Revamp the "[clangd] Format tweak's replacements"
Summary:
This patch contains two parts:

1) reverts commit r353306.
2) move the format logic out from tweaks, keep tweaks API unchanged.

Reviewers: sammccall, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58051

llvm-svn: 353712
2019-02-11 15:18:11 +00:00
Haojian Wu 12e194cbb7 [clangd] Format tweak's replacements.
llvm-svn: 353306
2019-02-06 15:24:50 +00:00
Kadir Cetinkaya c3dcd2673e [clangd] Enable include insertion for static index
Summary:
This enables include insertion by adding canonical includes into
preambledata.

Reviewers: ioeric, ilya-biryukov

Subscribers: javed.absar, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D57508

llvm-svn: 353054
2019-02-04 16:19:57 +00:00
Sam McCall 9c8f432617 [clangd] Expose SelectionTree to code tweaks, and use it for swap if branches.
Summary:
This reduces the per-check implementation burden and redundant work.
It also makes checks range-aware by default (treating the commonAncestor
as if it were a point selection should be good baseline behavior).

Reviewers: ilya-biryukov

Subscribers: ioeric, MaskRay, jkorous, arphaman, kadircet

Tags: #clang

Differential Revision: https://reviews.llvm.org/D57570

llvm-svn: 352876
2019-02-01 15:09:47 +00:00
Sam McCall 866ba2c924 [clangd] Implement textDocument/declaration from LSP 3.14
Summary:
LSP now reflects the declaration/definition distinction.

Language server changes:
 - textDocument/definition now returns a definition if one is found, otherwise
   the declaration. It no longer returns declaration + definition if they are
   distinct.
 - textDocument/declaration returns the best declaration we can find.
 - For macros, the active macro definition is returned for both methods.
 - For include directive, the top of the target file is returned for both.
There doesn't appear to be a discovery mechanism (we can't return everything to
clients that only know about definition), so this changes existing behavior.
In practice, it should greatly reduce the fraction of the time we need to show
the user a menu of options.

C++ API changes:
 - findDefinitions is replaced by locateSymbolAt, which returns a
   vector<LocatedSymbol> - one for each symbol under the cursor.
 - this contains the preferred declaration, the definition (if found), and
   the symbol name
This API enables some potentially-neat extensions, like swapping between decl
and def, and exposing the symbol name to the UI in the case of multiple symbols.

Reviewers: hokein

Subscribers: ilya-biryukov, javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D57388

llvm-svn: 352864
2019-02-01 11:26:13 +00:00
Sam McCall 8a32f3b7a0 [clangd] Fix crash in applyTweak, remove TweakID alias.
Strings are complicated, giving them opaque names makes us forget
they're complicated.

llvm-svn: 352837
2019-02-01 05:41:50 +00:00
Ilya Biryukov cce67a32cf [clangd] Interfaces for writing code tweaks
Summary:
The code tweaks are an implementation of mini-refactorings exposed
via the LSP code actions. They run in two stages:
  - Stage 1. Decides whether the action is available to the user and
    collects all the information required to finish the action.
    Should be cheap, since this will run over all the actions known to
    clangd on each textDocument/codeAction request from the client.

  - Stage 2. Uses information from stage 1 to produce the actual edits
    that the code action should perform. This stage can be expensive and
    will only run if the user chooses to perform the specified action in
    the UI.

One unfortunate consequence of this change is increased latency of
processing the textDocument/codeAction requests, which now wait for an
AST. However, we cannot avoid this with what we have available in the LSP
today.

Reviewers: kadircet, ioeric, hokein, sammccall

Reviewed By: sammccall

Subscribers: mgrang, mgorny, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D56267

llvm-svn: 352494
2019-01-29 14:17:36 +00:00
Eric Liu dd66277c36 [clangd] Suggest adding missing includes for incomplete type diagnostics.
Summary:
This enables clangd to intercept compiler diagnostics and attach fixes (e.g. by
querying index). This patch adds missing includes for incomplete types e.g.
member access into class with only forward declaration. This would allow adding
missing includes for user-typed symbol names that are missing declarations
(e.g. typos) in the future.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56903

llvm-svn: 352361
2019-01-28 14:01:55 +00:00
Haojian Wu 1ca0c58c81 [clangd] Support clang-tidy configuration in clangd.
Summary:
This patch adds some basic supports for clang-tidy configurations in clangd:
      - clangd will respect .clang-tidy configurations for each file
      - we don't aim to support all clang-tidy options in clangd, only a
        small subset of condfigurations (options related to which checks will be
        enabled) are supported.
      - add a `clang-tidy-checks` CLI option that can override options from
        .clang-tidy file

Reviewers: ilya-biryukov, sammccall

Reviewed By: sammccall

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D55256

llvm-svn: 351792
2019-01-22 09:39:05 +00:00
Kadir Cetinkaya be6b35dac4 [clangd] Filter out plugin related flags and move all commandline manipulations into OverlayCDB.
Summary:
Some projects make use of clang plugins when building, but clangd is
not aware of those plugins therefore can't work with the same compile command
arguments.

There were multiple places clangd performed commandline manipulations,
 this one also moves them all into OverlayCDB.

Reviewers: ilya-biryukov

Subscribers: klimek, sammccall, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D56841

llvm-svn: 351788
2019-01-22 09:10:20 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Haojian Wu c34f022bfe [clangd] Add Limit parameter for xref.
Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D56597

llvm-svn: 351081
2019-01-14 18:11:09 +00:00
Ilya Biryukov f2001aa743 [clangd] Remove 'using namespace llvm' from .cpp files. NFC
The new guideline is to qualify with 'llvm::' explicitly both in
'.h' and '.cpp' files. This simplifies moving the code between
header and source files and is easier to keep consistent.

llvm-svn: 350531
2019-01-07 15:45:19 +00:00
Ilya Biryukov 22fa465a8c [clangd] clang-format everything. NFC
llvm-svn: 350303
2019-01-03 13:28:05 +00:00
Eric Liu 667e8ef7e1 [clangd] BackgroundIndex rebuilds symbol index periodically.
Summary:
Currently, background index rebuilds symbol index on every indexed file,
which can be inefficient. This patch makes it only rebuild symbol index periodically.
As the rebuild no longer happens too often, we could also build more efficient
dex index.

Reviewers: ilya-biryukov, kadircet

Reviewed By: kadircet

Subscribers: dblaikie, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D55770

llvm-svn: 349496
2018-12-18 15:39:33 +00:00
Haojian Wu ec3d02013f [clangd] C++ API for emitting file status.
Introduce clangd C++ API to emit the current status of file.

llvm-svn: 348475
2018-12-06 09:41:04 +00:00
Jan Korous b406701f66 [clangd] textDocument/SymbolInfo extension
New method returning symbol info for given source position.

Differential Revision: https://reviews.llvm.org/D54799

rdar://problem/46050281

llvm-svn: 347675
2018-11-27 16:40:46 +00:00
Sam McCall 422c828dfc [clangd] Enable auto-index behind a flag.
Summary:
Ownership and configuration:
The auto-index (background index) is maintained by ClangdServer, like Dynamic.
(This means ClangdServer will be able to enqueue preamble indexing in future).
For now it's enabled by a simple boolean flag in ClangdServer::Options, but
we probably want to eventually allow injecting the storage strategy.

New 'sync' command:
In order to meaningfully test the integration (not just unit-test components)
we need a way for tests to ensure the asynchronous index reads/writes occur
before a certain point.
Because these tests and assertions are few, I think exposing an explicit "sync"
command for use in tests is simpler than allowing threading to be completely
disabled in the background index (as we do for TUScheduler).

Bugs:
I fixed a couple of trivial bugs I found while testing, but there's one I can't.
JSONCompilationDatabase::getAllFiles() may return relative paths, and currently
we trigger an assertion that assumes they are absolute.
There's no efficient way to resolve them (you have to retrieve the corresponding
command and then resolve against its directory property). In general I think
this behavior is broken and we should fix it in JSONCompilationDatabase and
require CompilationDatabase::getAllFiles() to be absolute.

Reviewers: kadircet

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D54894

llvm-svn: 347567
2018-11-26 16:00:11 +00:00
Ilya Biryukov 19d75608f8 [clangd] Add support for hierarchical documentSymbol
Reviewers: ioeric, sammccall, simark

Reviewed By: sammccall

Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52311

llvm-svn: 347498
2018-11-23 15:21:19 +00:00
Ilya Biryukov 16cc195e8d [clangd] Cleanup: make the diags callback global in TUScheduler
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54760

llvm-svn: 347474
2018-11-22 17:27:08 +00:00
Ilya Biryukov 27b83d230d [clangd] Cleanup error consumption code. NFC
- Remove reimplementations of llvm::consumeError.
- Simplify test code by using EXPECT_ERROR where it fits.

llvm-svn: 347472
2018-11-22 16:20:12 +00:00
Ilya Biryukov c76394bc4b [clangd] Cleanup: make diagnostics callbacks from TUScheduler non-racy
Summary:
Previously, removeDoc followed by an addDoc to TUScheduler resulted in
racy diagnostic responses, i.e. the old dianostics could be delivered
to the client after the new ones by TUScheduler.

To workaround this, we tracked a version number in ClangdServer and
discarded stale diagnostics. After this commit, the TUScheduler will
stop delivering diagnostics for removed files and the workaround in
ClangdServer is not required anymore.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, jfb, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54829

llvm-svn: 347468
2018-11-22 15:39:54 +00:00
Eric Liu c0ac4bb17c [clangd] Cleanup: stop passing around list of supported URI schemes.
Summary:
Instead of passing around a list of supported URI schemes in clangd, we
expose an interface to convert a path to URI using any compatible scheme
that has been registered. It favors customized schemes and falls
back to "file" when no other scheme works.

Changes in this patch are:
- URI::create(AbsPath, URISchemes) -> URI::create(AbsPath). The new API finds a
compatible scheme from the registry.
- Remove URISchemes option everywhere (ClangdServer, SymbolCollecter, FileIndex etc).
- Unit tests will use "unittest" by default.
- Move "test" scheme from ClangdLSPServer to ClangdMain.cpp, and only
register the test scheme when lit-test or enable-lit-scheme is set.
(The new flag is added to make lit protocol.test work; I wonder if there
is alternative here.)

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D54800

llvm-svn: 347467
2018-11-22 15:02:05 +00:00
Sam McCall 6d4eb46d0c [clangd] workspace/symbol should be async, it reads from the index.
Summary:
To enable this, TUScheduler has to provide a way to run async tasks without
needing a preamble or AST!

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, jfb, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53644

llvm-svn: 345268
2018-10-25 14:19:14 +00:00
Sam McCall c008af6466 [clangd] Namespace style cleanup in cpp files. NFC.
Standardize on the most common namespace setup in our *.cpp files:
  using namespace llvm;
  namespace clang {
  namespace clangd {
  void foo(StringRef) { ... }
And remove redundant llvm:: qualifiers. (Except for cases like
make_unique where this causes problems with std:: and ADL).

This choice is pretty arbitrary, but some broad consistency is nice.
This is going to conflict with everything. Sorry :-/

Squash the other configurations:

A)
  using namespace llvm;
  using namespace clang;
  using namespace clangd;
  void clangd::foo(StringRef);
This is in some of the older files. (It prevents accidentally defining a
new function instead of one in the header file, for what that's worth).

B)
  namespace clang {
  namespace clangd {
  void foo(llvm::StringRef) { ... }
This is fine, but in practice the using directive often gets added over time.

C)
  namespace clang {
  namespace clangd {
  using namespace llvm; // inside the namespace
This was pretty common, but is a bit misleading: name lookup preferrs
clang::clangd::foo > clang::foo > llvm:: foo (no matter where the using
directive is).

llvm-svn: 344850
2018-10-20 15:30:37 +00:00
Sam McCall 0d9b40f583 [clangd] Set workspace root when initializing ClangdServer, disallow mutation.
Summary:
Rename instance variable to WorkspaceRoot to match what we call it internally.

Add fixme to set it automatically. Don't do it yet, clients have assumptions
that the constructor won't access the FS.

Don't second-guess the provided root.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53404

llvm-svn: 344787
2018-10-19 15:42:23 +00:00
Sam McCall 3d0adbe636 [clangd] Enforce rules around "initialize" request, and create ClangdServer lazily.
Summary:
LSP is a slightly awkward map to C++ object lifetimes: the initialize request
is part of the protocol and provides information that doesn't change over the
lifetime of the server.

Until now, we handled this by initializing ClangdServer and ClangdLSPServer
right away, and making anything that can be set in the "initialize" request
mutable.
With this patch, we create ClangdLSPServer immediately, but defer creating
ClangdServer until "initialize". This opens the door to passing the relevant
initialize params in the constructor and storing them immutably.
(That change isn't actually done in this patch).

To make this safe, we have the MessageDispatcher enforce that the "initialize"
method is called before any other (as required by LSP). That way each method
handler can assume Server is initialized, as today.

As usual, while implementing this I found places where our test cases violated
the protocol.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53398

llvm-svn: 344741
2018-10-18 14:41:50 +00:00
Sam McCall 96f2489557 [clangd] Optionally use dex for the preamble parts of the dynamic index.
Summary:
Reuse the old -use-dex-index experiment flag for this.

To avoid breaking the tests, make Dex deduplicate symbols, addressing an old FIXME.

Reviewers: hokein

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D53288

llvm-svn: 344594
2018-10-16 08:53:52 +00:00
Benjamin Kramer c55e997556 Move some helpers from the global namespace into anonymous ones.
llvm-svn: 344468
2018-10-13 22:18:22 +00:00
Sam McCall 26fac469d2 [clangd] Make FSProvider const-correct. NFC
llvm-svn: 344118
2018-10-10 07:46:15 +00:00
Kirill Bobyrev 4a5ff88fdb [clangd] NFC: Migrate to LLVM STLExtras API where possible
This patch improves readability by migrating `std::function(ForwardIt
start, ForwardIt end, ...)` to LLVM's STLExtras range-based equivalent
`llvm::function(RangeT &&Range, ...)`.

Similar change in Clang: D52576.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D52650

llvm-svn: 343937
2018-10-07 14:49:41 +00:00
Sam McCall 422f724618 [clangd] expose MergedIndex class
Summary:
This allows inheriting from it, so index() can ga away and allowing
TestTU::index) to be fixed.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52250

llvm-svn: 343780
2018-10-04 14:20:22 +00:00
Eric Liu b1d7542dde [clangd] Cache FS stat() calls when building preamble.
Summary:
The file stats can be reused when preamble is reused (e.g. code
completion). It's safe to assume that cached status is not outdated as we
assume preamble files to remain unchanged.

On real file system, this made code completion ~20% faster on a measured file
(with big preamble). The preamble build time doesn't change much.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: mgorny, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52419

llvm-svn: 343576
2018-10-02 10:43:55 +00:00
Eric Liu 764f461f9c [clangd] Get rid of Decls parameter in indexMainDecls. NFC
It's already available in ParsedAST.

llvm-svn: 342473
2018-09-18 13:35:16 +00:00
Eric Liu 821a116818 [clangd] Merge ClangdServer::DynamicIndex into FileIndex. NFC.
Summary:
FileIndex now provides explicit interfaces for preamble and main file updates.
This avoids growing parameter list when preamble and main symbols diverge
further (e.g. D52078). This also gets rid of the hack in `indexAST` that
inferred main file index based on `TopLevelDecls`.

Also separate `indexMainDecls` from `indexAST`.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D52222

llvm-svn: 342460
2018-09-18 10:30:44 +00:00
Sam McCall 45be5cf0ed [clangd] Allow all LSP methods to signal cancellation via $/cancelRequest
Summary:
The cancelable scopes are managed by JSONRPCDispatcher so that all Handlers
run in cancelable contexts.
(Previously ClangdServer did this, for code completion only).

Cancellation request processing is therefore also in JSONRPCDispatcher.
(Previously it was in ClangdLSPServer).

This doesn't actually make any new commands *respect* cancellation - they'd
need to check isCancelled() and bail out. But it opens the door to doing
this incrementally, and putting such logic in common machinery like TUScheduler.

I also rewrote the ClangdServer class/threading comments because I wanted to
add to it and I got carried away.

Reviewers: ilya-biryukov, kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D52004

llvm-svn: 342135
2018-09-13 12:58:36 +00:00
Sam McCall c8758406b5 [clangd] Simplify cancellation public API
Summary:
Task is no longer exposed:
 - task cancellation is hidden as a std::function
 - task creation returns the new context directly
 - checking is via free function only, with no way to avoid the context lookup
The implementation is essentially the same, but a bit terser as it's hidden.

isCancelled() is now safe to use outside any task (it returns false).
This will leave us free to sprinkle cancellation in e.g. TUScheduler without
needing elaborate test setup, and lets callers that don't cancel "just work".

Updated the docs to describe the new expected use pattern.
One thing I noticed: there's nothing async-specific about the cancellation.
Async tasks can be cancelled from any thread (typically the one that created
them), sync tasks can be cancelled from any *other* thread in the same way.
So the docs now refer to "long-running" tasks instead of async ones.

Updated usage in code complete, without any structural changes.
I didn't update all the names of the helpers in ClangdLSPServer (these will
likely be moved to JSONRPCDispatcher anyway).

Reviewers: ilya-biryukov, kadircet

Subscribers: ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D51996

llvm-svn: 342130
2018-09-13 11:47:48 +00:00
Sam McCall 1ad142fe26 [clangd] Add xrefs LSP boilerplate implementation.
Reviewers: ilya-biryukov, ioeric

Subscribers: MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D50896

llvm-svn: 341462
2018-09-05 11:53:07 +00:00
Sam McCall b0138317d6 [clangd] SymbolOccurrences -> Refs and cleanup
Summary:
A few things that I noticed while merging the SwapIndex patch:
 - SymbolOccurrences and particularly SymbolOccurrenceSlab are unwieldy names,
   and these names appear *a lot*. Ref, RefSlab, etc seem clear enough
   and read/format much better.
 - The asymmetry between SymbolSlab and RefSlab (build() vs freeze()) is
   confusing and irritating, and doesn't even save much code.
   Avoiding RefSlab::Builder was my idea, but it was a bad one; add it.
 - DenseMap<SymbolID, ArrayRef<Ref>> seems like a reasonable compromise for
   constructing MemIndex - and means many less wasted allocations than the
   current DenseMap<SymbolID, vector<Ref*>> for FileIndex, and none for
   slabs.
 - RefSlab::find() is not actually used for anything, so we can throw
   away the DenseMap and keep the representation much more compact.
 - A few naming/consistency fixes: e.g. Slabs,Refs -> Symbols,Refs.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51605

llvm-svn: 341368
2018-09-04 14:39:56 +00:00
Sam McCall 046557bc03 [clangd] Some nitpicking around the new split (preamble/main) dynamic index
Summary:
- DynamicIndex doesn't implement ParsingCallbacks, to make its role clearer.
  ParsingCallbacks is a separate object owned by the receiving TUScheduler.
  (I tried to get rid of the "index-like-object that doesn't implement index"
  but it was too messy).
- Clarified(?) docs around DynamicIndex - fewer details up front, more details
  inside.
- Exposed dynamic index from ClangdServer for memory monitoring and more
  direct testing of its contents (actual tests not added here, wanted to get
  this out for review)
- Removed a redundant and sligthly confusing filename param in a callback

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51221

llvm-svn: 341325
2018-09-03 16:37:59 +00:00
Ilya Biryukov d9fe7538a3 [clangd] Handle errors before checking for cancelltion
To avoid hitting assertions in llvm::Expected destructor.

llvm-svn: 341319
2018-09-03 14:39:34 +00:00
Sam McCall e6ce8da025 [clangd] Run SignatureHelp using an up-to-date preamble, waiting if needed.
Summary:
After code completion inserts a header, running signature help using the old
preamble will usually fail. So we add support for consistent preamble reads.

Reviewers: ilya-biryukov

Subscribers: javed.absar, ioeric, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51438

llvm-svn: 341076
2018-08-30 15:07:34 +00:00
Ilya Biryukov c572dae405 [clangd] Add some trace::Spans. NFC
llvm-svn: 340815
2018-08-28 10:57:45 +00:00
Kadir Cetinkaya 689bf93b2f [clangd] Initial cancellation mechanism for LSP requests.
Reviewers: ilya-biryukov, ioeric, hokein

Reviewed By: ilya-biryukov

Subscribers: mgorny, ioeric, MaskRay, jkorous, arphaman, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D50502

llvm-svn: 340607
2018-08-24 13:09:41 +00:00
Eric Liu 25d74e9594 [clangd] Speculative code completion index request before Sema is run.
Summary:
For index-based code completion, send an asynchronous speculative index
request, based on the index request for the last code completion on the same
file and the filter text typed before the cursor, before sema code completion
is invoked. This can reduce the code completion latency (by roughly latency of
sema code completion) if the speculative request is the same as the one
generated for the ongoing code completion from sema. As a sequence of code
completions often have the same scopes and proximity paths etc, this should be
effective for a number of code completions.

Trace with speculative index request:{F6997544}

Reviewers: hokein, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: javed.absar, jfb, MaskRay, jkorous, arphaman, kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D50962

llvm-svn: 340604
2018-08-24 11:23:56 +00:00
Ilya Biryukov 8343baf686 [clangd] Make FileIndex aware of the main file
Summary:
It was previously only indexing the preamble decls. The new
implementation will index both the preamble and the main AST and
report both sets of symbols, preferring the ones from the main AST
whenever the symbol is present in both.
The symbols in the main AST slab always store all information
available in the preamble symbols, possibly adding more,
e.g. definition locations.

Reviewers: hokein, ioeric

Reviewed By: ioeric

Subscribers: kadircet, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D50889

llvm-svn: 340404
2018-08-22 12:43:17 +00:00
Ilya Biryukov c5e44c1805 [clangd] Add callbacks on parsed AST in addition to parsed preambles
Summary:
Will be used for updating the dynamic index on updates to the open files.
Currently we collect only information coming from the preamble
AST. This has a bunch of limitations:
  - Dynamic index misses important information from the body of the
    file, e.g. locations of definitions.
  - XRefs cannot be collected at all, since we can only obtain full
    information for the current file (preamble is parsed with skipped
    function bodies, therefore not reliable).

This patch only adds the new callback, actually updates to the index
will be done in a follow-up patch.

Reviewers: hokein

Reviewed By: hokein

Subscribers: kadircet, javed.absar, ioeric, MaskRay, jkorous, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D50847

llvm-svn: 340401
2018-08-22 11:39:16 +00:00
Ilya Biryukov 8a0f76b855 [clangd] Fetch documentation from the Index during signature help
Summary:
Sema can only be used for documentation in the current file, other doc
comments should be fetched from the index.

Reviewers: hokein, ioeric, kadircet

Reviewed By: hokein, kadircet

Subscribers: MaskRay, jkorous, mgrang, arphaman, cfe-commits

Differential Revision: https://reviews.llvm.org/D50727

llvm-svn: 340005
2018-08-17 09:32:30 +00:00
Sam McCall 6316e0d322 [clangd] Extract FileSystemProvider into a separate header. NFC
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: mgorny, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D49142

llvm-svn: 336909
2018-07-12 14:49:52 +00:00
Sam McCall bed5885d9e [clangd] Upgrade logging facilities with levels and formatv.
Summary:
log() is split into four functions:
 - elog()/log()/vlog() have different severity levels, allowing filtering
 - dlog() is a lazy macro which uses LLVM_DEBUG - it logs to the logger, but
   conditionally based on -debug-only flag and is omitted in release builds

All logging functions use formatv-style format strings now, e.g:
  log("Could not resolve URI {0}: {1}", URI, Result.takeError());

Existing log sites have been split between elog/log/vlog by best guess.

This includes a workaround for passing Error to formatv that can be
simplified when D49170 or similar lands.

Subscribers: ilya-biryukov, javed.absar, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D49008

llvm-svn: 336785
2018-07-11 10:35:11 +00:00
Marc-Andre Laperle 1be69701c3 [clangd] Implementation of textDocument/documentSymbol
Summary:
An AST-based approach is used to retrieve the document symbols rather than an
in-memory index query. The index is not an ideal fit to achieve this because of
the file-centric query being done here whereas the index is suited for
project-wide queries. Document symbols also includes more symbols and need to
keep the order as seen in the file.

Signed-off-by: Marc-Andre Laperle <marc-andre.laperle@ericsson.com>

Subscribers: tomgr, ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47846

llvm-svn: 336386
2018-07-05 19:35:01 +00:00
Benjamin Kramer c36c09fe8e [clangd] Replace UniqueFunction with llvm::unique_function.
One implementation of this ought to be enough for everyone.

llvm-svn: 336228
2018-07-03 20:59:33 +00:00
Sam McCall 3f0243fdaf [clangd] Incorporate transitive #includes into code complete proximity scoring.
Summary:
We now compute a distance from the main file to the symbol header, which
is a weighted count of:
 - some number of #include traversals from source file --> included file
 - some number of FS traversals from file --> parent directory
 - some number of FS traversals from parent directory --> child file/dir
This calculation is performed in the appropriate URI scheme.

This means we'll get some proximity boost from header files in main-file
contexts, even when these are in different directory trees.

This extended file proximity model is not yet incorporated in the index
interface/implementation.

Reviewers: ioeric

Subscribers: mgorny, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D48441

llvm-svn: 336177
2018-07-03 08:09:29 +00:00
Sam McCall e746a2b4f7 [clangd] ClangdServer::codeComplete return CodeCompleteResult, not LSP struct.
Summary:
This provides more structured information that embedders can use for rendering.
ClangdLSPServer continues to call render(), so NFC.

The patch is:
 - trivial changes to ClangdServer/ClangdLSPServer
 - mostly-mechanical updates to CodeCompleteTests etc for the new API
 - new direct tests of render() in CodeCompleteTests
 - tiny cleanups to CodeCompletionItem (operator<< and missing initializers)

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D48821

llvm-svn: 336094
2018-07-02 11:13:16 +00:00
Sam McCall 27c979ace0 [clangd] codeComplete returns more structured completion items, LSP. NFC.
Summary:
LSP has some presentational fields with limited semantics (e.g. 'detail') and
doesn't provide a good place to return information like namespace.

Some places where more detailed information is useful:
 - tools like quality analysis
 - alternate frontends that aren't strictly LSP
 - code completion unit tests

In this patch, ClangdServer::codeComplete still return LSP CompletionList, but
I plan to switch that soon (should be a no-op for ClangdLSPServer).

Deferring this makes it clear that we don't change behavior (tests stay the
same) and also keeps the API break to a small patch which can be reverted.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, cfe-commits, jkorous

Differential Revision: https://reviews.llvm.org/D48762

llvm-svn: 335980
2018-06-29 14:47:57 +00:00
Eric Liu bbadbe016f [clangd] Use default clang-format styles.
llvm-svn: 335598
2018-06-26 12:49:09 +00:00
Eric Liu 53425f297b [clangd] Use workspace root path as hint path for resolving URIs in workspace/symbol
Summary:
Some URI schemes require a hint path to be provided, and workspace root
path seems to be a good fit.

Reviewers: sammccall, malaperle

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D48290

llvm-svn: 335035
2018-06-19 09:33:53 +00:00
Eric Liu 13e503f68a [clangd] Customizable URI schemes for dynamic index.
Summary:
This allows dynamic index to have consistent URI schemes with the
static index which can have customized URI schemes, which would make file
proximity scoring based on URIs easier.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47931

llvm-svn: 334809
2018-06-15 08:55:00 +00:00
Ilya Biryukov b10ef47a68 [clangd] Move caching of compile args out of ClangdServer.
Summary:
Caching is now handled by ClangdLSPServer and hidden behind the
GlobalCompilationDatabase interface. This simplifies ClangdServer.
This change also removes the SkipCache flag from addDocument,
which is now obsolete.

No behavioral changes are intended, the clangd binary still caches the
compile commands on the first read.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: mgorny, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D48068

llvm-svn: 334585
2018-06-13 09:20:41 +00:00
Haojian Wu 6138b78672 [clangd] Fix using the incorrect Index for go-to-definition.
Reviewers: sammccall

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47869

llvm-svn: 334176
2018-06-07 08:49:55 +00:00
Marc-Andre Laperle d41bc70094 [clangd] Remove unused variables
Summary: Signed-off-by: Marc-Andre Laperle <marc-andre.laperle@ericsson.com>

Subscribers: klimek, ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47737

llvm-svn: 334018
2018-06-05 14:07:45 +00:00
Sam McCall 682cfe704d [clangd] Hover should return null when not hovering over anything.
Summary: Also made JSON serialize Optional<T> to simplify this.

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47701

llvm-svn: 333881
2018-06-04 10:37:16 +00:00
Ilya Biryukov 823b056f58 [clangd] Keep only a limited number of idle ASTs in memory
Summary:
After this commit, clangd will only keep the last 3 accessed ASTs in
memory. Preambles for each of the opened files are still kept in
memory to make completion and AST rebuilds fast.

AST rebuilds are usually fast enough, but having the last ASTs in
memory still considerably improves latency of operations like
findDefinition and documeneHighlight, which are often sent multiple
times a second when moving around the code. So keeping some of the last
accessed ASTs in memory seems like a reasonable tradeoff.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: malaperle, arphaman, klimek, javed.absar, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47063

llvm-svn: 333737
2018-06-01 10:08:43 +00:00
Ilya Biryukov 6c5e99ed46 [clangd] Build index on preamble changes instead of the AST changes
Summary:
This is more efficient and avoids data races when reading files that
come from the preamble. The staleness can occur when reading a file
from disk that changed after the preamble was built. This can lead to
crashes, e.g. when parsing comments.

We do not to rely on symbols from the main file anyway, since any info
that those provide can always be taken from the AST.

Reviewers: ioeric, sammccall

Reviewed By: ioeric

Subscribers: malaperle, klimek, javed.absar, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47272

llvm-svn: 333196
2018-05-24 15:50:15 +00:00
Ilya Biryukov 50a7969d6e [clangd] Fix a typo. NFC
llvm-svn: 332978
2018-05-22 13:29:37 +00:00
Eric Liu 63f419a5c6 [clangd] Populate #include insertions as additional edits in completion items.
Summary:
o Remove IncludeInsertion LSP command.
o Populate include insertion edits synchromously in completion items.
o Share the code completion compiler instance and precompiled preamble to get existing inclusions in main file.
o Include insertion logic lives only in CodeComplete now.
o Use tooling::HeaderIncludes for inserting new includes.
o Refactored tests.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46497

llvm-svn: 332363
2018-05-15 15:29:32 +00:00
Eric Liu 2c1905386c [clangd] Remove LSP command-based #include insertion.
Summary:
clangd will populate #include insertions as addtionalEdits in completion items.

The code completion tests in ClangdServerTest will be added back in D46497.

Reviewers: ilya-biryukov, sammccall

Reviewed By: ilya-biryukov

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46676

llvm-svn: 332362
2018-05-15 15:23:53 +00:00
Haojian Wu 88cfb66c5a [clangd] Using index for GoToDefinition.
Summary:
This patch adds index support for GoToDefinition -- when we don't get the
definition from local AST, we query our index (Static&Dynamic) index to
get it.

Since we currently collect top-level symbol in the index, it doesn't support all
cases (e.g. class members), we will extend the index to include more symbols in
the future.

Reviewers: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, MaskRay, cfe-commits

Differential Revision: https://reviews.llvm.org/D45717

llvm-svn: 331189
2018-04-30 15:24:17 +00:00
Sam McCall a4962cce49 [clangd] Fix unicode handling, using UTF-16 where LSP requires it.
Summary:
The Language Server Protocol unfortunately mandates that locations in files
be represented by line/column pairs, where the "column" is actually an index
into the UTF-16-encoded text of the line.
(This is because VSCode is written in JavaScript, which is UTF-16-native).

Internally clangd treats source files at UTF-8, the One True Encoding, and
generally deals with byte offsets (though there are exceptions).

Before this patch, conversions between offsets and LSP Position pretended
that Position.character was UTF-8 bytes, which is only true for ASCII lines.
Now we examine the text to convert correctly (but don't actually need to
transcode it, due to some nice details of the encodings).

The updated functions in SourceCode are the blessed way to interact with
the Position.character field, and anything else is likely to be wrong.
So I also updated the other accesses:
 - CodeComplete needs a "clang-style" line/column, with column in utf-8 bytes.
   This is now converted via Position -> offset -> clang line/column
   (a new function is added to SourceCode.h for the second conversion).
 - getBeginningOfIdentifier skipped backwards in UTF-16 space, which is will
   behave badly when it splits a surrogate pair. Skipping backwards in UTF-8
   coordinates gives the lexer a fighting chance of getting this right.
   While here, I clarified(?) the logic comments, fixed a bug with identifiers
   containing digits, simplified the signature slightly and added a test.

This seems likely to cause problems with editors that have the same bug, and
treat the protocol as if columns are UTF-8 bytes. But we can find and fix those.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46035

llvm-svn: 331029
2018-04-27 11:59:28 +00:00
Marc-Andre Laperle b387b6e6dc [clangd] Implementation of workspace/symbol request
Summary:
This is a basic implementation of the "workspace/symbol" request which is
used to find symbols by a string query. Since this is similar to code completion
in terms of result, this implementation reuses the "fuzzyFind" in order to get
matches. For now, the scoring algorithm is the same as code completion and
improvements could be done in the future.

The index model doesn't contain quite enough symbols for this to cover
common symbols like methods, enum class enumerators, functions in unamed
namespaces, etc. The index model will be augmented separately to achieve this.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: jkorous, hokein, simark, sammccall, klimek, mgorny, ilya-biryukov, mgrang, jkorous-apple, ioeric, MaskRay, cfe-commits

Differential Revision: https://reviews.llvm.org/D44882

llvm-svn: 330637
2018-04-23 20:00:52 +00:00
Haojian Wu e9a5a2f10c [clangd] Allow using customized include path in URI.
Summary:
Calculating the include path from absolute file path does not always
work for all build system, e.g. bazel uses symlink as the build working
directory. The absolute file path from editor and clang is diverged from
each other. We need to address it properly in build sysmtem integration.

This patch worksarounds the issue by providing a hook in URI which allows
clients to provide their customized include path.

Reviewers: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, MaskRay, cfe-commits

Differential Revision: https://reviews.llvm.org/D45426

llvm-svn: 329578
2018-04-09 15:09:44 +00:00
Simon Marchi 766338ad7f Make positionToOffset return llvm::Expected<size_t>
Summary:

To implement incremental document syncing, we want to verify that the
ranges provided by the front-end are valid.  Currently, positionToOffset
deals with invalid Positions by returning 0 or Code.size(), which are
two valid offsets.  Instead, return an llvm:Expected<size_t> with an
error if the position is invalid.

According to the LSP, if the character value exceeds the number of
characters of the given line, it should default back to the end of the
line.  It makes sense in some contexts to have this behavior, and does
not in other contexts.  The AllowColumnsBeyondLineLength parameter
allows to decide what to do in that case, default back to the end of the
line, or return an error.

Reviewers: ilya-biryukov

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44673

llvm-svn: 328100
2018-03-21 14:36:46 +00:00
Simon Marchi 9569fd51ac Move DraftMgr from ClangdServer to ClangdLSPServer
Summary:
This patch moves the draft manager closer to the edge of Clangd, from
ClangdServer to ClangdLSPServer.  This will make it easier to implement
incremental document sync, by making ClangdServer only deal with
complete documents.

As a result, DraftStore doesn't have to deal with versioning, and thus
its API can be simplified.  It is replaced by a StringMap in
ClangdServer holding a current version number for each file.

Signed-off-by: Simon Marchi <simon.marchi@ericsson.com>

Subscribers: klimek, mgorny, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44408

llvm-svn: 327711
2018-03-16 14:30:42 +00:00
Simon Marchi 5a48cf86f1 [clangd] Use Contents from inputs in codeComplete and signatureHelp
Summary:
ClangdServer::{codeComplete,signatureHelp} both use the Contents from
the draft manager.  Since we want to move the draft manager from
ClangdServer to ClangdLSPServer, this patch changes those methods to
find the file contents from InputsAndPreamble, which contains the source
passed in previously.

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44484

llvm-svn: 327550
2018-03-14 18:31:48 +00:00
Ilya Biryukov f1f3d57eb2 [clangd] Don't expose vfs in TUScheduler::runWithPreamble.
Summary:
It was previously an easy way to concurrently access a mutable vfs,
which is a recipe for disaster.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, jkorous-apple, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D44463

llvm-svn: 327537
2018-03-14 17:46:52 +00:00
Ilya Biryukov bec5df2d05 [clangd] Remove forceReparse, add a flag to addDocument instead
Summary: To make the removal of DraftMgr from ClangdServer easier (D44408).

Reviewers: sammccall, simark

Reviewed By: sammccall, simark

Subscribers: simark, klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44462

llvm-svn: 327532
2018-03-14 17:08:41 +00:00
Sam McCall a7bb0cc09e [clangd] Remove Tagged and some related APIs from ClangdServer.
Context can do what Tagged was intended to support (snapshot filesystems),
and less intrusively.
getTaggedFileSystem() no longer needs a filename.

Cleanups while here:
 - code-complete now returns errors as Expected, like other functions
 - added an alias Callback<T> for the usual callback function type

llvm-svn: 327344
2018-03-12 23:22:35 +00:00