Commit Graph

169 Commits

Author SHA1 Message Date
Sam McCall c18c280d23 [clangd] Add option to fold overloads into a single completion item.
Summary:
Adds a CodeCompleteOption to folds together compatible function/method overloads
into a single item. This feels pretty good (for editors with signatureHelp
support), but has limitations.

This happens in the code completion merge step, so there may be inconsistencies
(e.g. if only one overload made it into the index result list, no folding).

We don't want to bundle together completions that have different side-effects
(include insertion), because we can't constructo a coherent CompletionItem.
This may be confusing for users, as the reason for non-bundling may not
be immediately obvious. (Also, the implementation seems a little fragile)

Subscribers: ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47957

llvm-svn: 334822
2018-06-15 11:06:29 +00:00
Haojian Wu 7943d4fb19 [clangd] Fix buildbot error.
llvm-svn: 334812
2018-06-15 09:32:36 +00:00
Eric Liu 09c3c37b72 [clangd] Boost completion score according to file proximity.
Summary:
Also move unittest: URI scheme to TestFS so that it can be shared by
different tests.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47935

llvm-svn: 334810
2018-06-15 08:58:12 +00:00
Sam McCall 0f8df3e35a Revert "[clangd] Log completion context type. NFC"
This reverts commit r334572, which has a potential use-after-free.

llvm-svn: 334589
2018-06-13 11:31:20 +00:00
Sam McCall 2b45492247 [clangd] Log completion context type. NFC
llvm-svn: 334572
2018-06-13 06:25:36 +00:00
Eric Liu 6de95ece44 [clangd] Support proximity paths in index fuzzy find.
Reviewers: sammccall

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47937

llvm-svn: 334485
2018-06-12 08:48:20 +00:00
Sam McCall 8330589d59 [clangd] Deduplicate CompletionItemKind conversion.
llvm-svn: 334323
2018-06-08 21:17:19 +00:00
Sam McCall 80ad707288 [clangd] Require case-insensitive prefix match for macro completions.
Summary: Macros are terribly spammy at the moment and this offers some relief.

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47936

llvm-svn: 334287
2018-06-08 13:32:25 +00:00
Sam McCall 4caa85129f [clangd] Code completion: drop explicit injected names/operators, ignore Sema priority
Summary:
Now we have most of Sema's code completion signals incorporated in Quality,
which will allow us to give consistent ranking to sema/index results.

Therefore we can/should stop using Sema priority as an explicit signal.
This fixes some issues like namespaces always having a terrible score.

The most important missing signals are:
 - Really dumb/rarely useful completions like:
    SomeStruct().^SomeStruct
    SomeStruct().^operator=
    SomeStruct().~SomeStruct()
   We already filter out destructors, this patch adds injected names and
   operators to that list.
 - type matching the expression context.
   Ilya has a plan to add this in a way that's compatible with indexes
   (design doc should be shared real soon now!)

Reviewers: ioeric

Subscribers: ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47871

llvm-svn: 334192
2018-06-07 12:49:17 +00:00
Sam McCall d9b54f0025 [clangd] Boost code completion results that are narrowly scoped (local, members)
Summary:
This signal is considered a relevance rather than a quality signal because it's
dependent on the query (the fact that it's completion, and implicitly the query
context).

This is part of the effort to reduce reliance on Sema priority, so we can have
consistent ranking between Index and Sema results.

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47762

llvm-svn: 334026
2018-06-05 16:30:25 +00:00
Marc-Andre Laperle 945b5a3df0 [clangd] Add "member" symbols to the index
Summary:
This adds more symbols to the index:
- member variables and functions
- enum constants in scoped enums

The code completion behavior should remain intact but workspace symbols should
now provide much more useful symbols.
Other symbols should be considered such as the ones in "main files" (files not
being included) but this can be done separately as this introduces its fair
share of problems.

Signed-off-by: Marc-Andre Laperle <marc-andre.laperle@ericsson.com>

Reviewers: ioeric, sammccall

Reviewed By: ioeric, sammccall

Subscribers: hokein, sammccall, jkorous, klimek, ilya-biryukov, jkorous-apple, ioeric, MaskRay, cfe-commits

Differential Revision: https://reviews.llvm.org/D44954

llvm-svn: 334017
2018-06-05 14:01:40 +00:00
Eric Liu 9b3cba7923 [clangd] Avoid inserting new #include when declaration is present in the main file.
Summary:
Also fix USR generation for classes in unit tests. The previous USR
only works for class members, which happens to work when completing class name
inside the class, where constructors are suggested by sema.

Reviewers: sammccall, ilya-biryukov

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47466

llvm-svn: 333519
2018-05-30 09:03:39 +00:00
Kirill Bobyrev 5a267ed889 [clangd] Minor cleanup
This patch silences few clang-tidy warnings, removes unwanted trailing
whitespace and enforces coding guidelines.

The functionality is not affected since the cleanup is rather straightforward,
all clangd tests are still green.

Reviewers: ioeric, ilya-biryukov

Reviewed By: ioeric

Subscribers: MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47471

llvm-svn: 333411
2018-05-29 11:50:51 +00:00
Ilya Biryukov 981a35d81e [clangd] Fix leak sanitizers warnings in clangd
The commit includes two changes:
1. Set DisableFree to false when building the ParsedAST.
   This is sane default, since clangd never wants to leak the AST.
2. Make sure CompilerInstance created in code completion is passed to
   the FrontendAction::BeginSourceFile call.
   We have to do this to make sure the memory buffers of remapped
   files are properly freed.

Our tests do not produce any warnings under asan anymore.
The changes are mostly trivial, just moving the code around. So
sending without review.

llvm-svn: 333370
2018-05-28 12:11:37 +00:00
Ilya Biryukov be0eb8f498 [clangd] Serve comments for headers decls from dynamic index only
Summary:
To fix a crash in code completion that occurrs when reading doc
comments from files that were updated after the preamble was
computed. In that case, the files on disk could've been changed and we
can't rely on finding the comment text with the same range anymore.

The current workaround is to not provide comments from the headers at
all and rely on the dynamic index instead.

A more principled solution would be to store contents of the files
read inside the preamble, but it is way harder to implement properly,
given that it would definitely increase the sizes of the preamble.

Together with D47272, this should fix all preamble-related crashes
we're aware of.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47274

llvm-svn: 333189
2018-05-24 14:49:23 +00:00
Eric Liu 42abe419e2 [clangd] Fix code completion in MACROs with stringification.
Summary:
Currently, we only handle the first callback from sema code completion
and ignore results from potential following callbacks. This causes
causes loss of completion results when multiple contexts are tried by Sema.

For example, we wouldn't get any completion result in the following completion
as the first attemped context is natural language which has no
candidate. The parser would backtrack and tried a completion with AST
semantic, which would find candidate "::x".

```
void f(const char*, int);
#define F(x) f(#x, x)
int x;
void main() {
	F(::^);
}
```

To fix this, we only process a sema callback when it gives completion results or
the context supports index-based completion.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47256

llvm-svn: 333174
2018-05-24 11:20:19 +00:00
Nicola Zaghen c9fed1353a Replace last DEBUG occurrence with LLVM_DEBUG.
llvm-svn: 333083
2018-05-23 13:57:48 +00:00
Ilya Biryukov 06a485d390 [clangd] Remove ignored Preamble::CanReuse call from completion
Summary:
Now that the clients who relied on stats for files from preamble are
gone.

Reviewers: ioeric, sammccall

Reviewed By: ioeric

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D47066

llvm-svn: 332976
2018-05-22 13:10:09 +00:00
Eric Liu 4c0a27bb7d Second attempt to fix buildbot failure caused by r332363
http://lab.llvm.org:8011/builders/clang-x86_64-linux-abi-test/builds/26501

llvm-svn: 332520
2018-05-16 20:31:38 +00:00
Eric Liu f8947da9ce Attempt to fix buildbot failure caused by r332363
Log: http://lab.llvm.org:8011/builders/clang-x86_64-linux-abi-test/builds/26399
llvm-svn: 332515
2018-05-16 19:59:49 +00:00
Ilya Biryukov c22d344743 [clangd] Parse all comments in Sema and completion.
Summary:
And add tests for the comment extraction code.

clangd will now show non-doxygen comments in completion for results
coming from Sema and Dynamic index.
Static index does not include the comments yet, I will enable it in
a separate commit after investigating which implications it has for
the size of the index.

Reviewers: sammccall, hokein, ioeric

Reviewed By: sammccall

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46002

llvm-svn: 332460
2018-05-16 12:32:49 +00:00
Ilya Biryukov 43714504a8 [clangd] Retrieve minimally formatted comment text in completion.
Summary:
Previous implementation used to extract brief text from doxygen comments.
Brief text parsing slows down completion and is not suited for
non-doxygen comments.

This commit switches to providing comments that mimic the ones
originally written in the source code, doing minimal reindenting and
removing the comments markers to make the output more user-friendly.

It means we lose support for doxygen-specific features, e.g. extracting
brief text, but provide useful results for non-doxygen comments.
Switching the doxygen support back is an option, but I suggest to see
whether the current approach gives more useful results.

Reviewers: sammccall, hokein, ioeric

Reviewed By: sammccall

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D45999

llvm-svn: 332459
2018-05-16 12:32:44 +00:00
Sam McCall c5707b6c36 [clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
 - awkward to test. The mechanisms (extracting info from index/sema) can be
   unit-tested well, the policy (scoring) should be quantitatively measured.
   Neither was easily possible, and debugging was hard.
   The intermediate signal struct makes this easier.
 - hard to reuse. This is a bug in workspaceSymbols: it just presents the
   results in the index order, which is not sorted in practice, it needs to rank
   them!
   Also, index implementations care about scoring (both query-dependent and
   independent) in order to truncate result lists appropriately.

The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).

Reviewers: ilya-biryukov

Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D46524

llvm-svn: 332378
2018-05-15 17:43:27 +00:00
Eric Liu 63f419a5c6 [clangd] Populate #include insertions as additional edits in completion items.
Summary:
o Remove IncludeInsertion LSP command.
o Populate include insertion edits synchromously in completion items.
o Share the code completion compiler instance and precompiled preamble to get existing inclusions in main file.
o Include insertion logic lives only in CodeComplete now.
o Use tooling::HeaderIncludes for inserting new includes.
o Refactored tests.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46497

llvm-svn: 332363
2018-05-15 15:29:32 +00:00
Eric Liu 2c1905386c [clangd] Remove LSP command-based #include insertion.
Summary:
clangd will populate #include insertions as addtionalEdits in completion items.

The code completion tests in ClangdServerTest will be added back in D46497.

Reviewers: ilya-biryukov, sammccall

Reviewed By: ilya-biryukov

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46676

llvm-svn: 332362
2018-05-15 15:23:53 +00:00
Ilya Biryukov a6556e26ff [clangd] Fix warning after fully covered enum switch. NFC.
By adding llvm_unreachable.

llvm-svn: 332233
2018-05-14 11:47:30 +00:00
Ilya Biryukov a907ba4cb8 [clangd] Don't query index when completing inside classes
Summary:
We used to query the index when completing after class qualifiers,
e.g. 'ClassName::^'. We should not do that for the same reasons we
don't query the index for member access expressions.

Reviewers: sammccall, ioeric

Reviewed By: sammccall

Subscribers: klimek, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46795

llvm-svn: 332226
2018-05-14 10:50:04 +00:00
Sam McCall dc8abc45d2 [clangd] Incorporate #occurrences in scoring code complete results.
Summary: needs tests

Reviewers: ilya-biryukov

Subscribers: klimek, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46183

llvm-svn: 331457
2018-05-03 14:53:02 +00:00
Sam McCall a4962cce49 [clangd] Fix unicode handling, using UTF-16 where LSP requires it.
Summary:
The Language Server Protocol unfortunately mandates that locations in files
be represented by line/column pairs, where the "column" is actually an index
into the UTF-16-encoded text of the line.
(This is because VSCode is written in JavaScript, which is UTF-16-native).

Internally clangd treats source files at UTF-8, the One True Encoding, and
generally deals with byte offsets (though there are exceptions).

Before this patch, conversions between offsets and LSP Position pretended
that Position.character was UTF-8 bytes, which is only true for ASCII lines.
Now we examine the text to convert correctly (but don't actually need to
transcode it, due to some nice details of the encodings).

The updated functions in SourceCode are the blessed way to interact with
the Position.character field, and anything else is likely to be wrong.
So I also updated the other accesses:
 - CodeComplete needs a "clang-style" line/column, with column in utf-8 bytes.
   This is now converted via Position -> offset -> clang line/column
   (a new function is added to SourceCode.h for the second conversion).
 - getBeginningOfIdentifier skipped backwards in UTF-16 space, which is will
   behave badly when it splits a surrogate pair. Skipping backwards in UTF-8
   coordinates gives the lexer a fighting chance of getting this right.
   While here, I clarified(?) the logic comments, fixed a bug with identifiers
   containing digits, simplified the signature slightly and added a test.

This seems likely to cause problems with editors that have the same bug, and
treat the protocol as if columns are UTF-8 bytes. But we can find and fix those.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, ioeric, MaskRay, jkorous, cfe-commits

Differential Revision: https://reviews.llvm.org/D46035

llvm-svn: 331029
2018-04-27 11:59:28 +00:00
Ilya Biryukov 94da7bdde6 [clangd] Handle multiple callbacks from Sema's completion
Summary:
When parser backtracks, we might receive multiple code completion
callbacks.
Previously we had a failing assertion there, now we take first results
and hope they are good enough.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D44567

llvm-svn: 327717
2018-03-16 15:23:44 +00:00
Ilya Biryukov 53d6d9369b [clangd] Don't end completion item labels with '::'
llvm-svn: 326809
2018-03-06 16:45:21 +00:00
Ilya Biryukov ddf6a33306 [clang] Fix use-after-free on code completion
Summary:
Found by asan. Fiddling with code completion AST after
FrontendAction::Exceute can lead to errors.
Calling the callback in ProcessCodeCompleteResults to make sure we
don't access uninitialized state.

This particular issue comes from the fact that Sema::TUScope is
deleted when destructor of ~Parser runs, but still present in
Sema::TUScope and accessed when building completion items.

I'm still struggling to come up with a small repro. The relevant
stackframes reported by asan are:
ERROR: AddressSanitizer: heap-use-after-free on address
READ of size 8 at 0x61400020d090 thread T175
    #0 0x5632dff7821b in llvm::SmallPtrSetImplBase::isSmall() const include/llvm/ADT/SmallPtrSet.h:195:33
    #1 0x5632e0335901 in llvm::SmallPtrSetImplBase::insert_imp(void const*) include/llvm/ADT/SmallPtrSet.h:127:9
    #2 0x5632e067347d in llvm::SmallPtrSetImpl<clang::Decl*>::insert(clang::Decl*) include/llvm/ADT/SmallPtrSet.h:372:14
    #3 0x5632e065df80 in clang::Scope::AddDecl(clang::Decl*) tools/clang/include/clang/Sema/Scope.h:287:18
    #4 0x5632e0623eea in clang::ASTReader::pushExternalDeclIntoScope(clang::NamedDecl*, clang::DeclarationName) clang/lib/Serialization/ASTReader.cpp
    #5 0x5632e062ce74 in clang::ASTReader::finishPendingActions() tools/clang/lib/Serialization/ASTReader.cpp:9164:9
    ....
    #30 0x5632e02009c4 in clang::index::generateUSRForDecl(clang::Decl const*, llvm::SmallVectorImpl<char>&) tools/clang/lib/Index/USRGeneration.cpp:1037:6
    #31 0x5632dff73eab in clang::clangd::(anonymous namespace)::getSymbolID(clang::CodeCompletionResult const&) tools/clang/tools/extra/clangd/CodeComplete.cpp:326:20
    #32 0x5632dff6fe91 in clang::clangd::CodeCompleteFlow::mergeResults(std::vector<clang::CodeCompletionResult, std::allocator<clang::CodeCompletionResult> > const&, clang::clangd::SymbolSlab const&)::'lambda'(clang::CodeCompletionResult const&)::operator()(clang::CodeCompletionResult const&) tools/clang/tools/extra/clangd/CodeComplete.cpp:938:24
    #33 0x5632dff6e426 in clang::clangd::CodeCompleteFlow::mergeResults(std::vector<clang::CodeCompletionResult, std::allocator<clang::CodeCompletionResult> > const&, clang::clangd::SymbolSlab const&) third_party/llvm/llvm/tools/clang/tools/extra/clangd/CodeComplete.cpp:949:38
    #34 0x5632dff7a34d in clang::clangd::CodeCompleteFlow::runWithSema() llvm/tools/clang/tools/extra/clangd/CodeComplete.cpp:894:16
    #35 0x5632dff6df6a in clang::clangd::CodeCompleteFlow::run(clang::clangd::(anonymous namespace)::SemaCompleteInput const&) &&::'lambda'()::operator()() const third_party/llvm/llvm/tools/clang/tools/extra/clangd/CodeComplete.cpp:858:35
    #36 0x5632dff6cd42 in clang::clangd::(anonymous namespace)::semaCodeComplete(std::unique_ptr<clang::CodeCompleteConsumer, std::default_delete<clang::CodeCompleteConsumer> >, clang::CodeCompleteOptions const&, clang::clangd::(anonymous namespace)::SemaCompleteInput const&, llvm::function_ref<void ()>) tools/clang/tools/extra/clangd/CodeComplete.cpp:735:5

0x61400020d090 is located 80 bytes inside of 432-byte region [0x61400020d040,0x61400020d1f0)
freed by thread T175  here:
    #0 0x5632df74e115 in operator delete(void*, unsigned long) projects/compiler-rt/lib/asan/asan_new_delete.cc:161:3
    #1 0x5632e0b06973 in clang::Parser::~Parser() tools/clang/lib/Parse/Parser.cpp:410:3
    #2 0x5632e0b06ddd in clang::Parser::~Parser() clang/lib/Parse/Parser.cpp:408:19
    #3 0x5632e0b03286 in std::unique_ptr<clang::Parser, std::default_delete<clang::Parser> >::~unique_ptr() .../bits/unique_ptr.h:236:4
    #4 0x5632e0b021c4 in clang::ParseAST(clang::Sema&, bool, bool) tools/clang/lib/Parse/ParseAST.cpp:182:1
    #5 0x5632e0726544 in clang::FrontendAction::Execute() tools/clang/lib/Frontend/FrontendAction.cpp:904:8
    #6 0x5632dff6cd05 in clang::clangd::(anonymous namespace)::semaCodeComplete(std::unique_ptr<clang::CodeCompleteConsumer, std::default_delete<clang::CodeCompleteConsumer> >, clang::CodeCompleteOptions const&, clang::clangd::(anonymous namespace)::SemaCompleteInput const&, llvm::function_ref<void ()>) tools/clang/tools/extra/clangd/CodeComplete.cpp:728:15

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, jkorous-apple, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D44000

llvm-svn: 326569
2018-03-02 12:28:27 +00:00
Eric Liu 6c8e858551 [clangd] don't insert new includes if either original header or canonical header is already included.
Summary:
Changes:
o Store both the original header and the canonical header in LSP command.
o Also check that both original and canonical headers are not already included
by comparing both resolved header path and written literal includes.

This addresses the use case where private IWYU pragma is defined in a private
header while it would still be preferrable to include the private header, in the
internal implementation file. If we have seen that the priviate header is already
included, we don't try to insert the canonical include.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43510

llvm-svn: 326070
2018-02-26 08:32:13 +00:00
Haojian Wu b603a5e0e3 [clangd] Correct setting ignoreWarnings in CodeCompletion.
Summary:
We should set the flag before creating ComplierInstance -- when
CopmilerInstance gets initialized, it also initializes the DiagnosticsEngine
using the DiagnosticOptions.

This was hidden deeply -- as clang suppresses all diagnostics when we
hit the code-completion (but internally it does do unnecessary analysis stuff).

As a bonus point, this fix will optmize the completion speed -- clang won't do
any analysis (e.g. -Wunreachable-code, -Wthread-safety-analysisi) at all internally.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D43569

llvm-svn: 325779
2018-02-22 13:35:01 +00:00
Eric Liu 02ce01f1e8 [clangd] Not collect include headers for dynamic index for now.
Summary:
The new behaviors introduced by this patch:
o When include collection is enabled, we always set IncludeHeader field in Symbol
even if it's the same as FileURI in decl.
o Disable include collection in FileIndex which is currently only used to build
dynamic index. We should revisit when we actually want to use FileIndex to global
index.
o Code-completion only uses IncludeHeader to insert headers but not FileURI in
CanonicalDeclaration. This ensures that inserted headers are always canonicalized.
Note that include insertion can still be triggered for symbols that are already
included if they are merged from dynamic index and static index, but we would
only use includes that are already canonicalized (e.g. from static index).

Reason for change:
Collecting header includes in dynamic index enables inserting includes for headers
that are not indexed but opened in the editor. Comparing to inserting includes for
symbols in global/static index, this is nice-to-have but would probably require
non-trivial amount of work to get right. For example:
o Currently it's not easy to fully support CanonicalIncludes in dynamic index, given the way
we run dynamic index.
o It's also harder to reason about the correctness of include canonicalization for dynamic index
(i.e. symbols in the current file/TU) than static index where symbols are collected
offline and sanity check is possible before shipping to production.
o We have less control/flexibility over symbol info in the dynamic index
(e.g. URIs, path normalization), which could be used to help make decision when inserting includes.

As header collection (especially canonicalization) is relatively new, and enabling
it for dynamic index would immediately affect current users with only dynamic
index support, I propose we disable it for dynamic index for now to avoid
compromising other hot features like code completion and only support it for
static index where include insertion would likely to bring more value.

Reviewers: ilya-biryukov, sammccall, hokein

Subscribers: klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43550

llvm-svn: 325764
2018-02-22 10:14:05 +00:00
Sam McCall ab8e393f62 [clangd] Invert return value of fuzzyFind() (fix MemIndex's return value)
Have had way too many bugs by converting between "isComplete" and
"isIncomplete". LSP is immovable, so use isIncomplete everywhere.

llvm-svn: 325493
2018-02-19 13:04:41 +00:00
Ilya Biryukov 408657ce54 [clangd] Attach more information about Sema completion to traces
Reviewers: hokein, ioeric, sammccall

Reviewed By: sammccall

Subscribers: klimek, cfe-commits, jkorous-apple

Differential Revision: https://reviews.llvm.org/D43377

llvm-svn: 325491
2018-02-19 12:35:57 +00:00
Eric Liu c5105f9e3c [clangd] collect symbol #include & insert #include in global code completion.
Summary:
o Collect suitable #include paths for index symbols. This also does smart mapping
for STL symbols and IWYU pragma (code borrowed from include-fixer).
o For global code completion, add a command for inserting new #include in each code
completion item.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, jkorous-apple, hintonda, cfe-commits

Differential Revision: https://reviews.llvm.org/D42640

llvm-svn: 325343
2018-02-16 14:15:55 +00:00
Ilya Biryukov a9cf311d5b [clangd] Log if CWD could not be changed. NFC.
llvm-svn: 325024
2018-02-13 17:15:06 +00:00
Ilya Biryukov b6ad25cd4c [clangd] Fix crash when CompilerInvocation can't be created.
Summary:
This can happen if the CompileCommand provided by compilation database
is malformed.

Reviewers: hokein, ioeric, sammccall

Reviewed By: sammccall

Subscribers: klimek, jkorous-apple, cfe-commits

Differential Revision: https://reviews.llvm.org/D43122

llvm-svn: 324732
2018-02-09 13:51:57 +00:00
Sam McCall d1a7a37c22 [clangd] Pass Context implicitly using TLS.
Summary:
Instead of passing Context explicitly around, we now have a thread-local
Context object `Context::current()` which is an implicit argument to
every function.
Most manipulation of this should use the WithContextValue helper, which
augments the current Context to add a single KV pair, and restores the
old context on destruction.

Advantages are:
- less boilerplate in functions that just propagate contexts
- reading most code doesn't require understanding context at all, and
  using context as values in fewer places still
- fewer options to pass the "wrong" context when it changes within a
  scope (e.g. when using Span)
- contexts pass through interfaces we can't modify, such as VFS
- propagating contexts across threads was slightly tricky (e.g.
  copy vs move, no move-init in lambdas), and is now encapsulated in
  the threadpool

Disadvantages are all the usual TLS stuff - hidden magic, and
potential for higher memory usage on threads that don't use the
context. (In practice, it's just one pointer)

Reviewers: ilya-biryukov

Subscribers: klimek, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D42517

llvm-svn: 323872
2018-01-31 13:40:48 +00:00
Sam McCall 2b780160cd [clangd] Trace code completion.
Context passing is a little messy, but will go away with TLS soon.

llvm-svn: 323792
2018-01-30 17:20:54 +00:00
Haojian Wu 58d208d6e4 [clangd] Disable typo correction when doing code completion.
Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, jkorous-apple, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D42491

llvm-svn: 323410
2018-01-25 09:44:06 +00:00
Haojian Wu 48b4865683 [clangd] Limit completion results.
Summary:
* truncate symbols from static/dynamic index to the limited number
(which would save lots of cost in constructing the merged symbols).
* add an CLI option allowing to limit the number of returned completion results.
(default to 100)

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, jkorous-apple, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D42484

llvm-svn: 323408
2018-01-25 09:20:09 +00:00
Haojian Wu 061c73eb28 [clangd] Use accessible scopes to query indexes for global code completion.
Summary:
 * For qualified completion (foo::a^)
       * unresolved qualifier - use global namespace ("::")
       * resolved qualifier - use all accessible namespaces inside the resolved qualifier.
 * For unqualified completion (vec^), use scopes that are accessible from the
      scope from which code completion occurs.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: jkorous-apple, ioeric, klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D42073

llvm-svn: 323189
2018-01-23 11:37:26 +00:00
Sam McCall d2a9592526 [clangd] Drop ~destructor completions - rarely helpful and work inconsistently
llvm-svn: 323149
2018-01-22 21:05:00 +00:00
Simon Pilgrim 62dbdf1790 Fix MSVC implicit double-float truncation warning. NFCI.
llvm-svn: 323112
2018-01-22 13:15:16 +00:00
Sam McCall 8b2faeed02 [clangd] Change index scope convention from "outer::inner" to "outer::inner::"
Global scope is "" (was "")
Top-level namespace scope is "ns::" (was "ns")
Nested namespace scope is "ns::ns::" (was "ns::ns")

This composes more naturally:
- qname = scope + name
- full scope = resolved scope + unresolved scope (D42073 was the trigger)
It removes a wart from the old way: "foo::" has one more separator than "".

Another alternative that has these properties is "::ns", but that lacks
the property that both the scope and the name are substrings of the
qname as produced by clang.

This is re-landing r322996 which didn't build.

llvm-svn: 323000
2018-01-19 22:18:21 +00:00
Sam McCall 50584ea197 Revert "[clangd] Change index scope convention from "outer::inner" to "outer::inner::""
This reverts commit r322996.

llvm-svn: 322998
2018-01-19 22:09:34 +00:00
Sam McCall 1a8c55ecc6 [clangd] Change index scope convention from "outer::inner" to "outer::inner::"
Global scope is "" (was "")
Top-level namespace scope is "ns::" (was "ns")
Nested namespace scope is "ns::ns::" (was "ns::ns")

This composes more naturally:
 - qname = scope + name
 - full scope = resolved scope + unresolved scope (D42073 was the trigger)
It removes a wart from the old way: "foo::" has one more separator than "".

Another alternative that has these properties is "::ns", but that lacks
the property that both the scope and the name are substrings of the
qname as produced by clang.

llvm-svn: 322996
2018-01-19 21:58:58 +00:00
Sam McCall 545a20d90b [clangd] Merge index-provided completions with those from Sema.
Summary:
 - we match on USR, and do a field-by-field merge if both have results
 - scoring is post-merge, with both sets of information available
   (for now, sema priority is used if available, static score for index results)
 - limit is applied to the complete result set (previously index ignored limit)
 - CompletionItem is only produces for the returned results
 - If the user doesn't type a scope, we send the global scope for completion
   (we can improve this after D42073)

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D42181

llvm-svn: 322945
2018-01-19 14:34:02 +00:00
Sam McCall d8169a85cc [clangd] Remove unused IncludeGlobals completion option, always pass true to sema
llvm-svn: 322856
2018-01-18 15:31:30 +00:00
Ilya Biryukov 295c8e1e2d [clangd] Always use preamble (even stale) for code completion
Summary:
This improves performance of code completion, because we avoid stating
the files from the preamble and never attempt to parse the files
without using the preamble if it's provided.

However, the change comes at a cost of sometimes providing incorrect
results when doing code completion after making actually considerable
changes to the files used in the preamble or the preamble itself.
Eventually the preamble will get rebuilt and code completion will
be providing the correct results.

Reviewers: bkramer, sammccall, jkorous-apple

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41991

llvm-svn: 322854
2018-01-18 15:17:00 +00:00
Sam McCall 0faecf0c33 [clangd] Merge results from static/dynamic index.
Summary:
We now hide the static/dynamic split from the code completion, behind a
new implementation of the SymbolIndex interface. This will reduce the
complexity of the sema/index merging that needs to be done by
CodeComplete, at a fairly small cost in flexibility.

Reviewers: hokein

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D42049

llvm-svn: 322480
2018-01-15 12:33:00 +00:00
Sam McCall 3d139c527b [clangd] Code completion uses Sema for NS-level things in the current file.
Summary:
To stay fast, it avoids deserializing anything outside the current file, by
disabling the LoadExternal code completion option added in r322377, when the
index is enabled.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41996

llvm-svn: 322387
2018-01-12 18:30:08 +00:00
Sam McCall 93483e7f39 [clangd] Include debugging tags for both static and dynamic index. NFC
llvm-svn: 322379
2018-01-12 17:09:49 +00:00
Sam McCall 84652cc63c [clangd] Incorporate fuzzy-match into result rankings.
Summary: The scoring function is fuzzy-match-quality * existing quality score.

Reviewers: ioeric

Subscribers: klimek, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40780

llvm-svn: 322377
2018-01-12 16:16:09 +00:00
Haojian Wu ba28e9ac5d [clangd] Add static index for the global code completion.
Summary:
Use the YAML-format symbols (generated by the global-symbol-builder tool) to
do the global code completion.
It is **experimental** only , but it allows us to experience global code
completion on a relatively small project.

Tested with LLVM project.

Reviewers: sammccall, ioeric

Reviewed By: sammccall, ioeric

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41668

llvm-svn: 322191
2018-01-10 14:44:34 +00:00
Ilya Biryukov f60bf347c9 [clangd] Remove duplicates from code completion
Summary:
This patch removes hidden items from code completion.
Items can be hidden, e.g., by other items in the child scopes.
This patch addresses a particular problem of a duplicate completion
item for the class in the following example:

    struct Adapter { void method(); };
    void Adapter::method() {
       Adapter^
    }

We should probably investigate if there are other duplicates in
completion and remove them, possibly adding assertions that it never
happens.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D41901

llvm-svn: 322185
2018-01-10 13:51:09 +00:00
Eric Liu 76f6b44443 [clangd] Add more symbol information for code completion.
Reviewers: hokein, sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41345

llvm-svn: 322097
2018-01-09 17:32:00 +00:00
Ilya Biryukov 71590659ea [clangd] Fix memory leak in code completion
llvm-svn: 321867
2018-01-05 13:36:55 +00:00
Ilya Biryukov 5a5e1caf25 [clangd] Properly set filterText for index-based completion items
It was previously set to an identifier that the user typed, leading to
surprising behavior in VSCode (probably in other editors too).

llvm-svn: 321554
2017-12-29 14:59:22 +00:00
Eric Liu 63696e14e3 [clangd] Pull CodeCompletionString handling logic into its own file and add unit test.
Reviewers: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41450

llvm-svn: 321193
2017-12-20 17:24:31 +00:00
Eric Liu dc641ebfae [clangd] Supress a log warning by putting it behind a condition.
llvm-svn: 321094
2017-12-19 18:10:32 +00:00
Sam McCall e3e1570695 [clangd] Fix warnings/compiler pickiness after r321083
llvm-svn: 321086
2017-12-19 17:05:00 +00:00
Eric Liu 6f648df1b9 [clangd] Index-based code completion.
Summary: Use symbol index to populate completion results for qualfified IDs e.g. "nx::A^".

Reviewers: ilya-biryukov, sammccall

Reviewed By: ilya-biryukov, sammccall

Subscribers: rwols, klimek, mgorny, cfe-commits, sammccall

Differential Revision: https://reviews.llvm.org/D41281

llvm-svn: 321083
2017-12-19 16:50:37 +00:00
Ilya Biryukov 940901e8b1 [clangd] Implemented logging using Context
Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40486

llvm-svn: 320576
2017-12-13 12:51:22 +00:00
Sam McCall 9aad25f193 [clangd] Split code-completion tests out of ClangdTests. NFC.
Summary:
Common parts are mostly FS related, so pulled out TestFS.h for the common stuff.
Deliberately resisted cleaning up much here, so this is pretty mechanical.

Reviewers: hokein

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40784

llvm-svn: 319741
2017-12-05 07:20:26 +00:00
Sam McCall 98775c5055 [clangd] Split CodeComplete into a separate file. NFC
Summary: Shared details of ClangdUnit and CodeComplete moved to a new Compiler file.

Reviewers: ilya-biryukov

Subscribers: klimek, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D40719

llvm-svn: 319655
2017-12-04 13:49:59 +00:00