Commit Graph

272 Commits

Author SHA1 Message Date
Ilya Biryukov 82b59ae428 [clangd] Simplify code handling compile commands
Summary:
CppFile can now change compilation arguments during rebuild. This allows
simplifying code that manages CppFiles.

Reviewers: sammccall, bkramer, jkorous-apple

Reviewed By: sammccall

Subscribers: ioeric, jkorous-apple, klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D42173

llvm-svn: 323204
2018-01-23 15:07:52 +00:00
Haojian Wu 061c73eb28 [clangd] Use accessible scopes to query indexes for global code completion.
Summary:
 * For qualified completion (foo::a^)
       * unresolved qualifier - use global namespace ("::")
       * resolved qualifier - use all accessible namespaces inside the resolved qualifier.
 * For unqualified completion (vec^), use scopes that are accessible from the
      scope from which code completion occurs.

Reviewers: sammccall, ilya-biryukov

Reviewed By: sammccall

Subscribers: jkorous-apple, ioeric, klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D42073

llvm-svn: 323189
2018-01-23 11:37:26 +00:00
Sam McCall d2a9592526 [clangd] Drop ~destructor completions - rarely helpful and work inconsistently
llvm-svn: 323149
2018-01-22 21:05:00 +00:00
Simon Pilgrim 62dbdf1790 Fix MSVC implicit double-float truncation warning. NFCI.
llvm-svn: 323112
2018-01-22 13:15:16 +00:00
Eric Liu f5b8c82198 [clangd] Add support for different file URI schemas.
Summary: I will replace the existing URI struct in Protocol.h with the new URI and rename FileURI to URI in a followup patch.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: jkorous-apple, klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41946

llvm-svn: 323101
2018-01-22 11:48:20 +00:00
Sam McCall 8b2faeed02 [clangd] Change index scope convention from "outer::inner" to "outer::inner::"
Global scope is "" (was "")
Top-level namespace scope is "ns::" (was "ns")
Nested namespace scope is "ns::ns::" (was "ns::ns")

This composes more naturally:
- qname = scope + name
- full scope = resolved scope + unresolved scope (D42073 was the trigger)
It removes a wart from the old way: "foo::" has one more separator than "".

Another alternative that has these properties is "::ns", but that lacks
the property that both the scope and the name are substrings of the
qname as produced by clang.

This is re-landing r322996 which didn't build.

llvm-svn: 323000
2018-01-19 22:18:21 +00:00
Sam McCall 50584ea197 Revert "[clangd] Change index scope convention from "outer::inner" to "outer::inner::""
This reverts commit r322996.

llvm-svn: 322998
2018-01-19 22:09:34 +00:00
Sam McCall 1a8c55ecc6 [clangd] Change index scope convention from "outer::inner" to "outer::inner::"
Global scope is "" (was "")
Top-level namespace scope is "ns::" (was "ns")
Nested namespace scope is "ns::ns::" (was "ns::ns")

This composes more naturally:
 - qname = scope + name
 - full scope = resolved scope + unresolved scope (D42073 was the trigger)
It removes a wart from the old way: "foo::" has one more separator than "".

Another alternative that has these properties is "::ns", but that lacks
the property that both the scope and the name are substrings of the
qname as produced by clang.

llvm-svn: 322996
2018-01-19 21:58:58 +00:00
Sam McCall eb74ab8633 [clangd] Fix memcpy(?, null, 0) UB by switching to std::copy
llvm-svn: 322949
2018-01-19 15:03:49 +00:00
Sam McCall 545a20d90b [clangd] Merge index-provided completions with those from Sema.
Summary:
 - we match on USR, and do a field-by-field merge if both have results
 - scoring is post-merge, with both sets of information available
   (for now, sema priority is used if available, static score for index results)
 - limit is applied to the complete result set (previously index ignored limit)
 - CompletionItem is only produces for the returned results
 - If the user doesn't type a scope, we send the global scope for completion
   (we can improve this after D42073)

Reviewers: ioeric

Subscribers: klimek, ilya-biryukov, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D42181

llvm-svn: 322945
2018-01-19 14:34:02 +00:00
Haojian Wu 9873fdd4f5 [clangd] Collect enum constants in SymbolCollector
Summary:
* ignore nameless symbols
* include enum constant declarataion

Reviewers: ilya-biryukov, jkorous-apple

Reviewed By: ilya-biryukov

Subscribers: ioeric, cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D42074

llvm-svn: 322929
2018-01-19 09:35:55 +00:00
Sam McCall d8169a85cc [clangd] Remove unused IncludeGlobals completion option, always pass true to sema
llvm-svn: 322856
2018-01-18 15:31:30 +00:00
Ilya Biryukov 295c8e1e2d [clangd] Always use preamble (even stale) for code completion
Summary:
This improves performance of code completion, because we avoid stating
the files from the preamble and never attempt to parse the files
without using the preamble if it's provided.

However, the change comes at a cost of sometimes providing incorrect
results when doing code completion after making actually considerable
changes to the files used in the preamble or the preamble itself.
Eventually the preamble will get rebuilt and code completion will
be providing the correct results.

Reviewers: bkramer, sammccall, jkorous-apple

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41991

llvm-svn: 322854
2018-01-18 15:17:00 +00:00
Sam McCall 2082fe156d [clangd] Output log messages to stderr when not configured (e.g. in tests). NFC
llvm-svn: 322827
2018-01-18 10:24:01 +00:00
Sam McCall a5ce243fa3 [clangd] Use fuzzy match to select top N index results.
Summary:
This makes performance slower but more predictable (it always processes
every symbol). We need to find ways to make this fast, possibly by precomputing
short queries or capping the number of scored results. But our current approach
is too naive.

It also no longer returns results in a "good" order. In fact it's pathological:
the top N results are ranked from worst to best. Indexes aren't responsible for
ranking and MergedIndex can't do a good job, so I'm pleased that this will make
any hidden assumptions we have more noticeable :-)

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D42060

llvm-svn: 322821
2018-01-18 08:35:04 +00:00
Eric Liu 31300c7d8b [clangd] Deduplicate symbols collected in global-symbol-builder tool.
Summary: After D42111, the executor framework no longer deduplicate tool results.

Reviewers: bkramer, sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D42113

llvm-svn: 322722
2018-01-17 18:27:41 +00:00
Sam McCall 5668cd3e57 [clangd] Avoid divide-by-zero
llvm-svn: 322668
2018-01-17 15:25:55 +00:00
Ilya Biryukov 261c72e8f9 [clangd] Don't crash on LSP calls for non-added files
Summary:
We will return errors for non-added files for now.
Another alternative for clangd would be to read non-added files from
disk and provide useful features anyway.

There are still some cases that fail with assertion (e.g., code
complete). We should address those too, but they require more subtle
changes to the code and therefore out of scope of this patch.

Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D42164

llvm-svn: 322637
2018-01-17 12:30:24 +00:00
Sam McCall e2f43f500a [clangd] Improve const-correctness of Symbol->Detail. NFC
Summary:
This would have caught a bug I wrote in an early version of D42049, where
an index user could overwrite data internal to the index because the Symbol is
not deep-const.

The YAML traits are now a bit more verbose, but separate concerns a bit more
nicely: ArenaPtr can be reused for other similarly-allocated objects, including
scalars etc.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D42059

llvm-svn: 322509
2018-01-15 20:09:09 +00:00
Sam McCall 0faecf0c33 [clangd] Merge results from static/dynamic index.
Summary:
We now hide the static/dynamic split from the code completion, behind a
new implementation of the SymbolIndex interface. This will reduce the
complexity of the sema/index merging that needs to be done by
CodeComplete, at a fairly small cost in flexibility.

Reviewers: hokein

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D42049

llvm-svn: 322480
2018-01-15 12:33:00 +00:00
Sam McCall 09b4be51a8 [clangd] Fix uninitialized-read found by asan
llvm-svn: 322443
2018-01-13 16:46:26 +00:00
Sam McCall 3d139c527b [clangd] Code completion uses Sema for NS-level things in the current file.
Summary:
To stay fast, it avoids deserializing anything outside the current file, by
disabling the LoadExternal code completion option added in r322377, when the
index is enabled.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41996

llvm-svn: 322387
2018-01-12 18:30:08 +00:00
Sam McCall 93483e7f39 [clangd] Include debugging tags for both static and dynamic index. NFC
llvm-svn: 322379
2018-01-12 17:09:49 +00:00
Sam McCall 84652cc63c [clangd] Incorporate fuzzy-match into result rankings.
Summary: The scoring function is fuzzy-match-quality * existing quality score.

Reviewers: ioeric

Subscribers: klimek, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40780

llvm-svn: 322377
2018-01-12 16:16:09 +00:00
Haojian Wu f4374f087f [clangd] Don't navigate to forward class declaration when go to definition.
Summary:
For some cases, GoToDefinition will navigate to the forward class
declaration, we should always navigate to the class definition.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41661

llvm-svn: 322370
2018-01-12 14:21:10 +00:00
Ilya Biryukov 9555839228 [clangd] Pass Context to onDiagnosticsReady callback
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41907

llvm-svn: 322199
2018-01-10 17:59:27 +00:00
Eric Liu e5fe0ce45e Add a missing dependency for r322192
llvm-svn: 322194
2018-01-10 15:11:26 +00:00
Eric Liu 9af958fbf8 [clangd] Add more filters for collected symbols.
Summary:
o We only collect symbols in namespace or translation unit scopes.
o Add an option to only collect symbols in included headers.

Reviewers: hokein, ilya-biryukov

Reviewed By: hokein, ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41823

llvm-svn: 322193
2018-01-10 14:57:58 +00:00
Haojian Wu ba28e9ac5d [clangd] Add static index for the global code completion.
Summary:
Use the YAML-format symbols (generated by the global-symbol-builder tool) to
do the global code completion.
It is **experimental** only , but it allows us to experience global code
completion on a relatively small project.

Tested with LLVM project.

Reviewers: sammccall, ioeric

Reviewed By: sammccall, ioeric

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41668

llvm-svn: 322191
2018-01-10 14:44:34 +00:00
Ilya Biryukov f60bf347c9 [clangd] Remove duplicates from code completion
Summary:
This patch removes hidden items from code completion.
Items can be hidden, e.g., by other items in the child scopes.
This patch addresses a particular problem of a duplicate completion
item for the class in the following example:

    struct Adapter { void method(); };
    void Adapter::method() {
       Adapter^
    }

We should probably investigate if there are other duplicates in
completion and remove them, possibly adding assertions that it never
happens.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D41901

llvm-svn: 322185
2018-01-10 13:51:09 +00:00
Eric Liu 72b3855bf5 Try to fix build failure caused by r322097
Avoid mapping during output when Detail is nullptr; otherwise, an empty
"Detail" field will be populated in YAML output.

llvm-svn: 322116
2018-01-09 20:26:49 +00:00
Eric Liu 76f6b44443 [clangd] Add more symbol information for code completion.
Reviewers: hokein, sammccall

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41345

llvm-svn: 322097
2018-01-09 17:32:00 +00:00
Eric Liu 5732c0e6ea [clangd] Use ToolExecutor to write the global-symbol-builder tool.
Summary:
This enables more execution modes like standalone and Mapreduce-style execution.

See also D41729

Reviewers: hokein, sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41730

llvm-svn: 322084
2018-01-09 15:21:45 +00:00
Ilya Biryukov f6e2b4cb4b [clangd] Fix a bug in asynchronous code completion
A StringRef that goes out of scope was copied and used in a handler
running on a separate thread.
We didn't catch it because clangd does not use the async completion
API yet.

llvm-svn: 322080
2018-01-09 14:39:27 +00:00
Haojian Wu fe22b7474b [clangd] Catch more symbols in SymbolCollector.
Summary:
We currently only collect external-linkage symbols in the collector,
which results in missing some typical symbols (like no-linkage type alias symbols).

This patch relaxes the constraint a bit to allow collecting more symbols.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41759

llvm-svn: 322067
2018-01-09 10:44:09 +00:00
Ilya Biryukov 71590659ea [clangd] Fix memory leak in code completion
llvm-svn: 321867
2018-01-05 13:36:55 +00:00
Ilya Biryukov 5a5e1caf25 [clangd] Properly set filterText for index-based completion items
It was previously set to an identifier that the user typed, leading to
surprising behavior in VSCode (probably in other editors too).

llvm-svn: 321554
2017-12-29 14:59:22 +00:00
Benjamin Kramer 3a13ed60ba Avoid int to string conversion in Twine or raw_ostream contexts.
Some output changes from uppercase hex to lowercase hex, no other functionality change intended.

llvm-svn: 321526
2017-12-28 16:58:54 +00:00
Benjamin Kramer b4c5c2dd15 [clangd] Get rid of unnecessary global variable. No functionality change.
llvm-svn: 321525
2017-12-28 15:03:02 +00:00
Benjamin Kramer 50a967d601 [clangd] Simplify code. No functionality change intended.
llvm-svn: 321523
2017-12-28 14:47:01 +00:00
Ilya Biryukov da8daa3fe8 [clangd] Skip function bodies when building the preamble
Summary: To make building preambles faster and keep them smaller.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D41495

llvm-svn: 321521
2017-12-28 13:10:15 +00:00
Sam McCall 4b9bbb378b [clangd] Use Builder for symbol slabs, and use sorted-vector for storage
Summary:
This improves a few things:
 - the insert -> freeze -> read sequence is now enforced/communicated by the
   type system
 - SymbolSlab::const_iterator iterates over symbols, not over id-symbol pairs
 - we avoid permanently storing a second copy of the IDs, and the
   string map's hashtable

The slab size is now down to 21.8MB for the LLVM project.
Of this only 2.7MB is strings, the rest is #symbols * `sizeof(Symbol)`.
`sizeof(Symbol)` is currently 96, which seems too big - I think
SymbolInfo isn't efficiently packed. That's a topic for another patch!

Also added simple API to see the memory usage/#symbols of a slab, since
it seems likely we will continue to care about this.

Reviewers: ilya-biryukov

Subscribers: klimek, mgrang, cfe-commits

Differential Revision: https://reviews.llvm.org/D41506

llvm-svn: 321412
2017-12-23 19:38:03 +00:00
Haojian Wu 7f880c1850 [clangd] Add a tool to build YAML-format global symbols.
Summary:
The tools is used to generate global symbols for clangd (global code completion),
The format is YAML, which is only for **experiment**.

Usage:
./bin/global-symbol-builder </path/to/llvm-dir> > global-symbols.yaml

TEST:
used the tool to generate global symbols for LLVM (~72MB).

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41491

llvm-svn: 321358
2017-12-22 14:38:05 +00:00
Sam McCall c02ba7270e [clangd] Simplify GlobalCompilationDatabase, cache missing GCDs
llvm-svn: 321350
2017-12-22 09:47:34 +00:00
Sam McCall 98a7318113 [clangd] Improve packing of Symbol struct. NFC
llvm-svn: 321348
2017-12-22 08:12:39 +00:00
Sam McCall df898cc5ed [clangd] Don't re-hash SymbolID in maps, just use the SHA1 data
llvm-svn: 321302
2017-12-21 20:11:46 +00:00
Benjamin Kramer 6452efd07e [clangd] Fix use after free.
Found by asan.

llvm-svn: 321286
2017-12-21 17:51:35 +00:00
Sam McCall 6c0d0f5775 [clangd] Index symbols share storage within a slab.
Summary:
Symbols are not self-contained - it's only safe to hand them out if you
guarantee the lifetime of the underlying data.

Before this lands, I'm going to measure the before/after memory usage of the
LLVM index loaded into memory in a single slab.

Reviewers: hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41483

llvm-svn: 321272
2017-12-21 14:58:44 +00:00
Ilya Biryukov eaeea04edf [clangd] Log more info about preambles
llvm-svn: 321267
2017-12-21 14:05:28 +00:00
Haojian Wu 7093860aac [clangd] Use the clang-tools-extra as the official repo for `vscode-clangd` extension.
Summary:
Previously, we use a separate GitHub repository (https://github.com/llvm-vs-code-extensions/vscode-clangd)
for publishing `vscode-clangd` extension to marketplace.

To reduce the maintain burden, we will use the vscode extension in the
clang-tools-extra, and deprecate the one on GitHub.

Test in https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.clangd-vscode-test

Reviewers: sammccall, krasimir

Reviewed By: sammccall

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41391

llvm-svn: 321252
2017-12-21 08:45:18 +00:00