Commit Graph

207 Commits

Author SHA1 Message Date
Eric Liu eea1633878 [clangd] Build in-memory index on symbols in files.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41276

llvm-svn: 320807
2017-12-15 12:25:02 +00:00
Eric Liu b99d5e8b62 [clangd] Put all #includes in one block in clangd source files. NFC
Clang-format categorizes and sorts #includes with style. It doesn't make sense
to manually managing #include blocks.

llvm-svn: 320743
2017-12-14 21:22:03 +00:00
Ilya Biryukov 12cdb4fd33 [clangd] Changed tracing interfaces
Summary:
EventTracer interface now contains two methods:
- spanEvent for events that have duration,
- instant for events that are instant.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, luckygeck, cfe-commits

Differential Revision: https://reviews.llvm.org/D40489

llvm-svn: 320708
2017-12-14 15:33:38 +00:00
Ilya Biryukov ee27d2ebae [clangd] Implemented tracing using Context
Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall

Subscribers: klimek, luckygeck, cfe-commits

Differential Revision: https://reviews.llvm.org/D40488

llvm-svn: 320706
2017-12-14 15:04:59 +00:00
Eric Liu d293bf127a [clangd] Add a FileSymbols container that manages symbols from multiple files.
Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41232

llvm-svn: 320701
2017-12-14 14:50:58 +00:00
Haojian Wu 56a5fca473 [clangd] Construct SymbolSlab from YAML format.
Summary: This will be used together with D40548 for the global index source (experimental).

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits, ioeric

Differential Revision: https://reviews.llvm.org/D41178

llvm-svn: 320694
2017-12-14 12:17:14 +00:00
Eric Liu 3732cadc73 [clangd] Symbol index interfaces and an in-memory index implementation.
Summary:
o Index interfaces to support using different index sources (e.g. AST index, global index) for code completion, cross-reference finding etc. This patch focuses on code completion.

The following changes in the original patch has been split out.
o Implement an AST-based index.
o Add an option to replace sema code completion for qualified-id with index-based completion.
o Implement an initial naive code completion index which matches symbols that have the query string as substring.

Reviewers: malaperle, sammccall

Reviewed By: sammccall

Subscribers: hokein, klimek, malaperle, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40548

llvm-svn: 320688
2017-12-14 11:25:49 +00:00
Ilya Biryukov 3db40ffef3 [clangd] Fix bool conversion operator of UniqueFunction
Usages of it were giving compiler errors because of the missing
explicit conversion.

llvm-svn: 320591
2017-12-13 15:42:59 +00:00
Ilya Biryukov a1d324d8b7 [clangd] Try to workaround MSVC compilation failure.
llvm-svn: 320578
2017-12-13 13:43:47 +00:00
Ilya Biryukov 5a85b8e6dd [clangd] clang-format the source code. NFC
llvm-svn: 320577
2017-12-13 12:53:16 +00:00
Ilya Biryukov 940901e8b1 [clangd] Implemented logging using Context
Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40486

llvm-svn: 320576
2017-12-13 12:51:22 +00:00
Haojian Wu acc90d17c5 [clangd] Remove the const specifier of the takeSymbol method
otherwise we will copy an object.

llvm-svn: 320574
2017-12-13 12:39:06 +00:00
Sam McCall 8111d3b89c [clangd] Emit ranges for clangd diagnostics, and fix off-by-one positions
Summary:
 - when the diagnostic has an explicit range, we prefer that
 - if the diagnostic has a fixit, its RemoveRange is our next choice
 - otherwise we try to expand the diagnostic location into a whole token.
   (inspired by VSCode, which does this client-side when given an empty range)
 - if all else fails, we return the zero-width range as now.
   (clients react in different ways to this, highlighting a token or a char)
 - this includes the off-by-one fix from D40860, and borrows heavily from it

Reviewers: rwols, hokein

Subscribers: klimek, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D41118

llvm-svn: 320555
2017-12-13 08:48:42 +00:00
Sam McCall d404654987 [clangd] Overload hash_value for SymbolID, fix struct/class warning
llvm-svn: 320554
2017-12-13 08:34:48 +00:00
Raoul Wols 212bcf8370 [clangd] (Attempt to) read clang-format file for document formatting
Summary:
Takes into account the clang-format file of the project, if any.
Reverts to LLVM if nothing is found. Replies with an error if any error occured.
For instance, a parse error in the clang-format YAML file.

Reviewers: ilya-biryukov, sammccall, Nebiroth, malaperle, krasimir

Reviewed By: sammccall

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D41031

llvm-svn: 320524
2017-12-12 20:25:06 +00:00
Haojian Wu 4c1394d67d [clangd] Introduce a "Symbol" class.
Summary:
* The "Symbol" class represents a C++ symbol in the codebase, containing all the
  information of a C++ symbol needed by clangd. clangd will use it in clangd's
  AST/dynamic index and global/static index (code completion and code
  navigation).
* The SymbolCollector (another IndexAction) will be used to recollect the
  symbols when the source file is changed (for ASTIndex), or to generate
  all C++ symbols for the whole project.

In the long term (when index-while-building is ready), clangd should share a
same "Symbol" structure and IndexAction with index-while-building, but
for now we want to have some stuff working in clangd.

Reviewers: ioeric, sammccall, ilya-biryukov, malaperle

Reviewed By: sammccall

Subscribers: malaperle, klimek, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D40897

llvm-svn: 320486
2017-12-12 15:42:10 +00:00
Ilya Biryukov ef3191fcfd [clangd] Removed unused variable. NFC
llvm-svn: 320482
2017-12-12 14:15:01 +00:00
Ilya Biryukov 9b5ffc22f6 [clangd] clang-format the code. NFC
llvm-svn: 320476
2017-12-12 12:56:46 +00:00
Ilya Biryukov 0e6a51f4f3 [clangd] Document highlights for clangd
Summary: Implementation of Document Highlights Request as described in
LSP.

Contributed by William Enright (nebiroth).

Reviewers: malaperle, krasimir, bkramer, ilya-biryukov

Reviewed By: malaperle

Subscribers: mgrang, sammccall, klimek, ioeric, rwols, cfe-commits, arphaman, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D38425

llvm-svn: 320474
2017-12-12 12:27:47 +00:00
Ilya Biryukov 657159c273 [clangd] Introduced a Context that stores implicit data
Summary:
It will be used to pass around things like Logger and Tracer throughout
clangd classes.

Reviewers: sammccall, ioeric, hokein, bkramer

Reviewed By: sammccall

Subscribers: klimek, bkramer, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D40485

llvm-svn: 320468
2017-12-12 11:16:45 +00:00
Matt Morehouse 468d4da4e8 [clangd-fuzzer] Update contruction of LSPServer.
The constructor for ClangdLSPServer changed in r318412 and r318925,
breaking the clangd-fuzzer build.

llvm-svn: 320074
2017-12-07 19:04:27 +00:00
Matt Morehouse 5a5c1d1c69 [CMake] Use PRIVATE in target_link_libraries for fuzzers.
Several fuzzers were missed by r319840.

llvm-svn: 319948
2017-12-06 19:52:40 +00:00
Shoaib Meenai d806af3499 [CMake] Use PRIVATE in target_link_libraries for executables
We currently use target_link_libraries without an explicit scope
specifier (INTERFACE, PRIVATE or PUBLIC) when linking executables.
Dependencies added in this way apply to both the target and its
dependencies, i.e. they become part of the executable's link interface
and are transitive.

Transitive dependencies generally don't make sense for executables,
since you wouldn't normally be linking against an executable. This also
causes issues for generating install export files when using
LLVM_DISTRIBUTION_COMPONENTS. For example, clang has a lot of LLVM
library dependencies, which are currently added as interface
dependencies. If clang is in the distribution components but the LLVM
libraries it depends on aren't (which is a perfectly legitimate use case
if the LLVM libraries are being built static and there are therefore no
run-time dependencies on them), CMake will complain about the LLVM
libraries not being in export set when attempting to generate the
install export file for clang. This is reasonable behavior on CMake's
part, and the right thing is for LLVM's build system to explicitly use
PRIVATE dependencies for executables.

Unfortunately, CMake doesn't allow you to mix and match the keyword and
non-keyword target_link_libraries signatures for a single target; i.e.,
if a single call to target_link_libraries for a particular target uses
one of the INTERFACE, PRIVATE, or PUBLIC keywords, all other calls must
also be updated to use those keywords. This means we must do this change
in a single shot. I also fully expect to have missed some instances; I
tested by enabling all the projects in the monorepo (except dragonegg),
and configuring both with and without shared libraries, on both Darwin
and Linux, but I'm planning to rely on the buildbots for other
configurations (since it should be pretty easy to fix those).

Even after this change, we still have a lot of target_link_libraries
calls that don't specify a scope keyword, mostly for shared libraries.
I'm thinking about addressing those in a follow-up, but that's a
separate change IMO.

Differential Revision: https://reviews.llvm.org/D40823

llvm-svn: 319840
2017-12-05 21:49:56 +00:00
Ilya Biryukov d3b04e3517 [clangd] Set completion options per-request.
Summary:
Previously, completion options were set per ClangdServer instance.
It will allow to change completion preferences during the lifetime
of a single ClangdServer instance.

Also rewrote ClangdCompletionTest.CompletionOptions to reuse single
ClangdServer instance, the test now runs 2x faster on my machine.

Reviewers: sammccall, ioeric, hokein

Reviewed By: sammccall, ioeric

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40654

llvm-svn: 319753
2017-12-05 10:42:57 +00:00
Sam McCall 9aad25f193 [clangd] Split code-completion tests out of ClangdTests. NFC.
Summary:
Common parts are mostly FS related, so pulled out TestFS.h for the common stuff.
Deliberately resisted cleaning up much here, so this is pretty mechanical.

Reviewers: hokein

Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40784

llvm-svn: 319741
2017-12-05 07:20:26 +00:00
Sam McCall 98775c5055 [clangd] Split CodeComplete into a separate file. NFC
Summary: Shared details of ClangdUnit and CodeComplete moved to a new Compiler file.

Reviewers: ilya-biryukov

Subscribers: klimek, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D40719

llvm-svn: 319655
2017-12-04 13:49:59 +00:00
Sam McCall ecbeab0ea1 [clangd] GlobalCompilationDatabase interface changes
Summary:
- GlobalCompilationDatabase now returns a single command (that's all we use)
- fallback flags are now part of the GlobalCompilationDatabase.
  There's a default implementation that they can optionally customize.
- this allows us to avoid invoking the fallback logic on two separate codepaths
- race on extra flags fixed by locking the mutex
- made GCD const-correct (DBGCD does have mutating methods)

Reviewers: hokein

Subscribers: klimek, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40733

llvm-svn: 319647
2017-12-04 10:08:45 +00:00
Sam McCall 3ea964090a [clangd] Avoid enum in bitfields, can't satisfy old GCC and new MSVC
llvm-svn: 319608
2017-12-02 04:15:55 +00:00
Sam McCall 8e97cca149 [clangd] Fix FuzzyMatch tests on windows, NFC
Without specifying the signedness of the underlying type for Action,
packing it in a 1-bit field may restrict its range to [-1, 0] which
can't represent Match.

llvm-svn: 319606
2017-12-02 03:35:19 +00:00
Sam McCall a8c5d3af60 [clangd] Try to appease gcc constexpr bug (58541)
llvm-svn: 319604
2017-12-02 02:28:29 +00:00
Sam McCall 8fed634e21 [clangd] Define constants in the right namespace. NFC
llvm-svn: 319579
2017-12-01 20:03:19 +00:00
Sam McCall 87496417ff [clangd] Fuzzy match scorer
Summary:
This will be used for rescoring code completion results based on partial
identifiers.
Short-term use:
  - we want to limit the number of code completion results returned to
  improve performance of global completion. The scorer will be used to
  rerank the results to return when the user has applied a filter.
Long-term use case:
  - ranking of completion results from in-memory index
  - merging of completion results from multiple sources (merging usually
  works best when done at the component-score level, rescoring the
  fuzzy-match quality avoids different backends needing to have
  comparable scores)

Reviewers: ilya-biryukov

Subscribers: cfe-commits, mgorny

Differential Revision: https://reviews.llvm.org/D40060

llvm-svn: 319557
2017-12-01 17:08:02 +00:00
Sam McCall 9fbecd5b0d [clangd] Filter completion results by fuzzy-matching identifiers.
Summary:
This allows us to limit the number of results we return and still allow them
to be surfaced by refining a query (D39852).

The initial algorithm is very conservative - it accepts a completion if the
filter is any case-insensitive sub-sequence. It does not attempt to rank items
based on match quality.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39882

llvm-svn: 319552
2017-12-01 16:35:50 +00:00
Sam McCall 9c8f2caa01 [clangd] Remove no-op -fsyntax-only from fallback command. NFC
This has no effect because we explicitly choose our actions.
(If it had an effect, we'd want to add it to commands we get from a CDB)

llvm-svn: 319546
2017-12-01 14:35:17 +00:00
Sam McCall 318fbeb972 [clangd] Logger implicitly adds newline
llvm-svn: 319497
2017-11-30 23:21:34 +00:00
Sam McCall fae3b02520 [clangd] Log file compile commands
llvm-svn: 319496
2017-11-30 23:16:23 +00:00
Sam McCall ff8b874548 [clangd] New conventions for JSON-marshalling functions, centralize machinery
Summary:
 - JSON<->Obj interface is now ADL functions, so they play nicely with enums
 - recursive vector/map parsing and ObjectMapper moved to JSONExpr and tested
 - renamed (un)parse to (de)serialize, since text -> JSON is called parse
 - Protocol.cpp gets a bit shorter

Sorry for the giant patch, it's prety mechanical though

Reviewers: ilya-biryukov

Subscribers: klimek, cfe-commits

Differential Revision: https://reviews.llvm.org/D40596

llvm-svn: 319478
2017-11-30 21:32:29 +00:00
Sam McCall 38a0491c68 [clangd] Simplify common JSON-parsing patterns in Protocol.
Summary:
This makes the parse() functions about as short as they can be given the
current signature, and moves all array-traversal etc code to a
central location.

We keep the ability to distinguish between optional and required fields:
and we don't propagate parse errors for optional fields.

I've made most fields required per the LSP spec - the looseness we had
here was mostly a historical accident I think.

Reviewers: ioeric

Subscribers: klimek, cfe-commits, ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40564

llvm-svn: 319309
2017-11-29 11:36:46 +00:00
Sam McCall ec109029b1 [clangd] Switch from YAMLParser to JSONExpr
Summary:
 - Converted Protocol.h parse() functions to take JSON::Expr.
   These no longer detect and log unknown fields, as this is not that
   useful and no longer free.
   I haven't changed the error handling too much: fields that were
   treated as optional before are still optional, even when it's wrong.
   Exception: object properties with the wrong type are now ignored.
 - Made JSONRPCDispatcher parse using json::parse
 - The bug where 'method' must come before 'params' in the stream is
 fixed as a side-effect. (And the same bug in executeCommand).
 - Some parser crashers fixed as a side effect.
   e.g. https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3890
 - The debug stream now prettyprints the input messages with --pretty.
 - Request params are attached to traces when tracing is enabled.
 - Fixed some bugs in tests (errors tolerated by YAMLParser, and
 off-by-ones in Content-Length that our null-termination was masking)
 - Fixed a random double-escape bug in ClangdLSPServer (it was our last
 use of YAMLParser!)

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D40406

llvm-svn: 319159
2017-11-28 09:37:43 +00:00
Sam McCall 1bb272bc74 [clangd] Add missing (but documented!) JSONExpr typed accessors
Summary:
Noticed this when I tried to port the Protocol.h parsers.
And tests for the inspect API, which caught a small bug.

Reviewers: ioeric

Subscribers: ilya-biryukov

Differential Revision: https://reviews.llvm.org/D40399

llvm-svn: 319157
2017-11-28 09:25:09 +00:00
Benjamin Kramer 2d4f142726 Make helper function static. NFC.
llvm-svn: 319022
2017-11-27 12:48:26 +00:00
Simon Pilgrim 116e3b3ca0 Fix MSVC double-float implicit truncation warning. NFCI
llvm-svn: 318961
2017-11-24 18:18:42 +00:00
Ilya Biryukov 5fc214ec31 [clangd] Sort list of sources in CMakeLists.txt. NFC
llvm-svn: 318946
2017-11-24 13:13:41 +00:00
Ilya Biryukov 2660cc96dd [clangd] Ensure preamble outlives the AST
Summary:
In-memory preambles will not be copied anymore, so we need to make
sure they outlive the AST.

Reviewers: bkramer, sammccall, klimek

Reviewed By: sammccall

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D40301

llvm-svn: 318944
2017-11-24 13:04:21 +00:00
Sam McCall 9cfd9c9a9b [clangd] Tracing improvements
Summary:
[clangd] Tracing improvements

Compose JSON using JSONExpr
Allow attaching metadata to spans (and avoid it if tracing is off)
Attach IDs and responses of JSON RPCs to their spans

The downside is that large responses make the trace viewer sluggish.
We should make our responses less huge :-) Or fix trace viewer.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D40132

llvm-svn: 318928
2017-11-23 17:12:04 +00:00
Sam McCall b8d548a3de [clangd] Make completion scores use 0-1 floats internally.
Summary:
This scale is much easier to mix with other signals, such as fuzzy match strength.
Mostly NFC, but it does reorder some low-priority items that get folded together at a score of 0 (see completion-qualifiers.test).
Removed the exact sortText from the testcases, because it's the ranking that we want to test.

Reviewers: hokein

Subscribers: ilya-biryukov, cfe-commits

Differential Revision: https://reviews.llvm.org/D40089

llvm-svn: 318927
2017-11-23 17:09:04 +00:00
Sam McCall adccab64f2 [clangd] Drop impossible completions (unavailable or inaccessible)
Summary: (There must be some reason why D38077 didn't just do this, but I don't get it!)

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D39836

llvm-svn: 318925
2017-11-23 16:58:22 +00:00
Aaron Ballman 4cbef5e259 Fixing a typo; NFC.
llvm-svn: 318921
2017-11-23 14:57:24 +00:00
Aaron Ballman feeb0b705c Silence some MSVC warnings about not all control paths returning a value; NFC.
llvm-svn: 318809
2017-11-21 22:24:13 +00:00
Sam McCall bcfec0d6d5 [clangd] Include the right header for std::isxdigit
llvm-svn: 318782
2017-11-21 17:18:30 +00:00