Summary:
- Moved these functions to SourceCode.h
- added unit tests
- fix off by one in positionToOffset: Offset - 1 in final calculation was wrong
- fixed formatOnType which had an equal and opposite off-by-one
- positionToOffset and offsetToPosition both consistently clamp to beginning/end
of file when input is out of range
- gave variables more descriptive names
- removed windows line ending fixmes where there is nothing to fix
- elaborated on UTF-8 fixmes
This will conflict with Eric's D41281, but in a pretty easy-to-resolve way.
Reviewers: ioeric
Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits
Differential Revision: https://reviews.llvm.org/D41351
llvm-svn: 321073
Summary: When scopes are specified, only match symbols from scopes.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: klimek, ilya-biryukov, cfe-commits
Differential Revision: https://reviews.llvm.org/D41367
llvm-svn: 321067
Summary: This will be used together with D40548 for the global index source (experimental).
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits, ioeric
Differential Revision: https://reviews.llvm.org/D41178
llvm-svn: 320694
Summary:
o Index interfaces to support using different index sources (e.g. AST index, global index) for code completion, cross-reference finding etc. This patch focuses on code completion.
The following changes in the original patch has been split out.
o Implement an AST-based index.
o Add an option to replace sema code completion for qualified-id with index-based completion.
o Implement an initial naive code completion index which matches symbols that have the query string as substring.
Reviewers: malaperle, sammccall
Reviewed By: sammccall
Subscribers: hokein, klimek, malaperle, mgorny, ilya-biryukov, cfe-commits
Differential Revision: https://reviews.llvm.org/D40548
llvm-svn: 320688
Summary:
* The "Symbol" class represents a C++ symbol in the codebase, containing all the
information of a C++ symbol needed by clangd. clangd will use it in clangd's
AST/dynamic index and global/static index (code completion and code
navigation).
* The SymbolCollector (another IndexAction) will be used to recollect the
symbols when the source file is changed (for ASTIndex), or to generate
all C++ symbols for the whole project.
In the long term (when index-while-building is ready), clangd should share a
same "Symbol" structure and IndexAction with index-while-building, but
for now we want to have some stuff working in clangd.
Reviewers: ioeric, sammccall, ilya-biryukov, malaperle
Reviewed By: sammccall
Subscribers: malaperle, klimek, mgorny, cfe-commits
Differential Revision: https://reviews.llvm.org/D40897
llvm-svn: 320486
Summary:
It will be used to pass around things like Logger and Tracer throughout
clangd classes.
Reviewers: sammccall, ioeric, hokein, bkramer
Reviewed By: sammccall
Subscribers: klimek, bkramer, mgorny, cfe-commits
Differential Revision: https://reviews.llvm.org/D40485
llvm-svn: 320468
We currently use target_link_libraries without an explicit scope
specifier (INTERFACE, PRIVATE or PUBLIC) when linking executables.
Dependencies added in this way apply to both the target and its
dependencies, i.e. they become part of the executable's link interface
and are transitive.
Transitive dependencies generally don't make sense for executables,
since you wouldn't normally be linking against an executable. This also
causes issues for generating install export files when using
LLVM_DISTRIBUTION_COMPONENTS. For example, clang has a lot of LLVM
library dependencies, which are currently added as interface
dependencies. If clang is in the distribution components but the LLVM
libraries it depends on aren't (which is a perfectly legitimate use case
if the LLVM libraries are being built static and there are therefore no
run-time dependencies on them), CMake will complain about the LLVM
libraries not being in export set when attempting to generate the
install export file for clang. This is reasonable behavior on CMake's
part, and the right thing is for LLVM's build system to explicitly use
PRIVATE dependencies for executables.
Unfortunately, CMake doesn't allow you to mix and match the keyword and
non-keyword target_link_libraries signatures for a single target; i.e.,
if a single call to target_link_libraries for a particular target uses
one of the INTERFACE, PRIVATE, or PUBLIC keywords, all other calls must
also be updated to use those keywords. This means we must do this change
in a single shot. I also fully expect to have missed some instances; I
tested by enabling all the projects in the monorepo (except dragonegg),
and configuring both with and without shared libraries, on both Darwin
and Linux, but I'm planning to rely on the buildbots for other
configurations (since it should be pretty easy to fix those).
Even after this change, we still have a lot of target_link_libraries
calls that don't specify a scope keyword, mostly for shared libraries.
I'm thinking about addressing those in a follow-up, but that's a
separate change IMO.
Differential Revision: https://reviews.llvm.org/D40823
llvm-svn: 319840
Summary:
Previously, completion options were set per ClangdServer instance.
It will allow to change completion preferences during the lifetime
of a single ClangdServer instance.
Also rewrote ClangdCompletionTest.CompletionOptions to reuse single
ClangdServer instance, the test now runs 2x faster on my machine.
Reviewers: sammccall, ioeric, hokein
Reviewed By: sammccall, ioeric
Subscribers: klimek, cfe-commits
Differential Revision: https://reviews.llvm.org/D40654
llvm-svn: 319753
Summary:
Common parts are mostly FS related, so pulled out TestFS.h for the common stuff.
Deliberately resisted cleaning up much here, so this is pretty mechanical.
Reviewers: hokein
Subscribers: klimek, mgorny, ilya-biryukov, cfe-commits
Differential Revision: https://reviews.llvm.org/D40784
llvm-svn: 319741
Summary:
- GlobalCompilationDatabase now returns a single command (that's all we use)
- fallback flags are now part of the GlobalCompilationDatabase.
There's a default implementation that they can optionally customize.
- this allows us to avoid invoking the fallback logic on two separate codepaths
- race on extra flags fixed by locking the mutex
- made GCD const-correct (DBGCD does have mutating methods)
Reviewers: hokein
Subscribers: klimek, cfe-commits, ilya-biryukov
Differential Revision: https://reviews.llvm.org/D40733
llvm-svn: 319647
Summary:
This will be used for rescoring code completion results based on partial
identifiers.
Short-term use:
- we want to limit the number of code completion results returned to
improve performance of global completion. The scorer will be used to
rerank the results to return when the user has applied a filter.
Long-term use case:
- ranking of completion results from in-memory index
- merging of completion results from multiple sources (merging usually
works best when done at the component-score level, rescoring the
fuzzy-match quality avoids different backends needing to have
comparable scores)
Reviewers: ilya-biryukov
Subscribers: cfe-commits, mgorny
Differential Revision: https://reviews.llvm.org/D40060
llvm-svn: 319557
Summary:
This allows us to limit the number of results we return and still allow them
to be surfaced by refining a query (D39852).
The initial algorithm is very conservative - it accepts a completion if the
filter is any case-insensitive sub-sequence. It does not attempt to rank items
based on match quality.
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D39882
llvm-svn: 319552
Summary:
- JSON<->Obj interface is now ADL functions, so they play nicely with enums
- recursive vector/map parsing and ObjectMapper moved to JSONExpr and tested
- renamed (un)parse to (de)serialize, since text -> JSON is called parse
- Protocol.cpp gets a bit shorter
Sorry for the giant patch, it's prety mechanical though
Reviewers: ilya-biryukov
Subscribers: klimek, cfe-commits
Differential Revision: https://reviews.llvm.org/D40596
llvm-svn: 319478
Summary:
- Converted Protocol.h parse() functions to take JSON::Expr.
These no longer detect and log unknown fields, as this is not that
useful and no longer free.
I haven't changed the error handling too much: fields that were
treated as optional before are still optional, even when it's wrong.
Exception: object properties with the wrong type are now ignored.
- Made JSONRPCDispatcher parse using json::parse
- The bug where 'method' must come before 'params' in the stream is
fixed as a side-effect. (And the same bug in executeCommand).
- Some parser crashers fixed as a side effect.
e.g. https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3890
- The debug stream now prettyprints the input messages with --pretty.
- Request params are attached to traces when tracing is enabled.
- Fixed some bugs in tests (errors tolerated by YAMLParser, and
off-by-ones in Content-Length that our null-termination was masking)
- Fixed a random double-escape bug in ClangdLSPServer (it was our last
use of YAMLParser!)
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D40406
llvm-svn: 319159
Summary:
Noticed this when I tried to port the Protocol.h parsers.
And tests for the inspect API, which caught a small bug.
Reviewers: ioeric
Subscribers: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D40399
llvm-svn: 319157
Summary:
[clangd] Tracing improvements
Compose JSON using JSONExpr
Allow attaching metadata to spans (and avoid it if tracing is off)
Attach IDs and responses of JSON RPCs to their spans
The downside is that large responses make the trace viewer sluggish.
We should make our responses less huge :-) Or fix trace viewer.
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D40132
llvm-svn: 318928
Summary: (There must be some reason why D38077 didn't just do this, but I don't get it!)
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D39836
llvm-svn: 318925
Summary:
This will replace the places where we're using YAMLParser to parse JSON now:
- the new marshalling code (T::parse()) should handle fewer cases and require
fewer explicit casts
- we'll early-reject invalid JSON that YAMLParser accepts
- we'll be able to fix protocol-parsing bugs caused by the fact that YAML can
only parse forward
I plan to do the conversion as soon as this lands, but I don't want it in one
patch as the protocol.cpp changes are conflict-prone.
Reviewers: ioeric
Subscribers: ilya-biryukov, cfe-commits
Differential Revision: https://reviews.llvm.org/D40182
llvm-svn: 318774
Summary:
All results are scored, we only process CodeCompletionStrings for the winners.
We now return CompletionList rather than CompletionItem[] (both are valid).
sortText is now based on CodeCompletionResult::orderedName (mostly the same).
This is the first clangd-only completion option, so plumbing changed.
It requires a small clangd patch (exposing CodeCompletionResult::orderedName).
(This can't usefully be enabled yet: we don't support server-side filtering)
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D39852
llvm-svn: 318287
Summary:
This form can be created with a nice clang-format-friendly literal syntax,
and gets escaping right. It knows how to call unparse() on our Protocol types.
All the places where we pass around JSON internally now use this type.
Object properties are sorted (stored as std::map) and so serialization is
canonicalized, with optional prettyprinting (triggered by a -pretty flag).
This makes the lit tests much nicer to read and somewhat nicer to debug.
(Unfortunately the completion tests use CHECK-DAG, which only has
line-granularity, so pretty-printing is disabled there. In future we
could make completion ordering deterministic, or switch to unittests).
Compared to the current approach, it has some efficiencies like avoiding copies
of string literals used as object keys, but is probably slower overall.
I think the code/test quality benefits are worth it.
This patch doesn't attempt to do anything about JSON *parsing*.
It takes direction from the proposal in this doc[1], but is limited in scope
and visibility, for now.
I am of half a mind just to use Expr as the target of a parser, and maybe do a
little string deduplication, but not bother with clever memory allocation.
That would be simple, and fast enough for clangd...
[1] https://docs.google.com/document/d/1OEF9IauWwNuSigZzvvbjc1cVS1uGHRyGTXaoy3DjqM4/edit
+cc d0k so he can tell me not to use std::map.
Reviewers: ioeric, malaperle
Subscribers: bkramer, ilya-biryukov, mgorny, klimek
Differential Revision: https://reviews.llvm.org/D39435
llvm-svn: 317486
Summary:
This lets you visualize clangd's activity on different threads over time,
and understand critical paths of requests and object lifetimes.
The data produced can be visualized in Chrome (at chrome://tracing), or
in a standalone copy of catapult (http://github.com/catapult-project/catapult)
This patch consists of:
- a command line flag "-trace" that causes clangd to emit JSON trace data
- an API (in Trace.h) allowing clangd code to easily add events to the stream
- several initial uses of this API to capture JSON-RPC requests, builds, logs
Example result: https://photos.app.goo.gl/12L9swaz5REGQ1rm1
Caveats:
- JSON serialization is ad-hoc (isn't it everywhere?) so the API is
limited to naming events rather than attaching arbitrary metadata.
I'd like to fix this (I think we could use a JSON-object abstraction).
- The recording is very naive: events are written immediately by
locking a mutex. Contention on the mutex might disturb performance.
- For now it just traces instants or spans on the current thread.
There are other things that make sense to show (cross-thread flows,
non-thread resources such as ASTs). But we have to start somewhere.
Reviewers: ioeric, ilya-biryukov
Subscribers: cfe-commits, mgorny
Differential Revision: https://reviews.llvm.org/D39086
llvm-svn: 317193
Summary:
ClangdServer now provides async code completion API.
It is still used synchronously by ClangdLSPServer, more work is needed
to allow processing other requests in parallel while completion (or
any other request) is running.
Reviewers: klimek, bkramer, krasimir
Reviewed By: klimek
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D38583
llvm-svn: 314989
Summary:
Small extension to LSP to allow clients to use clangd to switch between C header files and source files.
Final version will use the completed clangd indexer to use the index of symbols to be able to switch from header to source file when the file names don't match.
Reviewers: malaperle, krasimir, bkramer, ilya-biryukov
Reviewed By: ilya-biryukov
Subscribers: ilya-biryukov, cfe-commits, arphaman
Patch by: William Enright
Differential Revision: https://reviews.llvm.org/D36150
llvm-svn: 314377
Summary:
Calls to onDiagnosticsReady were done concurrently before. This sometimes
led to older versions of diagnostics being reported to the user after
the newer versions.
Reviewers: klimek, bkramer, krasimir
Reviewed By: klimek
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D38032
llvm-svn: 313754
Summary:
ClangdServer owned objects passed to it in constructor for no good reason.
Lots of stuff was moved from the heap to the stack thanks to this change.
Reviewers: krasimir
Reviewed By: krasimir
Subscribers: klimek, cfe-commits
Differential Revision: https://reviews.llvm.org/D34148
llvm-svn: 305298
Summary:
This is a reapplied r305280 with a fix to the crash found by build bots
(StringRef to an out-of-scope local std::string).
Reviewers: krasimir
Reviewed By: krasimir
Subscribers: klimek, cfe-commits
Differential Revision: https://reviews.llvm.org/D34146
llvm-svn: 305291
Summary:
This allows an implementation of FileSystemProvider that can track which vfs::FileSystem
were used for each of the requests.
Reviewers: bkramer, krasimir
Reviewed By: bkramer
Subscribers: klimek, cfe-commits
Tags: #clang-tools-extra
Differential Revision: https://reviews.llvm.org/D33678
llvm-svn: 304214
Summary:
Custom vfs::FileSystem is currently used for unit tests.
This revision depends on https://reviews.llvm.org/D33397.
Reviewers: bkramer, krasimir
Reviewed By: bkramer, krasimir
Subscribers: klimek, cfe-commits, mgorny
Differential Revision: https://reviews.llvm.org/D33416
llvm-svn: 303977
Eric Liu