Summary:
This returns incremental highlights as a set of edits against the
previous highlights.
Server-side, we compute the full set of highlights, this just saves
wire-format size.
For now, the diff used is trivial: everything from the first change to
the last change is sent as a single edit.
The wire format is grungy - the replacement offset/length refer to
positions in the encoded array instead of the logical list of tokens.
We use token-oriented structs and translating to LSP forms when serializing.
This departs from LSP (but is consistent with semanticTokens today).
Tested in VSCode insiders (with a patched client to enable experimental
features).
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77225
llvm/Support/Base64, fix its implementation and provide a decent test suite.
Previous implementation code was using + operator instead of | to combine
results, which is a problem when shifting signed values. (0xFF << 16) is
implicitly converted to a (signed) int, and thus results in 0xffff0000,
h is
negative. Combining negative numbers with a + in that context is not what we
want to do.
This is a recommit of 5a1958f267 with UB removved.
This fixes https://github.com/llvm/llvm-project/issues/149.
Differential Revision: https://reviews.llvm.org/D75057
Move Base64 implementation from clangd/SemanticHighlighting to
llvm/Support/Base64, fix its implementation and provide a decent test suite.
Previous implementation code was using + operator instead of | to combine some
results, which is a problem when shifting signed values. (0xFF << 16) is
implicitly converted to a (signed) int, and thus results in 0xffff0000, which is
negative. Combining negative numbers with a + in that context is not what we
want to do.
This fixes https://github.com/llvm/llvm-project/issues/149.
Differential Revision: https://reviews.llvm.org/D75057
This patch adds the cross references for Macros in the MainFile.
We add references for the main file to the ParsedAST. We query the
references from it using the SymbolID.
Xref outside main file will be added to the index in a separate patch.
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
Summary:
Editors are good at highlightings the keywords themselves.
Note that this only affects highlightings of builtin types spelled out
as keywords in the source code. Highlightings of typedefs to builtin
types are unchanged.
Reviewers: hokein
Reviewed By: hokein
Subscribers: merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69431
Summary:
- store all macro references in the ParsedAST;
- unify the two variants of CollectMainFileMacros;
Reviewers: ilya-biryukov
Subscribers: MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67496
llvm-svn: 372725
Summary:
- Functions to compute highlighting kinds for things are separated from
the ones that add highlighting tokens.
This keeps each of them more focused on what they're doing: getting
locations and figuring out the kind of the entity, correspondingly.
- Less special cases in visitor for various nodes.
This change is an NFC.
Reviewers: hokein
Reviewed By: hokein
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67341
llvm-svn: 372008
Summary:
We still attempt to highlight them as underlying types, but fallback to
the generic 'typedef' highlighting kind if the underlying type is too
complicated.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67290
llvm-svn: 371402
Summary:
Template parameters were handled outside `addType`, this led to lack of highlightings for typedefs
to template types.
This was never desirable, we want to highlight our typedefs as their underlying type.
Note that typedefs to more complicated types, like pointers and references are still not highlighted.
Original patch by Johan Vikström.
Reviewers: hokein, jvikstrom
Reviewed By: hokein
Subscribers: nridge, javed.absar, kristof.beyls, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66516
llvm-svn: 371379
Summary:
The latter simplifies the client code by avoiding the need to handle it
as a separate case statement.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67277
llvm-svn: 371375
Summary:
Instead of matching lists of highlightings, we annotate input code with
resulting highlightings and diff it against the expected annotated input.
In case of failures, this produces much nicer output in form of text-based
diffs.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67274
llvm-svn: 371373
This much better reflects what is (now) in this header.
Maybe a rename to ParsedTU would be an improvement, but that's a much
more invasive change and life is too short.
ClangdUnit is dead, long live ClangdUnitTests!
llvm-svn: 370862
Summary: Structured bindings are in a BindingDecl. The decl the declRefExpr points to are the BindingDecls. So this adds an additional if statement in the addToken function to highlight them.
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66738
llvm-svn: 370473
Summary: This would make the client life (tracking the changes) easier.
Reviewers: jvikstrom
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66541
llvm-svn: 369884
Summary:
Adds semantic highlighting for tokens that are a macro argument.
Example:
```
D_V(SomeVar);
```
The "SomeVar" inside the macro is highlighted as a variable now.
Tokens that are in a macro body expansion are ignored in this patch for three reasons.
* The spelling loc is inside the macro "definition" meaning it would highlight inside the macro definition (could probably easily be fixed by using getExpansionLoc instead of getSpellingLoc?)
* If wanting to highlight the macro definition this could create duplicate tokens. And if the tokens are of different types there would be conflicts (tokens in the same range but with different types). Say a macro defines some name and both a variable declaration and a function use this, there would be two tokens in the macro definition but one with Kind "Variable" and the other with Kind "Function".
* Thirdly, macro body expansions could come from a file that is not the main file (easily fixed, just check that the Loc is in the main file and not even a problem if we wanted to highlight the actual macro "invocation")
Reviewers: hokein, sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64741
llvm-svn: 369275
Summary: This means that function parameters are no longer highlighted as variable.other.cpp but instead as variable.parameter.cpp which is the more "correct" TextMate scope for them.
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66335
llvm-svn: 369238
Summary: Non type templates were not being highlighted. This highlights
them as TemplateParameters.
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66221
llvm-svn: 369090
Summary: It is possible to write include code from other files so that the decls from there do not become topLevelDecls (For example by including methods for a class). These Decls are not filtered by topLevelDecls and therefore SemanticHighlighting must manually check that every SLoc belongs in the main file. Otherwise there can be highlightings appearing in places where they should not.
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66083
llvm-svn: 368563
Summary:
Done in VisitDeclaratorDecl as the AutoTypeLoc is not deduced.
Scoped to only work for variables.
auto function return values need to be handled in a special way (separate patch for that).
auto's that are in lambdas ar enot highlighted as we don't highlight their underlying type (it's a RecordDecl, but the name is not an identifier so it returns in addToken).
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65996
llvm-svn: 368546
Summary: Constructor initializers were not being highlighted. This adds highlighting for them by using TraverseConstructorInitializer. Uses the Traverse* because there is no visit for CXXCtorInitializer.
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66001
llvm-svn: 368434
Summary:
Adds a new HighlightingKind "Primitive". Adds a special case for TypeLocs that have an underlying TypePtr that is are builtin types, adding them as primitives.
The primary reason for this change is because otherwise typedefs that typedef primitives `typedef int A` would not get highlighted (so in the example `A` would not get any highlightings.)
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65943
llvm-svn: 368291
Summary:
Conversion operators contain invalid MemberLocs which caused SemanticHighlighting
to emit a lot of error logs in large files as they can occur fairly
often (for example converting StringRef to std string).
As the only thing happening was a lot of error logs being
emited there doesn't really seem to be any way to test this
(no erroneous tokens are added). But emiting as many logs as
were being emited is not wanted.
This also adds a test to guard against regressions for highlightings
disapearing from places where the conversion operators are used as their
behaviour differ from the other CXXMethodDecls.
Reviewers: hokein, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65928
llvm-svn: 368287
Summary: Added a class for diffing highlightings and removing duplicate lines. Integrated into the highlighting generation flow. Only works correctly if all tokens are on a single line. Also returns empty lines if the IDE should remove previous highlightings on a line.
Reviewers: hokein, sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64475
llvm-svn: 367521
Summary: Template parameters and specializations were not being highlighted before. This adds highlightings to those types of tokens by adding two Visit* methods.
Reviewers: hokein, sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64855
llvm-svn: 366420
Summary:
In `typedef int A` the `A` was not highlighted previously.
This patch gives `A` the same kind of highlighting that the underlying type has (class/enum) (which in this example is no special highlighting because builtins are not handled yet)
Will add highlightings for built ins in another patch.
Reviewers: hokein, sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64754
llvm-svn: 366207
Summary: The RecursiveASTVisitor sometimes visits exprs in initializer lists twice. Added deduplication to prevent duplicate highlighting tokens from appearing. Done by sorting and a linear search.
Reviewers: hokein, sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, mgrang, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64634
llvm-svn: 366070
Summary: VSCode does not have a scope for enum constants. So they were placed under "constant.other.enum" as that seems to be the most correct scope for enum constants. However, this makes theia color them blue (the same color it uses for keywords).
Reviewers: hokein, sammccall, ilya-biryukov
Subscribers: MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64624
llvm-svn: 366045
Nathan Ridge