2019-06-27 20:22:18 +08:00
|
|
|
//===--- SemanticHighlighting.cpp - ------------------------- ---*- C++ -*-===//
|
2019-06-26 21:08:36 +08:00
|
|
|
//
|
|
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
|
|
|
//
|
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
|
|
|
|
#include "SemanticHighlighting.h"
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
#include "FindTarget.h"
|
2019-09-04 17:46:06 +08:00
|
|
|
#include "ParsedAST.h"
|
2019-07-04 15:53:12 +08:00
|
|
|
#include "Protocol.h"
|
2019-06-26 21:08:36 +08:00
|
|
|
#include "SourceCode.h"
|
[clangd] Move non-clang base pieces into separate support/ lib. NFCI
Summary:
This enforces layering, reduces a sprawling clangd/ directory, and makes life
easier for embedders.
Reviewers: kbobyrev
Subscribers: mgorny, ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, jfb, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D79014
2020-04-28 23:49:17 +08:00
|
|
|
#include "support/Logger.h"
|
2019-06-26 21:08:36 +08:00
|
|
|
#include "clang/AST/ASTContext.h"
|
2019-08-19 15:51:39 +08:00
|
|
|
#include "clang/AST/Decl.h"
|
2019-08-09 20:19:10 +08:00
|
|
|
#include "clang/AST/DeclCXX.h"
|
2019-09-17 00:16:03 +08:00
|
|
|
#include "clang/AST/DeclarationName.h"
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
#include "clang/AST/ExprCXX.h"
|
2019-06-26 21:08:36 +08:00
|
|
|
#include "clang/AST/RecursiveASTVisitor.h"
|
[clangd] Highlight typedefs to template parameters as template parameters
Summary:
Template parameters were handled outside `addType`, this led to lack of highlightings for typedefs
to template types.
This was never desirable, we want to highlight our typedefs as their underlying type.
Note that typedefs to more complicated types, like pointers and references are still not highlighted.
Original patch by Johan Vikström.
Reviewers: hokein, jvikstrom
Reviewed By: hokein
Subscribers: nridge, javed.absar, kristof.beyls, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66516
llvm-svn: 371379
2019-09-09 17:37:17 +08:00
|
|
|
#include "clang/AST/Type.h"
|
|
|
|
#include "clang/AST/TypeLoc.h"
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
#include "clang/Basic/LangOptions.h"
|
2019-09-09 22:33:10 +08:00
|
|
|
#include "clang/Basic/SourceLocation.h"
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
#include "clang/Basic/SourceManager.h"
|
2020-03-01 23:04:07 +08:00
|
|
|
#include "clang/Tooling/Syntax/Tokens.h"
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
#include "llvm/ADT/None.h"
|
|
|
|
#include "llvm/ADT/Optional.h"
|
2019-09-25 06:17:55 +08:00
|
|
|
#include "llvm/ADT/STLExtras.h"
|
2020-03-03 18:10:38 +08:00
|
|
|
#include "llvm/Support/Base64.h"
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
#include "llvm/Support/Casting.h"
|
2019-08-01 16:08:44 +08:00
|
|
|
#include <algorithm>
|
2019-06-26 21:08:36 +08:00
|
|
|
|
|
|
|
namespace clang {
|
|
|
|
namespace clangd {
|
|
|
|
namespace {
|
|
|
|
|
2019-09-17 00:16:03 +08:00
|
|
|
/// Some names are not written in the source code and cannot be highlighted,
|
|
|
|
/// e.g. anonymous classes. This function detects those cases.
|
|
|
|
bool canHighlightName(DeclarationName Name) {
|
|
|
|
if (Name.getNameKind() == DeclarationName::CXXConstructorName ||
|
|
|
|
Name.getNameKind() == DeclarationName::CXXUsingDirective)
|
|
|
|
return true;
|
|
|
|
auto *II = Name.getAsIdentifierInfo();
|
|
|
|
return II && !II->getName().empty();
|
|
|
|
}
|
|
|
|
|
|
|
|
llvm::Optional<HighlightingKind> kindForType(const Type *TP);
|
|
|
|
llvm::Optional<HighlightingKind> kindForDecl(const NamedDecl *D) {
|
2019-10-28 20:42:20 +08:00
|
|
|
if (auto *USD = dyn_cast<UsingShadowDecl>(D)) {
|
|
|
|
if (auto *Target = USD->getTargetDecl())
|
|
|
|
D = Target;
|
|
|
|
}
|
2019-10-15 02:26:13 +08:00
|
|
|
if (auto *TD = dyn_cast<TemplateDecl>(D)) {
|
|
|
|
if (auto *Templated = TD->getTemplatedDecl())
|
|
|
|
D = Templated;
|
|
|
|
}
|
2019-09-17 00:16:03 +08:00
|
|
|
if (auto *TD = dyn_cast<TypedefNameDecl>(D)) {
|
|
|
|
// We try to highlight typedefs as their underlying type.
|
|
|
|
if (auto K = kindForType(TD->getUnderlyingType().getTypePtrOrNull()))
|
|
|
|
return K;
|
|
|
|
// And fallback to a generic kind if this fails.
|
|
|
|
return HighlightingKind::Typedef;
|
|
|
|
}
|
|
|
|
// We highlight class decls, constructor decls and destructor decls as
|
2019-10-28 18:31:06 +08:00
|
|
|
// `Class` type. The destructor decls are handled in `VisitTagTypeLoc` (we
|
2019-09-17 00:16:03 +08:00
|
|
|
// will visit a TypeLoc where the underlying Type is a CXXRecordDecl).
|
|
|
|
if (auto *RD = llvm::dyn_cast<RecordDecl>(D)) {
|
|
|
|
// We don't want to highlight lambdas like classes.
|
|
|
|
if (RD->isLambda())
|
|
|
|
return llvm::None;
|
|
|
|
return HighlightingKind::Class;
|
|
|
|
}
|
|
|
|
if (isa<ClassTemplateDecl>(D) || isa<RecordDecl>(D) ||
|
|
|
|
isa<CXXConstructorDecl>(D))
|
|
|
|
return HighlightingKind::Class;
|
|
|
|
if (auto *MD = dyn_cast<CXXMethodDecl>(D))
|
|
|
|
return MD->isStatic() ? HighlightingKind::StaticMethod
|
|
|
|
: HighlightingKind::Method;
|
|
|
|
if (isa<FieldDecl>(D))
|
|
|
|
return HighlightingKind::Field;
|
|
|
|
if (isa<EnumDecl>(D))
|
|
|
|
return HighlightingKind::Enum;
|
|
|
|
if (isa<EnumConstantDecl>(D))
|
|
|
|
return HighlightingKind::EnumConstant;
|
|
|
|
if (isa<ParmVarDecl>(D))
|
|
|
|
return HighlightingKind::Parameter;
|
|
|
|
if (auto *VD = dyn_cast<VarDecl>(D))
|
|
|
|
return VD->isStaticDataMember()
|
|
|
|
? HighlightingKind::StaticField
|
|
|
|
: VD->isLocalVarDecl() ? HighlightingKind::LocalVariable
|
|
|
|
: HighlightingKind::Variable;
|
|
|
|
if (isa<BindingDecl>(D))
|
|
|
|
return HighlightingKind::Variable;
|
|
|
|
if (isa<FunctionDecl>(D))
|
|
|
|
return HighlightingKind::Function;
|
|
|
|
if (isa<NamespaceDecl>(D) || isa<NamespaceAliasDecl>(D) ||
|
|
|
|
isa<UsingDirectiveDecl>(D))
|
|
|
|
return HighlightingKind::Namespace;
|
|
|
|
if (isa<TemplateTemplateParmDecl>(D) || isa<TemplateTypeParmDecl>(D) ||
|
|
|
|
isa<NonTypeTemplateParmDecl>(D))
|
|
|
|
return HighlightingKind::TemplateParameter;
|
2020-01-22 02:21:08 +08:00
|
|
|
if (isa<ConceptDecl>(D))
|
|
|
|
return HighlightingKind::Concept;
|
2019-09-17 00:16:03 +08:00
|
|
|
return llvm::None;
|
|
|
|
}
|
|
|
|
llvm::Optional<HighlightingKind> kindForType(const Type *TP) {
|
|
|
|
if (!TP)
|
|
|
|
return llvm::None;
|
|
|
|
if (TP->isBuiltinType()) // Builtins are special, they do not have decls.
|
|
|
|
return HighlightingKind::Primitive;
|
|
|
|
if (auto *TD = dyn_cast<TemplateTypeParmType>(TP))
|
|
|
|
return kindForDecl(TD->getDecl());
|
|
|
|
if (auto *TD = TP->getAsTagDecl())
|
|
|
|
return kindForDecl(TD);
|
|
|
|
return llvm::None;
|
|
|
|
}
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
|
|
|
|
llvm::Optional<HighlightingKind> kindForReference(const ReferenceLoc &R) {
|
2019-10-15 02:26:13 +08:00
|
|
|
llvm::Optional<HighlightingKind> Result;
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
for (const NamedDecl *Decl : R.Targets) {
|
|
|
|
if (!canHighlightName(Decl->getDeclName()))
|
|
|
|
return llvm::None;
|
2019-10-15 02:26:13 +08:00
|
|
|
auto Kind = kindForDecl(Decl);
|
|
|
|
if (!Kind || (Result && Kind != Result))
|
|
|
|
return llvm::None;
|
|
|
|
Result = Kind;
|
|
|
|
}
|
|
|
|
return Result;
|
|
|
|
}
|
2019-09-17 00:16:03 +08:00
|
|
|
|
2020-03-01 23:04:07 +08:00
|
|
|
// For a macro usage `DUMP(foo)`, we want:
|
|
|
|
// - DUMP --> "macro"
|
|
|
|
// - foo --> "variable".
|
|
|
|
SourceLocation getHighlightableSpellingToken(SourceLocation L,
|
|
|
|
const SourceManager &SM) {
|
|
|
|
if (L.isFileID())
|
|
|
|
return SM.isWrittenInMainFile(L) ? L : SourceLocation{};
|
|
|
|
// Tokens expanded from the macro body contribute no highlightings.
|
|
|
|
if (!SM.isMacroArgExpansion(L))
|
|
|
|
return {};
|
|
|
|
// Tokens expanded from macro args are potentially highlightable.
|
|
|
|
return getHighlightableSpellingToken(SM.getImmediateSpellingLoc(L), SM);
|
|
|
|
}
|
|
|
|
|
2020-03-20 03:42:10 +08:00
|
|
|
unsigned evaluateHighlightPriority(HighlightingKind Kind) {
|
|
|
|
enum HighlightPriority { Dependent = 0, Resolved = 1 };
|
|
|
|
return Kind == HighlightingKind::DependentType ||
|
|
|
|
Kind == HighlightingKind::DependentName
|
|
|
|
? Dependent
|
|
|
|
: Resolved;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Sometimes we get conflicts between findExplicitReferences() returning
|
|
|
|
// a heuristic result for a dependent name (e.g. Method) and
|
|
|
|
// CollectExtraHighlighting returning a fallback dependent highlighting (e.g.
|
|
|
|
// DependentName). In such cases, resolve the conflict in favour of the
|
|
|
|
// resolved (non-dependent) highlighting.
|
|
|
|
// With macros we can get other conflicts (if a spelled token has multiple
|
|
|
|
// expansions with different token types) which we can't usefully resolve.
|
|
|
|
llvm::Optional<HighlightingToken>
|
|
|
|
resolveConflict(ArrayRef<HighlightingToken> Tokens) {
|
|
|
|
if (Tokens.size() == 1)
|
|
|
|
return Tokens[0];
|
|
|
|
|
|
|
|
if (Tokens.size() != 2)
|
|
|
|
return llvm::None;
|
|
|
|
|
|
|
|
unsigned Priority1 = evaluateHighlightPriority(Tokens[0].Kind);
|
|
|
|
unsigned Priority2 = evaluateHighlightPriority(Tokens[1].Kind);
|
|
|
|
if (Priority1 == Priority2)
|
|
|
|
return llvm::None;
|
|
|
|
return Priority1 > Priority2 ? Tokens[0] : Tokens[1];
|
|
|
|
}
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
/// Consumes source locations and maps them to text ranges for highlightings.
|
|
|
|
class HighlightingsBuilder {
|
2019-06-26 21:08:36 +08:00
|
|
|
public:
|
2020-03-01 23:04:07 +08:00
|
|
|
HighlightingsBuilder(const ParsedAST &AST)
|
|
|
|
: TB(AST.getTokens()), SourceMgr(AST.getSourceManager()),
|
|
|
|
LangOpts(AST.getLangOpts()) {}
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
|
|
|
|
void addToken(HighlightingToken T) { Tokens.push_back(T); }
|
|
|
|
|
|
|
|
void addToken(SourceLocation Loc, HighlightingKind Kind) {
|
2020-03-01 23:04:07 +08:00
|
|
|
Loc = getHighlightableSpellingToken(Loc, SourceMgr);
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
if (Loc.isInvalid())
|
|
|
|
return;
|
2020-03-01 23:04:07 +08:00
|
|
|
const auto *Tok = TB.spelledTokenAt(Loc);
|
|
|
|
assert(Tok);
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
|
2020-03-01 23:04:07 +08:00
|
|
|
auto Range = halfOpenToRange(SourceMgr,
|
|
|
|
Tok->range(SourceMgr).toCharRange(SourceMgr));
|
|
|
|
Tokens.push_back(HighlightingToken{Kind, std::move(Range)});
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
}
|
|
|
|
|
2019-09-25 06:17:55 +08:00
|
|
|
std::vector<HighlightingToken> collect(ParsedAST &AST) && {
|
2019-07-15 23:08:27 +08:00
|
|
|
// Initializer lists can give duplicates of tokens, therefore all tokens
|
|
|
|
// must be deduplicated.
|
2019-08-01 16:08:44 +08:00
|
|
|
llvm::sort(Tokens);
|
2019-07-15 23:08:27 +08:00
|
|
|
auto Last = std::unique(Tokens.begin(), Tokens.end());
|
|
|
|
Tokens.erase(Last, Tokens.end());
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
|
2019-08-20 00:27:49 +08:00
|
|
|
// Macros can give tokens that have the same source range but conflicting
|
|
|
|
// kinds. In this case all tokens sharing this source range should be
|
|
|
|
// removed.
|
|
|
|
std::vector<HighlightingToken> NonConflicting;
|
|
|
|
NonConflicting.reserve(Tokens.size());
|
|
|
|
for (ArrayRef<HighlightingToken> TokRef = Tokens; !TokRef.empty();) {
|
|
|
|
ArrayRef<HighlightingToken> Conflicting =
|
|
|
|
TokRef.take_while([&](const HighlightingToken &T) {
|
|
|
|
// TokRef is guaranteed at least one element here because otherwise
|
|
|
|
// this predicate would never fire.
|
|
|
|
return T.R == TokRef.front().R;
|
|
|
|
});
|
2020-03-20 03:42:10 +08:00
|
|
|
if (auto Resolved = resolveConflict(Conflicting))
|
|
|
|
NonConflicting.push_back(*Resolved);
|
2019-08-20 00:27:49 +08:00
|
|
|
// TokRef[Conflicting.size()] is the next token with a different range (or
|
|
|
|
// the end of the Tokens).
|
|
|
|
TokRef = TokRef.drop_front(Conflicting.size());
|
|
|
|
}
|
2019-09-25 06:17:55 +08:00
|
|
|
// Add tokens indicating lines skipped by the preprocessor.
|
|
|
|
for (const Range &R : AST.getMacros().SkippedRanges) {
|
|
|
|
// Create one token for each line in the skipped range, so it works
|
|
|
|
// with line-based diffing.
|
|
|
|
assert(R.start.line <= R.end.line);
|
2020-07-10 05:56:06 +08:00
|
|
|
for (int Line = R.start.line; Line <= R.end.line; ++Line) {
|
2019-09-25 06:17:55 +08:00
|
|
|
// Don't bother computing the offset for the end of the line, just use
|
|
|
|
// zero. The client will treat this highlighting kind specially, and
|
|
|
|
// highlight the entire line visually (i.e. not just to where the text
|
|
|
|
// on the line ends, but to the end of the screen).
|
|
|
|
NonConflicting.push_back({HighlightingKind::InactiveCode,
|
|
|
|
{Position{Line, 0}, Position{Line, 0}}});
|
|
|
|
}
|
|
|
|
}
|
|
|
|
// Re-sort the tokens because that's what the diffing expects.
|
|
|
|
llvm::sort(NonConflicting);
|
2019-08-20 00:27:49 +08:00
|
|
|
return NonConflicting;
|
2019-06-26 21:08:36 +08:00
|
|
|
}
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
private:
|
2020-03-01 23:04:07 +08:00
|
|
|
const syntax::TokenBuffer &TB;
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
const SourceManager &SourceMgr;
|
|
|
|
const LangOptions &LangOpts;
|
|
|
|
std::vector<HighlightingToken> Tokens;
|
|
|
|
};
|
2019-07-05 21:06:03 +08:00
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
/// Produces highlightings, which are not captured by findExplicitReferences,
|
|
|
|
/// e.g. highlights dependent names and 'auto' as the underlying type.
|
|
|
|
class CollectExtraHighlightings
|
|
|
|
: public RecursiveASTVisitor<CollectExtraHighlightings> {
|
|
|
|
public:
|
|
|
|
CollectExtraHighlightings(HighlightingsBuilder &H) : H(H) {}
|
2019-10-28 20:42:20 +08:00
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
bool VisitDecltypeTypeLoc(DecltypeTypeLoc L) {
|
|
|
|
if (auto K = kindForType(L.getTypePtr()))
|
|
|
|
H.addToken(L.getBeginLoc(), *K);
|
2019-07-16 21:23:12 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
bool VisitDeclaratorDecl(DeclaratorDecl *D) {
|
|
|
|
auto *AT = D->getType()->getContainedAutoType();
|
|
|
|
if (!AT)
|
|
|
|
return true;
|
|
|
|
if (auto K = kindForType(AT->getDeducedType().getTypePtrOrNull()))
|
|
|
|
H.addToken(D->getTypeSpecStartLoc(), *K);
|
2019-07-18 17:56:38 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
bool VisitOverloadExpr(OverloadExpr *E) {
|
|
|
|
if (!E->decls().empty())
|
|
|
|
return true; // handled by findExplicitReferences.
|
|
|
|
H.addToken(E->getNameLoc(), HighlightingKind::DependentName);
|
2019-07-18 17:56:38 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
bool VisitCXXDependentScopeMemberExpr(CXXDependentScopeMemberExpr *E) {
|
|
|
|
H.addToken(E->getMemberNameInfo().getLoc(),
|
|
|
|
HighlightingKind::DependentName);
|
2019-10-28 18:31:06 +08:00
|
|
|
return true;
|
2019-09-17 00:16:03 +08:00
|
|
|
}
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
bool VisitDependentScopeDeclRefExpr(DependentScopeDeclRefExpr *E) {
|
|
|
|
H.addToken(E->getNameInfo().getLoc(), HighlightingKind::DependentName);
|
2019-09-17 00:16:03 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2019-10-28 18:31:06 +08:00
|
|
|
bool VisitDependentNameTypeLoc(DependentNameTypeLoc L) {
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
H.addToken(L.getNameLoc(), HighlightingKind::DependentType);
|
2019-08-12 15:45:12 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2019-12-06 03:28:56 +08:00
|
|
|
bool VisitDependentTemplateSpecializationTypeLoc(
|
|
|
|
DependentTemplateSpecializationTypeLoc L) {
|
|
|
|
H.addToken(L.getTemplateNameLoc(), HighlightingKind::DependentType);
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
// findExplicitReferences will walk nested-name-specifiers and
|
|
|
|
// find anything that can be resolved to a Decl. However, non-leaf
|
|
|
|
// components of nested-name-specifiers which are dependent names
|
|
|
|
// (kind "Identifier") cannot be resolved to a decl, so we visit
|
|
|
|
// them here.
|
|
|
|
bool TraverseNestedNameSpecifierLoc(NestedNameSpecifierLoc Q) {
|
|
|
|
if (NestedNameSpecifier *NNS = Q.getNestedNameSpecifier()) {
|
|
|
|
if (NNS->getKind() == NestedNameSpecifier::Identifier)
|
|
|
|
H.addToken(Q.getLocalBeginLoc(), HighlightingKind::DependentType);
|
|
|
|
}
|
|
|
|
return RecursiveASTVisitor::TraverseNestedNameSpecifierLoc(Q);
|
|
|
|
}
|
|
|
|
|
2019-08-12 15:45:12 +08:00
|
|
|
private:
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
HighlightingsBuilder &H;
|
2019-06-26 21:08:36 +08:00
|
|
|
};
|
|
|
|
|
2019-07-04 15:53:12 +08:00
|
|
|
void write32be(uint32_t I, llvm::raw_ostream &OS) {
|
|
|
|
std::array<char, 4> Buf;
|
|
|
|
llvm::support::endian::write32be(Buf.data(), I);
|
|
|
|
OS.write(Buf.data(), Buf.size());
|
|
|
|
}
|
|
|
|
|
|
|
|
void write16be(uint16_t I, llvm::raw_ostream &OS) {
|
|
|
|
std::array<char, 2> Buf;
|
|
|
|
llvm::support::endian::write16be(Buf.data(), I);
|
|
|
|
OS.write(Buf.data(), Buf.size());
|
|
|
|
}
|
2019-08-01 16:08:44 +08:00
|
|
|
|
|
|
|
// Get the highlightings on \c Line where the first entry of line is at \c
|
|
|
|
// StartLineIt. If it is not at \c StartLineIt an empty vector is returned.
|
|
|
|
ArrayRef<HighlightingToken>
|
|
|
|
takeLine(ArrayRef<HighlightingToken> AllTokens,
|
|
|
|
ArrayRef<HighlightingToken>::iterator StartLineIt, int Line) {
|
|
|
|
return ArrayRef<HighlightingToken>(StartLineIt, AllTokens.end())
|
|
|
|
.take_while([Line](const HighlightingToken &Token) {
|
|
|
|
return Token.R.start.line == Line;
|
|
|
|
});
|
|
|
|
}
|
2019-06-26 21:08:36 +08:00
|
|
|
} // namespace
|
|
|
|
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
std::vector<HighlightingToken> getSemanticHighlightings(ParsedAST &AST) {
|
|
|
|
auto &C = AST.getASTContext();
|
|
|
|
// Add highlightings for AST nodes.
|
2020-03-01 23:04:07 +08:00
|
|
|
HighlightingsBuilder Builder(AST);
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
// Highlight 'decltype' and 'auto' as their underlying types.
|
|
|
|
CollectExtraHighlightings(Builder).TraverseAST(C);
|
|
|
|
// Highlight all decls and references coming from the AST.
|
|
|
|
findExplicitReferences(C, [&](ReferenceLoc R) {
|
|
|
|
if (auto Kind = kindForReference(R))
|
|
|
|
Builder.addToken(R.NameLoc, *Kind);
|
|
|
|
});
|
2019-11-07 19:14:38 +08:00
|
|
|
// Add highlightings for macro references.
|
|
|
|
for (const auto &SIDToRefs : AST.getMacros().MacroRefs) {
|
|
|
|
for (const auto &M : SIDToRefs.second)
|
|
|
|
Builder.addToken({HighlightingKind::Macro, M});
|
|
|
|
}
|
|
|
|
for (const auto &M : AST.getMacros().UnknownMacros)
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
Builder.addToken({HighlightingKind::Macro, M});
|
2019-11-07 19:14:38 +08:00
|
|
|
|
2019-09-25 06:17:55 +08:00
|
|
|
return std::move(Builder).collect(AST);
|
[clangd] Implement semantic highlightings via findExplicitReferences
Summary:
To keep the logic of finding locations of interesting AST nodes in one
place.
The advantage is better coverage of various AST nodes, both now and in
the future: as new nodes get added to `findExplicitReferences`, semantic
highlighting will automatically pick them up.
The drawback of this change is that we have to traverse declarations
inside our file twice in order to highlight dependent names, 'auto'
and 'decltype'. Hopefully, this should not affect the actual latency
too much, most time should be spent in building the AST and not
traversing it.
Reviewers: hokein
Reviewed By: hokein
Subscribers: nridge, merge_guards_bot, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69673
2019-11-06 02:06:12 +08:00
|
|
|
}
|
|
|
|
|
2019-09-09 16:47:05 +08:00
|
|
|
llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, HighlightingKind K) {
|
|
|
|
switch (K) {
|
|
|
|
case HighlightingKind::Variable:
|
|
|
|
return OS << "Variable";
|
|
|
|
case HighlightingKind::LocalVariable:
|
|
|
|
return OS << "LocalVariable";
|
|
|
|
case HighlightingKind::Parameter:
|
|
|
|
return OS << "Parameter";
|
|
|
|
case HighlightingKind::Function:
|
|
|
|
return OS << "Function";
|
|
|
|
case HighlightingKind::Method:
|
|
|
|
return OS << "Method";
|
|
|
|
case HighlightingKind::StaticMethod:
|
|
|
|
return OS << "StaticMethod";
|
|
|
|
case HighlightingKind::Field:
|
|
|
|
return OS << "Field";
|
|
|
|
case HighlightingKind::StaticField:
|
|
|
|
return OS << "StaticField";
|
|
|
|
case HighlightingKind::Class:
|
|
|
|
return OS << "Class";
|
|
|
|
case HighlightingKind::Enum:
|
|
|
|
return OS << "Enum";
|
|
|
|
case HighlightingKind::EnumConstant:
|
|
|
|
return OS << "EnumConstant";
|
2019-09-09 22:33:10 +08:00
|
|
|
case HighlightingKind::Typedef:
|
|
|
|
return OS << "Typedef";
|
2019-10-15 02:26:13 +08:00
|
|
|
case HighlightingKind::DependentType:
|
|
|
|
return OS << "DependentType";
|
|
|
|
case HighlightingKind::DependentName:
|
|
|
|
return OS << "DependentName";
|
2019-09-09 16:47:05 +08:00
|
|
|
case HighlightingKind::Namespace:
|
|
|
|
return OS << "Namespace";
|
|
|
|
case HighlightingKind::TemplateParameter:
|
|
|
|
return OS << "TemplateParameter";
|
2020-01-22 02:21:08 +08:00
|
|
|
case HighlightingKind::Concept:
|
|
|
|
return OS << "Concept";
|
2019-09-09 16:47:05 +08:00
|
|
|
case HighlightingKind::Primitive:
|
|
|
|
return OS << "Primitive";
|
|
|
|
case HighlightingKind::Macro:
|
|
|
|
return OS << "Macro";
|
2019-09-25 06:17:55 +08:00
|
|
|
case HighlightingKind::InactiveCode:
|
|
|
|
return OS << "InactiveCode";
|
2019-09-09 16:47:05 +08:00
|
|
|
}
|
2019-09-09 16:57:17 +08:00
|
|
|
llvm_unreachable("invalid HighlightingKind");
|
2019-09-09 16:47:05 +08:00
|
|
|
}
|
|
|
|
|
2019-08-01 16:08:44 +08:00
|
|
|
std::vector<LineHighlightings>
|
|
|
|
diffHighlightings(ArrayRef<HighlightingToken> New,
|
2019-08-26 16:38:45 +08:00
|
|
|
ArrayRef<HighlightingToken> Old) {
|
|
|
|
assert(std::is_sorted(New.begin(), New.end()) &&
|
|
|
|
"New must be a sorted vector");
|
|
|
|
assert(std::is_sorted(Old.begin(), Old.end()) &&
|
|
|
|
"Old must be a sorted vector");
|
2019-08-01 16:08:44 +08:00
|
|
|
|
|
|
|
// FIXME: There's an edge case when tokens span multiple lines. If the first
|
|
|
|
// token on the line started on a line above the current one and the rest of
|
|
|
|
// the line is the equal to the previous one than we will remove all
|
|
|
|
// highlights but the ones for the token spanning multiple lines. This means
|
|
|
|
// that when we get into the LSP layer the only highlights that will be
|
|
|
|
// visible are the ones for the token spanning multiple lines.
|
|
|
|
// Example:
|
|
|
|
// EndOfMultilineToken Token Token Token
|
|
|
|
// If "Token Token Token" don't differ from previously the line is
|
|
|
|
// incorrectly removed. Suggestion to fix is to separate any multiline tokens
|
|
|
|
// into one token for every line it covers. This requires reading from the
|
|
|
|
// file buffer to figure out the length of each line though.
|
|
|
|
std::vector<LineHighlightings> DiffedLines;
|
|
|
|
// ArrayRefs to the current line in the highlightings.
|
|
|
|
ArrayRef<HighlightingToken> NewLine(New.begin(),
|
2019-08-01 17:08:41 +08:00
|
|
|
/*length*/ static_cast<size_t>(0));
|
2019-08-01 16:08:44 +08:00
|
|
|
ArrayRef<HighlightingToken> OldLine(Old.begin(),
|
2019-08-01 17:08:41 +08:00
|
|
|
/*length*/ static_cast<size_t>(0));
|
2019-08-01 16:08:44 +08:00
|
|
|
auto NewEnd = New.end();
|
|
|
|
auto OldEnd = Old.end();
|
|
|
|
auto NextLineNumber = [&]() {
|
|
|
|
int NextNew = NewLine.end() != NewEnd ? NewLine.end()->R.start.line
|
2019-08-28 03:39:11 +08:00
|
|
|
: std::numeric_limits<int>::max();
|
2019-08-01 16:08:44 +08:00
|
|
|
int NextOld = OldLine.end() != OldEnd ? OldLine.end()->R.start.line
|
2019-08-28 03:39:11 +08:00
|
|
|
: std::numeric_limits<int>::max();
|
2019-08-01 16:08:44 +08:00
|
|
|
return std::min(NextNew, NextOld);
|
|
|
|
};
|
|
|
|
|
2019-08-26 16:38:45 +08:00
|
|
|
for (int LineNumber = 0; NewLine.end() < NewEnd || OldLine.end() < OldEnd;
|
2019-08-01 16:08:44 +08:00
|
|
|
LineNumber = NextLineNumber()) {
|
|
|
|
NewLine = takeLine(New, NewLine.end(), LineNumber);
|
|
|
|
OldLine = takeLine(Old, OldLine.end(), LineNumber);
|
2019-09-25 06:17:55 +08:00
|
|
|
if (NewLine != OldLine) {
|
|
|
|
DiffedLines.push_back({LineNumber, NewLine, /*IsInactive=*/false});
|
|
|
|
|
|
|
|
// Turn a HighlightingKind::InactiveCode token into the IsInactive flag.
|
|
|
|
auto &AddedLine = DiffedLines.back();
|
|
|
|
llvm::erase_if(AddedLine.Tokens, [&](const HighlightingToken &T) {
|
|
|
|
if (T.Kind == HighlightingKind::InactiveCode) {
|
|
|
|
AddedLine.IsInactive = true;
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
return false;
|
|
|
|
});
|
|
|
|
}
|
2019-08-01 16:08:44 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
return DiffedLines;
|
|
|
|
}
|
|
|
|
|
|
|
|
bool operator==(const HighlightingToken &L, const HighlightingToken &R) {
|
|
|
|
return std::tie(L.R, L.Kind) == std::tie(R.R, R.Kind);
|
|
|
|
}
|
|
|
|
bool operator<(const HighlightingToken &L, const HighlightingToken &R) {
|
|
|
|
return std::tie(L.R, L.Kind) < std::tie(R.R, R.Kind);
|
|
|
|
}
|
|
|
|
bool operator==(const LineHighlightings &L, const LineHighlightings &R) {
|
|
|
|
return std::tie(L.Line, L.Tokens) == std::tie(R.Line, R.Tokens);
|
2019-06-26 21:08:36 +08:00
|
|
|
}
|
|
|
|
|
2020-03-24 09:24:47 +08:00
|
|
|
std::vector<SemanticToken>
|
|
|
|
toSemanticTokens(llvm::ArrayRef<HighlightingToken> Tokens) {
|
|
|
|
assert(std::is_sorted(Tokens.begin(), Tokens.end()));
|
|
|
|
std::vector<SemanticToken> Result;
|
|
|
|
const HighlightingToken *Last = nullptr;
|
|
|
|
for (const HighlightingToken &Tok : Tokens) {
|
|
|
|
// FIXME: support inactive code - we need to provide the actual bounds.
|
|
|
|
if (Tok.Kind == HighlightingKind::InactiveCode)
|
|
|
|
continue;
|
|
|
|
Result.emplace_back();
|
|
|
|
SemanticToken &Out = Result.back();
|
|
|
|
// deltaStart/deltaLine are relative if possible.
|
|
|
|
if (Last) {
|
|
|
|
assert(Tok.R.start.line >= Last->R.start.line);
|
|
|
|
Out.deltaLine = Tok.R.start.line - Last->R.start.line;
|
|
|
|
if (Out.deltaLine == 0) {
|
|
|
|
assert(Tok.R.start.character >= Last->R.start.character);
|
|
|
|
Out.deltaStart = Tok.R.start.character - Last->R.start.character;
|
|
|
|
} else {
|
|
|
|
Out.deltaStart = Tok.R.start.character;
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
Out.deltaLine = Tok.R.start.line;
|
|
|
|
Out.deltaStart = Tok.R.start.character;
|
|
|
|
}
|
|
|
|
assert(Tok.R.end.line == Tok.R.start.line);
|
|
|
|
Out.length = Tok.R.end.character - Tok.R.start.character;
|
|
|
|
Out.tokenType = static_cast<unsigned>(Tok.Kind);
|
|
|
|
|
|
|
|
Last = &Tok;
|
|
|
|
}
|
|
|
|
return Result;
|
|
|
|
}
|
|
|
|
llvm::StringRef toSemanticTokenType(HighlightingKind Kind) {
|
|
|
|
switch (Kind) {
|
|
|
|
case HighlightingKind::Variable:
|
|
|
|
case HighlightingKind::LocalVariable:
|
|
|
|
case HighlightingKind::StaticField:
|
|
|
|
return "variable";
|
|
|
|
case HighlightingKind::Parameter:
|
|
|
|
return "parameter";
|
|
|
|
case HighlightingKind::Function:
|
|
|
|
return "function";
|
|
|
|
case HighlightingKind::Method:
|
|
|
|
return "member";
|
|
|
|
case HighlightingKind::StaticMethod:
|
|
|
|
// FIXME: better function/member with static modifier?
|
|
|
|
return "function";
|
|
|
|
case HighlightingKind::Field:
|
|
|
|
return "member";
|
|
|
|
case HighlightingKind::Class:
|
|
|
|
return "class";
|
|
|
|
case HighlightingKind::Enum:
|
|
|
|
return "enum";
|
|
|
|
case HighlightingKind::EnumConstant:
|
|
|
|
return "enumConstant"; // nonstandard
|
|
|
|
case HighlightingKind::Typedef:
|
|
|
|
return "type";
|
|
|
|
case HighlightingKind::DependentType:
|
|
|
|
return "dependent"; // nonstandard
|
|
|
|
case HighlightingKind::DependentName:
|
|
|
|
return "dependent"; // nonstandard
|
|
|
|
case HighlightingKind::Namespace:
|
|
|
|
return "namespace";
|
|
|
|
case HighlightingKind::TemplateParameter:
|
|
|
|
return "typeParameter";
|
|
|
|
case HighlightingKind::Concept:
|
|
|
|
return "concept"; // nonstandard
|
|
|
|
case HighlightingKind::Primitive:
|
|
|
|
return "type";
|
|
|
|
case HighlightingKind::Macro:
|
|
|
|
return "macro";
|
|
|
|
case HighlightingKind::InactiveCode:
|
|
|
|
return "comment";
|
|
|
|
}
|
2020-04-01 21:08:25 +08:00
|
|
|
llvm_unreachable("unhandled HighlightingKind");
|
2020-03-24 09:24:47 +08:00
|
|
|
}
|
|
|
|
|
2020-03-24 07:31:14 +08:00
|
|
|
std::vector<TheiaSemanticHighlightingInformation>
|
|
|
|
toTheiaSemanticHighlightingInformation(
|
|
|
|
llvm::ArrayRef<LineHighlightings> Tokens) {
|
2019-07-04 15:53:12 +08:00
|
|
|
if (Tokens.size() == 0)
|
|
|
|
return {};
|
|
|
|
|
|
|
|
// FIXME: Tokens might be multiple lines long (block comments) in this case
|
|
|
|
// this needs to add multiple lines for those tokens.
|
2020-03-24 07:31:14 +08:00
|
|
|
std::vector<TheiaSemanticHighlightingInformation> Lines;
|
2019-08-01 16:08:44 +08:00
|
|
|
Lines.reserve(Tokens.size());
|
|
|
|
for (const auto &Line : Tokens) {
|
2019-07-04 15:53:12 +08:00
|
|
|
llvm::SmallVector<char, 128> LineByteTokens;
|
|
|
|
llvm::raw_svector_ostream OS(LineByteTokens);
|
2019-08-01 16:08:44 +08:00
|
|
|
for (const auto &Token : Line.Tokens) {
|
2019-07-04 15:53:12 +08:00
|
|
|
// Writes the token to LineByteTokens in the byte format specified by the
|
|
|
|
// LSP proposal. Described below.
|
|
|
|
// |<---- 4 bytes ---->|<-- 2 bytes -->|<--- 2 bytes -->|
|
|
|
|
// | character | length | index |
|
|
|
|
|
|
|
|
write32be(Token.R.start.character, OS);
|
|
|
|
write16be(Token.R.end.character - Token.R.start.character, OS);
|
|
|
|
write16be(static_cast<int>(Token.Kind), OS);
|
|
|
|
}
|
|
|
|
|
2019-09-25 06:17:55 +08:00
|
|
|
Lines.push_back({Line.Line, encodeBase64(LineByteTokens), Line.IsInactive});
|
2019-07-04 15:53:12 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
return Lines;
|
|
|
|
}
|
|
|
|
|
2019-07-04 20:27:21 +08:00
|
|
|
llvm::StringRef toTextMateScope(HighlightingKind Kind) {
|
2019-07-04 15:53:12 +08:00
|
|
|
// FIXME: Add scopes for C and Objective C.
|
2019-07-04 20:27:21 +08:00
|
|
|
switch (Kind) {
|
|
|
|
case HighlightingKind::Function:
|
|
|
|
return "entity.name.function.cpp";
|
2019-07-15 16:12:21 +08:00
|
|
|
case HighlightingKind::Method:
|
|
|
|
return "entity.name.function.method.cpp";
|
2019-08-30 11:37:24 +08:00
|
|
|
case HighlightingKind::StaticMethod:
|
|
|
|
return "entity.name.function.method.static.cpp";
|
2019-07-04 20:27:21 +08:00
|
|
|
case HighlightingKind::Variable:
|
2019-07-15 16:12:21 +08:00
|
|
|
return "variable.other.cpp";
|
2019-08-28 03:39:11 +08:00
|
|
|
case HighlightingKind::LocalVariable:
|
|
|
|
return "variable.other.local.cpp";
|
2019-08-19 15:51:39 +08:00
|
|
|
case HighlightingKind::Parameter:
|
|
|
|
return "variable.parameter.cpp";
|
2019-07-15 16:12:21 +08:00
|
|
|
case HighlightingKind::Field:
|
|
|
|
return "variable.other.field.cpp";
|
2019-08-30 11:37:24 +08:00
|
|
|
case HighlightingKind::StaticField:
|
|
|
|
return "variable.other.field.static.cpp";
|
2019-07-10 16:41:25 +08:00
|
|
|
case HighlightingKind::Class:
|
|
|
|
return "entity.name.type.class.cpp";
|
|
|
|
case HighlightingKind::Enum:
|
|
|
|
return "entity.name.type.enum.cpp";
|
2019-07-15 15:41:12 +08:00
|
|
|
case HighlightingKind::EnumConstant:
|
|
|
|
return "variable.other.enummember.cpp";
|
2019-09-09 22:33:10 +08:00
|
|
|
case HighlightingKind::Typedef:
|
|
|
|
return "entity.name.type.typedef.cpp";
|
2019-10-15 02:26:13 +08:00
|
|
|
case HighlightingKind::DependentType:
|
|
|
|
return "entity.name.type.dependent.cpp";
|
|
|
|
case HighlightingKind::DependentName:
|
|
|
|
return "entity.name.other.dependent.cpp";
|
2019-07-11 17:29:16 +08:00
|
|
|
case HighlightingKind::Namespace:
|
|
|
|
return "entity.name.namespace.cpp";
|
2019-07-18 17:56:38 +08:00
|
|
|
case HighlightingKind::TemplateParameter:
|
|
|
|
return "entity.name.type.template.cpp";
|
2020-01-22 02:21:08 +08:00
|
|
|
case HighlightingKind::Concept:
|
|
|
|
return "entity.name.type.concept.cpp";
|
2019-08-08 21:10:30 +08:00
|
|
|
case HighlightingKind::Primitive:
|
|
|
|
return "storage.type.primitive.cpp";
|
2019-08-30 23:47:27 +08:00
|
|
|
case HighlightingKind::Macro:
|
|
|
|
return "entity.name.function.preprocessor.cpp";
|
2019-09-25 06:17:55 +08:00
|
|
|
case HighlightingKind::InactiveCode:
|
|
|
|
return "meta.disabled";
|
2019-07-04 20:27:21 +08:00
|
|
|
}
|
|
|
|
llvm_unreachable("unhandled HighlightingKind");
|
2019-07-04 15:53:12 +08:00
|
|
|
}
|
|
|
|
|
[clangd] Support textDocument/semanticTokens/edits
Summary:
This returns incremental highlights as a set of edits against the
previous highlights.
Server-side, we compute the full set of highlights, this just saves
wire-format size.
For now, the diff used is trivial: everything from the first change to
the last change is sent as a single edit.
The wire format is grungy - the replacement offset/length refer to
positions in the encoded array instead of the logical list of tokens.
We use token-oriented structs and translating to LSP forms when serializing.
This departs from LSP (but is consistent with semanticTokens today).
Tested in VSCode insiders (with a patched client to enable experimental
features).
Reviewers: hokein
Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77225
2020-04-01 22:21:44 +08:00
|
|
|
std::vector<SemanticTokensEdit>
|
|
|
|
diffTokens(llvm::ArrayRef<SemanticToken> Old,
|
|
|
|
llvm::ArrayRef<SemanticToken> New) {
|
|
|
|
// For now, just replace everything from the first-last modification.
|
|
|
|
// FIXME: use a real diff instead, this is bad with include-insertion.
|
|
|
|
|
|
|
|
unsigned Offset = 0;
|
|
|
|
while (!Old.empty() && !New.empty() && Old.front() == New.front()) {
|
|
|
|
++Offset;
|
|
|
|
Old = Old.drop_front();
|
|
|
|
New = New.drop_front();
|
|
|
|
}
|
|
|
|
while (!Old.empty() && !New.empty() && Old.back() == New.back()) {
|
|
|
|
Old = Old.drop_back();
|
|
|
|
New = New.drop_back();
|
|
|
|
}
|
|
|
|
|
|
|
|
if (Old.empty() && New.empty())
|
|
|
|
return {};
|
|
|
|
SemanticTokensEdit Edit;
|
|
|
|
Edit.startToken = Offset;
|
|
|
|
Edit.deleteTokens = Old.size();
|
|
|
|
Edit.tokens = New;
|
|
|
|
return {std::move(Edit)};
|
|
|
|
}
|
|
|
|
|
2019-06-26 21:08:36 +08:00
|
|
|
} // namespace clangd
|
|
|
|
} // namespace clang
|