Summary:
The exclusive-claim model is successful at resolving conflicts over tokens
between parent/child or siblings. However claims at the spelled-token
level do the wrong thing for macro expansions, where siblings can be
equally associated with the macro invocation.
Moreover, any model that only uses the endpoints in a range can fail when
a macro invocation occurs inside the node.
To address this, we use the existing TokenBuffer in more depth.
Claims are expressed in terms of expanded tokens, so there is no need to worry
about macros, includes etc.
Once we know which expanded tokens were claimed, they are mapped onto
spelled tokens for hit-testing.
This mapping is fairly flexible, currently the handling of macros is
pretty simple (map macro args onto spellings, other macro expansions onto the
macro name token).
This mapping is in principle token-by-token for correctness (though
there's some batching for performance).
The aggregation of the selection enum is now more principled as we need to be
able to aggregate several hit-test results together.
For simplicity i removed the ability to determine selectedness of TUDecl.
(That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very
important or worth the complexity any longer).
The expandedTokens(SourceLocation) helper could be added locally, but seems to
make sense on TokenBuffer.
Fixes https://github.com/clangd/clangd/issues/202
Fixes https://github.com/clangd/clangd/issues/126
Reviewers: hokein
Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70512
Summary:
Returns the first token in every mapping where the token is an identifier.
This API is required to be able to highlight macro expansions in clangd.
Reviewers: hokein, ilya-biryukov
Subscribers: kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66470
llvm-svn: 369385
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368942
Summary:
Preivously we would only discard it if we failed to parse parameter lists.
If we do not consume the body, parser sees tokens inside directive. In
turn, this leads to spurious diagnostics and a crash in TokenBuffer, see
the added tests.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65517
llvm-svn: 367530
Summary:
This change makes sure we have a single mapping for each macro expansion,
even if the result of expansion was empty.
To achieve that, we take information from PPCallbacks::MacroExpands into
account. Previously we relied only on source locations of expanded tokens.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62953
llvm-svn: 364236
Summary: Used in clangd for a code tweak that expands a macro.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62954
llvm-svn: 363698
Summary:
TokenBuffer stores the list of tokens for a file obtained after
preprocessing. This is a base building block for syntax trees,
see [1] for the full proposal on syntax trees.
This commits also starts a new sub-library of ClangTooling, which
would be the home for the syntax trees and syntax-tree-based refactoring
utilities.
[1]: https://lists.llvm.org/pipermail/cfe-dev/2019-February/061414.html
Reviewers: gribozavr, sammccall
Reviewed By: sammccall
Subscribers: mgrang, riccibruno, Eugene.Zelenko, mgorny, jdoerfert, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D59887
llvm-svn: 361148
Sam McCall