Commit Graph

473 Commits

Author SHA1 Message Date
Benjamin Kramer 757bdc64d3 Fix clang unnittest build with GCC 5 2020-01-29 10:30:36 +01:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Yitzhak Mandelbaum b9d2bf38e8 [libTooling] Fix bug in Stencil handling of macro ranges
Summary: Currently, an attempt to rewrite source code inside a macro expansion succeeds, but results in empty text, rather than failing with an error.  This patch restructures to the code to explicitly validate ranges before attempting to edit them.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72274
2020-01-17 12:11:25 -05:00
Dmitri Gribenko 0406b4fab9 Renamed traverseDecl to TraverseDecl in a test
RecursiveASTVisitor expects TraverseDecl to be implemented by
subclasses.
2020-01-17 17:12:23 +01:00
Ilya Biryukov 013c07f697 [Syntax] Unset IsOriginal flag on nodes removed from the tree
And add a corresponding test.
Only nodes inside the TranslationUnit subtree can be marked as original,
computeReplacements() relies on this.
2020-01-14 17:00:33 +01:00
Ilya Biryukov 07a41018e9 [Syntax] Mark synthesized nodes as modifiable
This was an oversight in the original patch.
Also add corresponding tests.
2020-01-14 16:41:09 +01:00
Ilya Biryukov 04f627f6b9 [Syntax] Build spanning SimpleDecalration for classes, structs, etc
When they are free-standing, e.g. `struct X;` or `struct X {};`.
Although this complicates the common case (of free-standing class
declarations), this ensures the less common case (e.g. `struct X {} a;`)
are handled uniformly and produce similar syntax trees.
2020-01-03 12:33:11 +01:00
Alexey Bader 128f39da93 Fix crash in getFullyQualifiedName for inline namespace
Summary: The ICE happens when the most outer namespace is an inline namespace.

Reviewers: bkramer, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ebevhan, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71962
2019-12-28 16:35:51 +03:00
Kadir Cetinkaya c2377eae28
[clang][Tooling] Prefer -x over -std when interpolating
Summary:
Currently interpolation logic prefers -std over -x. But the latter is a
more strong signal, so this patch inverts the order and only makes use of -std
if -x didn't exist.

Fixes https://github.com/clangd/clangd/issues/185

Thanks @sammccall for tracking this down!

Reviewers: sammccall

Subscribers: ilya-biryukov, usaxena95, cfe-commits, sammccall

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71727
2019-12-20 11:10:36 +01:00
Ilya Biryukov c1bbefef9d [Syntax] Use a hash table to search for tokens by their location
This is both more efficient and avoids corner cases in
`SourceManager::isBeforeInTranslationUnit`.

The change is trivial and clearly a performance improvement on the hot
path of building the syntax tree, so sending without review.
2019-12-18 12:24:00 +01:00
Ilya Biryukov 038f53882e [Syntax] Uppercase the first letter of the test name. NFC
To match naming style of other tests.
2019-12-18 12:20:30 +01:00
Ilya Biryukov 1ad15046dc [Syntax] Allow to mutate syntax trees
Summary:
This patch adds facilities to mutate the syntax trees and produce
corresponding text replacements.

The public interface of the syntax library now includes facilities to:
    1. perform type-safe modifications of syntax trees,
    2. compute textual replacements to apply the modifications,
    3. create syntax trees not backed by the source code.

For each of the three, we only add a few example transformations in this
patch to illustrate the idea, support for more kinds of nodes and
transformations will be done in follow-up patches.

The high-level mutation operations are implemented on top of operations
that allow to arbitrarily change the trees. They are considered to be
implementation details and are not available to the users of the
library.

Reviewers: sammccall, gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64573
2019-12-18 12:19:03 +01:00
Sam McCall 3f8da5d091 [Tooling/Syntax] Helpers to find spelled tokens touching a location.
Summary: Useful when positions are used to target nodes, with before/after ambiguity.

Reviewers: ilya-biryukov, kbobyrev

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71356
2019-12-12 12:59:50 +01:00
Ilya Biryukov 7d7789899f [Tooling] Move raw string literal out of a macro call. NFC
Should fix buildbots with some older gcc versions.
2019-12-12 10:53:20 +01:00
Ilya Biryukov be14a22b47 [Syntax] Build nodes for simple cases of top level declarations
Summary:
More complicated nodes (e.g. template declarations) will be implemented
in the follow-up patches.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70856
2019-12-12 08:04:22 +01:00
Haojian Wu 6d5c273500 Reland "[AST] Traverse the class type loc inside the member type loc.""
Summary: added a unittest which causes "TL.getClassTInfo" is null.

Reviewers: ilya-biryukov

Subscribers: mgorny, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71186
2019-12-09 11:18:12 +01:00
Sterling Augustine 6470497817 Revert "[AST] Traverse the class type loc inside the member type loc."
This reverts commit 7f93cb6228.

The assertion at RecursiveASTVisitor.h:1169 fails when passed a TypeLocNode.
Not sure if the correct fix is to use getTypeLocClass or something else.
2019-12-05 16:48:18 -08:00
Ilya Biryukov 242649c17c [Syntax] Show input code on tests failures. NFC 2019-12-05 16:25:35 +01:00
Haojian Wu 7f93cb6228 [AST] Traverse the class type loc inside the member type loc.
Summary:
We are missing this currently.

This would fix https://github.com/clangd/clangd/issues/216.

Reviewers: ilya-biryukov

Subscribers: mgorny, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70849
2019-12-05 15:03:30 +01:00
Kadir Cetinkaya 45ef055d4f
[clang][Tooling] Add support for .rsp files in compile_commands.json
Summary:
Add support for .rsp files.

Fixes https://github.com/clangd/clangd/issues/81

Patch By: liu hui(@lh123)

Reviewers: sammccall, ilya-biryukov, hokein, kadircet

Reviewed By: kadircet

Subscribers: merge_guards_bot, mgorny, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang-tools-extra, #clang

Differential Revision: https://reviews.llvm.org/D70222
2019-12-04 15:13:12 +01:00
Sam McCall c9c714c705 Reland [clangd] Rethink how SelectionTree deals with macros and #includes.
This reverts commit 905b002c13.

Avoid tricky (and invalid) comparator for std::set.
2019-12-03 17:53:43 +01:00
Sam McCall 905b002c13 Revert "[clangd] Rethink how SelectionTree deals with macros and #includes."
This reverts commit 19daa21f84.

It causes a bunch of failures on a bot that I've been unable to
reproduce so far:
http://45.33.8.238/mac/3308/step_7.txt
2019-11-29 19:59:02 +01:00
Sam McCall 19daa21f84 [clangd] Rethink how SelectionTree deals with macros and #includes.
Summary:
The exclusive-claim model is successful at resolving conflicts over tokens
between parent/child or siblings. However claims at the spelled-token
level do the wrong thing for macro expansions, where siblings can be
equally associated with the macro invocation.
Moreover, any model that only uses the endpoints in a range can fail when
a macro invocation occurs inside the node.

To address this, we use the existing TokenBuffer in more depth.
Claims are expressed in terms of expanded tokens, so there is no need to worry
about macros, includes etc.

Once we know which expanded tokens were claimed, they are mapped onto
spelled tokens for hit-testing.
This mapping is fairly flexible, currently the handling of macros is
pretty simple (map macro args onto spellings, other macro expansions onto the
macro name token).
This mapping is in principle token-by-token for correctness (though
there's some batching for performance).

The aggregation of the selection enum is now more principled as we need to be
able to aggregate several hit-test results together.

For simplicity i removed the ability to determine selectedness of TUDecl.
(That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very
important or worth the complexity any longer).

The expandedTokens(SourceLocation) helper could be added locally, but seems to
make sense on TokenBuffer.

Fixes https://github.com/clangd/clangd/issues/202
Fixes https://github.com/clangd/clangd/issues/126

Reviewers: hokein

Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70512
2019-11-29 15:21:13 +01:00
Ilya Biryukov e702bdb859 [Syntax] Build SimpleDeclaration node that groups multiple declarators
Summary:
Also remove the temporary TopLevelDeclaration node and add
UnknownDeclaration to represent other unknown nodes.

See the follow-up change for building more top-level declarations.
Adding declarators is also pretty involved and will be done in another
follow-up patch.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70787
2019-11-29 11:39:45 +01:00
Yitzhak Mandelbaum 01e8dd2e7a [libTooling] Add stencil combinators for nodes that may be pointers or values.
Summary:
Adds combinators `maybeDeref` and `maybeAddressOf` to provide a uniform way to handle
nodes which may be bound to either a pointer or a value (most often in the
context of member expressions). Such polymorphism is already supported by
`access`; these combinators extend it to more general uses.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70554
2019-11-22 12:36:40 -05:00
Kadir Cetinkaya 16bdcc809c
[clang][Tooling] Filter flags that generate output in SyntaxOnlyAdjuster
Summary:
Flags that generate output could result in failures when creating
syntax only actions. This patch introduces initial logic for filtering out
those. The first such flag is "save-temps", which saves intermediate
files(bitcode, assembly, etc.) into a specified directory.

Fixes https://github.com/clangd/clangd/issues/191

Reviewers: hokein

Subscribers: ilya-biryukov, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70173
2019-11-13 15:03:30 +01:00
Yitzhak Mandelbaum 489449c28a [libTooling] Further simplify `Stencil` type and introduce `MatchComputation`.
Summary:
This revision introduces a new interface `MatchComputation` which generalizes
the `Stencil` interface and replaces the `std::function` interface of
`MatchConsumer`. With this revision, `Stencil` (as an abstraction) becomes just
one collection of implementations of
`MatchComputation<std::string>`. Correspondingly, we remove the `Stencil` class
entirely in favor of a simple type alias, deprecate `MatchConsumer` and change
all functions that accepted `MatchConsumer<std::string>` to use
`MatchComputation<std::string>` instead.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69802
2019-11-11 12:44:15 -05:00
Yitzhak Mandelbaum 6c683aa8d7 [libTooling] Fix breakage from change #84922 2019-11-06 11:31:35 -05:00
Yitzhak Mandelbaum 9f97480cdd [libTooling] Small changes in Transformer API.
Summary:
* Rename `transformer::change` to `transformer::changeTo`, make `change` forward
  to `changeTo` and mark it deprecated.

* Mark `transformer::text` and `transformer::selection` deprecated and migrate
  references to them in tests.

Reviewers: ilya-biryukov

Subscribers: gribozavr, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69896
2019-11-06 11:16:50 -05:00
Yitzhak Mandelbaum ce2b5cb6de [libTooling] Simplify type structure of `Stencil`s.
Summary:
Currently, stencils are defined as a sequence of `StencilParts`. This
differentiation adds an unneeded layer of complexity to the definition of
Stencils. This change significantly simplifies the type structure: a stencil is
now conceptually any object implementing `StencilInterface` and `Stencil` is
just a thin wrapper for pointers to this interface.

To account for the sequencing that was supported by the old `Stencil` type, we
introduce a sequencing class that implements `StencilInterface`. That is,
sequences are just another kind of Stencil and no longer have any special
status.

Corresponding to this change in the type structure, we change the way `cat` is
used (and defined). `cat` bundles multiple features: it builds a stencil from a
sequence of subcomponents and admits multiple different types for its arguments,
while coercing them into the right type. Previously, `cat` was also used to
coerce a single `StencilPart` into a `Stencil`. With that distinction gone, many
uses of `cat` (e.g. in the tests) are unnecessary and have, therefore, been
removed.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69613
2019-11-06 10:28:34 -05:00
Ilya Biryukov 58fa50f437 [Syntax] Add nodes for most common statements
Summary:
Most of the statements mirror the ones provided by clang AST.
Major differences are:
  - expressions are wrapped into 'ExpressionStatement' instead of being
    a subclass of statement,
  - semicolons are always consumed by the leaf expressions (return,
    expression satement, etc),
  - some clang statements are not handled yet, we wrap those into an
    UnknownStatement class, which is not present in clang.

We also define an 'Expression' and 'UnknownExpression' classes in order
to produce 'ExpressionStatement' where needed. The actual implementation
of expressions is not yet ready, it will follow later.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63835
2019-11-06 10:56:06 +01:00
Yitzhak Mandelbaum c14f1ea25e [libTooling] Fix r374962: add more Transformer forwarding decls.
Summary:
The move to a new, single namespace in r374962 left out some type definitions
from the old namespace and resulted in one naming conflict (`text`).  This
revision adds aliases for those definitions and removes one of the `text`
functions from the new namespace.

Reviewers: alexfh

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69036

llvm-svn: 375003
2019-10-16 14:26:20 +00:00
Yitzhak Mandelbaum 8bb47cd8c3 [libTooling] Put all Transformer declarations in a single namespace.
Summary:
This revision introduces a new namespace, `clang::transformer`, to hold
the declarations for the Transformer library.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68876

llvm-svn: 374962
2019-10-16 01:06:46 +00:00
Yitzhak Mandelbaum cf2438ec13 [libTooling] Change Stencil equality to use `toString()`
Summary:
Removes the `isEqual` method from StencilPartInterface and modifies equality to
use the string representation returned by the `toString` method for comparison.

This means the `run` and `selection` stencils return true by default, and
clients should be cautious in relying on equality operator for comparison of
stencils containing parts generated by these functions.

It also means we no longer need the custom RTTI support (typeId() and
down_cast()), so it has been removed.

Patch by Harshal T. Lehri.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68825

llvm-svn: 374552
2019-10-11 14:02:03 +00:00
Yitzhak Mandelbaum fbdf83521b [libTooling] Move Transformer files to their own directory/library.
Summary:
The Transformer library has been growing inside of
lib/Tooling/Refactoring. However, it's not really related to anything else in
that directory. This revision moves all Transformer-related files into their own
include & lib directories.  A followup revision will (temporarily) add
forwarding headers to help any users migrate their code to the new location.

Reviewers: gribozavr

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68637

llvm-svn: 374271
2019-10-10 02:34:47 +00:00
Yitzhak Mandelbaum e4cec2d3c7 [libTooling][NFC] Fix build break in r373916.
r373916 used raw strings inside macro calls, which breaks some builds.

llvm-svn: 373928
2019-10-07 17:24:23 +00:00
Yitzhak Mandelbaum d5b983555f [libTooling] Add `toString` method to the Stencil class
Summary:
`toString` generates a string representation of the stencil.

Patch by Harshal T. Lehri.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68574

llvm-svn: 373916
2019-10-07 16:20:22 +00:00
Yitzhak Mandelbaum e80e889695 [libTooling] Add various Stencil combinators for expressions.
Summary:
This revision adds three new Stencil combinators:
* `expression`, which idiomatically constructs the source for an expression,
  including wrapping the expression's source in parentheses if needed.
* `deref`, which constructs an idiomatic dereferencing expression.
* `addressOf`, which constructs an idiomatic address-taking expression.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68315

llvm-svn: 373593
2019-10-03 13:01:00 +00:00
Yitzhak Mandelbaum db24ef509e [libTooling] Transformer: refine `SourceLocation` specified as anchor of changes.
Summary: Every change triggered by a rewrite rule is anchored at a particular
location in the source code.  This patch refines how that location is chosen and
defines it as an explicit function so it can be shared by other Transformer
implementations.

This patch was inspired by a bug found by a clang tidy, wherein two changes were
anchored at the same location (the expansion loc of the macro) resulting in the
discarding of the second change.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66652

llvm-svn: 373093
2019-09-27 15:26:04 +00:00
Yitzhak Mandelbaum ea966c1bc0 [libTooling] Add `run` combinator to Stencils.
Summary:
This revision adds `run`, a StencilPart that runs a user-defined function that
computes a result over `MatchFinder::MatchResult`.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67969

llvm-svn: 372936
2019-09-26 00:53:56 +00:00
Yitzhak Mandelbaum 185f56bbbe [libTooling][NFC] Switch StencilTest.cpp to use EXPECT_THAT_EXPECTED
Summary:
Currently, some tests use homegrown matchers to handle `llvm::Expected`
values. This revision standardizes on EXPECT_THAT_EXPECTED and `HasValue`.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67973

llvm-svn: 372918
2019-09-25 20:04:25 +00:00
Yitzhak Mandelbaum 0a81b4ed97 [libTooling] Add `access` and `ifBound` combinators to Stencil library.
Summary:
This revision add the `access` and `ifBound` combinators to the Stencil library:
* `access` -- constructs an idiomatic expression for accessing a member (a
  `MemberExpr`).
* `ifBound` -- chooses between two `StencilParts` based on the whether an id is
  bound in the match (corresponds to the combinator of the same name in
  RangeSelector).

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67633

llvm-svn: 372605
2019-09-23 13:21:42 +00:00
Yitzhak Mandelbaum 1588c08735 [libTooling] Introduce new library of source-code builders.
Summary:
Introduces facilities for easily building source-code strings, including
idiomatic use of parentheses and the address-of, dereference and member-access
operators (dot and arrow) and queries about need for parentheses.

Reviewers: gribozavr

Subscribers: mgorny, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67632

llvm-svn: 372595
2019-09-23 12:40:10 +00:00
Yitzhak Mandelbaum eff88e42f7 [libTooling] Add `ifBound`, `elseBranch` RangeSelector combinators.
Summary:
Adds two new combinators and corresponding tests to the RangeSelector library.
* `ifBound` -- conditional evaluation of range-selectors, based on whether a
   given node id is bound in the match.
* `elseBranch` -- selects the source range of the else and its statement.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67621

llvm-svn: 372410
2019-09-20 17:11:03 +00:00
Dmitri Gribenko b22804b354 [Tooling] Migrated APIs that take ownership of objects to unique_ptr
Subscribers: jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66960

llvm-svn: 370451
2019-08-30 09:29:34 +00:00
Dmitri Gribenko 907452107d Changed FrontendActionFactory::create to return a std::unique_ptr
Subscribers: jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66947

llvm-svn: 370379
2019-08-29 16:38:36 +00:00
Alex Lorenz 1c8a4b7204 Use FileEntryRef for PPCallbacks::HasInclude
This fixes the issue where a filename dependendency was missing if the file that
was referenced with __has_include() was accessed through a symlink in an earlier run,
if the file manager was reused between runs.

llvm-svn: 370081
2019-08-27 17:32:42 +00:00
Alex Lorenz 67d25fede9 Use FileEntryRef for PPCallbacks::FileSkipped
This fixes the issue where a filename dependendency was missing if the file that
was skipped was included through a symlink in an earlier run, if the file
manager was reused between runs.

llvm-svn: 369998
2019-08-27 01:03:25 +00:00
Benjamin Kramer 1e4241cad7 Remove unused variables.
llvm-svn: 369965
2019-08-26 20:51:23 +00:00
Alex Lorenz a39e6490af Re-enable DependencyScannerTest on windows with the right fixes
It should now pass.

llvm-svn: 369832
2019-08-24 01:53:40 +00:00