Commit Graph

657 Commits

Author SHA1 Message Date
Eduardo Caldas eca41919d2 Improve test infrastructure in SyntaxTree
Summary:
* Test if the code sourcing the SyntaxTree compiles
* Output compiler errors and warnings to err
* Fix tests with code that did not compile

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80731
2020-05-28 21:35:12 +02:00
Yitzhak Mandelbaum ce5780b88c [libTooling] Fix Transformer to work with ambient traversal kinds.
Summary:
`RewriteRule`'s `applyFirst` was brittle with respect to the default setting of the
`TraversalKind`. This patch builds awareness of traversal kinds directly into
rewrite rules so that they are insensitive to any changes in defaults.

Reviewers: steveire, gribozavr

Subscribers: hokein, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80606
2020-05-28 11:42:07 -04:00
Eduardo Caldas 461af57de7 Add support for UnaryOperator in SyntaxTree
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80624
2020-05-27 17:12:46 +02:00
Yitzhak Mandelbaum ff2743bf04 [libTooling] In Transformer, allow atomic changes to span multiple files.
Summary:
Currently, all changes returned by a single application of a rule must fit in
one atomic change and therefore must apply to one file. However, there are
patterns in which a single rule will want to modify multiple files; for example,
a header and implementation to change a declaration and its definition. This
patch relaxes Transformer, libTooling's interpreter of RewriteRules, to support
multiple changes.

Reviewers: gribozavr

Subscribers: mgrang, jfb, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80239
2020-05-26 09:17:35 -04:00
Eduardo Caldas 3785eb83af Add support for binary operators in Syntax Trees
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80540
2020-05-26 12:25:58 +02:00
mydeveloperday 447ea9b4f5 [AST] default implementation is possible for non-member functions in C++20.
Summary:
Make RAV not visit the default function decl by default.
Also update some stale comments on FunctionDecl::isDefault.

Fixes https://github.com/clangd/clangd/issues/383

Reviewers: sammccall, rsmith

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80288
2020-05-25 10:45:12 +02:00
Stephen Kelly a30d411629 Set traversal explicitly where needed in tests
Reviewers: aaron.ballman, shafik

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72531
2020-05-21 22:34:38 +01:00
Arthur Eubanks dab1326a5a Strip -showIncludes in getClangStripDependencyFileAdjuster()
Summary:
Follow-up to https://reviews.llvm.org/D78836.

Also consolidate some test cases.

Reviewers: thakis

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D78939
2020-04-27 11:20:08 -07:00
Arthur Eubanks 8000d506af [clangd] Strip /showIncludes in clangd compile commands
In command lines with /showIncludes, clangd would output includes to stdout, breaking clients.

Summary: Fixes https://github.com/clangd/clangd/issues/322.

Reviewers: sammccall, kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D78836
2020-04-26 18:57:39 -07:00
Johannes Doerfert f9d558c871 [OpenMP] "UnFix" layering problem with FrontendOpenMP
This reverts commit 97aa593a83 as it
causes problems (PR45453) https://reviews.llvm.org/D77574#1966321.

This additionally adds an explicit reference to FrontendOpenMP to
clang-tidy where ASTMatchers is used.

This is hopefully just a temporary solution. The dependence on
`FrontendOpenMP` from `ASTMatchers` should be handled by CMake
implicitly, not us explicitly.

Reviewed By: aheejin

Differential Revision: https://reviews.llvm.org/D77666
2020-04-07 14:41:18 -05:00
Marcel Hlopko 1bf055c989 [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
Summary:
Same restrictions apply as in the other direction: macro arguments are
not supported yet, only full macro expansions can be mapped.

Taking over from https://reviews.llvm.org/D72581.

Reviewers: gribozavr2, sammccall

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77209
2020-04-07 15:07:16 +02:00
Sam McCall d66afd6dde [Syntax] Merge overlapping top-level macros in TokenBuffer
Summary:
Our previous definition of "top-level" was too informal, and didn't
allow for overlapping macros that each directly produce expanded tokens.
See D77507 for previous discussion.

Fixes http://bugs.llvm.org/show_bug.cgi?id=45428

Reviewers: kadircet, vabridgers

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77615
2020-04-07 15:06:04 +02:00
Johannes Doerfert 97aa593a83 [OpenMP] Fix layering problem with FrontendOpenMP
Summary:
ASTMatchers is used in various places and it now exposes the
LLVMFrontendOpenMP library to its users without them needing to depend
on it explicitly.

Reviewers: lebedev.ri

Subscribers: mgorny, yaxunl, bollu, guansong, martong, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77574
2020-04-06 13:04:26 -05:00
Johannes Doerfert 931c0cd713 [OpenMP][NFC] Move and simplify directive -> allowed clause mapping
Move the listing of allowed clauses per OpenMP directive to the new
macro file in `llvm/Frontend/OpenMP`. Also, use a single generic macro
that specifies the directive and one allowed clause explicitly instead
of a dedicated macro per directive.

We save 800 loc and boilerplate for all new directives/clauses with no
functional change. We also need to include the macro file only once and
not once per directive.

Depends on D77112.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D77113
2020-04-06 00:04:08 -05:00
Marcel Hlopko cdce2fe561 [Syntax] Remove delayed folding from tree building.
Summary:
This patch removes delayed folding and replaces it with forward peeking.

Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:

```
int a,b;
```

is expressed in the AST as:

```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```

And in the syntax tree we need:

```
*: TranslationUnit
`-SimpleDeclaration
  |-int
  |-SimpleDeclarator
  | `-a
  |-,
  |-SimpleDeclarator
  | `-b
  |-;
```

So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.

This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).

This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
  method in all places.
* Adds a bunch of tests

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76922
2020-03-31 09:47:50 +02:00
Kadir Cetinkaya 9619c2cc9a
[clang][Syntax] Handle macro arguments in spelledForExpanded
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75446
2020-03-28 16:35:46 +01:00
Marcel Hlopko eddede9d51 [Syntax] Test both the default and windows target platforms in unittests
Summary:
This increases the coverage for things that differ between Linux and Windows, such as -fdelayed-template-parsing. This would have prevented the rollback of https://reviews.llvm.org/D76346.

While at it, update -std=c++11 to c++17 for the test.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76497
2020-03-20 16:33:58 +01:00
Dmitri Gribenko 9967352a03 Revert "[Syntax] Test both the default and windows target platforms in unittests"
This reverts commit fd7300f717. The fix
in this patch didn't help and the Windows buildbot broke:
http://45.33.8.238/win/10881/step_7.txt
2020-03-20 12:13:49 +01:00
Marcel Hlopko fd7300f717 [Syntax] Test both the default and windows target platforms in unittests
Summary:
This increases the coverage for things that differ between Linux and Windows, such as `-fdelayed-template-parsing`. This would have prevented the rollback of https://reviews.llvm.org/D76346.

While at it, update -std=c++11 to c++17 for the test.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76433
2020-03-20 11:42:18 +01:00
Marcel Hlopko e9630630ff [Syntax] Split syntax tests
Summary:
This patch split Basic test into multple individual tests to allow simpler
filtering and clearer signal into what's broken when it's broken.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76366
2020-03-20 09:53:43 +01:00
Marcel Hlopko 88bf9b3d26 [Syntax] Build template declaration nodes
Summary:
Rollforward of
https://reviews.llvm.org/rGdd12826808f9079e164b82e64b0697a077379241 after
temporarily adding -fno-delayed-template-parsing to the TreeTest.

Original summary:

> Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
>
> Handles template declaration of all kinds.
>
> Also builds template declaration nodes for specializations and explicit
> instantiations of classes.
>
> Some missing things will be addressed in the follow-up patches:
>
> * specializations of functions and variables,
> * template parameters.

Reviewers: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76418
2020-03-19 17:43:07 +01:00
Nico Weber 881f5b5a7b Revert "[Syntax] Build template declaration nodes"
This reverts commit dd12826808.
Breaks tests on Windows, see https://reviews.llvm.org/D76346#1929208
2020-03-18 12:57:55 -04:00
Marcel Hlopko dd12826808 [Syntax] Build template declaration nodes
Summary:
Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.

Handles template declaration of all kinds.

Also builds template declaration nodes for specializations and explicit
instantiations of classes.

Some missing things will be addressed in the follow-up patches:

specializations of functions and variables,
template parameters.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76346
2020-03-18 16:16:59 +01:00
Marcel Hlopko 7d382dcd46 [Syntax] Build declarator nodes
Summary:
Copy of https://reviews.llvm.org/D72089 with Ilya's permission. See
https://reviews.llvm.org/D72089 for the first batch of comments.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76220
2020-03-16 19:13:59 +01:00
Reid Kleckner 213aea4c58 Remove unused Endian.h includes, NFC
Mainly avoids including Host.h everywhere:

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
    | grep '^[-+] ' | sort | uniq -c | sort -nr
   3141 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Host.h
2020-03-11 15:45:34 -07:00
Kadir Cetinkaya cd9b2e18bd
[clang][Syntax] Add spelledTokenAt helper to TokenBuffer
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75503
2020-03-03 14:30:41 +01:00
Adrian McCarthy 0a3117832c Need /bigobj to build SourceCodeTest.cpp with MSVC. 2020-02-27 14:56:29 -08:00
Joe Turner b26c88e3c6 [clang-tidy] Store all ranges in clang::tooling::Diagnostic
Summary: Instead of dropping all the ranges associated with a Diagnostic when
converting them to a ClangTidy error, instead attach them to the ClangTidyError,
so they can be consumed by other APIs.

Patch by Joe Turner <joturner@google.com>.
Differential Revision: https://reviews.llvm.org/D69782
2020-02-27 19:39:42 +01:00
Yitzhak Mandelbaum 38b4516de8 [libTooling] Add function to determine associated text of a declaration.
Summary:
Second attempt -- the first was reverted in commit 0e480b39c6, because of test breakages. This revision fixes the cause of the test breakages.

Original description follows:
This patch adds `getAssociatedRange` which, for a given decl, computes preceding
and trailing text that would conceptually be associated with the decl by the
reader. This includes comments, whitespace, and separators like ';'.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72153
2020-02-26 09:56:48 -05:00
Kadir Cetinkaya 8c2cf499e6
[clang][Tooling] Add a way to tokenize a FileRange
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74962
2020-02-26 13:50:41 +01:00
Nico Weber 0e480b39c6 Revert "[libTooling] Add function to determine associated text of a declaration."
This reverts commit 9c54f6154f.
Breaks two tests on Windows, see e.g.
http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/14505/steps/stage%201%20check/logs/stdio
2020-02-25 19:21:52 -05:00
Yitzhak Mandelbaum 9c54f6154f [libTooling] Add function to determine associated text of a declaration.
Summary:
This patch adds `getAssociatedRange` which, for a given decl, computes preceding
and trailing text that would conceptually be associated with the decl by the
reader. This includes comments, whitespace, and separators like ';'.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72153
2020-02-25 16:37:20 -05:00
Yitzhak Mandelbaum 523cae324d [libTooling] Add option for `buildAST` to report diagnostics.
Summary:
Currently, `buildAST[WithArgs]` either succeeds or fails.  This patch adds
support for the caller to pass a `DiagnosticConsumer` to receive all relevant
diagnostics.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74763
2020-02-18 09:11:51 -05:00
Justin Lebar 027eb71696 Use std::foo_t rather than std::foo in clang.
Summary: No functional change.

Reviewers: bkramer, MaskRay, martong, shafik

Subscribers: martong, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D74414
2020-02-11 10:37:08 -08:00
Benjamin Kramer 757bdc64d3 Fix clang unnittest build with GCC 5 2020-01-29 10:30:36 +01:00
Benjamin Kramer adcd026838 Make llvm::StringRef to std::string conversions explicit.
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.

This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.

This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 23:25:25 +01:00
Yitzhak Mandelbaum b9d2bf38e8 [libTooling] Fix bug in Stencil handling of macro ranges
Summary: Currently, an attempt to rewrite source code inside a macro expansion succeeds, but results in empty text, rather than failing with an error.  This patch restructures to the code to explicitly validate ranges before attempting to edit them.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72274
2020-01-17 12:11:25 -05:00
Dmitri Gribenko 0406b4fab9 Renamed traverseDecl to TraverseDecl in a test
RecursiveASTVisitor expects TraverseDecl to be implemented by
subclasses.
2020-01-17 17:12:23 +01:00
Ilya Biryukov 013c07f697 [Syntax] Unset IsOriginal flag on nodes removed from the tree
And add a corresponding test.
Only nodes inside the TranslationUnit subtree can be marked as original,
computeReplacements() relies on this.
2020-01-14 17:00:33 +01:00
Ilya Biryukov 07a41018e9 [Syntax] Mark synthesized nodes as modifiable
This was an oversight in the original patch.
Also add corresponding tests.
2020-01-14 16:41:09 +01:00
Ilya Biryukov 04f627f6b9 [Syntax] Build spanning SimpleDecalration for classes, structs, etc
When they are free-standing, e.g. `struct X;` or `struct X {};`.
Although this complicates the common case (of free-standing class
declarations), this ensures the less common case (e.g. `struct X {} a;`)
are handled uniformly and produce similar syntax trees.
2020-01-03 12:33:11 +01:00
Alexey Bader 128f39da93 Fix crash in getFullyQualifiedName for inline namespace
Summary: The ICE happens when the most outer namespace is an inline namespace.

Reviewers: bkramer, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ebevhan, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71962
2019-12-28 16:35:51 +03:00
Kadir Cetinkaya c2377eae28
[clang][Tooling] Prefer -x over -std when interpolating
Summary:
Currently interpolation logic prefers -std over -x. But the latter is a
more strong signal, so this patch inverts the order and only makes use of -std
if -x didn't exist.

Fixes https://github.com/clangd/clangd/issues/185

Thanks @sammccall for tracking this down!

Reviewers: sammccall

Subscribers: ilya-biryukov, usaxena95, cfe-commits, sammccall

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71727
2019-12-20 11:10:36 +01:00
Ilya Biryukov c1bbefef9d [Syntax] Use a hash table to search for tokens by their location
This is both more efficient and avoids corner cases in
`SourceManager::isBeforeInTranslationUnit`.

The change is trivial and clearly a performance improvement on the hot
path of building the syntax tree, so sending without review.
2019-12-18 12:24:00 +01:00
Ilya Biryukov 038f53882e [Syntax] Uppercase the first letter of the test name. NFC
To match naming style of other tests.
2019-12-18 12:20:30 +01:00
Ilya Biryukov 1ad15046dc [Syntax] Allow to mutate syntax trees
Summary:
This patch adds facilities to mutate the syntax trees and produce
corresponding text replacements.

The public interface of the syntax library now includes facilities to:
    1. perform type-safe modifications of syntax trees,
    2. compute textual replacements to apply the modifications,
    3. create syntax trees not backed by the source code.

For each of the three, we only add a few example transformations in this
patch to illustrate the idea, support for more kinds of nodes and
transformations will be done in follow-up patches.

The high-level mutation operations are implemented on top of operations
that allow to arbitrarily change the trees. They are considered to be
implementation details and are not available to the users of the
library.

Reviewers: sammccall, gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64573
2019-12-18 12:19:03 +01:00
Sam McCall 3f8da5d091 [Tooling/Syntax] Helpers to find spelled tokens touching a location.
Summary: Useful when positions are used to target nodes, with before/after ambiguity.

Reviewers: ilya-biryukov, kbobyrev

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71356
2019-12-12 12:59:50 +01:00
Ilya Biryukov 7d7789899f [Tooling] Move raw string literal out of a macro call. NFC
Should fix buildbots with some older gcc versions.
2019-12-12 10:53:20 +01:00
Ilya Biryukov be14a22b47 [Syntax] Build nodes for simple cases of top level declarations
Summary:
More complicated nodes (e.g. template declarations) will be implemented
in the follow-up patches.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70856
2019-12-12 08:04:22 +01:00
Haojian Wu 6d5c273500 Reland "[AST] Traverse the class type loc inside the member type loc.""
Summary: added a unittest which causes "TL.getClassTInfo" is null.

Reviewers: ilya-biryukov

Subscribers: mgorny, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D71186
2019-12-09 11:18:12 +01:00
Sterling Augustine 6470497817 Revert "[AST] Traverse the class type loc inside the member type loc."
This reverts commit 7f93cb6228.

The assertion at RecursiveASTVisitor.h:1169 fails when passed a TypeLocNode.
Not sure if the correct fix is to use getTypeLocClass or something else.
2019-12-05 16:48:18 -08:00
Ilya Biryukov 242649c17c [Syntax] Show input code on tests failures. NFC 2019-12-05 16:25:35 +01:00
Haojian Wu 7f93cb6228 [AST] Traverse the class type loc inside the member type loc.
Summary:
We are missing this currently.

This would fix https://github.com/clangd/clangd/issues/216.

Reviewers: ilya-biryukov

Subscribers: mgorny, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70849
2019-12-05 15:03:30 +01:00
Kadir Cetinkaya 45ef055d4f
[clang][Tooling] Add support for .rsp files in compile_commands.json
Summary:
Add support for .rsp files.

Fixes https://github.com/clangd/clangd/issues/81

Patch By: liu hui(@lh123)

Reviewers: sammccall, ilya-biryukov, hokein, kadircet

Reviewed By: kadircet

Subscribers: merge_guards_bot, mgorny, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang-tools-extra, #clang

Differential Revision: https://reviews.llvm.org/D70222
2019-12-04 15:13:12 +01:00
Sam McCall c9c714c705 Reland [clangd] Rethink how SelectionTree deals with macros and #includes.
This reverts commit 905b002c13.

Avoid tricky (and invalid) comparator for std::set.
2019-12-03 17:53:43 +01:00
Sam McCall 905b002c13 Revert "[clangd] Rethink how SelectionTree deals with macros and #includes."
This reverts commit 19daa21f84.

It causes a bunch of failures on a bot that I've been unable to
reproduce so far:
http://45.33.8.238/mac/3308/step_7.txt
2019-11-29 19:59:02 +01:00
Sam McCall 19daa21f84 [clangd] Rethink how SelectionTree deals with macros and #includes.
Summary:
The exclusive-claim model is successful at resolving conflicts over tokens
between parent/child or siblings. However claims at the spelled-token
level do the wrong thing for macro expansions, where siblings can be
equally associated with the macro invocation.
Moreover, any model that only uses the endpoints in a range can fail when
a macro invocation occurs inside the node.

To address this, we use the existing TokenBuffer in more depth.
Claims are expressed in terms of expanded tokens, so there is no need to worry
about macros, includes etc.

Once we know which expanded tokens were claimed, they are mapped onto
spelled tokens for hit-testing.
This mapping is fairly flexible, currently the handling of macros is
pretty simple (map macro args onto spellings, other macro expansions onto the
macro name token).
This mapping is in principle token-by-token for correctness (though
there's some batching for performance).

The aggregation of the selection enum is now more principled as we need to be
able to aggregate several hit-test results together.

For simplicity i removed the ability to determine selectedness of TUDecl.
(That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very
important or worth the complexity any longer).

The expandedTokens(SourceLocation) helper could be added locally, but seems to
make sense on TokenBuffer.

Fixes https://github.com/clangd/clangd/issues/202
Fixes https://github.com/clangd/clangd/issues/126

Reviewers: hokein

Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70512
2019-11-29 15:21:13 +01:00
Ilya Biryukov e702bdb859 [Syntax] Build SimpleDeclaration node that groups multiple declarators
Summary:
Also remove the temporary TopLevelDeclaration node and add
UnknownDeclaration to represent other unknown nodes.

See the follow-up change for building more top-level declarations.
Adding declarators is also pretty involved and will be done in another
follow-up patch.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: merge_guards_bot, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70787
2019-11-29 11:39:45 +01:00
Yitzhak Mandelbaum 01e8dd2e7a [libTooling] Add stencil combinators for nodes that may be pointers or values.
Summary:
Adds combinators `maybeDeref` and `maybeAddressOf` to provide a uniform way to handle
nodes which may be bound to either a pointer or a value (most often in the
context of member expressions). Such polymorphism is already supported by
`access`; these combinators extend it to more general uses.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70554
2019-11-22 12:36:40 -05:00
Kadir Cetinkaya 16bdcc809c
[clang][Tooling] Filter flags that generate output in SyntaxOnlyAdjuster
Summary:
Flags that generate output could result in failures when creating
syntax only actions. This patch introduces initial logic for filtering out
those. The first such flag is "save-temps", which saves intermediate
files(bitcode, assembly, etc.) into a specified directory.

Fixes https://github.com/clangd/clangd/issues/191

Reviewers: hokein

Subscribers: ilya-biryukov, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D70173
2019-11-13 15:03:30 +01:00
Yitzhak Mandelbaum 489449c28a [libTooling] Further simplify `Stencil` type and introduce `MatchComputation`.
Summary:
This revision introduces a new interface `MatchComputation` which generalizes
the `Stencil` interface and replaces the `std::function` interface of
`MatchConsumer`. With this revision, `Stencil` (as an abstraction) becomes just
one collection of implementations of
`MatchComputation<std::string>`. Correspondingly, we remove the `Stencil` class
entirely in favor of a simple type alias, deprecate `MatchConsumer` and change
all functions that accepted `MatchConsumer<std::string>` to use
`MatchComputation<std::string>` instead.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69802
2019-11-11 12:44:15 -05:00
Yitzhak Mandelbaum 6c683aa8d7 [libTooling] Fix breakage from change #84922 2019-11-06 11:31:35 -05:00
Yitzhak Mandelbaum 9f97480cdd [libTooling] Small changes in Transformer API.
Summary:
* Rename `transformer::change` to `transformer::changeTo`, make `change` forward
  to `changeTo` and mark it deprecated.

* Mark `transformer::text` and `transformer::selection` deprecated and migrate
  references to them in tests.

Reviewers: ilya-biryukov

Subscribers: gribozavr, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69896
2019-11-06 11:16:50 -05:00
Yitzhak Mandelbaum ce2b5cb6de [libTooling] Simplify type structure of `Stencil`s.
Summary:
Currently, stencils are defined as a sequence of `StencilParts`. This
differentiation adds an unneeded layer of complexity to the definition of
Stencils. This change significantly simplifies the type structure: a stencil is
now conceptually any object implementing `StencilInterface` and `Stencil` is
just a thin wrapper for pointers to this interface.

To account for the sequencing that was supported by the old `Stencil` type, we
introduce a sequencing class that implements `StencilInterface`. That is,
sequences are just another kind of Stencil and no longer have any special
status.

Corresponding to this change in the type structure, we change the way `cat` is
used (and defined). `cat` bundles multiple features: it builds a stencil from a
sequence of subcomponents and admits multiple different types for its arguments,
while coercing them into the right type. Previously, `cat` was also used to
coerce a single `StencilPart` into a `Stencil`. With that distinction gone, many
uses of `cat` (e.g. in the tests) are unnecessary and have, therefore, been
removed.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69613
2019-11-06 10:28:34 -05:00
Ilya Biryukov 58fa50f437 [Syntax] Add nodes for most common statements
Summary:
Most of the statements mirror the ones provided by clang AST.
Major differences are:
  - expressions are wrapped into 'ExpressionStatement' instead of being
    a subclass of statement,
  - semicolons are always consumed by the leaf expressions (return,
    expression satement, etc),
  - some clang statements are not handled yet, we wrap those into an
    UnknownStatement class, which is not present in clang.

We also define an 'Expression' and 'UnknownExpression' classes in order
to produce 'ExpressionStatement' where needed. The actual implementation
of expressions is not yet ready, it will follow later.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63835
2019-11-06 10:56:06 +01:00
Yitzhak Mandelbaum c14f1ea25e [libTooling] Fix r374962: add more Transformer forwarding decls.
Summary:
The move to a new, single namespace in r374962 left out some type definitions
from the old namespace and resulted in one naming conflict (`text`).  This
revision adds aliases for those definitions and removes one of the `text`
functions from the new namespace.

Reviewers: alexfh

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D69036

llvm-svn: 375003
2019-10-16 14:26:20 +00:00
Yitzhak Mandelbaum 8bb47cd8c3 [libTooling] Put all Transformer declarations in a single namespace.
Summary:
This revision introduces a new namespace, `clang::transformer`, to hold
the declarations for the Transformer library.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68876

llvm-svn: 374962
2019-10-16 01:06:46 +00:00
Yitzhak Mandelbaum cf2438ec13 [libTooling] Change Stencil equality to use `toString()`
Summary:
Removes the `isEqual` method from StencilPartInterface and modifies equality to
use the string representation returned by the `toString` method for comparison.

This means the `run` and `selection` stencils return true by default, and
clients should be cautious in relying on equality operator for comparison of
stencils containing parts generated by these functions.

It also means we no longer need the custom RTTI support (typeId() and
down_cast()), so it has been removed.

Patch by Harshal T. Lehri.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68825

llvm-svn: 374552
2019-10-11 14:02:03 +00:00
Yitzhak Mandelbaum fbdf83521b [libTooling] Move Transformer files to their own directory/library.
Summary:
The Transformer library has been growing inside of
lib/Tooling/Refactoring. However, it's not really related to anything else in
that directory. This revision moves all Transformer-related files into their own
include & lib directories.  A followup revision will (temporarily) add
forwarding headers to help any users migrate their code to the new location.

Reviewers: gribozavr

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68637

llvm-svn: 374271
2019-10-10 02:34:47 +00:00
Yitzhak Mandelbaum e4cec2d3c7 [libTooling][NFC] Fix build break in r373916.
r373916 used raw strings inside macro calls, which breaks some builds.

llvm-svn: 373928
2019-10-07 17:24:23 +00:00
Yitzhak Mandelbaum d5b983555f [libTooling] Add `toString` method to the Stencil class
Summary:
`toString` generates a string representation of the stencil.

Patch by Harshal T. Lehri.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68574

llvm-svn: 373916
2019-10-07 16:20:22 +00:00
Yitzhak Mandelbaum e80e889695 [libTooling] Add various Stencil combinators for expressions.
Summary:
This revision adds three new Stencil combinators:
* `expression`, which idiomatically constructs the source for an expression,
  including wrapping the expression's source in parentheses if needed.
* `deref`, which constructs an idiomatic dereferencing expression.
* `addressOf`, which constructs an idiomatic address-taking expression.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68315

llvm-svn: 373593
2019-10-03 13:01:00 +00:00
Yitzhak Mandelbaum db24ef509e [libTooling] Transformer: refine `SourceLocation` specified as anchor of changes.
Summary: Every change triggered by a rewrite rule is anchored at a particular
location in the source code.  This patch refines how that location is chosen and
defines it as an explicit function so it can be shared by other Transformer
implementations.

This patch was inspired by a bug found by a clang tidy, wherein two changes were
anchored at the same location (the expansion loc of the macro) resulting in the
discarding of the second change.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66652

llvm-svn: 373093
2019-09-27 15:26:04 +00:00
Yitzhak Mandelbaum ea966c1bc0 [libTooling] Add `run` combinator to Stencils.
Summary:
This revision adds `run`, a StencilPart that runs a user-defined function that
computes a result over `MatchFinder::MatchResult`.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67969

llvm-svn: 372936
2019-09-26 00:53:56 +00:00
Yitzhak Mandelbaum 185f56bbbe [libTooling][NFC] Switch StencilTest.cpp to use EXPECT_THAT_EXPECTED
Summary:
Currently, some tests use homegrown matchers to handle `llvm::Expected`
values. This revision standardizes on EXPECT_THAT_EXPECTED and `HasValue`.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67973

llvm-svn: 372918
2019-09-25 20:04:25 +00:00
Yitzhak Mandelbaum 0a81b4ed97 [libTooling] Add `access` and `ifBound` combinators to Stencil library.
Summary:
This revision add the `access` and `ifBound` combinators to the Stencil library:
* `access` -- constructs an idiomatic expression for accessing a member (a
  `MemberExpr`).
* `ifBound` -- chooses between two `StencilParts` based on the whether an id is
  bound in the match (corresponds to the combinator of the same name in
  RangeSelector).

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67633

llvm-svn: 372605
2019-09-23 13:21:42 +00:00
Yitzhak Mandelbaum 1588c08735 [libTooling] Introduce new library of source-code builders.
Summary:
Introduces facilities for easily building source-code strings, including
idiomatic use of parentheses and the address-of, dereference and member-access
operators (dot and arrow) and queries about need for parentheses.

Reviewers: gribozavr

Subscribers: mgorny, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67632

llvm-svn: 372595
2019-09-23 12:40:10 +00:00
Yitzhak Mandelbaum eff88e42f7 [libTooling] Add `ifBound`, `elseBranch` RangeSelector combinators.
Summary:
Adds two new combinators and corresponding tests to the RangeSelector library.
* `ifBound` -- conditional evaluation of range-selectors, based on whether a
   given node id is bound in the match.
* `elseBranch` -- selects the source range of the else and its statement.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67621

llvm-svn: 372410
2019-09-20 17:11:03 +00:00
Dmitri Gribenko b22804b354 [Tooling] Migrated APIs that take ownership of objects to unique_ptr
Subscribers: jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66960

llvm-svn: 370451
2019-08-30 09:29:34 +00:00
Dmitri Gribenko 907452107d Changed FrontendActionFactory::create to return a std::unique_ptr
Subscribers: jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66947

llvm-svn: 370379
2019-08-29 16:38:36 +00:00
Alex Lorenz 1c8a4b7204 Use FileEntryRef for PPCallbacks::HasInclude
This fixes the issue where a filename dependendency was missing if the file that
was referenced with __has_include() was accessed through a symlink in an earlier run,
if the file manager was reused between runs.

llvm-svn: 370081
2019-08-27 17:32:42 +00:00
Alex Lorenz 67d25fede9 Use FileEntryRef for PPCallbacks::FileSkipped
This fixes the issue where a filename dependendency was missing if the file that
was skipped was included through a symlink in an earlier run, if the file
manager was reused between runs.

llvm-svn: 369998
2019-08-27 01:03:25 +00:00
Benjamin Kramer 1e4241cad7 Remove unused variables.
llvm-svn: 369965
2019-08-26 20:51:23 +00:00
Alex Lorenz a39e6490af Re-enable DependencyScannerTest on windows with the right fixes
It should now pass.

llvm-svn: 369832
2019-08-24 01:53:40 +00:00
Alex Lorenz 7a490c5b06 Disable the ScanDepsReuseFilemanager test on Windows
Right now it fails.
I'm going to investigate it and fix it in follow-up commits.

llvm-svn: 369688
2019-08-22 19:00:08 +00:00
Alex Lorenz 4dc5573acc Introduce FileEntryRef and use it when handling includes to report correct dependencies
when the FileManager is reused across invocations

This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.

The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.

Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.

Differential Revision: https://reviews.llvm.org/D65907

llvm-svn: 369680
2019-08-22 18:15:50 +00:00
Johan Vikstrom 6687fde07a [Syntax] Added function to get macro expansion tokens to TokenBuffer.
Summary:
Returns the first token in every mapping where the token is an identifier.
This API is required to be able to highlight macro expansions in clangd.

Reviewers: hokein, ilya-biryukov

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66470

llvm-svn: 369385
2019-08-20 13:34:01 +00:00
Dmitri Gribenko dee011b7f4 Removed the 'id' AST matcher, which is superseded by '.bind()'
Summary:
The 'id' matcher is not even included in the AST Matchers Reference
document, so I don't expect there to be a significant number of users.

There's no reason to provide two ways to do the exact same thing that
only have a minor syntactic difference.

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66462

llvm-svn: 369380
2019-08-20 13:02:28 +00:00
Jonas Devlieghere 2b3d49b610 [Clang] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942
2019-08-14 23:04:18 +00:00
Dmitri Gribenko 030409020c Removed ToolExecutor::isSingleProcess, it is not used by anything
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66212

llvm-svn: 368832
2019-08-14 11:35:04 +00:00
Yitzhak Mandelbaum 42b957aad7 [libTooling] In Transformer, generalize `applyFirst` to admit rules with incompatible matchers.
Summary:
This patch removes an (artificial) limitation of `applyFirst`, which requires
that all of the rules' matchers can be grouped together in a single `anyOf()`.
This change generalizes the code to group the matchers into separate `anyOf`s
based on compatibility. Correspondingly, `buildMatcher` is changed to
`buildMatchers`, to allow for returning a set of matchers rather than just one.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65877

llvm-svn: 368681
2019-08-13 12:31:29 +00:00
Johan Vikstrom d639f6dff1 [AST] No longer visiting CXXMethodDecl bodies created by compiler when method was default created.
Summary:
Clang generates function bodies and puts them in the AST for default methods if it is defaulted outside the class definition.

`
struct A {
   A &operator=(A &&O);
};

A &A::operator=(A &&O) = default;
`

This will generate a function body for the `A &A::operator=(A &&O)` and put it in the AST. This body should not be visited if implicit code is not visited as it is implicit.

This was causing SemanticHighlighting in clangd to generate duplicate tokens and putting them in weird places.

Reviewers: hokein, ilya-biryukov, gribozavr

Subscribers: mgorny, jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65938

llvm-svn: 368402
2019-08-09 07:30:28 +00:00
Ilya Biryukov bfbf6b6cab [Syntax] Do not add a node for 'eof' into the tree
Summary:
While useful as a sentinel value when iterating over tokens, having
'eof' in the tree, seems to do more harm than good.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, kristof.beyls, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64576

llvm-svn: 368062
2019-08-06 17:07:58 +00:00
Johan Vikstrom 86a98baa13 [AST] Fix buildbot failure because of raw string inside macro from 367839.
llvm-svn: 367892
2019-08-05 17:14:46 +00:00
David Green a96cfee98a [AST] Fix RecursiveASTVisitorTest multiline string literal. NFC
Some compiler, notably older gccs (< 8) can have trouble with multiline raw
string literals inside macros. This just moves the code outsize the macro, to
attempt to appease the bots.

llvm-svn: 367885
2019-08-05 16:27:36 +00:00
Johan Vikstrom be60f97d23 [AST] Fix RecursiveASTVisitor visiting implicit constructor initializers.
Summary: RecursiveASTVisitor was visiting implcit constructor initializers. This caused semantic highlighting in clangd to emit error logs. Fixes this by checking if the constructor is written or if the visitor should visit implicit decls.

Reviewers: hokein, ilya-biryukov

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65735

llvm-svn: 367839
2019-08-05 12:20:43 +00:00
Harlan Haskins 8d323d1506 [clang] Adopt new FileManager error-returning APIs
Update the callers of FileManager::getFile and FileManager::getDirectory to handle the new llvm::ErrorOr-returning methods.

Signed-off-by: Harlan Haskins <harlan@apple.com>
llvm-svn: 367616
2019-08-01 21:31:56 +00:00
Ilya Biryukov b455fc429f [Preprocessor] Always discard body of #define if we failed to parse it
Summary:
Preivously we would only discard it if we failed to parse parameter lists.
If we do not consume the body, parser sees tokens inside directive. In
turn, this leads to spurious diagnostics and a crash in TokenBuffer, see
the added tests.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65517

llvm-svn: 367530
2019-08-01 09:10:37 +00:00
Ilya Biryukov 0a42fe70a5 [AST] Treat semantic form of InitListExpr as implicit code in traversals
Summary:
In particular, do not traverse the semantic form if shouldVisitImplicitCode()
returns false.

This simplifies the common case of traversals, avoiding the need to
worry about some expressions being traversed twice.

No tests break after the change, the change would allow to simplify at
least one of the usages, i.e. r366070 which had to handle this in
clangd.

Reviewers: gribozavr

Reviewed By: gribozavr

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64762

llvm-svn: 366672
2019-07-22 09:58:53 +00:00
Sunil Srivastava f4038e75d2 Disallow most calling convention attributes on PS4
PS4 now only allows "cdecl", and its equivalent on PS4, "sysv_abi".

Differential Revision: https://reviews.llvm.org/D64780

llvm-svn: 366617
2019-07-19 21:38:34 +00:00
Yitzhak Mandelbaum 3f1ab737e2 [LibTooling] Relax Transformer to allow rewriting macro expansions
Summary:
Currently, Transformer rejects any changes to source locations inside macro
expansions. This change relaxes that constraint to allow rewrites when the
entirety of the expansion is replaced, since that can be mapped to replacing the
entirety of the expansion range in the file source.  This change makes
Transformer consistent with the handling of edit ranges in `clang::edit::Commit`
(which is used, for example, for applying `FixItHint`s from diagnostics).

Reviewers: ilya-biryukov

Subscribers: gribozavr, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64518

llvm-svn: 366473
2019-07-18 17:44:54 +00:00
Yitzhak Mandelbaum 2e97a1e19e [LibTooling] Add function to translate and validate source range for editing
Summary:
Adds the function `getRangeForEdit` to validate that a given source range is
editable and, if needed, translate it into a range in the source file (for
example, if it's sourced in macro expansions).

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64924

llvm-svn: 366469
2019-07-18 17:26:57 +00:00
Sam McCall 5d9d7c59ee Re-land [JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.
Use //net/dir like other test cases for windows compatibility

llvm-svn: 365975
2019-07-12 23:38:31 +00:00
Russell Gallop 614a78c15a Revert "[JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands."
New test is failing on Windows bot

This reverts commit 9c0391b36a.

llvm-svn: 365906
2019-07-12 15:15:56 +00:00
Sam McCall 9c0391b36a [JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.
Summary:
It's common to use compiler wrappers by setting CC="gomacc clang++".
This results in both args appearing in compile_commands.json, and clang's driver
can't handle this.

This patch attempts to recognize this pattern (by looking for well-known
wrappers) and dropping argv0 in this case.

It conservatively ignores other cases for now:
 - wrappers with unknown names
 - wrappers that accept -flags
 - wrappers where the compiler to use is implied (usually cc or gcc)

This is done at the JSONCompilationDatabase level rather than somewhere more
fundamental, as (hopefully) this isn't a general conceptual problem, but a messy
aspect of the nature of the ecosystem around compile_commands.json.
i.e. compilation databases more tightly tied to the build system should not have
this problem.

Reviewers: phosek, klimek

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64297

llvm-svn: 365887
2019-07-12 10:11:40 +00:00
Fangrui Song ff6836f429 [test] Delete trailing spaces from YAML tests
llvm-svn: 365873
2019-07-12 05:59:28 +00:00
Ilya Biryukov 51dad4196e [Syntax] Move roles into a separate enum
To align with reviewer's suggestions.

llvm-svn: 365479
2019-07-09 13:31:43 +00:00
Ilya Biryukov 9b3f38f990 Reland r365355: [Syntax] Introduce syntax trees
With a fix to a PS4 buildbot crash.

llvm-svn: 365466
2019-07-09 11:32:13 +00:00
Simon Pilgrim 7e3b22758d Revert rL365355 : [Syntax] Introduce syntax trees
Summary:
A tooling-focused alternative to the AST. This commit focuses on the
memory-management strategy and the structure of the AST.

More to follow later:
  - Operations to mutate the syntax trees and corresponding textual
    replacements.
  - Mapping between clang AST nodes and syntax tree nodes.
  - More node types corresponding to the language constructs.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: llvm-commits, mgorny, cfe-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D61637
........
Fixes buildbots which were crashing on SyntaxTests.exe

llvm-svn: 365465
2019-07-09 11:26:35 +00:00
Ilya Biryukov b736969edd [Syntax] Introduce syntax trees
Summary:
A tooling-focused alternative to the AST. This commit focuses on the
memory-management strategy and the structure of the AST.

More to follow later:
  - Operations to mutate the syntax trees and corresponding textual
    replacements.
  - Mapping between clang AST nodes and syntax tree nodes.
  - More node types corresponding to the language constructs.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: llvm-commits, mgorny, cfe-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D61637

llvm-svn: 365355
2019-07-08 17:25:02 +00:00
Tom Stellard 2e97d2aa1b cmake: Add CLANG_LINK_CLANG_DYLIB option
Summary:
Setting CLANG_LINK_CLANG_DYLIB=ON causes clang tools to link against
libclang_shared.so instead of the individual component libraries.

Reviewers: mgorny, beanz, smeenai, phosek, sylvestre.ledru

Subscribers: arphaman, cfe-commits, llvm-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63503

llvm-svn: 365092
2019-07-03 22:45:55 +00:00
Ivan Donchevskii 080014ee6d [clang-tidy] Fix the YAML created for checks like modernize-pass-by-value
Currently this check generates the replacement with the newline in the end.
The proper way to export it to YAML is to have two \n\n instead of one.
Without this fix clients should reinterpret the replacement as
"#include <utility> " instead of "#include <utility>\n"

Differential Revision: https://reviews.llvm.org/D63482

llvm-svn: 365017
2019-07-03 10:21:50 +00:00
Yitzhak Mandelbaum 727bdcb237 [LibTooling] Extend `RewriteRule` with support for adding includes.
Summary:
This revision allows users to specify the insertion of an included directive (at
the top of the file being rewritten) as part of a rewrite rule.  These
directives are bundled with `RewriteRule` cases, so that different cases can
potentially result in different include actions.

Reviewers: ilya-biryukov, gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63892

llvm-svn: 364917
2019-07-02 13:11:04 +00:00
Kadir Cetinkaya 7d719b75ea [clang][ArgumentAdjusters] Do not add fsyntax-only if already exists
Reviewers: hokein

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64063

llvm-svn: 364904
2019-07-02 10:45:53 +00:00
Kadir Cetinkaya 0a43d1fa71 [clang][Tooling] Fix windows build-bots after rL364386
llvm-svn: 364396
2019-06-26 08:39:42 +00:00
Kadir Cetinkaya c3a7302397 [clang][Tooling] Infer target and mode from argv[0] when using JSONCompilationDatabase
Summary:
Wraps JSON compilation database with a target and mode adding database
wrapper. So that driver can correctly figure out which toolchain to use.

Note that clients that wants to make use of this target discovery mechanism
needs to link in TargetsInfos and initialize them at startup.

Reviewers: ilya-biryukov

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63755

llvm-svn: 364386
2019-06-26 07:39:03 +00:00
Ilya Biryukov 5e69f27ef7 [Syntax] Do not glue multiple empty PP expansions to a single mapping
Summary:
This change makes sure we have a single mapping for each macro expansion,
even if the result of expansion was empty.

To achieve that, we take information from PPCallbacks::MacroExpands into
account. Previously we relied only on source locations of expanded tokens.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62953

llvm-svn: 364236
2019-06-24 21:39:51 +00:00
Jordan Rupprecht ff384a2d1c [clang][test] Add missing LambdaTemplateParams test and migrate from getLocStart
These were removed a long time ago in r341573, but this test was missed because it was not in cmake

llvm-svn: 363848
2019-06-19 17:43:58 +00:00
Ilya Biryukov 26c066d66d [Syntax] Fix a crash when dumping empty token buffer
llvm-svn: 363801
2019-06-19 13:56:36 +00:00
Ilya Biryukov 5aed309a4f [Syntax] Add a helper to find expansion by its first spelled token
Summary: Used in clangd for a code tweak that expands a macro.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62954

llvm-svn: 363698
2019-06-18 16:27:27 +00:00
Yitzhak Mandelbaum 2e4a628c06 [LibTooling] Add insert/remove convenience functions for creating `ASTEdit`s.
Summary: `change()` is an all purpose function; the revision adds simple shortcuts for the specific operations of inserting (before/after) or removing source.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62621

llvm-svn: 362707
2019-06-06 14:20:29 +00:00
Pengxuan Zheng e34d1a4e07 [cmake] Remove duplicate TestingSupport library for linking
Summary: This patch cleans up a duplicate use of TestingSupport library.

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62924

llvm-svn: 362637
2019-06-05 19:44:08 +00:00
Ilya Biryukov f4302ad35e [Syntax] Do not depend on llvm targets for Syntax tests. NFC
They are not required and only slow down the build.

llvm-svn: 362530
2019-06-04 17:15:48 +00:00
Yitzhak Mandelbaum e1312c54d3 [LibTooling] Add `before` and `after` selectors for selecting point-ranges relative to nodes.
Summary:
The `before` and `after` selectors allow users to specify a zero-length range --
a point -- at the relevant location in an AST-node's source.  Point ranges can
be useful, for example, to insert a change using an API that takes a range to be
modified (e.g. `tooling::change()`).

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62419

llvm-svn: 361955
2019-05-29 12:40:36 +00:00
Duncan P. N. Exon Smith d4a9cae965 Add missing newline at end of file
llvm-svn: 361708
2019-05-25 22:38:02 +00:00
Nico Weber bab1d8edcf Rename clangToolingRefactor to clangToolingRefactoring for consistency with its directory
See "[cfe-dev] The name of clang/lib/Tooling/Refactoring".

Differential Revision: https://reviews.llvm.org/D62420

llvm-svn: 361684
2019-05-25 00:27:19 +00:00
Yitzhak Mandelbaum fab7205ac4 [LibTooling] Add Explanation parameter to `makeRule`.
Summary:
Conceptually, a single-case RewriteRule has a matcher, edit(s) and an (optional)
explanation. `makeRule` previously only took the matcher and edit(s). This
change adds (optional) support for the explanation.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62390

llvm-svn: 361643
2019-05-24 15:11:45 +00:00
Thomas Lively a8e0d49c0c Fix unresolved symbols when linking tools/clang/unittests/Tooling/ToolingTests
Summary: Add correct cmake dependencies so that `ToolingTests` link
successfully.

Patch by Guanzhong Chen

Reviewers: tlively, aheejin

Reviewed By: tlively

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62333

llvm-svn: 361536
2019-05-23 18:55:00 +00:00
Yitzhak Mandelbaum 1f46d524a1 [LibTooling] Update Stencil to use RangeSelector
Add support for creating a `StencilPart` from any `RangeSelector`, which
broadens the scope of `Stencil`.

Correspondingly, deprecate Stencil's specialized combinators `node` and `sNode`
in favor of using the new `selection` combinator directly (with the appropriate
range selector).

Reviewers: sbenza

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62160

llvm-svn: 361413
2019-05-22 18:03:00 +00:00
Yitzhak Mandelbaum 3ec50e292f [LibTooling] Update Transformer to use RangeSelector instead of NodePart enum.
Transformer provides an enum to indicate the range of source text to be edited.
That support is now redundant with the new (and more general) RangeSelector
library, so we remove the custom enum support in favor of supporting any
RangeSelector.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62149

llvm-svn: 361392
2019-05-22 14:48:19 +00:00
Ilya Biryukov e7230ea7c9 Reland r361148 with a fix to the buildbot failure.
Reverted in r361377.
Also reland the '.gn' files (reverted in r361389).

llvm-svn: 361391
2019-05-22 14:44:45 +00:00
Russell Gallop fd22d7f861 Revert r361148 "[Syntax] Introduce TokenBuffer, start clangToolingSyntax library"
Also reverted r361264 "[Syntax] Rename TokensTest to SyntaxTests. NFC"
which built on it. This is because there were hitting an assert on bots

http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast

llvm-svn: 361377
2019-05-22 12:50:52 +00:00
Yitzhak Mandelbaum f0d9cfa591 [LibTooling] Address post-commit feedback for r361152
Fixes a redundant dependency and moves another to its proper place.

Reviewers: thakis

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62201

llvm-svn: 361285
2019-05-21 18:48:58 +00:00
Ilya Biryukov 6c0531222a [Syntax] Rename TokensTest to SyntaxTests. NFC
To be more consistent with conventions used in the codebase. The new
name will be a better fit when more bits of the syntax library land.

llvm-svn: 361264
2019-05-21 14:37:41 +00:00
Nico Weber 4522e26696 Try to fix build with older gccs after r361152
Also merge the cmake change there to the gn build.

llvm-svn: 361209
2019-05-21 00:27:26 +00:00
Dmitri Gribenko 76e3a04951 Renamed `apply` to `select` to avoid ADL conflict with `std::apply`
Summary:
`std::apply` in C++14 and above is defined with two unrestricted arguments, and
it wins overload resolution in this case.

Reviewers: ilya-biryukov

Subscribers: cfe-commits, ymandel

Tags: #clang

Differential Revision: https://reviews.llvm.org/D62150

llvm-svn: 361170
2019-05-20 16:30:49 +00:00
Yitzhak Mandelbaum ef29c6a63e [LibTooling] Fix build break in test after r361152.
r361152 broke gcc builds.

llvm-svn: 361160
2019-05-20 14:44:40 +00:00
Yitzhak Mandelbaum 27872b8d55 [LibTooling] Add RangeSelector library for defining source ranges based on bound AST nodes.
Summary:

The RangeSelector library defines a combinator language for specifying source
ranges based on bound ids for AST nodes.  The combinator approach follows the
design of the AST matchers.  The RangeSelectors defined here will be used in
both RewriteRule, for specifying source affected by edit, and in Stencil for
specifying source to use constructively in a replacement.

Reviewers: ilya-biryukov

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61774

llvm-svn: 361152
2019-05-20 13:15:14 +00:00
Ilya Biryukov ddd5d5dbc8 [Syntax] Introduce TokenBuffer, start clangToolingSyntax library
Summary:
TokenBuffer stores the list of tokens for a file obtained after
preprocessing. This is a base building block for syntax trees,
see [1] for the full proposal on syntax trees.

This commits also starts a new sub-library of ClangTooling, which
would be the home for the syntax trees and syntax-tree-based refactoring
utilities.

[1]: https://lists.llvm.org/pipermail/cfe-dev/2019-February/061414.html

Reviewers: gribozavr, sammccall

Reviewed By: sammccall

Subscribers: mgrang, riccibruno, Eugene.Zelenko, mgorny, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59887

llvm-svn: 361148
2019-05-20 13:00:42 +00:00
Yitzhak Mandelbaum 8369a9beb7 [LibTooling] Add support to Transformer for composing rules as an ordered choice.
This revision updates `RewriteRule` to support multiple subrules that are
interpreted as an ordered-choice (apply the first one that matches). With this
feature, users can write the rules that appear later in the list of subrules
knowing that previous rules' patterns *have not matched*, freeing them from
reasoning about those cases in the current pattern.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61335

llvm-svn: 361037
2019-05-17 14:23:33 +00:00
Sam McCall 87ad30be5f [Tooling] Add -x flags when inferring compile commands for files with no/invalid extension.
Summary: We treat them as headers, as the motivating case is C++ standard library.

Reviewers: kadircet

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61633

llvm-svn: 360153
2019-05-07 14:34:06 +00:00
Hamza Sood 8205a814a6 [c++20] Implement P0428R2 - Familiar template syntax for generic lambdas
Differential Revision: https://reviews.llvm.org/D36527

llvm-svn: 359967
2019-05-04 10:49:46 +00:00
Yitzhak Mandelbaum a5dadbe1aa [LibTooling] Fix broken test after r359574.
r359574 changed the way that failures are reported, which broke the test TransformerTest.NodePartNameDeclRefFailure which detects a faiure.

llvm-svn: 359578
2019-04-30 17:24:36 +00:00
Yitzhak Mandelbaum aecc59c5f9 [LibTooling] Change Transformer's TextGenerator to a partial function.
Summary:
Changes the signature of the TextGenerator std::function to return an Expected<std::string>
instead of std::string to allow for (non-fatal) failures.  Previously, we
expected that any failures would be expressed with assertions. However, that's
unfriendly to running the code in servers or other places that don't want their
library calls to crash the program.

Correspondingly, updates Transformer's handling of failures in TextGenerators
and the signature of `ChangeConsumer`.

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D61015

llvm-svn: 359574
2019-04-30 16:48:33 +00:00
Simon Pilgrim 648a8cfe70 Fix gcc "-Wdangling-else" warning. NFCI.
llvm-svn: 359551
2019-04-30 10:57:37 +00:00
Bjorn Pettersson 18b0442560 [LibTooling] Fix -Wsign-compare after r358697
llvm-svn: 358745
2019-04-19 09:10:42 +00:00
Yitzhak Mandelbaum fa1552e8c9 [LibTooling] Extend Transformer to support multiple simultaneous changes.
Summary: This revision allows users to specify independent changes to multiple (related) sections of the input.  Previously, only a single section of input could be selected for replacement.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: jfb, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60408

llvm-svn: 358697
2019-04-18 17:52:24 +00:00
Yitzhak Mandelbaum e955f8bac4 [LibTooling] Add Stencil library for format-string style codegen.
Summary:
This file defines the *Stencil* abstraction: a code-generating object, parameterized by named references to (bound) AST nodes.  Given a match result, a stencil can be evaluated to a string of source code.

A stencil is similar in spirit to a format string: it is composed of a series of raw text strings, references to nodes (the parameters) and helper code-generation operations.

See thread on cfe-dev list with subject "[RFC] Easier source-to-source transformations with clang tooling" for background.

Reviewers: sbenza

Reviewed By: sbenza

Subscribers: ilya-biryukov, mgorny, jfb, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59371

llvm-svn: 358691
2019-04-18 17:23:01 +00:00
Haojian Wu f2879d8a48 [clang-tidy] Add fix descriptions to clang-tidy checks.
Summary:
Motivation/Context: in the code review system integrating with clang-tidy,
clang-tidy doesn't provide a human-readable description of the fix. Usually
developers have to preview a code diff (before vs after apply the fix) to
understand what the fix does before applying a fix.

This patch proposes that each clang-tidy check provides a short and
actional fix description that can be shown in the UI, so that users can know
what the fix does without previewing diff.

This patch extends clang-tidy framework to support fix descriptions (will add implementations for
existing checks in the future). Fix descriptions and fixes are emitted via diagnostic::Note (rather than
attaching the main warning diagnostic).

Before this patch:

```
void MyCheck::check(...) {
   ...
   diag(loc, "my check warning") <<  FixtItHint::CreateReplacement(...);
}
```

After:

```
void MyCheck::check(...) {
   ...
   diag(loc, "my check warning"); // Emit a check warning
   diag(loc, "fix description", DiagnosticIDs::Note) << FixtItHint::CreateReplacement(...); // Emit a diagnostic note and a fix
}
```

Reviewers: sammccall, alexfh

Reviewed By: alexfh

Subscribers: MyDeveloperDay, Eugene.Zelenko, aaron.ballman, JonasToth, xazax.hun, jdoerfert, cfe-commits

Tags: #clang-tools-extra, #clang

Differential Revision: https://reviews.llvm.org/D59932

llvm-svn: 358576
2019-04-17 12:53:59 +00:00
Eric Liu dc8d8fb20b [Lookup] Invisible decls should not be ambiguous when renaming.
Summary:
For example, a renamed type in a header file can conflict with declaration in
a random file that includes the header, but we should not consider the decl ambiguous if
it's not visible at the rename location. This improves consistency of generated replacements
when header file is included in different TUs.

Reviewers: hokein

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60257

llvm-svn: 358378
2019-04-15 08:46:34 +00:00
Sam McCall 36913e3f8f [Tooling] add a Heuristic field indicating that a CompileCommand was guessed.
Summary:
Use cases:
 - a tool that dumps the heuristic used for each header in a project can
   be used to evaluate changes to the heuristic
 - we want to expose this information to users in clangd as it affects
   accuracy/reliability of editor features
 - express interpolation tests more directly

Reviewers: ilya-biryukov, klimek

Subscribers: ioeric, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60194

llvm-svn: 357770
2019-04-05 15:22:20 +00:00
Yitzhak Mandelbaum fdd98782aa [LibTooling] Add Transformer, a library for source-to-source transformations.
Summary: Adds a basic version of Transformer, a library supporting the concise specification of clang-based source-to-source transformations.  A full discussion of the end goal can be found on the cfe-dev list with subject "[RFC] Easier source-to-source transformations with clang tooling".

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: ioeric, ABataev, mgorny, jfb, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59376

llvm-svn: 357768
2019-04-05 15:14:05 +00:00
Yitzhak Mandelbaum 84f2271acd [LibTooling] Add "SourceCode" library for functions relating to source-code manipulation.
Summary:
Introduces a utility library in Refactoring/ to collect routines related to
source-code manipulation.  In this change, we move "extended-range" functions
from the FixIt library (in clangTooling) to this new library.

We need to use this functionality in Refactoring/ and cannot access it if it
resides in Tooling/, because that would cause clangToolingRefactor to depend on
clangTooling, which would be a circular dependency.

Reviewers: ilya-biryukov, ioeric

Reviewed By: ilya-biryukov

Subscribers: mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D60269

llvm-svn: 357764
2019-04-05 14:05:03 +00:00
Alexey Bataev 35ccd864e0 Revert "[LibTooling] Add Transformer, a library for source-to-source transformations."
This reverts commit r357576 to fix the problem with the cyclic
dependencies between libTooling and libToolingRefactor.

llvm-svn: 357612
2019-04-03 17:34:04 +00:00
Yitzhak Mandelbaum d5856302f7 [LibTooling] Add Transformer, a library for source-to-source transformations.
Summary: Adds a basic version of Transformer, a library supporting the concise specification of clang-based source-to-source transformations.  A full discussion of the end goal can be found on the cfe-dev list with subject "[RFC] Easier source-to-source transformations with clang tooling".

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: mgorny, jfb, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59376

llvm-svn: 357576
2019-04-03 13:05:19 +00:00
Fangrui Song 75e74e077c Range-style std::find{,_if} -> llvm::find{,_if}. NFC
llvm-svn: 357359
2019-03-31 08:48:19 +00:00
Eric Liu 52b49ab3fd [Tooling] Add more scope specifiers until spelling is not ambiguous.
Summary:
Previously, when the renamed spelling is ambiguous, we simply use the
full-qualfied name (with leading "::"). This patch makes it try adding
additional specifiers one at a time until name is no longer ambiguous,
which allows us to find better disambuguated spelling.

Reviewers: kadircet, gribozavr

Subscribers: jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59487

llvm-svn: 356446
2019-03-19 10:12:15 +00:00
Benjamin Kramer a65ec7eec8 Make getFullyQualifiedName qualify both the pointee and class type for member ptr types
We already handle pointers and references, member ptrs are just another
special case. Fixes PR40732.

Differential Revision: https://reviews.llvm.org/D59387

llvm-svn: 356250
2019-03-15 11:09:41 +00:00
Yitzhak Mandelbaum 60a4163f6d [LibTooling] Add retrieval of extended AST-node source to FixIt library
Summary:
Introduces variants of `getText` and `getSourceRange` that extract the source text of an AST node potentially with a trailing token.

Some of the new functions manipulate `CharSourceRange`s, rather than `SourceRange`s, because they document and dynamically enforce their type.  So, this revision also updates the corresponding existing FixIt functions to manipulate `CharSourceRange`s.  This change is not strictly necessary, but seems like the correct choice, to keep the API self-consistent.

This revision is the first in a series intended to improve the abstractions available to users for writing source-to-source transformations.  A full discussion of the end goal can be found on the cfe-dev list with subject "[RFC] Easier source-to-source transformations with clang tooling".

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: kimgr, riccibruno, JonasToth, jdoerfert, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D58556

llvm-svn: 356095
2019-03-13 19:48:51 +00:00
Kadir Cetinkaya 7ec1ec10c9 [clang][Tooling] Delete dots and dotdots when generating absolute paths
Summary:
GetAllFiles interface returns absolute paths, but keeps dots and dot
dots. This patch makes those paths canonical by deleting them.

Reviewers: hokein

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59079

llvm-svn: 355678
2019-03-08 09:42:04 +00:00
Fangrui Song 9ac13a1244 Use llvm::is_contained. NFC
llvm-svn: 353635
2019-02-10 05:54:57 +00:00
Eric Liu 2473961305 [Tooling] Handle #pragma once header guard in include insertion.
Reviewers: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D57223

llvm-svn: 352503
2019-01-29 14:40:01 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Kadir Cetinkaya 2eff821bbd [tooling] Add a new argument adjuster for deleting plugin related command line args
Summary:
Currently both clangd and clang-tidy makes use of this mechanism so
putting it into tooling so that all tools can make use of it.

Reviewers: ilya-biryukov, sammccall

Subscribers: ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D56856

llvm-svn: 351531
2019-01-18 09:00:31 +00:00
Sam McCall f2e25e708c Reapply [Tooling] Make clang-tool find libc++ dir on mac when running on a file without compilation database.
This reverts commit r351282, and re-lands r351222 and r351229 with the
use-after-free fixed.

llvm-svn: 351316
2019-01-16 09:41:26 +00:00
Vlad Tsyrklevich e3226737ce Revert "[Tooling] Make clang-tool find libc++ dir on mac when running on a file without compilation database."
This reverts commits r351222 and r351229, they were causing ASan/MSan failures
on the sanitizer bots.

llvm-svn: 351282
2019-01-16 00:37:39 +00:00
Haojian Wu ccb98a63c2 [Tooling] Fix broken compliation databse tests.
I forgot to update the unittest in r351222.

llvm-svn: 351229
2019-01-15 19:51:39 +00:00
Sam McCall 32ef52063c [AST] Fix double-traversal of code in top-level lambdas in RAV(implicit = yes).
Summary:
Prior to r351069, lambda classes were traversed or not depending on the
{Function, Class, Namespace, TU} DeclContext containing them.
If it was a function (common case) they were not traversed.
If it was a namespace or TU (top-level lambda) they were traversed as part of
that DeclContext traversal.

r351069 "fixed" RAV to traverse these as part of the LambdaExpr, which is the
right place. But top-level lambdas are now traversed twice.
We fix that as blocks and block captures were apparently fixed in the past.

Maybe it would be nicer to avoid adding the lambda classes to the DeclContext
in the first place, but I can't work out the implications of that.

Reviewers: bkramer, klimek

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D56665

llvm-svn: 351075
2019-01-14 17:16:00 +00:00
Sam McCall e60151c915 [AST] RecursiveASTVisitor visits lambda classes when implicit visitation is on.
Summary:
This fixes ASTContext's parent map for nodes in such classes (e.g. operator()).
https://bugs.llvm.org/show_bug.cgi?id=39949

This also changes the observed shape of the AST for implicit RAVs.
- this includes AST MatchFinder: cxxRecordDecl() now matches lambda classes,
functionDecl() matches the call operator, and the parent chain is body -> call
operator -> lambda class -> lambdaexpr rather than body -> lambdaexpr.
- this appears not to matter for the ASTImporterLookupTable builder
- this doesn't matter for the other RAVs in-tree.

In order to do this, we remove the TraverseLambdaBody hook. The problem is it's
hard/weird to ensure this hook is called when traversing via the implicit class.
There were just two users of this hook in-tree, who use it to skip bodies.
I replaced these with explicitly traversing the captures only. Another approach
would be recording the bodies when the lambda is visited, and then recognizing
them later.
I'd be open to suggestion on how to preserve this hook, instead.

Reviewers: aaron.ballman, JonasToth

Subscribers: cfe-commits, rsmith, jdennett

Differential Revision: https://reviews.llvm.org/D56444

llvm-svn: 351047
2019-01-14 10:31:42 +00:00
Fangrui Song 5313327f61 Add explicit dependency on clangSerialization for a bunch of components to fix -DBUILD_SHARED_LIBS=on build
This is a more thorough fix of rC348911.
The story about -DBUILD_SHARED_LIBS=on build after rC348907 (Move PCHContainerOperations from Frontend to Serialization) is:

1. libclangSerialization.so defines PCHContainerReader dtor, ...
2. clangFrontend and clangTooling define classes inheriting from PCHContainerReader, thus their DSOs have undefined references on PCHContainerReader dtor
3. Components depending on either clangFrontend or clangTooling cannot be linked unless they have explicit dependency on clangSerialization due to the default linker option -z defs. The explicit dependency could be avoided if libclang{Frontend,Tooling}.so had these undefined references.

This patch adds the explicit dependency on clangSerialization to make them build.

llvm-svn: 348915
2018-12-12 08:02:18 +00:00
Hamza Sood 2d9d026c00 [unittests] Add C++17 and C++2a support to the tooling tests
llvm-svn: 348603
2018-12-07 15:46:29 +00:00
Alexander Kornienko af17a3866b Implement YAML serialization of notes in clang::tooling::Diagnostic.
llvm-svn: 347371
2018-11-21 01:06:32 +00:00
Scott Linder c0830f5577 [Support] Teach YAMLIO about polymorphic types
Add support for "polymorphic" types to YAMLIO.

PolymorphicTraits can dynamically switch between other traits (Scalar, Map, or
Sequence). When inputting, the PolymorphicTraits type is told which type to
become, and when outputting the PolymorphicTraits type is asked which type it
currently is.

Also add support for TaggedScalarTraits to allow dynamically differentiating
between multiple scalar types using YAML tags.

Serialize empty maps as "{}" and empty sequences as "[]", so that types
are preserved when round-tripping PolymorphicTraits. This change has
equivalent semantics, but may break e.g. tests which compare output
verbatim.

Differential Revision: https://reviews.llvm.org/D48144

llvm-svn: 346884
2018-11-14 19:39:59 +00:00
Sam McCall 814e7974c6 [AST] Allow limiting the scope of common AST traversals (getParents, RAV).
Summary:
The goal is to allow analyses such as clang-tidy checks to run on a
subset of the AST, e.g. "only on main-file decls" for interactive tools.

Today, these become "problematically global" by running RecursiveASTVisitors
rooted at the TUDecl, or by navigating up via ASTContext::getParent().

The scope is restricted using a set of top-level-decls that RecursiveASTVisitors
should be rooted at. This also applies to the visitor that populates the
parent map, and so the top-level-decls are considered to have no parents.

This patch makes the traversal scope a mutable property of ASTContext.
The more obvious way to do this is to pass the top-level decls to
relevant functions directly, but this has some problems:
 - it's error-prone: accidentally mixing restricted and unrestricted
   scopes is a performance trap. Interleaving multiple analyses is
   common (many clang-tidy checks run matchers or RAVs from matcher callbacks)
 - it doesn't map well to the actual use cases, where we really do want
   *all* traversals to be restricted.
 - it involves a lot of plumbing in parts of the code that don't care
   about traversals.
This approach was tried out in D54259 and D54261, I wanted to like it
but it feels pretty awful in practice.

Caveats: to get scope-limiting behavior of RecursiveASTVisitors, callers
have to call the new TraverseAST(Ctx) function instead of TraverseDecl(TU).
I think this is an improvement to the API regardless.

Reviewers: klimek, ioeric

Subscribers: mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D54309

llvm-svn: 346847
2018-11-14 10:33:30 +00:00
Haojian Wu cd5e59f552 [Tooling] Add "-filter" option to AllTUsExecution
Summary: We can run the tools on a subset files of compilation database.

Reviewers: ioeric

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D54092

llvm-svn: 346131
2018-11-05 13:42:05 +00:00
Zachary Turner 9f169afab2 Make YAML quote forward slashes.
If you have the string /usr/bin, prior to this patch it would not
be quoted by our YAML serializer.  But a string like C:\src would
be, due to the presence of a backslash.  This makes the quoting
rules of basically every single file path different depending on
the path syntax (posix vs. Windows).

While technically not required by the YAML specification to quote
forward slashes, when the behavior of paths is inconsistent it
makes it difficult to portably write FileCheck lines that will
work with either kind of path.

Differential Revision: https://reviews.llvm.org/D53169

llvm-svn: 344359
2018-10-12 16:31:20 +00:00
Zachary Turner 9c544199cf Revert "Make YAML quote forward slashes."
This reverts commit b86c16ad8c97dadc1f529da72a5bb74e9eaed344.

This is being reverted because I forgot to write a useful
commit message, so I'm going to resubmit it with an actual
commit message.

llvm-svn: 344358
2018-10-12 16:31:08 +00:00
Zachary Turner ec234052a6 Make YAML quote forward slashes.
llvm-svn: 344357
2018-10-12 16:24:09 +00:00
Jonas Devlieghere fc51490baf Lift VFS from clang to llvm (NFC)
This patch moves the virtual file system form clang to llvm so it can be
used by more projects.

Concretely the patch:
 - Moves VirtualFileSystem.{h|cpp} from clang/Basic to llvm/Support.
 - Moves the corresponding unit test from clang to llvm.
 - Moves the vfs namespace from clang::vfs to llvm::vfs.
 - Formats the lines affected by this change, mostly this is the result of
   the added llvm namespace.

RFC on the mailing list:
http://lists.llvm.org/pipermail/llvm-dev/2018-October/126657.html

Differential revision: https://reviews.llvm.org/D52783

llvm-svn: 344140
2018-10-10 13:27:25 +00:00
Vedant Kumar 35d303adbf [Sema] Remove location from implicit capture init expr
A lambda's closure is initialized when the lambda is declared. For
implicit captures, the initialization code emitted from EmitLambdaExpr
references source locations *within the lambda body* in the function
containing the lambda. This results in a poor debugging experience: we
step to the line containing the lambda, then into lambda, out again,
over and over, until every capture's field is initialized.

To improve stepping behavior, assign the starting location of the lambda
to expressions which initialize an implicit capture within it.

rdar://39807527

Differential Revision: https://reviews.llvm.org/D50927

llvm-svn: 342194
2018-09-13 23:28:25 +00:00
Hamza Sood e05bf60631 [Tooling] Improve handling of CL-style options
This patch fixes the handling of clang-cl options in InterpolatingCompilationDatabase.
They were previously ignored completely, which led to a lot of bugs:

Additional options were being added with the wrong syntax. E.g. a file was
specified as C++ by adding -x c++, which causes an error in CL mode.

The args were parsed and then rendered, which means that the aliasing information
was lost. E.g. /W4 was rendered to -Wall, which in CL mode means -Weverything.

CL options were ignored when checking things like -std=, so a lot of logic was
being bypassed.

Differential Revision: https://reviews.llvm.org/D51321

llvm-svn: 341760
2018-09-09 12:06:35 +00:00
Ilya Biryukov 5167e2d1af Parse compile commands lazily in InterpolatingCompilationDatabase
Summary:
This greatly reduces the time to read 'compile_commands.json'.
For Chromium on my machine it's now 0.7 seconds vs 30 seconds before the
change.

Reviewers: sammccall, jfb

Reviewed By: sammccall

Subscribers: mgrang, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D51314

llvm-svn: 340838
2018-08-28 16:15:56 +00:00
Ilya Biryukov dbdd8e5238 [Tooling] Add a isSingleProcess() helper to ToolExecutor
Summary:
Used in clangd's symbol builder to optimize for the common
shared-memory executor case.

Reviewers: ioeric

Reviewed By: ioeric

Subscribers: kadircet, cfe-commits

Differential Revision: https://reviews.llvm.org/D51164

llvm-svn: 340599
2018-08-24 09:03:29 +00:00
Stephen Kelly f2ceec4811 Port getLocStart -> getBeginLoc
Reviewers: teemperor!

Subscribers: jholewinski, whisperity, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D50350

llvm-svn: 339385
2018-08-09 21:08:08 +00:00
Eric Liu db887cab7a Fully qualify the renamed symbol if the shortened name is ambiguous.
Summary:
For example, when renaming `a:🅱️:foo` to `y::foo` below, replacing
`x::foo()` with `y::foo()` can cause ambiguity. In such cases, we simply fully
qualify the name with leading `::`.
```
namespace a {
namespace b {
namespace x { void foo() {} }
namespace y { void foo() {} }
}
}

namespace a {
namespace b {
void f() { x::foo(); }
}
}
```

Reviewers: ilya-biryukov, hokein

Reviewed By: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D50189

llvm-svn: 338832
2018-08-03 09:16:07 +00:00
Simon Marchi 6567c9e5e0 [Tooling] Add operator== to CompileCommand
Summary:
It does the obvious thing of comparing all fields.  This will be needed
for a clangd patch I have in the pipeline.

Subscribers: dblaikie, ilya-biryukov, ioeric, cfe-commits

Differential Revision: https://reviews.llvm.org/D49265

llvm-svn: 337284
2018-07-17 14:13:05 +00:00
Roman Lebedev ed88ed81be [clang][tooling] Don't forget to link to clangToolingInclusions.
Fixes build with shared libs, broken by rL333874.
Some buildbot converage is sorely missing.

llvm-svn: 333890
2018-06-04 12:04:41 +00:00
Eric Liu 44564ac7eb Reland "Move #include manipulation code to new lib/Tooling/Inclusions."
This reverts commit r333534 (i.e. reland r332720) after fixing module build.

Differential Revision: https://reviews.llvm.org/D47068

llvm-svn: 333874
2018-06-04 09:04:12 +00:00
Eric Liu cfd01f94b2 Revert "Reland "Move #include manipulation code to new lib/Tooling/Inclusions.""
This reverts commit r333532. Revert for now to fix an internal bot issue.

llvm-svn: 333534
2018-05-30 12:09:58 +00:00
Eric Liu 4f20e9de0a Reland "Move #include manipulation code to new lib/Tooling/Inclusions."
This reverts commit r332751 (i.e. reland r332720) after fixing module
build.

Differential Revision: https://reviews.llvm.org/D47068

llvm-svn: 333532
2018-05-30 11:51:48 +00:00
Eric Liu 1edf4bc68c Revert "Move #include manipulation code to new lib/Tooling/Inclusions."
This reverts commit r332720. This break build bot with modules. Need to
investigate. Revert the change to unbreak bots.

llvm-svn: 332751
2018-05-18 18:33:08 +00:00
Eric Liu e0a3f5b1a9 Move #include manipulation code to new lib/Tooling/Inclusions.
Summary:
clangToolingCore is linked into almost everything (incl. clang), but
not few tools need #include manipulation at this point. So pull this into a
separate library in Tooling.

Reviewers: ilya-biryukov

Subscribers: klimek, mgorny, cfe-commits, thakis

Differential Revision: https://reviews.llvm.org/D47068

llvm-svn: 332720
2018-05-18 14:16:37 +00:00
Eric Liu 7129e63bcc [Tooling] Pull #include manipulation code from clangFormat into libToolingCore.
Summary: Also pull #include related style out of FormatStyle as tooling::IncludeStyle.

Reviewers: ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: klimek, mgorny, cfe-commits, djasper

Differential Revision: https://reviews.llvm.org/D46496

llvm-svn: 332287
2018-05-14 20:17:53 +00:00
Sterling Augustine 5891064692 Allow modifying the PrintingPolicy for fully qualified names.
Author: mikhail.ramalho@gmail.com
llvm-svn: 331552
2018-05-04 20:12:39 +00:00
Nico Weber d637c05986 IWYU for llvm-config.h in clang. See r331124 for details.
llvm-svn: 331177
2018-04-30 13:52:15 +00:00
Nico Weber 1865df4996 s/LLVM_ON_WIN32/_WIN32/, clang
LLVM_ON_WIN32 is set exactly with MSVC and MinGW (but not Cygwin) in
HandleLLVMOptions.cmake, which is where _WIN32 defined too.  Just use the
default macro instead of a reinvented one.

See thread "Replacing LLVM_ON_WIN32 with just _WIN32" on llvm-dev and cfe-dev.
No intended behavior change.

llvm-svn: 331069
2018-04-27 19:11:14 +00:00
David Zarzycki d488daa032 [UnitTests] NFC/build-perf: Break up nontrivial compile jobs
RecursiveASTVisitorTest.cpp is one of the longest compile jobs and a
build bottleneck on many-core machines. This patch breaks that file and
some peer files up into smaller files to increase build concurrency and
overall rebuild performance.

llvm-svn: 330353
2018-04-19 18:19:02 +00:00
Haojian Wu 27695da998 [Tooling] Correct the "-std" compile command option.
Summary:
"-std c++11" is not valid in compiler, we have to use "-std=c++11".

Test in vscode with this patch, code completion for header works as expected.

Reviewers: sammccall

Subscribers: cfe-commits, klimek

Differential Revision: https://reviews.llvm.org/D45512

llvm-svn: 329786
2018-04-11 09:18:18 +00:00
Sam McCall ea9773ac69 [Tooling] A CompilationDatabase wrapper that infers header commands.
Summary:
The wrapper finds the closest matching compile command using filename heuristics
and makes minimal tweaks so it can be used with the header.

Subscribers: klimek, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D45006

llvm-svn: 329580
2018-04-09 15:17:39 +00:00
Ilya Biryukov 749788eab6 Updated a usage of createTemporaryFile that does not expect file to be created.
Summary:
This fixes a usage of createTemporaryFile in clang repo after
a change in llvm repo.

Reviewers: klimek, bkramer, krasimir, espindola, ilya-biryukov

Reviewed By: ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D36828

llvm-svn: 327852
2018-03-19 14:20:25 +00:00