Commit Graph

631 Commits

Author SHA1 Message Date
Eduardo Caldas ed83095254 [SyntaxTree] Use annotations to reduce noise on member function tests
Differential Revision: https://reviews.llvm.org/D86439
2020-08-24 14:31:45 +00:00
Eduardo Caldas 4e8dd506e6 [SyntaxTree] Split array declarator tests
Differential Revision: https://reviews.llvm.org/D86437
2020-08-24 14:31:45 +00:00
Eduardo Caldas 1beb11c61a [SyntaxTree] Use annotations in Statement tests
Differential Revision: https://reviews.llvm.org/D86345
2020-08-21 14:42:33 +00:00
Eduardo Caldas 85c15f17cc [SyntaxTree] Add support for `this`
Differential Revision: https://reviews.llvm.org/D86298
2020-08-21 08:01:29 +00:00
Eduardo Caldas e4e983e240 [SyntaxTree] Split tests related to Namespace
Differential Revision: https://reviews.llvm.org/D86139
2020-08-20 15:14:56 +00:00
Eduardo Caldas ba32915db2 [SyntaxTree] Add support for `MemberExpression`
Differential Revision: https://reviews.llvm.org/D86227
2020-08-20 14:57:35 +00:00
Eduardo Caldas c8c92b54d7 [SyntaxTree] Use Annotations based tests for expressions
In this process we also create some other tests, in order to not lose
coverage when focusing on the annotated code

Differential Revision: https://reviews.llvm.org/D85962
2020-08-18 13:00:56 +00:00
Eduardo Caldas ab58c9ee8a [SyntaxTree] Implement annotation-based test infrastructure
We add the method `SyntaxTreeTest::treeDumpEqualOnAnnotations`, which
allows us to compare the treeDump of only annotated code. This will reduce a
lot of noise from our `BuildTreeTest` and make them short and easier to
read.
2020-08-18 13:00:56 +00:00
Nathan Ridge 00d7b7d014 [clang] Fix visitation of ConceptSpecializationExpr in constrained-parameter
Summary: RecursiveASTVisitor needs to traverse TypeConstraint::ImmediatelyDeclaredConstraint

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D84136
2020-08-18 00:32:34 -04:00
Eduardo Caldas 2e4a20fd70 [SyntaxTree] Split `TreeTestBase` into header and source
* Switch to using directive on source files.
* Remove unused `SyntaxTreeTest::addFile`

Differential Revision: https://reviews.llvm.org/D85913
2020-08-14 07:29:07 +00:00
Eduardo Caldas 9c2e708f0d [SyntaxTree] Clean `#includes` in `TreeTestBase.h`
Differential Revision: https://reviews.llvm.org/D85898
2020-08-13 13:30:57 +00:00
Eduardo Caldas d17437d2bd [SyntaxTree] Split `TreeTest.cpp`
We extract the test infrastructure into `TreeTestBase.h` and split the
tests into `MutationsTest.cpp` and `BuildTreeTest.cpp`
2020-08-13 13:30:57 +00:00
Eduardo Caldas 833c2b6be2 [SyntaxTree] Rename tests following `TestSuite_TestCase` + nits 2020-08-13 08:18:14 +00:00
Eduardo Caldas d1211fd1ec [SyntaxTree] Split tests for expressions
We do that because:
* Big tests generated big tree dumps that could hardly serve as documentation.
* In most cases the tests didn't share setup, thus there was not much addition in lines of code.

We split tests for:
* `UserDefinedLiteral`
* `NestedBinaryOperator`
* `UserDefinedBinaryOperator`
* `UserDefinedPrefixOperator`
* `QualifiedId`

Differential Revision: https://reviews.llvm.org/D85819
2020-08-13 08:18:14 +00:00
Eduardo Caldas ac37afa650 [SyntaxTree] Unbox operators into tokens for nodes generated from `CXXOperatorCallExpr`
For an user define `<`, `x < y` would yield the syntax tree:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
|   `-x
|-IdExpression
| `-UnqualifiedId
|   `-<
`-IdExpression
  `-UnqualifiedId
    `-y
```
But there is no syntatic difference at call site between call site or
built-in `<`. As such they should generate the same syntax tree, namely:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
|   `-x
|-<
`-IdExpression
  `-UnqualifiedId
    `-y
```

Differential Revision: https://reviews.llvm.org/D85750
2020-08-12 08:01:18 +00:00
Yitzhak Mandelbaum d8c1f43dcc [libTooling] Move RewriteRule include edits to ASTEdit granularity.
Currently, changes to includes are applied to an entire rule. However,
include changes may be specific to particular edits within a rule (for example,
they may apply to one file but not another). Also, include changes may need to
carry metadata, just like other changes. So, we make include changes first-class
edits.

Reviewed By: tdl-g

Differential Revision: https://reviews.llvm.org/D85734
2020-08-11 16:47:14 +00:00
Kadir Cetinkaya ea8e71c3da
[clang][HeaderInsert] Do not treat defines with values as header guards
This was resulting in inserting headers at bogus locations, see
https://github.com/ycm-core/YouCompleteMe/issues/3736 for an example.

Differential Revision: https://reviews.llvm.org/D85590
2020-08-11 16:02:11 +02:00
Eduardo Caldas f9500cc487 [SyntaxTree] Expand support for `NestedNameSpecifier`
Summary:
We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.

To achieve this we:
* Use the `RecursiveASTVisitor`'s API to traverse
`NestedNameSpecifierLoc`s and automatically create its syntax nodes
* Add links from the `NestedNameSpecifierLoc`s to their syntax nodes.

In this way, from any semantic construct that has a `NestedNameSpecifier`,
we implicitly generate its syntax node via RAV and we can easily access
this syntax node via the links we added.
2020-08-10 15:47:20 +00:00
Eduardo Caldas 8abb5fb68f [SyntaxTree] Use simplified grammar rule for `NestedNameSpecifier` grammar nodes
This is our grammar rule for nested-name-specifiers:
globalbal-specifier:
  /*empty*/
simple-template-specifier:
  template_opt simple-template-id
name-specifier:
  global-specifier
  decltype-specifier
  identifier
  simple-template-specifier
nested-name-specifier:
  list(name-specifier, ::, non-empty, terminated)

It is a relaxed version of C++ [expr.prim.id] and quite simpler to map to our API.

TODO: refine name specifiers, `simple-template-name-specifier` and
decltype-name-specifier` are token soup for now.
2020-08-07 18:05:47 +00:00
Eduardo Caldas ba41a0f733 [SyntaxTree][NFC] remove redundant namespace-specifiers
Differential Revision: https://reviews.llvm.org/D85427
2020-08-07 08:45:29 +00:00
Eduardo Caldas c5cdc3e801 [SyntaxTree] Add test coverage for `->*` operator
This was the last binary operator that we supported but didn't have any
test coverage. The recent fix in a crash in member pointers allowed us
to add this test.

Differential Revision: https://reviews.llvm.org/D85185
2020-08-05 07:36:39 +00:00
Eduardo Caldas 8ce15f7eeb [SyntaxTree] Fix crash on pointer to member function
Differential Revision: https://reviews.llvm.org/D85146
2020-08-04 14:31:12 +00:00
Kadir Cetinkaya 87de54dbb6
[clang][Tooling] Fix addTargetAndModeForProgramName to use correct flag names
The logic was using incorrect flag versions. For example:
- `-target=` can't be a prefix, it must be `--target=`.
- `--driver-mode` can't appear on its own, value must be attached to it.

While fixing those, also changes the append logic to make use of new
`--target=X` format instead of the legacy `-target X` version.

In addition to that makes use of the OPTTable instead of hardcoded strings to
make sure helper also gets updated if clang's options are modified.

Differential Revision: https://reviews.llvm.org/D85076
2020-08-03 11:46:58 +02:00
Kadir Cetinkaya 1618828165
[clang][Syntax] syntax::Arena doesnt own TokenBuffer
Currently an Arena can only be built while consuming a TokenBuffer,
some users (like clangd) might want to share a TokenBuffer with multiple
compenents. This patch changes Arena's TokenBuffer member to be a reference so
that it can be created with read-only token buffers.

Differential Revision: https://reviews.llvm.org/D84973
2020-07-31 11:50:01 +02:00
Zahira Ammarguellat 80bd6ae13e On Windows build, making the /bigobj flag global , instead of passing it per file.
To avoid having this flag be passed in per/file manner, we are instead
passing it globally.

This fixes this bug: https://bugs.llvm.org/show_bug.cgi?id=46733

Reviewed-by: aaron.ballman, beanz, meinersbur

Differential Revision: https://reviews.llvm.org/D84038
2020-07-28 18:04:36 -05:00
Yitzhak Mandelbaum 04a21318b5 [libTooling] Add a `between` range-selector combinator.
Adds the `between` combinator and registers it with the parser. As a driveby, updates some deprecated names to their current versions.

Reviewed By: gribozavr2

Differential Revision: https://reviews.llvm.org/D84315
2020-07-28 17:26:12 +00:00
Yitzhak Mandelbaum c332a984ae [libTooling] Add an `EditGenerator` that applies a rule throughout a bound node.
The new combinator, `rewriteDescendants`, applies a rewrite rule to all
descendants of a specified bound node.  That rewrite rule can refer to nodes
bound by the parent, both in the matcher and in the edits.

Reviewed By: gribozavr2

Differential Revision: https://reviews.llvm.org/D84409
2020-07-24 14:38:17 +00:00
Yitzhak Mandelbaum cf42877812 [libTooling] Add assorted `EditGenerator` combinators.
Summary:
This patch adds various combinators that help in constructing `EditGenerator`s:
   * `noEdits`
   * `ifBound`, specialized to `ASTEdit`
   * `flatten` and `flattenVector` which allow for easy construction from a set
     of sub edits.
   * `shrinkTo`, which generates edits to shrink a given range to another that
     it encloses.

Reviewers: asoffer, gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D84310
2020-07-24 12:51:54 +00:00
Andy Soffer e5b3202b6f [libTooling] In Clang Transformer, change `Metadata` field to deferred evaluation.
`Metadata` is being changed from an `llvm::Any` to a `MatchConsumer<llvm::Any>`
so that it's evaluation can be be dependent on on `MatchResult`s passed in.

Reviewed By: ymandel, gribozavr2

Differential Revision: https://reviews.llvm.org/D83820
2020-07-21 18:05:49 +00:00
Yitzhak Mandelbaum bd994b81d3 Revert "[libTooling] In Clang Transformer, change `Metadata` field to deferred evalutaion"
This reverts commit c0b8954ecb.

The commit has broken various builds. Reverting while I investigate the cause.
2020-07-20 21:24:58 +00:00
Yitzhak Mandelbaum c0b8954ecb [libTooling] In Clang Transformer, change `Metadata` field to deferred evalutaion
`Metadata` is being changed from an `llvm::Any` to a `MatchConsumer<llvm;:Any>`, so that it's evaluation can be be dependent on `MatchResult`s passed in.

Reviewed By: ymandel, gribozavr2

Differential Revision: https://reviews.llvm.org/D83820
2020-07-20 21:17:09 +00:00
Sam McCall f0ab336e74 [Syntax] expose API for expansions overlapping a spelled token range.
Summary:
This allows efficiently accessing all expansions (without iterating over each
token and searching), and also identifying tokens within a range that are
affected by the preprocessor (which is how clangd will use it).

Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D84009
2020-07-20 14:48:12 +02:00
Aleksandr Platonov d19f0666bc [clang][Tooling] Try to avoid file system access if there is no record for the file in compile_commads.json
Summary:
If there is no record in compile_commands.json, we try to find suitable record with `MatchTrie.findEquivalent()` call.
This is very expensive operation with a lot of `llvm::sys::fs::equivalent()` calls in some cases.

This patch disables file symlinks for performance reasons.

Example scenario without this patch:
- compile_commands.json generated at clangd build (contains ~3000 files).
- it tooks more than 1 second to get compile command for newly created file in the root folder of LLVM project.
- we wait for 1 second every time when clangd requests compile command for this file (at file change).

Reviewers: sammccall, kadircet, hokein

Reviewed By: sammccall

Subscribers: chandlerc, djasper, klimek, ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D83621
2020-07-17 18:49:14 +02:00
Eduardo Caldas 1db5b348c4 Add kinded UDL for raw literal operator and numeric literal operator template 2020-07-10 16:21:11 +00:00
Eduardo Caldas f33c2c27a8 Fix crash on `user defined literals`
Summary:
Given an UserDefinedLiteral `1.2_w`:
Problem: Lexer generates one Token for the literal, but ClangAST
references two source locations
Fix: Ignore the operator and interpret it as the underlying literal.
e.g.: `1.2_w` token generates syntax node IntegerLiteral(1.2_w)

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82157
2020-07-10 16:21:11 +00:00
Dmitry Polukhin 9e7fddbd36 [yaml][clang-tidy] Fix multiline YAML serialization
Summary:
New line duplication logic introduced in https://reviews.llvm.org/D63482
has two issues: (1) there is no logic that removes duplicate newlines
when clang-apply-replacment reads YAML and (2) in general such logic
should be applied to all strings and should happen on string
serialization level instead in YAML parser.

This diff changes multiline strings quotation from single quote `'` to
double `"`. It solves problems with internal newlines because now they are
escaped. Also double quotation solves the problem with leading whitespace after
newline. In case of single quotation YAML parsers should remove leading
whitespace according to specification. In case of double quotation these
leading are internal space and they are preserved. There is no way to
instruct YAML parsers to preserve leading whitespaces after newline so
double quotation is the only viable option that solves all problems at
once.

Test Plan: check-all

Reviewers: gribozavr, mgehre, yvvan

Subscribers: xazax.hun, hiraditya, cfe-commits, llvm-commits

Tags: #clang-tools-extra, #clang, #llvm

Differential Revision: https://reviews.llvm.org/D80301
2020-07-09 02:41:58 -07:00
Eduardo Caldas ea8bba7e8d Fix crash on overloaded postfix unary operators due to invalid sloc
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82954
2020-07-08 14:09:40 +00:00
Dmitri Gribenko 5689b38c6a Removed a RecursiveASTVisitor feature to visit operator kinds with different methods
Summary:
This feature was only used in two places, but contributed a non-trivial
amount to the complexity of RecursiveASTVisitor, and was buggy (see my
recent patches where I was fixing the bugs that I noticed). I don't
think the convenience benefit of this feature is worth the complexity.

Besides complexity, another issue with the current state of
RecursiveASTVisitor is the non-uniformity in how it handles different
AST nodes. All AST nodes follow a regular pattern, but operators are
special -- and this special behavior not documented. Correct usage of
RecursiveASTVisitor relies on shadowing member functions with specific
names and signatures. Near misses don't cause any compile-time errors,
incorrectly named or typed methods are just silently ignored. Therefore,
predictability of RecursiveASTVisitor API is quite important.

This change reduces the size of the `clang` binary by 38 KB (0.2%) in
release mode, and by 7 MB (0.3%) in debug mode. The `clang-tidy` binary
is reduced by 205 KB (0.3%) in release mode, and by 5 MB (0.4%) in debug
mode. I don't think these code size improvements are significant enough
to justify this change on its own (for me, the primary motivation is
reducing code complexity), but they I think are a nice side-effect.

Reviewers: rsmith, sammccall, ymandel, aaron.ballman

Reviewed By: rsmith, sammccall, ymandel, aaron.ballman

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82921
2020-07-06 13:38:01 +02:00
Dmitri Gribenko 8e750b1f0a Make RecursiveASTVisitor call WalkUpFrom for operators when the data recursion queue is absent
Reviewers: eduucaldas, ymandel, rsmith

Reviewed By: eduucaldas

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82889
2020-07-06 13:38:01 +02:00
Dmitri Gribenko c19c6b1722 Make RecursiveASTVisitor call WalkUpFrom for unary and binary operators in post-order traversal mode
Reviewers: ymandel, eduucaldas, rsmith

Reviewed By: eduucaldas, rsmith

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82787
2020-07-06 13:38:01 +02:00
Dmitri Gribenko 7349479f22 RecursiveASTVisitor: don't call WalkUp unnecessarily in post-order traversal
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?

* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
  callback from the default implementation of Traverse callbacks.

* In post-order traversal mode when we don't have a DataRecursionQueue,
  RecursiveASTVisitor also calls the WalkUp callback from the default
  implementation of Traverse callbacks.

* However, in post-order traversal mode when we have a DataRecursionQueue,
  RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.

As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.

I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.

In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.

Reviewers: eduucaldas, ymandel

Reviewed By: eduucaldas, ymandel

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82486
2020-07-06 13:38:01 +02:00
Dmitri Gribenko 19eaff650c Revert RecursiveASTVisitor fixes.
This reverts commit 8bf4c40af8.
This reverts commit 7b0be962d6.
This reverts commit 94454442c3.

Some compilers on some buildbots didn't accept the specialization of
is_same_method_impl in a non-namespace scope.
2020-07-03 13:48:24 +02:00
Dmitri Gribenko 8bf4c40af8 Make RecursiveASTVisitor call WalkUpFrom for operators when the data recursion queue is absent
Reviewers: eduucaldas, ymandel, rsmith

Reviewed By: eduucaldas

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82889
2020-07-03 13:03:19 +02:00
Dmitri Gribenko 7b0be962d6 Make RecursiveASTVisitor call WalkUpFrom for unary and binary operators in post-order traversal mode
Reviewers: ymandel, eduucaldas, rsmith

Reviewed By: eduucaldas, rsmith

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82787
2020-07-03 13:03:19 +02:00
Dmitri Gribenko 94454442c3 RecursiveASTVisitor: don't call WalkUp unnecessarily in post-order traversal
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?

* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
  callback from the default implementation of Traverse callbacks.

* In post-order traversal mode when we don't have a DataRecursionQueue,
  RecursiveASTVisitor also calls the WalkUp callback from the default
  implementation of Traverse callbacks.

* However, in post-order traversal mode when we have a DataRecursionQueue,
  RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.

As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.

I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.

In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.

Reviewers: eduucaldas, ymandel

Reviewed By: eduucaldas, ymandel

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82486
2020-07-03 13:03:19 +02:00
Dmitri Gribenko 7988969143 Added tests for RecursiveASTVisitor for AST nodes that are special cased
Summary:
RecursiveASTVisitor has special code for handling operator AST nodes,
specifically, unary, binary, and compound assignment operators. In this
change I'm adding tests for operator AST nodes that follow the existing
pattern of tests for the CallExpr node (an AST node that triggers the
common code path).

Reviewers: ymandel, eduucaldas

Reviewed By: ymandel, eduucaldas

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82875
2020-07-03 13:03:18 +02:00
Eduardo Caldas fdbd78333f Add parenthesized expression to SyntaxTree
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82960
2020-07-02 06:28:41 +00:00
Gabriel Matute ecfa0b2418 [libTooling] Fix `maybeExtendRange` to support `CharRange`s.
Currently, `maybeExtendRange` takes a `CharSourceRange`, but only works
correctly for the `TokenRange` case. This change adds proper support for the
`CharRange` case.

Reviewed By: gribozavr2

Differential Revision: https://reviews.llvm.org/D82901
2020-07-01 20:40:48 +00:00
Andy Soffer 9945bd5911 Add Metadata to Transformer tooling
This change adds a Metadata field to ASTEdit, Edit, and AtomicChange so that
edits can have associated metadata and that metadata can be constructed with
Transformer-based RewriteRules. Metadata is ignored when applying edits to
source, but other consumers of AtomicChange can use this metadata to direct how
they want to consume each edit.

Reviewed By: ymandel, gribozavr2

Differential Revision: https://reviews.llvm.org/D82226
2020-06-30 15:03:07 +00:00
Dmitri Gribenko 1cf2e45c19 Compile the RecursiveASTVisitor callbacks test with "/bigobj"
Summary:
This file was exceeding a limit in MSVC:

fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj

Reviewers: erichkeane

Reviewed By: erichkeane

Subscribers: jmorse, gribozavr2, mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82766
2020-06-29 17:04:45 +02:00
Dmitri Gribenko 339ed1e042 Move TestClangConfig into libClangTesting and use it in AST Matchers tests
Summary:
Previously, AST Matchers tests were using a custom way to run a test
with a specific C++ standard version. I'm migrating them to a shared
infrastructure to specify a Clang target from libClangTesting. I'm also
changing tests for AST Matchers to run in multiple language standards
versions, and under multiple triples that have different behavior with
regards to templates.

To keep the size of the patch manageable, in this patch I'm only
migrating one file to get the process started and get feedback on this
approach.

One caveat is that increasing the number of test configuration does
significantly increase the runtime of AST Matchers tests. On my machine,
the test runtime increases from 2.0 to 6.0s. I think it is worth the
improved test coverage.

Reviewers: jdoerfert, ymandel

Reviewed By: ymandel

Subscribers: gribozavr2, jfb, sstefan1, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82179
2020-06-29 12:50:15 +02:00
Dmitri Gribenko 8e5a56865f Add tests for sequences of callbacks that RecursiveASTVisitor produces
Summary:
These tests show a bug: post-order traversal introduces an extra call to
WalkUp*, that is not present in pre-order traversal. I'm fixing this bug
in a follow-up commit.

Reviewers: ymandel, eduucaldas

Reviewed By: ymandel, eduucaldas

Subscribers: gribozavr2, mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82485
2020-06-29 12:36:01 +02:00
Yitzhak Mandelbaum 30deabf89f [libTooling] Improve error message from failure in selection Stencil
This patch improves the error message provided by the stencil that handles
source from a range selector.

Reviewed By: gribozavr2

Differential Revision: https://reviews.llvm.org/D82654
2020-06-26 16:17:28 +00:00
Dmitri Gribenko fa1b488776 Work around a bug in MSVC in the syntax tree test
Summary:
MSVC does not handle raw string literals with embedded double quotes
correctly. I switched the affected test case to use regular string
literals insetad.

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82636
2020-06-26 16:43:30 +02:00
Yitzhak Mandelbaum 056a539e57 [libTooling] Rename overloaded `range` range selector.
Renames the overloaded `RangeSelector` combinator `range` to the more
descriptive `enclose` and `encloseNodes`. The old overloads are left in place
and marked deprected and will be deleted at a future time.

Reviewed By: tdl-g

Differential Revision: https://reviews.llvm.org/D82592
2020-06-26 14:23:25 +00:00
Eduardo Caldas 7b404b6d00 Add `FloatingLiteral` to SyntaxTree
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82318
2020-06-25 17:05:08 +00:00
Eduardo Caldas 466e8b7ea6 Add StringLiteral to SyntaxTree
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82360
2020-06-25 17:05:08 +00:00
Eduardo Caldas 221d7bbe49 Add `CharLiteral` to SyntaxTree
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82312
2020-06-25 17:05:08 +00:00
Eduardo Caldas 7f7f856480 Add `BoolLiteralExpression` to SyntaxTree
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82310
2020-06-25 15:37:53 +00:00
Yitzhak Mandelbaum 87340a2bf1 [libTooling] Delete deprecated `Stencil` combinators.
Summary: Deletes `text()` and `selection()` combinators, since they have been deprecated for months.

Reviewers: tdl-g

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82225
2020-06-24 16:45:24 +00:00
Yitzhak Mandelbaum d81d69f1c0 [libTooling] Change Transformer's `cat` to handle some cases of text in macros.
Summary:
Currently, `cat` validates range selections before extracting the corresponding
source text. However, this means that any range inside a macro is rejected as an
error. This patch changes the implementation to first try to map the range to
something reasonable. This makes the behavior consistent with handling of ranges
used for selecting portions of the source to edit.

Also updates a clang-tidy lit-test for one of the checks which was affected by
this change.

Reviewers: gribozavr2, tdl-g

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D82126
2020-06-19 18:48:54 +00:00
Yitzhak Mandelbaum 9ca50e887d [libTooling] Add parser for string representation of `RangeSelector`.
This patch adds a parser for a `RangeSelector` written as a string. The format
is closely based on the way one would right the selector in C++. This should
enable use of `RangeSelector`s from tools like clang-query and web UIs.
2020-06-19 01:11:29 +00:00
Eduardo Caldas 1b2f6b4a08 Add support for DeclRefExpr in SyntaxTree, by generating IdExpressions
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: hlopko, gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81168
2020-06-18 21:05:23 +02:00
Dmitry Polukhin c98c94d85f [clang-tidy] Add diagnostics level to YAML output
Summary:
Also added BuildDirectory for completness and removed unused `Fix`.

Test Plan: check-all

Reviewers: alexfh, gribozavr2

Subscribers: xazax.hun, cfe-commits

Tags: #clang-tools-extra, #clang

Differential Revision: https://reviews.llvm.org/D79285
2020-06-15 07:40:53 -07:00
Shengchen Kan 2c63ea6ede [TEST] TreeTest.cpp - Add a comma to avoid build error with -werror
Summary:
The macro `INSTANTIATE_TEST_CASE_P` is defined as
```
\# define INSTANTIATE_TEST_CASE_P(prefix, test_case_name, generator, ...) \
...
```

If we build the test case with -werror, we will get an error like
```
error: ISO C++11 requires at least one argument for the "..." in a
variadic macro

testing::ValuesIn(TestClangConfig::allConfigs()));
                                                ^
```
This patch fixes that.

Reviewers: gribozavr, hlopko, eduucaldas, gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81388
2020-06-08 23:14:16 +08:00
Eduardo Caldas 3b739690b0 Add support for IntegerLiteral in SyntaxTree
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81135
2020-06-04 14:05:31 +02:00
Dmitri Gribenko 06cf7adcc8 Make syntax tree test print the line number when it fails
Summary:
The syntax tree test uses a helper function that executes all testing
assertions. When an assertion fails, the only line number that gets
printed to the log refers to the helper function. After this change, we
would also get the line number of the EXPECT_TRUE macro invocation
(unfortunately, the line number of the last token of it, not the first
one, but there's not much I can do about it).

Reviewers: hlopko, eduucaldas

Reviewed By: hlopko, eduucaldas

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81107
2020-06-04 10:42:44 +02:00
Eduardo Caldas 007098d7e6 Add support for `nullptr` in SyntaxTrees
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81092
2020-06-03 15:52:33 +02:00
Dmitri Gribenko d7d5dd31fc Split syntax tree tests into more granular ones
Summary:
Doing so allows us to increase test coverage by removing unnecessary
language restrictions.

Reviewers: hlopko, eduucaldas

Reviewed By: hlopko, eduucaldas

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81040
2020-06-03 11:17:11 +02:00
Dmitri Gribenko b34b7691fa Syntax tree: ignore implicit expressions at the top level of statements
Summary:
I changed `markStmtChild` to ignore implicit expressions the same way as
`markExprChild` does it already. The test that I modified crashes
without this change.

Reviewers: hlopko, eduucaldas

Reviewed By: hlopko, eduucaldas

Subscribers: gribozavr2, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81019
2020-06-03 10:58:12 +02:00
Dmitri Gribenko 53c29a42d0 Reinstate the syntax tree test for 'static' in an array subscript
Reviewers: eduucaldas

Reviewed By: eduucaldas

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81009
2020-06-02 17:01:51 +02:00
Dmitri Gribenko d559185aae Renamed Lang_C to Lang_C99, Lang_CXX to Lang_CXX03, and 2a to 20
Summary:
I think we would be better off with tests explicitly specifying the
language mode. Right now Lang_C means C99, but reads as "any C version",
or as "unspecified C version".

I also changed '-std=c++98' to '-std=c++03' because they are aliases (so
there is no difference in practice), because Clang implements C++03
rules in practice, and because 03 makes a nice sortable progression
between 03, 11, 14, 17, 20.

Reviewers: shafik, hlopko

Reviewed By: hlopko

Subscribers: jfb, martong, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D81000
2020-06-02 16:31:20 +02:00
Dmitri Gribenko 44f989e780 Run syntax tree tests in many language modes
Reviewers: hlopko, eduucaldas

Reviewed By: hlopko, eduucaldas

Subscribers: gribozavr2, mgorny, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80822
2020-06-02 10:30:01 +02:00
Eduardo Caldas 3a574a6cb3 Add support for Overloaded Binary Operators in SyntaxTree
Reviewers: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80812
2020-05-29 20:03:59 +02:00
Eduardo Caldas eca41919d2 Improve test infrastructure in SyntaxTree
Summary:
* Test if the code sourcing the SyntaxTree compiles
* Output compiler errors and warnings to err
* Fix tests with code that did not compile

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80731
2020-05-28 21:35:12 +02:00
Yitzhak Mandelbaum ce5780b88c [libTooling] Fix Transformer to work with ambient traversal kinds.
Summary:
`RewriteRule`'s `applyFirst` was brittle with respect to the default setting of the
`TraversalKind`. This patch builds awareness of traversal kinds directly into
rewrite rules so that they are insensitive to any changes in defaults.

Reviewers: steveire, gribozavr

Subscribers: hokein, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80606
2020-05-28 11:42:07 -04:00
Eduardo Caldas 461af57de7 Add support for UnaryOperator in SyntaxTree
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80624
2020-05-27 17:12:46 +02:00
Yitzhak Mandelbaum ff2743bf04 [libTooling] In Transformer, allow atomic changes to span multiple files.
Summary:
Currently, all changes returned by a single application of a rule must fit in
one atomic change and therefore must apply to one file. However, there are
patterns in which a single rule will want to modify multiple files; for example,
a header and implementation to change a declaration and its definition. This
patch relaxes Transformer, libTooling's interpreter of RewriteRules, to support
multiple changes.

Reviewers: gribozavr

Subscribers: mgrang, jfb, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80239
2020-05-26 09:17:35 -04:00
Eduardo Caldas 3785eb83af Add support for binary operators in Syntax Trees
Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80540
2020-05-26 12:25:58 +02:00
mydeveloperday 447ea9b4f5 [AST] default implementation is possible for non-member functions in C++20.
Summary:
Make RAV not visit the default function decl by default.
Also update some stale comments on FunctionDecl::isDefault.

Fixes https://github.com/clangd/clangd/issues/383

Reviewers: sammccall, rsmith

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80288
2020-05-25 10:45:12 +02:00
Stephen Kelly a30d411629 Set traversal explicitly where needed in tests
Reviewers: aaron.ballman, shafik

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D72531
2020-05-21 22:34:38 +01:00
Arthur Eubanks dab1326a5a Strip -showIncludes in getClangStripDependencyFileAdjuster()
Summary:
Follow-up to https://reviews.llvm.org/D78836.

Also consolidate some test cases.

Reviewers: thakis

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D78939
2020-04-27 11:20:08 -07:00
Arthur Eubanks 8000d506af [clangd] Strip /showIncludes in clangd compile commands
In command lines with /showIncludes, clangd would output includes to stdout, breaking clients.

Summary: Fixes https://github.com/clangd/clangd/issues/322.

Reviewers: sammccall, kadircet

Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D78836
2020-04-26 18:57:39 -07:00
Johannes Doerfert f9d558c871 [OpenMP] "UnFix" layering problem with FrontendOpenMP
This reverts commit 97aa593a83 as it
causes problems (PR45453) https://reviews.llvm.org/D77574#1966321.

This additionally adds an explicit reference to FrontendOpenMP to
clang-tidy where ASTMatchers is used.

This is hopefully just a temporary solution. The dependence on
`FrontendOpenMP` from `ASTMatchers` should be handled by CMake
implicitly, not us explicitly.

Reviewed By: aheejin

Differential Revision: https://reviews.llvm.org/D77666
2020-04-07 14:41:18 -05:00
Marcel Hlopko 1bf055c989 [Syntax] Add mapping from spelled to expanded tokens for TokenBuffer
Summary:
Same restrictions apply as in the other direction: macro arguments are
not supported yet, only full macro expansions can be mapped.

Taking over from https://reviews.llvm.org/D72581.

Reviewers: gribozavr2, sammccall

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77209
2020-04-07 15:07:16 +02:00
Sam McCall d66afd6dde [Syntax] Merge overlapping top-level macros in TokenBuffer
Summary:
Our previous definition of "top-level" was too informal, and didn't
allow for overlapping macros that each directly produce expanded tokens.
See D77507 for previous discussion.

Fixes http://bugs.llvm.org/show_bug.cgi?id=45428

Reviewers: kadircet, vabridgers

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77615
2020-04-07 15:06:04 +02:00
Johannes Doerfert 97aa593a83 [OpenMP] Fix layering problem with FrontendOpenMP
Summary:
ASTMatchers is used in various places and it now exposes the
LLVMFrontendOpenMP library to its users without them needing to depend
on it explicitly.

Reviewers: lebedev.ri

Subscribers: mgorny, yaxunl, bollu, guansong, martong, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D77574
2020-04-06 13:04:26 -05:00
Johannes Doerfert 931c0cd713 [OpenMP][NFC] Move and simplify directive -> allowed clause mapping
Move the listing of allowed clauses per OpenMP directive to the new
macro file in `llvm/Frontend/OpenMP`. Also, use a single generic macro
that specifies the directive and one allowed clause explicitly instead
of a dedicated macro per directive.

We save 800 loc and boilerplate for all new directives/clauses with no
functional change. We also need to include the macro file only once and
not once per directive.

Depends on D77112.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D77113
2020-04-06 00:04:08 -05:00
Marcel Hlopko cdce2fe561 [Syntax] Remove delayed folding from tree building.
Summary:
This patch removes delayed folding and replaces it with forward peeking.

Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:

```
int a,b;
```

is expressed in the AST as:

```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```

And in the syntax tree we need:

```
*: TranslationUnit
`-SimpleDeclaration
  |-int
  |-SimpleDeclarator
  | `-a
  |-,
  |-SimpleDeclarator
  | `-b
  |-;
```

So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.

This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).

This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
  method in all places.
* Adds a bunch of tests

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76922
2020-03-31 09:47:50 +02:00
Kadir Cetinkaya 9619c2cc9a
[clang][Syntax] Handle macro arguments in spelledForExpanded
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75446
2020-03-28 16:35:46 +01:00
Marcel Hlopko eddede9d51 [Syntax] Test both the default and windows target platforms in unittests
Summary:
This increases the coverage for things that differ between Linux and Windows, such as -fdelayed-template-parsing. This would have prevented the rollback of https://reviews.llvm.org/D76346.

While at it, update -std=c++11 to c++17 for the test.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76497
2020-03-20 16:33:58 +01:00
Dmitri Gribenko 9967352a03 Revert "[Syntax] Test both the default and windows target platforms in unittests"
This reverts commit fd7300f717. The fix
in this patch didn't help and the Windows buildbot broke:
http://45.33.8.238/win/10881/step_7.txt
2020-03-20 12:13:49 +01:00
Marcel Hlopko fd7300f717 [Syntax] Test both the default and windows target platforms in unittests
Summary:
This increases the coverage for things that differ between Linux and Windows, such as `-fdelayed-template-parsing`. This would have prevented the rollback of https://reviews.llvm.org/D76346.

While at it, update -std=c++11 to c++17 for the test.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76433
2020-03-20 11:42:18 +01:00
Marcel Hlopko e9630630ff [Syntax] Split syntax tests
Summary:
This patch split Basic test into multple individual tests to allow simpler
filtering and clearer signal into what's broken when it's broken.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76366
2020-03-20 09:53:43 +01:00
Marcel Hlopko 88bf9b3d26 [Syntax] Build template declaration nodes
Summary:
Rollforward of
https://reviews.llvm.org/rGdd12826808f9079e164b82e64b0697a077379241 after
temporarily adding -fno-delayed-template-parsing to the TreeTest.

Original summary:

> Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
>
> Handles template declaration of all kinds.
>
> Also builds template declaration nodes for specializations and explicit
> instantiations of classes.
>
> Some missing things will be addressed in the follow-up patches:
>
> * specializations of functions and variables,
> * template parameters.

Reviewers: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76418
2020-03-19 17:43:07 +01:00
Nico Weber 881f5b5a7b Revert "[Syntax] Build template declaration nodes"
This reverts commit dd12826808.
Breaks tests on Windows, see https://reviews.llvm.org/D76346#1929208
2020-03-18 12:57:55 -04:00
Marcel Hlopko dd12826808 [Syntax] Build template declaration nodes
Summary:
Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.

Handles template declaration of all kinds.

Also builds template declaration nodes for specializations and explicit
instantiations of classes.

Some missing things will be addressed in the follow-up patches:

specializations of functions and variables,
template parameters.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76346
2020-03-18 16:16:59 +01:00
Marcel Hlopko 7d382dcd46 [Syntax] Build declarator nodes
Summary:
Copy of https://reviews.llvm.org/D72089 with Ilya's permission. See
https://reviews.llvm.org/D72089 for the first batch of comments.

Reviewers: gribozavr2

Reviewed By: gribozavr2

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76220
2020-03-16 19:13:59 +01:00
Reid Kleckner 213aea4c58 Remove unused Endian.h includes, NFC
Mainly avoids including Host.h everywhere:

$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
    | grep '^[-+] ' | sort | uniq -c | sort -nr
   3141 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Host.h
2020-03-11 15:45:34 -07:00
Kadir Cetinkaya cd9b2e18bd
[clang][Syntax] Add spelledTokenAt helper to TokenBuffer
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D75503
2020-03-03 14:30:41 +01:00