Commit Graph

657 Commits

Author SHA1 Message Date
Shu-Chun Weng 1c5717225e [libTooling] Add smart pointer support to the `access` Stencil
This extends smart pointer support beyond the existing `maybeDeref` and
`maybeAddressOf`.

Differential Revision: https://reviews.llvm.org/D100450
2021-04-14 10:45:59 -07:00
Balázs Kéri 6e51991049 [clang][AST] Handle overload callee type in CallExpr::getCallReturnType.
The function did not handle every case. In some cases this
caused assertion failure.
After the fix the function returns DependentTy if the exact
return type can not be determined.

It seems that clang itself does not call the function in the
affected cases but some checker or other code may call it.

Reviewed By: hokein

Differential Revision: https://reviews.llvm.org/D95244
2021-04-12 09:44:17 +02:00
Whisperity 3b677b81ce [libtooling][clang-tidy] Fix diagnostics not highlighting fed SourceRanges
Fixes bug http://bugs.llvm.org/show_bug.cgi?id=49000.

This patch allows Clang-Tidy checks to do

    diag(X->getLocation(), "text") << Y->getSourceRange();

and get the highlight of `Y` as expected:

    warning: text [blah-blah]
        xxx(something)
        ^   ~~~~~~~~~

Reviewed-By: aaron.ballman, njames93

Differential Revision: http://reviews.llvm.org/D98635
2021-04-10 16:43:44 +02:00
Jan Svoboda 2935737da3 [clang][tooling] Create SourceManager for DiagnosticsEngine before command-line parsing
In D84673, we started using `DiagnosticsEngine` during command-line parsing in more contexts.

When using `ToolInvocation`, a custom `DiagnosticsConsumer` can be specified and it might expect `SourceManager` to be present on the emitted diagnostics.

This patch ensures the `SourceManager` is set up in such scenarios.

Test authored by Jordan Rupprecht.

Reviewed By: rupprecht

Differential Revision: https://reviews.llvm.org/D99414
2021-04-06 10:40:47 +02:00
Utkarsh Saxena aa979084df [clang][Syntax] Optimize expandedTokens for token ranges.
`expandedTokens(SourceRange)` used to do a binary search to get the
expanded tokens belonging to a source range. Each binary search uses
`isBeforeInTranslationUnit` to order two source locations. This is
inherently very slow.
By profiling clangd we found out that users like clangd::SelectionTree
spend 95% of time in `isBeforeInTranslationUnit`. Also it is worth
noting that users of `expandedTokens(SourceRange)` majorly use ranges
provided by AST to query this funciton. The ranges provided by AST are
token ranges (starting at the beginning of a token and ending at the
beginning of another token).

Therefore we can avoid the binary search in majority of the cases by
maintaining an index of ExpandedToken by their SourceLocations. We still
do binary search for ranges which are not token ranges but such
instances are quite low.

Performance:
`~/build/bin/clangd --check=clang/lib/Serialization/ASTReader.cpp`
Before: Took 2:10s to complete.
Now: Took 1:13s to complete.

Differential Revision: https://reviews.llvm.org/D99086
2021-03-25 18:54:15 +01:00
Janusz Nykiel e030ce3ec7 [Tooling] Handle compilation databases containing commands with double dashes
As of CMake commit https://gitlab.kitware.com/cmake/cmake/-/commit/d993ebd4,
which first appeared in CMake 3.19.x series, in the compile commands for
clang-cl, CMake puts `--` before the input file. When operating on such a
database, the `InterpolatingCompilationDatabase` - specifically, the
`TransferableCommand` constructor - does not recognize that pattern and so, does
not strip the input, or the double dash when 'transferring' the compile command.
This results in a incorrect compile command - with the double dash and old input
file left in, and the language options and new input file appended after them,
where they're all treated as inputs, including the language version option.

Test files for some tests have names similar enough to be matched to commands
from the database, e.g.:

`.../path-mappings.test.tmp/server/bar.cpp`

can be matched to:

`.../Driver/ToolChains/BareMetal.cpp`

etc. When that happens, the tool being tested tries to use the matched, and
incorrectly 'transferred' compile command, and fails, reporting errors similar
to:

`error: no such file or directory: '/std:c++14'; did you mean '/std:c++14'? [clang-diagnostic-error]`

This happens in at least 4 tests:

  Clang Tools :: clang-tidy/checkers/performance-trivially-destructible.cpp
  Clangd :: check-fail.test
  Clangd :: check.test
  Clangd :: path-mappings.test

The fix for `TransferableCommand` removes the `--` and everything after it when
determining the arguments that apply to the new file. `--` is inserted in the
'transferred' command if the new file name starts with `-` and when operating in
clang-cl mode, also `/`. Additionally, other places in the code known to do
argument adjustment without accounting for the `--` and causing the tests to
fail are fixed as well.

Differential Revision: https://reviews.llvm.org/D98824
2021-03-24 16:01:47 -04:00
Kadir Cetinkaya f43ff34ae6
[clang] Mark re-injected tokens appropriately during pragma handling
This hides such tokens from TokenWatcher, preventing crashes in clients
trying to match spelled and expanded tokens.

Fixes https://github.com/clangd/clangd/issues/712

Differential Revision: https://reviews.llvm.org/D98483
2021-03-12 18:18:17 +01:00
Sam McCall 588db1ccff [clangd] Use flags from open files when opening headers they include
Currently our strategy for getting header compile flags is something like:

A) look for flags for the header in compile_commands.json
   This basically never works, build systems don't generate this info.
B) try to match to an impl file in compile_commands.json and use its flags
   This only (mostly) works if the headers are in the same project.
C) give up and use fallback flags
   This kind of works for stdlib in the default configuration, and
   otherwise doesn't.

Obviously there are big gaps here.

This patch inserts a new attempt between A and B: if the header is
transitively included by any open file (whether same project or not),
then we use its compile command.

This doesn't make any attempt to solve some related problems:
 - parsing non-self-contained header files in context (importing PP state)
 - using the compile flags of non-opened candidate files found in the index

Fixes https://github.com/clangd/clangd/issues/123
Fixes https://github.com/clangd/clangd/issues/695
See https://github.com/clangd/clangd/issues/519

Differential Revision: https://reviews.llvm.org/D97351
2021-03-01 09:43:59 +01:00
Haojian Wu 780ead41e0 [Syntax] No crash on OpaqueValueExpr.
OpaqueValueExpr doesn't correspond to the concrete syntax, it has
invalid source location, ignore them.

Reviewed By: kbobyrev

Differential Revision: https://reviews.llvm.org/D96112
2021-02-18 10:32:04 +01:00
Haojian Wu e159a3ced4 [Syntax] Remove a strict valid source location assertion for TypeLoc.
The EndLoc of a type loc can be invalid for broken code.

Also extend the existing test to support error code with `error-ok`
annotation.

Differential Revision: https://reviews.llvm.org/D96261
2021-02-11 09:53:52 +01:00
Haojian Wu 35a5e88390 [Syntax] NFC, Simplify a test with annotations 2021-02-11 09:49:06 +01:00
Haojian Wu 6c1a23303d [Syntax] Support condition for IfStmt.
Differential Revision: https://reviews.llvm.org/D95782
2021-02-04 09:15:30 +01:00
Stephan Bergmann 215ed9b33c Adapt CastExpr::getSubExprAsWritten to ConstantExpr
Differential Revision: https://reviews.llvm.org/D87030
2021-01-12 09:41:03 +01:00
Yitzhak Mandelbaum 675a2973ee [libTooling] Add support for smart pointers to relevant Transformer `Stencil`s.
Stencils `maybeDeref` and `maybeAddressOf` are designed to handle nodes that may
be pointers. Currently, they only handle native pointers. This patch extends the
support to recognize smart pointers and handle them as well.

Differential Revision: https://reviews.llvm.org/D93637
2021-01-05 17:57:41 +00:00
Duncan P. N. Exon Smith 494aacd72c Tooling: Migrate some tests to FileEntryRef, NFC
Migrate to the `FileEntryRef` overload of `SourceManager::createFileID`
(using `FileManager::getOptionalFileRef`) in RefactoringTest.cpp and
RewriterTestContext.h.

No functionality change.

Differential Revision: https://reviews.llvm.org/D92967
2020-12-10 18:05:03 -08:00
Alexander Kornienko 027899dab6 Remove references to the ast_type_traits namespace
Follow up to cd62511496 /
https://reviews.llvm.org/D74499

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D92994
2020-12-11 00:58:46 +01:00
Yitzhak Mandelbaum e6bc4a71e3 [libTooling] Add `describe` combinator for formatting AST nodes for diagnostics.
This new stencil combinator is intended for use in diagnostics and the like.

Differential Revision: https://reviews.llvm.org/D92658
2020-12-07 16:08:05 +00:00
Sam McCall a1cb9cbf5c Add ability to load a FixedCompilationDatabase from a buffer.
Previously, loading one from a file meant allowing the library to do the IO.
Clangd would prefer to do such IO itself (e.g. to allow caching).

Differential Revision: https://reviews.llvm.org/D92640
2020-12-07 13:07:10 +01:00
Sam McCall 650e04e179 [Tooling] JSONCompilationDatabase::loadFromBuffer retains the buffer, copy it.
This function doesn't seem to be used in-tree outside tests.
However clangd wants to use it soon, and having the CDB be self-contained seems
reasonable.

Differential Revision: https://reviews.llvm.org/D92646
2020-12-04 21:54:55 +01:00
Arthur O'Dwyer e181a6aedd s/instantate/instantiate/ throughout. NFCI.
The static_assert in "libcxx/include/memory" was the main offender here,
but then I figured I might as well `git grep -i instantat` and fix all
the instances I found. One was in user-facing HTML documentation;
the rest were in comments or tests.
2020-12-01 22:13:40 -05:00
Yitzhak Mandelbaum fdff677a95 [libTooling] Remove deprecated Clang Transformer declarations
A number of declarations were leftover after the move from `clang::tooling` to
`clang::transformer`. This patch removes those declarations and upgrades the
handful of references to the deprecated declarations.

Differential Revision: https://reviews.llvm.org/D92340
2020-11-30 20:15:26 +00:00
Yitzhak Mandelbaum 88e6208562 [libTooling] Update Transformer's `node` combinator to include the trailing semicolon for decls.
Currently, `node` only includes the semicolon for (some) statements. However,
declarations have the same issue of (potentially) trailing semicolons, so `node`
should behave the same for them.

Differential Revision: https://reviews.llvm.org/D91872
2020-11-20 18:11:50 +00:00
Stephen Kelly ef1a4169e8 [Transformer] Split ForStmt test into two
It is apparently not possible to have two rewrites in one gtest function
because atomic changes in the test harness accumulate.
2020-11-17 18:16:10 +00:00
Stephen Kelly 6ef6beaa57 Comment out new test while I figure out what is wrong with it 2020-11-17 17:48:48 +00:00
Stephen Kelly 4cadb66b49 [AST] Update matchers to be traverse-aware
Don't match Stmt or Decl nodes not spelled in the source when using
TK_IgnoreUnlessSpelledInSource.  This prevents accidental modification
of source code at incorrect locations.

Differential Revision: https://reviews.llvm.org/D90984
2020-11-17 16:31:08 +00:00
Stephen Kelly 246b428fb3 [AST] Ignore implicit nodes in IgnoreUnlessSpelledInSource mode
Update the ASTNodeTraverser to dump only nodes spelled in source.  There
are only a few which need to be handled, but Decl nodes for which
isImplicit() is true are handled together.

Update the RAV instances used in ASTMatchFinder to ignore the nodes too.
As with handling of template instantiations, it is necessary to allow
the RAV to process the implicit nodes because they need to be visitable
before the first traverse() matcher is encountered.  An exception to
this is in the MatchChildASTVisitor, because we sometimes wish to make a
node matchable but make its children not-matchable.  This is the case
for defaulted CXXMethodDecls for example.

Extend TransformerTests to illustrate the kinds of problems that can
arise when performing source code rewriting due to matching implicit
nodes.

This change accounts for handling nodes not spelled in source when using
direct matching of nodes, and when using the has() and hasDescendant()
matchers.  Other matchers such as
cxxRecordDecl(hasMethod(cxxMethodDecl())) still succeed for
compiler-generated methods for example after this change.  Updating the
implementations of hasMethod() and other matchers is for a follow-up
patch.

Differential Revision: https://reviews.llvm.org/D90982
2020-11-17 16:30:07 +00:00
Kirill Bobyrev 142c6f82fd
[clang] Simplify buildSyntaxTree API
Follow-up on https://reviews.llvm.org/D88553#inline-837013

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D90672
2020-11-09 22:49:54 +01:00
Matt Morehouse a6d15d4070 Undo Revert "Ignore template instantiations if not in AsIs mode"
MaskRay already fixed the ASan bug.
2020-11-03 13:59:01 -08:00
Matt Morehouse 72531ae6e6 Revert "Ignore template instantiations if not in AsIs mode"
This reverts commit 53df3beb62 due to
check-asan failure on the buildbot.
2020-11-03 13:57:31 -08:00
Fangrui Song 96ed6793b3 [unittest][TrasnformerTest] Fix asan stack-use-after-return 2020-11-03 12:34:45 -08:00
Stephen Kelly 53df3beb62 Ignore template instantiations if not in AsIs mode
Summary:
IgnoreUnlessSpelledInSource mode should ignore these because they are
not written in the source.  This matters for example when trying to
replace types or values which are templated.  The new test in
TransformerTest.cpp in this commit demonstrates the problem.

In existing matcher code, users can write
`unless(isInTemplateInstantiation())` or `unless(isInstantiated())` (the
user must know which to use).  The point of the
TK_IgnoreUnlessSpelledInSource mode is to allow the novice to avoid such
details.  This patch changes the IgnoreUnlessSpelledInSource mode to
skip over implicit template instantiations.

This patch does not change the TK_AsIs mode.

Note: An obvious attempt at an alternative implementation would simply
change the shouldVisitTemplateInstantiations() in ASTMatchFinder.cpp to
return something conditional on the operational TraversalKind.  That
does not work because shouldVisitTemplateInstantiations() is called
before a possible top-level traverse() matcher changes the operational
TraversalKind.

Reviewers: sammccall, aaron.ballman, gribozavr2, ymandel, klimek

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D80961
2020-11-02 20:21:48 +00:00
Sam McCall d4934eb5f8 [Syntax] Add iterators over children of syntax trees.
This gives us slightly nicer syntax (foreach) for idioms currently expressed
as a loop, and the option to use range algorithms where it makes sense
(e.g. llvm::all_of et al encapsulate the needed flow control in a useful way).

It's also a building block for iteration over filtered views (e.g. iterate over
all Stmt children, with the right type):
for (const Statement &S : filter<Statement>(N.children()))
  ...

I realize the recent direction has been mostly towards strongly-typed
node-specific facilities, but I think it's important we have convenient
generic facilities too.

Differential Revision: https://reviews.llvm.org/D90023
2020-10-28 12:37:57 +01:00
Nathan Ridge 2756e2ee0b [libTooling] Recognize sccache as a compiler wrapper in compilation database commands
sccache is a compiler caching tool similar to ccache.

Differential Revision: https://reviews.llvm.org/D88790
2020-10-27 01:46:32 -04:00
Yitzhak Mandelbaum 6f8f5cb77e [libTooling] Add function to Transformer to create a no-op edit.
This functionality is commonly needed in clang tidy checks (based on
transformer) that only print warnings, without suggesting any edits. The no-op
edit allows the user to associate a diagnostic message with a source location.

Differential Revision: https://reviews.llvm.org/D89961
2020-10-22 21:29:03 +00:00
Alex Richardson 66a3b9073a [clang-format] Drop clangFrontend dependency for FormatTests
This allows building the clang-format unit tests in only 657 ninja steps
rather than 1257 which allows for much faster incremental builds after a
git pull.

Reviewed By: MyDeveloperDay
Differential Revision: https://reviews.llvm.org/D89709
2020-10-20 10:13:28 +01:00
Alex Richardson a2214757e2 Move clang/Tooling/Core/Lookup.h to clang/Tooling/Refactoring/Lookup.h
This allows removing the clangAST dependency from libclangToolingCore and
therefore allows clang-format to be built without depending on clangAST.
Before 1166 files had to be compiled for clang-format, now only 796.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D89708
2020-10-20 10:13:28 +01:00
Yitzhak Mandelbaum 65cb4fdd69 [libTooling] Change `after` range-selector to operate only on source ranges
Currently, `after` fails when applied to locations in macro arguments.  This
change projects the subrange into a file source range and then applies `after`.

Differential Revision: https://reviews.llvm.org/D89468
2020-10-15 20:58:30 +00:00
Stefan Pintilie 0741a2c9ca [Clang][unittests][NFC] Break up test in Callbacks.cpp
The Callbacks.cpp test was taking a long time to compile on some build bots
causing timeouts. This patch splits up that test into five separate cpp
files and a header file.

Reviewed By: gribozavr2

Differential Revision: https://reviews.llvm.org/D88886
2020-10-09 08:53:50 -05:00
Eduardo Caldas 5011d43108 Migrate Declarators to use the List API
After this change all nodes that have a delimited-list are using the
`List` API.

Implementation details:
Let's look at a declaration with multiple declarators:
`int a, b;`
To generate a declarator list node we need to have the range of
declarators: `a, b`:
However, the `ClangAST` actually stores them as separate declarations:
`int a   ;`
`int    b;`
We solve that by appropriately marking the declarators on each separate
declaration in the `ClangAST` and then for the final declarator `int
b`, shrinking its range to fit to the already marked declarators.

Differential Revision: https://reviews.llvm.org/D88403
2020-10-01 13:56:31 +00:00
Eduardo Caldas c3c08bfdfd [SyntaxTree] Test the List API
Differential Revision: https://reviews.llvm.org/D87839
2020-09-22 17:07:41 +00:00
Eduardo Caldas 6dc06fa09d [SyntaxTree] Add tests for the assignment of the `canModify` tag.
Differential Revision: https://reviews.llvm.org/D88077
2020-09-22 13:17:33 +00:00
Eduardo Caldas 66bcb14312 [SyntaxTree][Synthesis] Fix: `deepCopy` -> `deepCopyExpandingMacros`.
There can be Macros that are tagged with `modifiable`. Thus verifying
`canModifyAllDescendants` is not sufficient to avoid macros when deep
copying.

We think the `TokenBuffer` could inform us whether a `Token` comes from
a macro. We'll look into that when we can surface this information
easily, for instance in unit tests for `ComputeReplacements`.

Differential Revision: https://reviews.llvm.org/D88034
2020-09-22 09:15:21 +00:00
Eduardo Caldas af582c9b0f [SyntaxTree] Test `findFirstLeaf` and `findLastLeaf`
* Introduce `TreeTest.cpp` to unit test `Tree.h`
* Add `generateAllTreesWithShape` to generating test cases
* Add tests for `findFirstLeaf` and `findLastLeaf`
* Fix implementations of `findFirstLeaf` and `findLastLeaf` that had
been broken when empty `Tree` were present.

Differential Revision: https://reviews.llvm.org/D87779
2020-09-22 06:47:36 +00:00
Eduardo Caldas 4a5cc389c5 [SyntaxTree][Synthesis] Implement `deepCopy`
Differential Revision: https://reviews.llvm.org/D87749
2020-09-21 09:27:15 +00:00
Eduardo Caldas e616a42598 [SyntaxTree] Test for '\' inside token.
Differential Revision: https://reviews.llvm.org/D87895
2020-09-21 06:56:14 +00:00
Eduardo Caldas bb5b28f12f [SyntaxTree][Synthesis] Improve testing `createLeaf`
The new test shows that `createLeaf` depends on the C++ version.

Differential Revision: https://reviews.llvm.org/D87896
2020-09-21 06:11:46 +00:00
Eduardo Caldas 7c37b82f5b [SyntaxTree][Synthesis] Add support for Tree.
In a future patch
* Implement helper function to generate Trees for tests
* and test Tree methods, namely `findFirstLeaf` and `findLastLeaf`

Differential Revision: https://reviews.llvm.org/D87533
2020-09-11 20:37:23 +00:00
Eduardo Caldas 5d152127d4 [SyntaxTree][Synthesis] Add support for simple Leafs and test based on tree dump
Differential Revision: https://reviews.llvm.org/D87495
2020-09-11 18:22:00 +00:00
Eduardo Caldas 4c14ee61b7 [SyntaxTree] Rename functions to start with verb
According to LLVM coding standards:
https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly

Differential Revision: https://reviews.llvm.org/D87498
2020-09-11 14:54:18 +00:00
Eduardo Caldas c01d28dc51 [SyntaxTree] Specialize `TreeTestBase` for `BuildTreeTest`, `MutationsTest` and `SynthesisTest`
Differential Revision: https://reviews.llvm.org/D87374
2020-09-10 16:44:14 +00:00