Motivating use case is ".cu.cc" extensions used in some bazel projects.
Alternative is to work around this with IncludeIsMainRegex in styles.
I proposed this approach because it seems like a better default.
Differential Revision: https://reviews.llvm.org/D86597
Previously a NodeRole would generally be prefixed with the `NodeKind`,
we remove this prefix, as it we redundant and made tests more noisy.
Differential Revision: https://reviews.llvm.org/D86636
* Generate `CallExpression` syntax node for all semantic nodes inheriting from
`CallExpr` with call-expression syntax - except `CUDAKernelCallExpr`.
* Implement all the accessors
* Arguments of `CallExpression` have their own syntax node which is based on
the `List` base API
Differential Revision: https://reviews.llvm.org/D86544
We should see `NodeRole` information in the dump because that exposes how the
accessors will behave.
Functional changes in the dump:
* Surround Leaf tokens with `'`
* Append `Node` dumps with `NodeRole` information, except for unknown roles
* Append marks to `Node` dumps, instead of prepending
Non-functional changes:
* `::dumpTokens(llvm::raw_ostream, ArrayRef<syntax::Token>, const
SourceManager &SM)` always received as parameter a `syntax::Token *`
pointing to `Leaf::token()`. Changed the function to
`dumpLeaf(llvm::raw_ostream, syntax::Leaf *, const SourceManager&)`
* `dumpTree` acted on a Node, rename to `dumpNode`
Differential Revision: https://reviews.llvm.org/D85330
For an user define `<`, `x < y` would yield the syntax tree:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-IdExpression
| `-UnqualifiedId
| `-<
`-IdExpression
`-UnqualifiedId
`-y
```
But there is no syntatic difference at call site between call site or
built-in `<`. As such they should generate the same syntax tree, namely:
```
BinaryOperatorExpression
|-IdExpression
| `-UnqualifiedId
| `-x
|-<
`-IdExpression
`-UnqualifiedId
`-y
```
Differential Revision: https://reviews.llvm.org/D85750
Currently, changes to includes are applied to an entire rule. However,
include changes may be specific to particular edits within a rule (for example,
they may apply to one file but not another). Also, include changes may need to
carry metadata, just like other changes. So, we make include changes first-class
edits.
Reviewed By: tdl-g
Differential Revision: https://reviews.llvm.org/D85734
This patch lifts `RootID` out of the `RewriteRule` class so that constructs
(e.g. inline functions) can that refer to the root id don't need to depend on
the `RewriteRule` class.
With this dependency, the patch is able to collect all `ASTEdit` helper function
declarations together with the class declaration, before the introduction of the
`RewriteRule` class. In the process, we also adjust some of the comments.
This patch is essentially a NFC.
Reviewed By: gribozavr2
Differential Revision: https://reviews.llvm.org/D85733
Summary:
We want NestedNameSpecifier syntax nodes to be generally supported, not
only for `DeclRefExpr` and `DependentScopedDeclRefExpr`.
To achieve this we:
* Use the `RecursiveASTVisitor`'s API to traverse
`NestedNameSpecifierLoc`s and automatically create its syntax nodes
* Add links from the `NestedNameSpecifierLoc`s to their syntax nodes.
In this way, from any semantic construct that has a `NestedNameSpecifier`,
we implicitly generate its syntax node via RAV and we can easily access
this syntax node via the links we added.
We defined a List construct to help with the implementation of list-like
grammar rules. This is a first implementation of this API.
Differential Revision: https://reviews.llvm.org/D85295
This is our grammar rule for nested-name-specifiers:
globalbal-specifier:
/*empty*/
simple-template-specifier:
template_opt simple-template-id
name-specifier:
global-specifier
decltype-specifier
identifier
simple-template-specifier
nested-name-specifier:
list(name-specifier, ::, non-empty, terminated)
It is a relaxed version of C++ [expr.prim.id] and quite simpler to map to our API.
TODO: refine name specifiers, `simple-template-name-specifier` and
decltype-name-specifier` are token soup for now.
`OS << ND->getDeclName();` is equivalent to `OS << ND->getNameAsString();`
without the extra temporary string.
This is not quite a NFC since two uses of `getNameAsString` in a
diagnostic are replaced, which results in the named entity being
quoted with additional "'"s (ie: 'var' instead of var).
We use inheritance to model the grammar's disjunction rule:
literal:
integer-literal
character-literal
floating-point-literal
string-literal
boolean-literal
pointer-literal
user-defined-literal
Differential Revision: https://reviews.llvm.org/D85186
This skips searching for `target` related flags in the existing args if
we don't have a valid target to insert.
Depends on D85076
Differential Revision: https://reviews.llvm.org/D85077
The logic was using incorrect flag versions. For example:
- `-target=` can't be a prefix, it must be `--target=`.
- `--driver-mode` can't appear on its own, value must be attached to it.
While fixing those, also changes the append logic to make use of new
`--target=X` format instead of the legacy `-target X` version.
In addition to that makes use of the OPTTable instead of hardcoded strings to
make sure helper also gets updated if clang's options are modified.
Differential Revision: https://reviews.llvm.org/D85076
Currently an Arena can only be built while consuming a TokenBuffer,
some users (like clangd) might want to share a TokenBuffer with multiple
compenents. This patch changes Arena's TokenBuffer member to be a reference so
that it can be created with read-only token buffers.
Differential Revision: https://reviews.llvm.org/D84973
Adds the `between` combinator and registers it with the parser. As a driveby, updates some deprecated names to their current versions.
Reviewed By: gribozavr2
Differential Revision: https://reviews.llvm.org/D84315
The new combinator, `rewriteDescendants`, applies a rewrite rule to all
descendants of a specified bound node. That rewrite rule can refer to nodes
bound by the parent, both in the matcher and in the edits.
Reviewed By: gribozavr2
Differential Revision: https://reviews.llvm.org/D84409
Summary:
This patch adds various combinators that help in constructing `EditGenerator`s:
* `noEdits`
* `ifBound`, specialized to `ASTEdit`
* `flatten` and `flattenVector` which allow for easy construction from a set
of sub edits.
* `shrinkTo`, which generates edits to shrink a given range to another that
it encloses.
Reviewers: asoffer, gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D84310
`Metadata` is being changed from an `llvm::Any` to a `MatchConsumer<llvm::Any>`
so that it's evaluation can be be dependent on on `MatchResult`s passed in.
Reviewed By: ymandel, gribozavr2
Differential Revision: https://reviews.llvm.org/D83820
`Metadata` is being changed from an `llvm::Any` to a `MatchConsumer<llvm;:Any>`, so that it's evaluation can be be dependent on `MatchResult`s passed in.
Reviewed By: ymandel, gribozavr2
Differential Revision: https://reviews.llvm.org/D83820
Summary:
This allows efficiently accessing all expansions (without iterating over each
token and searching), and also identifying tokens within a range that are
affected by the preprocessor (which is how clangd will use it).
Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D84009
Summary:
If there is no record in compile_commands.json, we try to find suitable record with `MatchTrie.findEquivalent()` call.
This is very expensive operation with a lot of `llvm::sys::fs::equivalent()` calls in some cases.
This patch disables file symlinks for performance reasons.
Example scenario without this patch:
- compile_commands.json generated at clangd build (contains ~3000 files).
- it tooks more than 1 second to get compile command for newly created file in the root folder of LLVM project.
- we wait for 1 second every time when clangd requests compile command for this file (at file change).
Reviewers: sammccall, kadircet, hokein
Reviewed By: sammccall
Subscribers: chandlerc, djasper, klimek, ilya-biryukov, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D83621
* as we are using them only for integer and floating literals they have
the same behavior
* FileRange::text is simpler to call and is within the context of
syntax trees
Summary:
Given an UserDefinedLiteral `1.2_w`:
Problem: Lexer generates one Token for the literal, but ClangAST
references two source locations
Fix: Ignore the operator and interpret it as the underlying literal.
e.g.: `1.2_w` token generates syntax node IntegerLiteral(1.2_w)
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82157
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?
* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
callback from the default implementation of Traverse callbacks.
* In post-order traversal mode when we don't have a DataRecursionQueue,
RecursiveASTVisitor also calls the WalkUp callback from the default
implementation of Traverse callbacks.
* However, in post-order traversal mode when we have a DataRecursionQueue,
RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.
As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.
I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.
In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.
Reviewers: eduucaldas, ymandel
Reviewed By: eduucaldas, ymandel
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82486
This reverts commit 8bf4c40af8.
This reverts commit 7b0be962d6.
This reverts commit 94454442c3.
Some compilers on some buildbots didn't accept the specialization of
is_same_method_impl in a non-namespace scope.
Summary:
How does RecursiveASTVisitor call the WalkUp callback for expressions?
* In pre-order traversal mode, RecursiveASTVisitor calls the WalkUp
callback from the default implementation of Traverse callbacks.
* In post-order traversal mode when we don't have a DataRecursionQueue,
RecursiveASTVisitor also calls the WalkUp callback from the default
implementation of Traverse callbacks.
* However, in post-order traversal mode when we have a DataRecursionQueue,
RecursiveASTVisitor calls the WalkUp callback from PostVisitStmt.
As a result, when the user overrides the Traverse callback, in pre-order
traversal mode they never get the corresponding WalkUp callback. However
in the post-order traversal mode the WalkUp callback is invoked or not
depending on whether the data recursion optimization could be applied.
I had to adjust the implementation of TraverseCXXForRangeStmt in the
syntax tree builder to call the WalkUp method directly, as it was
relying on this behavior. There is an existing test for this
functionality and it prompted me to make this extra fix.
In addition, I had to fix the default implementation implementation of
RecursiveASTVisitor::TraverseSynOrSemInitListExpr to call WalkUpFrom in
the same manner as the implementation generated by the DEF_TRAVERSE_STMT
macro. Without this fix, the InitListExprIsPostOrderNoQueueVisitedTwice
test was failing because WalkUpFromInitListExpr was never called.
Reviewers: eduucaldas, ymandel
Reviewed By: eduucaldas, ymandel
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82486
Currently, `maybeExtendRange` takes a `CharSourceRange`, but only works
correctly for the `TokenRange` case. This change adds proper support for the
`CharRange` case.
Reviewed By: gribozavr2
Differential Revision: https://reviews.llvm.org/D82901
This change adds a Metadata field to ASTEdit, Edit, and AtomicChange so that
edits can have associated metadata and that metadata can be constructed with
Transformer-based RewriteRules. Metadata is ignored when applying edits to
source, but other consumers of AtomicChange can use this metadata to direct how
they want to consume each edit.
Reviewed By: ymandel, gribozavr2
Differential Revision: https://reviews.llvm.org/D82226
This patch improves the error message provided by the stencil that handles
source from a range selector.
Reviewed By: gribozavr2
Differential Revision: https://reviews.llvm.org/D82654
Renames the overloaded `RangeSelector` combinator `range` to the more
descriptive `enclose` and `encloseNodes`. The old overloads are left in place
and marked deprected and will be deleted at a future time.
Reviewed By: tdl-g
Differential Revision: https://reviews.llvm.org/D82592
Summary: Deletes `text()` and `selection()` combinators, since they have been deprecated for months.
Reviewers: tdl-g
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82225
Summary:
As discussed previously when landing patch for OpenMP in Flang, the idea is
to share common part of the OpenMP declaration between the different Frontend.
While doing this it was thought that moving to tablegen instead of Macros will also
give a cleaner and more powerful way of generating these declaration.
This first part of a future series of patches is setting up the base .td file for
DirectiveLanguage as well as the OpenMP version of it. The base file is meant to
be used by other directive language such as OpenACC.
In this first patch, the Directive and Clause enums are generated with tablegen
instead of the macros on OMPConstants.h. The next pacth will extend this
to other enum and move the Flang frontend to use it.
Reviewers: jdoerfert, DavidTruby, fghanim, ABataev, jdenny, hfinkel, jhuber6, kiranchandramohan, kiranktp
Reviewed By: jdoerfert, jdenny
Subscribers: arphaman, martong, cfe-commits, mgorny, yaxunl, hiraditya, guansong, jfb, sstefan1, aaron.ballman, llvm-commits
Tags: #llvm, #openmp, #clang
Differential Revision: https://reviews.llvm.org/D81736
Summary:
As discussed previously when landing patch for OpenMP in Flang, the idea is
to share common part of the OpenMP declaration between the different Frontend.
While doing this it was thought that moving to tablegen instead of Macros will also
give a cleaner and more powerful way of generating these declaration.
This first part of a future series of patches is setting up the base .td file for
DirectiveLanguage as well as the OpenMP version of it. The base file is meant to
be used by other directive language such as OpenACC.
In this first patch, the Directive and Clause enums are generated with tablegen
instead of the macros on OMPConstants.h. The next pacth will extend this
to other enum and move the Flang frontend to use it.
Reviewers: jdoerfert, DavidTruby, fghanim, ABataev, jdenny, hfinkel, jhuber6, kiranchandramohan, kiranktp
Reviewed By: jdoerfert, jdenny
Subscribers: cfe-commits, mgorny, yaxunl, hiraditya, guansong, jfb, sstefan1, aaron.ballman, llvm-commits
Tags: #llvm, #openmp, #clang
Differential Revision: https://reviews.llvm.org/D81736
Summary:
Currently, `cat` validates range selections before extracting the corresponding
source text. However, this means that any range inside a macro is rejected as an
error. This patch changes the implementation to first try to map the range to
something reasonable. This makes the behavior consistent with handling of ranges
used for selecting portions of the source to edit.
Also updates a clang-tidy lit-test for one of the checks which was affected by
this change.
Reviewers: gribozavr2, tdl-g
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82126
This patch adds a parser for a `RangeSelector` written as a string. The format
is closely based on the way one would right the selector in C++. This should
enable use of `RangeSelector`s from tools like clang-query and web UIs.
When targetting CodeView, the goal is to store argv0 & cc1 cmd-line in the emitted .OBJ, in order to allow a reproducer from the .OBJ alone.
This patch is to simplify https://reviews.llvm.org/D80833
Summary:
I changed `markStmtChild` to ignore implicit expressions the same way as
`markExprChild` does it already. The test that I modified crashes
without this change.
Reviewers: hlopko, eduucaldas
Reviewed By: hlopko, eduucaldas
Subscribers: gribozavr2, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D81019
Summary:
`RewriteRule`'s `applyFirst` was brittle with respect to the default setting of the
`TraversalKind`. This patch builds awareness of traversal kinds directly into
rewrite rules so that they are insensitive to any changes in defaults.
Reviewers: steveire, gribozavr
Subscribers: hokein, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80606
Summary:
Currently, all changes returned by a single application of a rule must fit in
one atomic change and therefore must apply to one file. However, there are
patterns in which a single rule will want to modify multiple files; for example,
a header and implementation to change a declaration and its definition. This
patch relaxes Transformer, libTooling's interpreter of RewriteRules, to support
multiple changes.
Reviewers: gribozavr
Subscribers: mgrang, jfb, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80239
Summary:
These files tend to be hand-authored, and people get very confused.
I can't think of any reason that such whitespace would be intended.
Reviewers: kadircet
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D80160
It can be used to avoid passing the begin and end of a range.
This makes the code shorter and it is consistent with another
wrappers we already have.
Differential revision: https://reviews.llvm.org/D78016
Summary:
* accept -x cu to indicate language is CUDA
* transfer CUDA language flag to header-file arguments
Differential Revision: https://reviews.llvm.org/D77451
Summary:
This revision simplifies the representation of edits in rewrite rules. The
simplified form is more general, allowing the user more flexibility in building
custom edit specifications.
The changes extend the API, without changing the signature of existing
functions. So this only risks breaking users that directly accessed the
`RewriteRule` struct.
Reviewers: gribozavr2
Subscribers: jfb, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77419
This reverts commit 97aa593a83 as it
causes problems (PR45453) https://reviews.llvm.org/D77574#1966321.
This additionally adds an explicit reference to FrontendOpenMP to
clang-tidy where ASTMatchers is used.
This is hopefully just a temporary solution. The dependence on
`FrontendOpenMP` from `ASTMatchers` should be handled by CMake
implicitly, not us explicitly.
Reviewed By: aheejin
Differential Revision: https://reviews.llvm.org/D77666
Summary:
Same restrictions apply as in the other direction: macro arguments are
not supported yet, only full macro expansions can be mapped.
Taking over from https://reviews.llvm.org/D72581.
Reviewers: gribozavr2, sammccall
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77209
Summary:
Our previous definition of "top-level" was too informal, and didn't
allow for overlapping macros that each directly produce expanded tokens.
See D77507 for previous discussion.
Fixes http://bugs.llvm.org/show_bug.cgi?id=45428
Reviewers: kadircet, vabridgers
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77615
Summary:
The motivation here is fixing https://bugs.llvm.org/show_bug.cgi?id=45428, see
D77507. The fundamental problem is that a "top-level" expansion wasn't precisely
defined. Repairing this concept means that TokenBuffer's "top-level expansion"
may not correspond to a single macro expansion. Example:
```
M(2); // expands to 1+2
```
The expansions overlap, but neither expansion alone yields all the tokens.
We need a TokenBuffer::Mapping that corresponds to their union.
This is fairly easy to fix in CollectPPExpansions, but the current design of
TokenCollector::Builder needs a fix too as it relies on the macro's expansion
range rather than the captured expansion bounds. This fix is hard to make due
to the way code is reused within Builder. And honestly, I found that code pretty
hard to reason about too.
The new approach doesn't use the expansion range, but only the expansion
location: it assumes an expansion is the contiguous set of expanded tokens with
the same expansion location, which seems like a reasonable formalization of
the "top-level" notion.
And hopefully the control flow is easier to follow too, it's considerably
shorter even with more documentation.
Reviewers: kadircet
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77614
Summary:
ASTMatchers is used in various places and it now exposes the
LLVMFrontendOpenMP library to its users without them needing to depend
on it explicitly.
Reviewers: lebedev.ri
Subscribers: mgorny, yaxunl, bollu, guansong, martong, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D77574
Saves only 36 includes of ASTContext.h and related headers.
There are two deps on ASTContext.h:
- C++ method overrides iterator types (TinyPtrVector)
- getting LangOptions
For #1, duplicate the iterator type, which is
TinyPtrVector<>::const_iterator.
For #2, add an out-of-line accessor to get the language options. Getting
the ASTContext from a Decl is already an out of line method that loops
over the parent DeclContexts, so if it is ever performance critical, the
proper fix is to pass the context (or LangOpts) into the predicate in
question.
Other changes are just header fixups.
Move the listing of allowed clauses per OpenMP directive to the new
macro file in `llvm/Frontend/OpenMP`. Also, use a single generic macro
that specifies the directive and one allowed clause explicitly instead
of a dedicated macro per directive.
We save 800 loc and boilerplate for all new directives/clauses with no
functional change. We also need to include the macro file only once and
not once per directive.
Depends on D77112.
Reviewed By: JonChesterfield
Differential Revision: https://reviews.llvm.org/D77113
See rational here: https://reviews.llvm.org/D76173#1922916
Time to compile Attr.h in isolation goes from 2.6s to 1.8s.
Original patch by Johannes, plus some additions from Reid to fix some
clang tooling targets.
Effect on transitive includes is marginal, though:
$ diff -u <(sort thedeps-before.txt) <(sort thedeps-after.txt) \
| grep '^[-+] ' | sort | uniq -c | sort -nr
104 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/AST/OpenMPClause.h
87 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Frontend/OpenMP/OMPContext.h
19 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/ADT/SmallSet.h
19 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/ADT/SetVector.h
14 - /usr/include/c++/9/set
...
Differential Revision: https://reviews.llvm.org/D76184
Summary:
This patch removes delayed folding and replaces it with forward peeking.
Delayed folding was previously used as a solution to the problem that
declaration doesn't have a representation in the AST. For example following
code:
```
int a,b;
```
is expressed in the AST as:
```
TranslationUnitDecl
|-...
|-VarDecl `int a`
`-VarDecl `int b`
```
And in the syntax tree we need:
```
*: TranslationUnit
`-SimpleDeclaration
|-int
|-SimpleDeclarator
| `-a
|-,
|-SimpleDeclarator
| `-b
|-;
```
So in words, we need to create SimpleDeclaration to be a parent of
SimpleDeclarator nodes. Previously we used delayed folding to make sure SimpleDeclarations will be
eventually created. And in case multiple declarators requested declaration
creation, declaration range was extended to cover all declarators.
This design started to be hard to reason about, so we decided to replace it with
forward peeking. The last declarator node in the chain is responsible for creating
SimpleDeclaration for the whole chain. Range of the declaration corresponds to
the source range of the declarator node. Declarator decides whether its the last
one by peeking to the next AST node (see `isResponsibleForCreatingDeclaration`).
This patch does following:
* Removed delayed folding logic
* Tweaks Token.dumpForTests
* Moves getQualifiedNameStart inside BuildTreeVisitor
* Extracts BuildTreeVisitor.ProcessDeclaratorAndDeclaration
* Renames Builder.getDeclRange to Builder.getDeclarationRange and uses the
method in all places.
* Adds a bunch of tests
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76922
Summary:
Copy of https://reviews.llvm.org/D72446, submitting with Ilya's permission.
Only used to assign roles to child nodes for now. This is more efficient
than doing range-based queries.
In the future, will be exposed in the public API of syntax trees.
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76355
Summary:
Rollforward of
https://reviews.llvm.org/rGdd12826808f9079e164b82e64b0697a077379241 after
temporarily adding -fno-delayed-template-parsing to the TreeTest.
Original summary:
> Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
>
> Handles template declaration of all kinds.
>
> Also builds template declaration nodes for specializations and explicit
> instantiations of classes.
>
> Some missing things will be addressed in the follow-up patches:
>
> * specializations of functions and variables,
> * template parameters.
Reviewers: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76418
Summary:
Copy of https://reviews.llvm.org/D72334, submitting with Ilya's permission.
Handles template declaration of all kinds.
Also builds template declaration nodes for specializations and explicit
instantiations of classes.
Some missing things will be addressed in the follow-up patches:
specializations of functions and variables,
template parameters.
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D76346
Most clients of SourceManager.h need to do things like turning source
locations into file & line number pairs, but this doesn't require
bringing in FileManager.h and LLVM's FS headers.
The main code change here is to sink SM::createFileID into the cpp file.
I reason that this is not performance critical because it doesn't happen
on the diagnostic path, it happens along the paths of macro expansion
(could be hot) and new includes (less hot).
Saves some includes:
309 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileManager.h
272 - /usr/local/google/home/rnk/llvm-project/clang/include/clang/Basic/FileSystemOptions.h
271 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/VirtualFileSystem.h
267 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/FileSystem.h
266 - /usr/local/google/home/rnk/llvm-project/llvm/include/llvm/Support/Chrono.h
Differential Revision: https://reviews.llvm.org/D75406
Summary: Instead of dropping all the ranges associated with a Diagnostic when
converting them to a ClangTidy error, instead attach them to the ClangTidyError,
so they can be consumed by other APIs.
Patch by Joe Turner <joturner@google.com>.
Differential Revision: https://reviews.llvm.org/D69782
Summary:
Second attempt -- the first was reverted in commit 0e480b39c6, because of test breakages. This revision fixes the cause of the test breakages.
Original description follows:
This patch adds `getAssociatedRange` which, for a given decl, computes preceding
and trailing text that would conceptually be associated with the decl by the
reader. This includes comments, whitespace, and separators like ';'.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D72153
Summary: Fixes https://github.com/clangd/clangd/issues/279. We were removing the color options but not the preceeding -Xclang which causes errors since the -Xclang would now apply to the next option in the list of options. Now, when removing a color option, we check if there was a preceeding -Xclang and remove it as well.
Patch By @DaanDeMeyer !
Reviewers: sammccall, kadircet
Reviewed By: sammccall
Subscribers: ilya-biryukov, usaxena95
Differential Revision: https://reviews.llvm.org/D75019
Summary:
This patch adds `getAssociatedRange` which, for a given decl, computes preceding
and trailing text that would conceptually be associated with the decl by the
reader. This includes comments, whitespace, and separators like ';'.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D72153
Summary:
This fixes a clangd rename issue, which is missing the reference of
an incomplete specialization.
Unfortunately, I didn't reproduce this issue in clang-rename, I guess
the input `FoundDecl` of AdditionalUSRFinder is different in clangd vs
clang-rename, clang-rename uses the underlying CXXRecordDecl of the
ClassTemplateDecl, which is fixed in 5d862c042b;
while clangd-rename uses the ClassTemplateDecl.
Reviewers: kbobyrev
Reviewed By: kbobyrev
Subscribers: ilya-biryukov, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74829
Summary:
Currently, `buildAST[WithArgs]` either succeeds or fails. This patch adds
support for the caller to pass a `DiagnosticConsumer` to receive all relevant
diagnostics.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74763
The goal of this patch is to maximize CPU utilization on multi-socket or high core count systems, so that parallel computations such as LLD/ThinLTO can use all hardware threads in the system. Before this patch, on Windows, a maximum of 64 hardware threads could be used at most, in some cases dispatched only on one CPU socket.
== Background ==
Windows doesn't have a flat cpu_set_t like Linux. Instead, it projects hardware CPUs (or NUMA nodes) to applications through a concept of "processor groups". A "processor" is the smallest unit of execution on a CPU, that is, an hyper-thread if SMT is active; a core otherwise. There's a limit of 32-bit processors on older 32-bit versions of Windows, which later was raised to 64-processors with 64-bit versions of Windows. This limit comes from the affinity mask, which historically is represented by the sizeof(void*). Consequently, the concept of "processor groups" was introduced for dealing with systems with more than 64 hyper-threads.
By default, the Windows OS assigns only one "processor group" to each starting application, in a round-robin manner. If the application wants to use more processors, it needs to programmatically enable it, by assigning threads to other "processor groups". This also means that affinity cannot cross "processor group" boundaries; one can only specify a "preferred" group on start-up, but the application is free to allocate more groups if it wants to.
This creates a peculiar situation, where newer CPUs like the AMD EPYC 7702P (64-cores, 128-hyperthreads) are projected by the OS as two (2) "processor groups". This means that by default, an application can only use half of the cores. This situation could only get worse in the years to come, as dies with more cores will appear on the market.
== The problem ==
The heavyweight_hardware_concurrency() API was introduced so that only *one hardware thread per core* was used. Once that API returns, that original intention is lost, only the number of threads is retained. Consider a situation, on Windows, where the system has 2 CPU sockets, 18 cores each, each core having 2 hyper-threads, for a total of 72 hyper-threads. Both heavyweight_hardware_concurrency() and hardware_concurrency() currently return 36, because on Windows they are simply wrappers over std:🧵:hardware_concurrency() -- which can only return processors from the current "processor group".
== The changes in this patch ==
To solve this situation, we capture (and retain) the initial intention until the point of usage, through a new ThreadPoolStrategy class. The number of threads to use is deferred as late as possible, until the moment where the std::threads are created (ThreadPool in the case of ThinLTO).
When using hardware_concurrency(), setting ThreadCount to 0 now means to use all the possible hardware CPU (SMT) threads. Providing a ThreadCount above to the maximum number of threads will have no effect, the maximum will be used instead.
The heavyweight_hardware_concurrency() is similar to hardware_concurrency(), except that only one thread per hardware *core* will be used.
When LLVM_ENABLE_THREADS is OFF, the threading APIs will always return 1, to ensure any caller loops will be exercised at least once.
Differential Revision: https://reviews.llvm.org/D71775
DynTypedNode and ASTNodeKind are implemented as part of the clang AST
library, which uses the main clang namespace. There doesn't seem to be a
need for this extra level of namespacing.
I left behind aliases in the ast_type_traits namespace for out of tree
clients of these APIs. To provide aliases for the enumerators, I used
this pattern:
namespace ast_type_traits {
constexpr TraversalKind TK_AsIs = ::clang::TK_AsIs;
}
I think the typedefs will be useful for migration, but we might be able
to drop these enumerator aliases.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D74499
Summary:
When renaming a class with template constructors, we are missing the
occurrences of the template constructors, because getUSRsForDeclaration doesn't
give USRs of the templated constructors (they are not in the normal `ctors()`
method).
Reviewers: kbobyrev
Subscribers: jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D74216
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
The only part of ASTContext.h that requires most AST types to be
complete is the parent map. Nothing in Clang proper uses the ParentMap,
so split it out into its own class. Make ASTContext own the
ParentMapContext so there is still a one-to-one relationship.
After this change, 562 fewer files depend on ASTTypeTraits.h, and 66
fewer depend on TypeLoc.h:
$ diff -u deps-before.txt deps-after.txt | \
grep '^[-+] ' | sort | uniq -c | sort -nr | less
562 - ../clang/include/clang/AST/ASTTypeTraits.h
340 + ../clang/include/clang/AST/ParentMapContext.h
66 - ../clang/include/clang/AST/TypeLocNodes.def
66 - ../clang/include/clang/AST/TypeLoc.h
15 - ../clang/include/clang/AST/TemplateBase.h
...
I computed deps-before.txt and deps-after.txt with `ninja -t deps`.
This removes a common and key dependency on TemplateBase.h and
TypeLoc.h.
This also has the effect of breaking the ParentMap RecursiveASTVisitor
instantiation into its own file, which roughly halves the compilation
time of ASTContext.cpp (29.75s -> 17.66s). The new file takes 13.8s to
compile.
I left behind forwarding methods for getParents(), but clients will need
to include a new header to make them work:
#include "clang/AST/ParentMapContext.h"
I noticed that this parent map functionality is unfortunately duplicated
in ParentMap.h, which only works for Stmt nodes.
Reviewed By: rsmith
Differential Revision: https://reviews.llvm.org/D71313
Differential Revision: https://reviews.llvm.org/D70268
This is a recommit of f978ea4983 with a fix for the PowerPC failure.
The issue was that:
* `CompilerInstance::ExecuteAction` calls
`getTarget().adjust(getLangOpts());`.
* `PPCTargetInfo::adjust` changes `LangOptions::HasAltivec`.
* This happens after the first few calls to `getModuleHash`.
There’s even a FIXME saying:
```
// FIXME: We shouldn't need to do this, the target should be immutable once
// created. This complexity should be lifted elsewhere.
```
This only showed up on PowerPC because it's one of the few targets that
almost always changes a hashed langopt.
I looked into addressing the fixme, but that would be a much larger
change, and it's not the only thing that happens in `ExecuteAction` that
can change the module context hash. Instead I changed the code to not
call `getModuleHash` until after it has been modified in `ExecuteAction`.
Summary: Currently, an attempt to rewrite source code inside a macro expansion succeeds, but results in empty text, rather than failing with an error. This patch restructures to the code to explicitly validate ranges before attempting to edit them.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D72274
Add checks for some structural invariants when building and mutating
the syntax trees.
Fix a bug failing the invariants after mutations: the parent of nodes
added into the tree was null.
When they are free-standing, e.g. `struct X;` or `struct X {};`.
Although this complicates the common case (of free-standing class
declarations), this ensures the less common case (e.g. `struct X {} a;`)
are handled uniformly and produce similar syntax trees.
Summary:
Currently interpolation logic prefers -std over -x. But the latter is a
more strong signal, so this patch inverts the order and only makes use of -std
if -x didn't exist.
Fixes https://github.com/clangd/clangd/issues/185
Thanks @sammccall for tracking this down!
Reviewers: sammccall
Subscribers: ilya-biryukov, usaxena95, cfe-commits, sammccall
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71727
This is both more efficient and avoids corner cases in
`SourceManager::isBeforeInTranslationUnit`.
The change is trivial and clearly a performance improvement on the hot
path of building the syntax tree, so sending without review.
Summary:
This patch adds facilities to mutate the syntax trees and produce
corresponding text replacements.
The public interface of the syntax library now includes facilities to:
1. perform type-safe modifications of syntax trees,
2. compute textual replacements to apply the modifications,
3. create syntax trees not backed by the source code.
For each of the three, we only add a few example transformations in this
patch to illustrate the idea, support for more kinds of nodes and
transformations will be done in follow-up patches.
The high-level mutation operations are implemented on top of operations
that allow to arbitrarily change the trees. They are considered to be
implementation details and are not available to the users of the
library.
Reviewers: sammccall, gribozavr2
Reviewed By: gribozavr2
Subscribers: merge_guards_bot, mgorny, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64573
Summary: Useful when positions are used to target nodes, with before/after ambiguity.
Reviewers: ilya-biryukov, kbobyrev
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71356
Summary: Useful when positions are used to target nodes, with before/after ambiguity.
Reviewers: ilya-biryukov, kbobyrev
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71356
Summary:
This is a follow-up to D70769 and D70222, which allows propagation of
current directory down to ExpandResponseFiles for handling of relative paths.
Previously clients had to mutate FS to achieve that, which is not thread-safe
and can even be thread-hostile in the case of real file system.
Reviewers: sammccall
Subscribers: hiraditya, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D70857
Summary:
The exclusive-claim model is successful at resolving conflicts over tokens
between parent/child or siblings. However claims at the spelled-token
level do the wrong thing for macro expansions, where siblings can be
equally associated with the macro invocation.
Moreover, any model that only uses the endpoints in a range can fail when
a macro invocation occurs inside the node.
To address this, we use the existing TokenBuffer in more depth.
Claims are expressed in terms of expanded tokens, so there is no need to worry
about macros, includes etc.
Once we know which expanded tokens were claimed, they are mapped onto
spelled tokens for hit-testing.
This mapping is fairly flexible, currently the handling of macros is
pretty simple (map macro args onto spellings, other macro expansions onto the
macro name token).
This mapping is in principle token-by-token for correctness (though
there's some batching for performance).
The aggregation of the selection enum is now more principled as we need to be
able to aggregate several hit-test results together.
For simplicity i removed the ability to determine selectedness of TUDecl.
(That was originally implemented in 90a5bf92ff97b1, but doesn't seem to be very
important or worth the complexity any longer).
The expandedTokens(SourceLocation) helper could be added locally, but seems to
make sense on TokenBuffer.
Fixes https://github.com/clangd/clangd/issues/202
Fixes https://github.com/clangd/clangd/issues/126
Reviewers: hokein
Subscribers: MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits, ilya-biryukov
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70512
Summary:
Also remove the temporary TopLevelDeclaration node and add
UnknownDeclaration to represent other unknown nodes.
See the follow-up change for building more top-level declarations.
Adding declarators is also pretty involved and will be done in another
follow-up patch.
Reviewers: gribozavr2
Reviewed By: gribozavr2
Subscribers: merge_guards_bot, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70787
Fix the confusing naming of `CompilerInstance::ModuleManager`. This is
actually an instance of `ASTReader`, which contains an instance of
`ModuleManager`. I have to assume there was a point in the past where
they were just one class, but it's been pretty confusing for a while. I
think it's time to fix it.
The new name is `TheASTReader`; the annoying `The` prefix is so that we
don't shadow the `ASTReader` class. I tried out `ASTRdr` but that
seemed less clear, and this choice matches `ThePCHContainerOperations`
just a couple of declarations below.
Also rename `CompilerInstance::getModuleManager` and
`CompilerInstance::createModuleManager` to `*ASTReader`, making some
cases of `getModuleManager().getModuleManager()` a little more clear.
https://reviews.llvm.org/D70583
Summary:
Adds combinators `maybeDeref` and `maybeAddressOf` to provide a uniform way to handle
nodes which may be bound to either a pointer or a value (most often in the
context of member expressions). Such polymorphism is already supported by
`access`; these combinators extend it to more general uses.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70554
Summary:
Adds an optional parameter to `buildASTFromCodeWithArgs` that allows the user to
pass additional files that the main code needs to compile. This change makes
`buildASTFromCodeWithArgs` consistent with `runToolOnCodeWithArgs`.
Patch by Alexey Eremin.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70175
Summary:
Flags that generate output could result in failures when creating
syntax only actions. This patch introduces initial logic for filtering out
those. The first such flag is "save-temps", which saves intermediate
files(bitcode, assembly, etc.) into a specified directory.
Fixes https://github.com/clangd/clangd/issues/191
Reviewers: hokein
Subscribers: ilya-biryukov, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D70173
Summary:
By additional regex match, grouping of main include can be enabled in files that are not normally considered as a C/C++ source code.
For example, this might be useful in templated code, where template implementations are being held in *Impl.hpp files.
On the occassion, 'assume-filename' option description was reworded as it was misleading. It has nothing to do with `style=file` option and it does not influence sourced style filename.
Reviewers: rsmith, ioeric, krasimir, sylvestre.ledru, MyDeveloperDay
Reviewed By: MyDeveloperDay
Subscribers: MyDeveloperDay, cfe-commits
Patch by: furdyna
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67750
Summary:
This revision introduces a new interface `MatchComputation` which generalizes
the `Stencil` interface and replaces the `std::function` interface of
`MatchConsumer`. With this revision, `Stencil` (as an abstraction) becomes just
one collection of implementations of
`MatchComputation<std::string>`. Correspondingly, we remove the `Stencil` class
entirely in favor of a simple type alias, deprecate `MatchConsumer` and change
all functions that accepted `MatchConsumer<std::string>` to use
`MatchComputation<std::string>` instead.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69802
Summary:
This should be NFC to clang-rename, by default the traversal scope is
TUDecl. Traversing the TUDecl in clangd is a performance cliff, we should
avoid it.
Reviewers: ilya-biryukov
Subscribers: kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69892
Summary:
Currently, stencils are defined as a sequence of `StencilParts`. This
differentiation adds an unneeded layer of complexity to the definition of
Stencils. This change significantly simplifies the type structure: a stencil is
now conceptually any object implementing `StencilInterface` and `Stencil` is
just a thin wrapper for pointers to this interface.
To account for the sequencing that was supported by the old `Stencil` type, we
introduce a sequencing class that implements `StencilInterface`. That is,
sequences are just another kind of Stencil and no longer have any special
status.
Corresponding to this change in the type structure, we change the way `cat` is
used (and defined). `cat` bundles multiple features: it builds a stencil from a
sequence of subcomponents and admits multiple different types for its arguments,
while coercing them into the right type. Previously, `cat` was also used to
coerce a single `StencilPart` into a `Stencil`. With that distinction gone, many
uses of `cat` (e.g. in the tests) are unnecessary and have, therefore, been
removed.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69613
Summary:
Most of the statements mirror the ones provided by clang AST.
Major differences are:
- expressions are wrapped into 'ExpressionStatement' instead of being
a subclass of statement,
- semicolons are always consumed by the leaf expressions (return,
expression satement, etc),
- some clang statements are not handled yet, we wrap those into an
UnknownStatement class, which is not present in clang.
We also define an 'Expression' and 'UnknownExpression' classes in order
to produce 'ExpressionStatement' where needed. The actual implementation
of expressions is not yet ready, it will follow later.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63835
Previously, given a CompilationDatabase with two commands for the same
source file we would report that file twice with the union of the
dependencies for each command both times.
This was due to the way `ClangTool` runs actions given an input source
file (see the comment in `DependencyScanningTool.cpp`). This commit adds
a `SingleCommandCompilationDatabase` that is created with each
`CompileCommand` in the original CDB, which is then used for each
`ClangTool` invocation. This gives us a single run of
`DependencyScanningAction` per `CompileCommand`.
I looked at using `AllTUsToolExecutor` which is a parallel tool
executor, but I'm not sure it's suitable for `clang-scan-deps` as it
does a lot more sharing of state than `AllTUsToolExecutor` expects.
Differential Revision: https://reviews.llvm.org/D69643
This is a recommit of d8a4ef0e68 with the nondeterminism fixed.
This adds experimental support for extracting a Clang module dependency graph
from a compilation database. The output format is experimental and will change.
It is currently a concatenation of JSON outputs for each compilation. Future
patches will change this to deduplicate modules between compilations.
Differential Revision: https://reviews.llvm.org/D69420
This adds experimental support for extracting a Clang module dependency graph
from a compilation database. The output format is experimental and will change.
It is currently a concatenation of JSON outputs for each compilation. Future
patches will change this to deduplicate modules between compilations.
Differential Revision: https://reviews.llvm.org/D69420
This fixes two issues that prevent simple uses of modules from working.
* We would previously minimize _every_ file opened by clang, even module maps
and module pcm files. Now we only minimize files with known extensions. It
would be better if we knew which files clang intended to open as a source
file, but this works for now.
* We previously cached every lookup, even failed lookups. This is a problem
because clang stats the module cache directory before building a module and
creating that directory. If we cache that failure then the subsequent pcm
load doesn't see the module cache and fails.
Overall this still leaves us building minmized modules on disk during scanning.
This will need to be improved eventually for performance, but this is correct,
and works for now.
Differential Revision: https://reviews.llvm.org/D68835
Summary:
There's no behavior change - just moving DependencyScanningTool to its own file
since this tool can be reused across both clang-scan-deps binary and an interface
exposed as part of libClang APIs.
Reviewers: arphaman, jkorous, Bigcheese, dexonsmith
Subscribers: mgorny, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69186
llvm-svn: 375483
Summary:
Reduce include dependencies by no longer including Pass.h from
DataLayout.h. That include seemed irrelevant to DataLayout, as
well as being irrelevant to several users of DataLayout.
Reviewers: rnk
Reviewed By: rnk
Subscribers: mehdi_amini, hiraditya, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D69261
llvm-svn: 375436
This test is failing on Windows bots, revert for now (will check the right fix and retry the patch).
Summary: This reverts commit 962ca076e5.
Reviewers: Bigcheese, jkorous, arphaman
Subscribers: dexonsmith, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69079
llvm-svn: 375079
Summary:
Clang's -M mode includes these extra dependencies in its output and clang-scan-deps
should have equivalent behavior, so adding these extradeps to output just like
how its being done for ".d" file generation mode.
Reviewers: arphaman, dexonsmith, Bigcheese, jkorous
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69017
llvm-svn: 375074
Summary:
This revision introduces a new namespace, `clang::transformer`, to hold
the declarations for the Transformer library.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68876
llvm-svn: 374962
Summary: Move the `RewriteRule` class and related declarations into its own set
of files (header, implementation). Only the `Transformer` class is left in the
Transformer-named files. This change clarifies the distinction between the
`RewriteRule` class, which is essential to the Transformer library, and the
`Transformer` class, which is only one possible `RewriteRule` interpreter
(compare to `TransformerClangTidyCheck`, a clang-tidy based interpreter).
Reviewers: gribozavr
Subscribers: jfb, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68795
llvm-svn: 374558
Summary:
Removes the `isEqual` method from StencilPartInterface and modifies equality to
use the string representation returned by the `toString` method for comparison.
This means the `run` and `selection` stencils return true by default, and
clients should be cautious in relying on equality operator for comparison of
stencils containing parts generated by these functions.
It also means we no longer need the custom RTTI support (typeId() and
down_cast()), so it has been removed.
Patch by Harshal T. Lehri.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68825
llvm-svn: 374552
Summary:
- So far, we only recognize the host compilation with offloading and
skip the offloading part.
Reviewers: tra, yaxunl
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68660
llvm-svn: 374470
Summary:
It seems that when the CachingFileSystem is first given a file to open that is actually a directory, it incorrectly
caches that path to be errenous and throws an error when subsequently a directory open call is made for the same
path.
This change makes it so that we do NOT cache a path if it turns out we asked for a file when its a directory.
Reviewers: arphaman
Subscribers: dexonsmith, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68193
llvm-svn: 374366
Summary:
The Transformer library has been growing inside of
lib/Tooling/Refactoring. However, it's not really related to anything else in
that directory. This revision moves all Transformer-related files into their own
include & lib directories. A followup revision will (temporarily) add
forwarding headers to help any users migrate their code to the new location.
Reviewers: gribozavr
Subscribers: mgorny, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68637
llvm-svn: 374271
Summary:
`toString` generates a string representation of the stencil.
Patch by Harshal T. Lehri.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68574
llvm-svn: 373916
Summary:
This revision adds three new Stencil combinators:
* `expression`, which idiomatically constructs the source for an expression,
including wrapping the expression's source in parentheses if needed.
* `deref`, which constructs an idiomatic dereferencing expression.
* `addressOf`, which constructs an idiomatic address-taking expression.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68315
llvm-svn: 373593
Summary: Every change triggered by a rewrite rule is anchored at a particular
location in the source code. This patch refines how that location is chosen and
defines it as an explicit function so it can be shared by other Transformer
implementations.
This patch was inspired by a bug found by a clang tidy, wherein two changes were
anchored at the same location (the expansion loc of the macro) resulting in the
discarding of the second change.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66652
llvm-svn: 373093
Summary:
This revision adds `run`, a StencilPart that runs a user-defined function that
computes a result over `MatchFinder::MatchResult`.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67969
llvm-svn: 372936
Summary:
This new Style rule is made as a part of adding support for NetBSD KNF in clang-format. NetBSD have it's own priority of includes which should be followed while formatting NetBSD code. This style sorts the Cpp Includes according to the priorities of NetBSD, as mentioned in the [Style Guide](http://cvsweb.netbsd.org/bsdweb.cgi/src/share/misc/style?rev=HEAD&content-type=text/x-cvsweb-markup)
The working of this Style rule shown below:
**Configuration:**
This revision introduces a new field under IncludeCategories named `SortPriority` which defines the priority of ordering the `#includes` and the `Priority` will define the categories for grouping the `#include blocks`.
Reviewers: cfe-commits, mgorny, christos, MyDeveloperDay
Reviewed By: MyDeveloperDay
Subscribers: lebedev.ri, rdwampler, christos, mgorny, krytarowski
Patch By: Manikishan
Tags: #clang, #clang-format
Differential Revision: https://reviews.llvm.org/D64695
llvm-svn: 372919
Summary:
This revision introduces a separate (small) library for the `MatchConsumer`
abstraction: computations over AST match results. This abstraction is central
to the Transformer framework, and there deserves being defined explicitly.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67961
llvm-svn: 372870
Summary:
This revision add the `access` and `ifBound` combinators to the Stencil library:
* `access` -- constructs an idiomatic expression for accessing a member (a
`MemberExpr`).
* `ifBound` -- chooses between two `StencilParts` based on the whether an id is
bound in the match (corresponds to the combinator of the same name in
RangeSelector).
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67633
llvm-svn: 372605
Summary:
Introduces facilities for easily building source-code strings, including
idiomatic use of parentheses and the address-of, dereference and member-access
operators (dot and arrow) and queries about need for parentheses.
Reviewers: gribozavr
Subscribers: mgorny, cfe-commits, ilya-biryukov
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67632
llvm-svn: 372595
Summary:
Adds two new combinators and corresponding tests to the RangeSelector library.
* `ifBound` -- conditional evaluation of range-selectors, based on whether a
given node id is bound in the match.
* `elseBranch` -- selects the source range of the else and its statement.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67621
llvm-svn: 372410
Scan deps tool crashes when called on a C++ file, containing an include
that has the same name as a directory.
The tool crashes since it finds foo/dir and tries to read that as a file and fails.
Patch by: kousikk (Kousik Kumar)
Differential Revision: https://reviews.llvm.org/D67091
llvm-svn: 371903
This resolves differences observed on LLVM + Clang when running the comparison between canonical
dependencies (full preprocessing, no file manager reused), and dependencies obtained
when the file manager was reused between the full preprocessing invocations.
llvm-svn: 371751
This commit adds an optimization to clang-scan-deps and clang's preprocessor that skips excluded preprocessor
blocks by bumping the lexer pointer, and not lexing the tokens until reaching appropriate #else/#endif directive.
The skip positions and lexer offsets are computed when the file is minimized, directly from the minimized tokens.
On an 18-core iMacPro with macOS Catalina Beta I got 10-15% speed-up from this optimization when running clang-scan-deps on
the compilation database for a recent LLVM and Clang (3511 files).
Differential Revision: https://reviews.llvm.org/D67127
llvm-svn: 371656
Summary:
This significantly reduces the time required to run clangd tests, by
~10%.
Should also have an effect on other tests that run command-line parsing
multiple times inside a single invocation.
Reviewers: gribozavr, sammccall
Reviewed By: sammccall
Subscribers: kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D67163
llvm-svn: 370908
There's no need to purge symlinked entries in the FileManager,
as the new FileEntryRef API allows us to compute dependencies more
accurately when the FileManager is reused.
llvm-svn: 370493
to report the dependencies to the client
This will allow the scanner to report modular dependencies to the consumer.
This will also allow the scanner to accept regular cc1 clang invocations, e.g.
in an implementation of a libclang C API for clang-scan-deps, that I will add
follow-up patches for in the future.
llvm-svn: 370425
the dependency scanner on a single worker thread
This behavior can be controlled using the new `-reuse-filemanager` clang-scan-deps
option. By default the file manager is reused.
The added test/ClangScanDeps/symlink.cpp is able to pass with
the reused filemanager after the related FileEntryRef changes
landed earlier. The test test/ClangScanDeps/subframework_header_dir_symlink.m
still fails when the file manager is reused (I run the FileCheck with not to
make it PASS). I will address this in a follow-up patch that improves
the DirectoryEntry name modelling in the FileManager.
llvm-svn: 370420
Summary:
Returns the first token in every mapping where the token is an identifier.
This API is required to be able to highlight macro expansions in clangd.
Reviewers: hokein, ilya-biryukov
Subscribers: kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66470
llvm-svn: 369385
The change in r368681 contains a (probably unintentional) behavioral change for
rewrite rules with a single matcher. Previously, the single matcher would not
need to be bound (`joinCaseMatchers` returned it directly), even though a final
DynTypeMatcher was created and bound by `buildMatcher`. With the new change, a
single matcher will be bound, in addition to the final binding (which is now in
`buildMatchers`, but happens roughly at the same point in the overall flow).
This patch simply duplicates the "final matcher" trick: it creates an extra
DynTypedMatcher for each rewrite rule case matcher, and unconditionally makes it
bindable. This is probably not the right long-term fix, but it does allow
existing code to continue to work with this interface.
Subscribers: cfe-commits, gribozavr, ymandel
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66273
llvm-svn: 368958
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368942
Summary:
- Moved the SourceExtraction header from lib to include so that it can be used in clangd.
Reviewers: arphaman
Subscribers: ilya-biryukov, dexonsmith, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65878
llvm-svn: 368743
Summary:
This patch removes an (artificial) limitation of `applyFirst`, which requires
that all of the rules' matchers can be grouped together in a single `anyOf()`.
This change generalizes the code to group the matchers into separate `anyOf`s
based on compatibility. Correspondingly, `buildMatcher` is changed to
`buildMatchers`, to allow for returning a set of matchers rather than just one.
Reviewers: gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65877
llvm-svn: 368681
D65628 requires a flag to specify the number of threads for a clang-doc step. It would be good to use ExecutorConcurrency after exposing it instead of creating a new one that has the same purpose.
Differential Revision: https://reviews.llvm.org/D65833
llvm-svn: 368196
This commit implements the fast dependency scanning mode in clang-scan-deps: the
preprocessing is done on files that are minimized using the dependency directives source minimizer.
A shared file system cache is used to ensure that the file system requests and source minimization
is performed only once. The cache assumes that the underlying filesystem won't change during the course
of the scan (or if it will, it will not affect the output), and it can't be evicted. This means that the
service and workers can be used for a single run of a dependency scanner, and can't be reused across multiple,
incremental runs. This is something that we'll most likely support in the future though.
Note that the driver still utilizes the underlying real filesystem.
This commit is also still missing the fast skipped PP block skipping optimization that I mentioned at EuroLLVM talk.
Additionally, the file manager is still not reused by the threads as well.
Differential Revision: https://reviews.llvm.org/D63907
llvm-svn: 368086
Summary:
While useful as a sentinel value when iterating over tokens, having
'eof' in the tree, seems to do more harm than good.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: javed.absar, kristof.beyls, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64576
llvm-svn: 368062
This patch is a prerequisite for using LangStandard from Driver in
https://reviews.llvm.org/D64793.
It moves LangStandard* and InputKind::Language to Basic. It is mostly
mechanical, with only a few changes of note:
- enum Language has been changed into enum class Language : uint8_t to
avoid a clash between OpenCL in enum Language and OpenCL in enum
LangFeatures and not to increase the size of class InputKind.
- Now that getLangStandardForName, which is currently unused, also checks
both canonical and alias names, I've introduced a helper getLangKind
which factors out a code pattern already used 3 times.
The patch has been tested on x86_64-pc-solaris2.11, sparcv9-sun-solaris2.11,
and x86_64-pc-linux-gnu.
There's a companion patch for lldb which uses LangStandard.h
(https://reviews.llvm.org/D65717).
While polly includes isl which in turn uses InputKind::C, that part of the
code isn't even built inside the llvm tree. I've posted a patch to allow
for both InputKind::C and Language::C upstream
(https://groups.google.com/forum/#!topic/isl-development/6oEvNWOSQFE).
Differential Revision: https://reviews.llvm.org/D65562
llvm-svn: 367864
Update the callers of FileManager::getFile and FileManager::getDirectory to handle the new llvm::ErrorOr-returning methods.
Signed-off-by: Harlan Haskins <harlan@apple.com>
llvm-svn: 367616
Summary:
Currently, Transformer rejects any changes to source locations inside macro
expansions. This change relaxes that constraint to allow rewrites when the
entirety of the expansion is replaced, since that can be mapped to replacing the
entirety of the expansion range in the file source. This change makes
Transformer consistent with the handling of edit ranges in `clang::edit::Commit`
(which is used, for example, for applying `FixItHint`s from diagnostics).
Reviewers: ilya-biryukov
Subscribers: gribozavr, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64518
llvm-svn: 366473
Summary:
Adds the function `getRangeForEdit` to validate that a given source range is
editable and, if needed, translate it into a range in the source file (for
example, if it's sourced in macro expansions).
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64924
llvm-svn: 366469
Summary:
It's common to use compiler wrappers by setting CC="gomacc clang++".
This results in both args appearing in compile_commands.json, and clang's driver
can't handle this.
This patch attempts to recognize this pattern (by looking for well-known
wrappers) and dropping argv0 in this case.
It conservatively ignores other cases for now:
- wrappers with unknown names
- wrappers that accept -flags
- wrappers where the compiler to use is implied (usually cc or gcc)
This is done at the JSONCompilationDatabase level rather than somewhere more
fundamental, as (hopefully) this isn't a general conceptual problem, but a messy
aspect of the nature of the ecosystem around compile_commands.json.
i.e. compilation databases more tightly tied to the build system should not have
this problem.
Reviewers: phosek, klimek
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64297
llvm-svn: 365887
Summary:
A tooling-focused alternative to the AST. This commit focuses on the
memory-management strategy and the structure of the AST.
More to follow later:
- Operations to mutate the syntax trees and corresponding textual
replacements.
- Mapping between clang AST nodes and syntax tree nodes.
- More node types corresponding to the language constructs.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: llvm-commits, mgorny, cfe-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D61637
........
Fixes buildbots which were crashing on SyntaxTests.exe
llvm-svn: 365465
Summary:
A tooling-focused alternative to the AST. This commit focuses on the
memory-management strategy and the structure of the AST.
More to follow later:
- Operations to mutate the syntax trees and corresponding textual
replacements.
- Mapping between clang AST nodes and syntax tree nodes.
- More node types corresponding to the language constructs.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: llvm-commits, mgorny, cfe-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D61637
llvm-svn: 365355
Summary:
This revision allows users to specify the insertion of an included directive (at
the top of the file being rewritten) as part of a rewrite rule. These
directives are bundled with `RewriteRule` cases, so that different cases can
potentially result in different include actions.
Reviewers: ilya-biryukov, gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63892
llvm-svn: 364917
thread worker code and better error handling
This commit extracts out the code that will powers the fast scanning
worker into a new file in a new DependencyScanning library. The error
and output handling is improved so that the clients can gather
errors/results from the worker directly.
Differential Revision: https://reviews.llvm.org/D63681
llvm-svn: 364474
Summary:
Wraps JSON compilation database with a target and mode adding database
wrapper. So that driver can correctly figure out which toolchain to use.
Note that clients that wants to make use of this target discovery mechanism
needs to link in TargetsInfos and initialize them at startup.
Reviewers: ilya-biryukov
Subscribers: mgorny, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63755
llvm-svn: 364386
Summary:
Previously, we performed rename for all kinds of symbols (local, global).
This patch narrows the scope by only renaming symbols not being used
outside of the main file (with index asisitance). Renaming global
symbols is not supported at the moment (return an error).
Reviewers: sammccall
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63426
llvm-svn: 364283
Summary:
This change makes sure we have a single mapping for each macro expansion,
even if the result of expansion was empty.
To achieve that, we take information from PPCallbacks::MacroExpands into
account. Previously we relied only on source locations of expanded tokens.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62953
llvm-svn: 364236
Summary: Used in clangd for a code tweak that expands a macro.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62954
llvm-svn: 363698
Summary:
The `before` and `after` selectors allow users to specify a zero-length range --
a point -- at the relevant location in an AST-node's source. Point ranges can
be useful, for example, to insert a change using an API that takes a range to be
modified (e.g. `tooling::change()`).
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62419
llvm-svn: 361955
Summary:
Conceptually, a single-case RewriteRule has a matcher, edit(s) and an (optional)
explanation. `makeRule` previously only took the matcher and edit(s). This
change adds (optional) support for the explanation.
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62390
llvm-svn: 361643
Summary:
RangeSelector had a number of cases of capturing a StringRef in a lambda, which
lead to dangling references. This change converts all uses in the API of
`StringRef` to `std::string` to avoid this problem. `std::string` in the API is
a reasonable choice, because the combinators are always storing the string
beyond the life of the combinator construction.
Reviewers: ilya-biryukov, gribozavr
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62328
llvm-svn: 361514
Using a static function as a template parameter gets a bogus compile-time
error with Visual Studio 2017, prior to version 15.8. Our current
minimum-version requirement is a particular update to VS2015, and we
assume all Visual Studio 2017 versions are usable. This patch makes the
code buildable with older versions of VS2017, and can be reverted after
we upgrade the minimum version sometime in the future.
Description of the Microsoft bug:
https://developercommunity.visualstudio.com/content/problem/25334/error-code-c2971-when-specifying-a-function-as-the.html
Differential Revision: https://reviews.llvm.org/D62202
llvm-svn: 361502
Add support for creating a `StencilPart` from any `RangeSelector`, which
broadens the scope of `Stencil`.
Correspondingly, deprecate Stencil's specialized combinators `node` and `sNode`
in favor of using the new `selection` combinator directly (with the appropriate
range selector).
Reviewers: sbenza
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62160
llvm-svn: 361413
Transformer provides an enum to indicate the range of source text to be edited.
That support is now redundant with the new (and more general) RangeSelector
library, so we remove the custom enum support in favor of supporting any
RangeSelector.
Reviewers: ilya-biryukov
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62149
llvm-svn: 361392
Eduardo Caldas