Commit Graph

527 Commits

Author SHA1 Message Date
Yitzhak Mandelbaum d5b983555f [libTooling] Add `toString` method to the Stencil class
Summary:
`toString` generates a string representation of the stencil.

Patch by Harshal T. Lehri.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68574

llvm-svn: 373916
2019-10-07 16:20:22 +00:00
Haojian Wu 5d862c042b [clang-rename] Fix a crash when renaming a class without definition.
Reviewers: sammccall

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68459

llvm-svn: 373748
2019-10-04 14:09:31 +00:00
Yitzhak Mandelbaum e80e889695 [libTooling] Add various Stencil combinators for expressions.
Summary:
This revision adds three new Stencil combinators:
* `expression`, which idiomatically constructs the source for an expression,
  including wrapping the expression's source in parentheses if needed.
* `deref`, which constructs an idiomatic dereferencing expression.
* `addressOf`, which constructs an idiomatic address-taking expression.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D68315

llvm-svn: 373593
2019-10-03 13:01:00 +00:00
Benjamin Kramer 340406a740 [Stencil] Hide implementaion detai. NFC.
llvm-svn: 373504
2019-10-02 18:33:44 +00:00
Yitzhak Mandelbaum db24ef509e [libTooling] Transformer: refine `SourceLocation` specified as anchor of changes.
Summary: Every change triggered by a rewrite rule is anchored at a particular
location in the source code.  This patch refines how that location is chosen and
defines it as an explicit function so it can be shared by other Transformer
implementations.

This patch was inspired by a bug found by a clang tidy, wherein two changes were
anchored at the same location (the expansion loc of the macro) resulting in the
discarding of the second change.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66652

llvm-svn: 373093
2019-09-27 15:26:04 +00:00
Yitzhak Mandelbaum ea966c1bc0 [libTooling] Add `run` combinator to Stencils.
Summary:
This revision adds `run`, a StencilPart that runs a user-defined function that
computes a result over `MatchFinder::MatchResult`.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67969

llvm-svn: 372936
2019-09-26 00:53:56 +00:00
Paul Hoad 52e44b1423 [clang-format] Modified SortIncludes and IncludeCategories to priority for sorting #includes within the Group Category.
Summary:
This new Style rule is made as a part of adding support for NetBSD KNF in clang-format. NetBSD have it's own priority of includes which should be followed while formatting NetBSD code. This style sorts the Cpp Includes according to the priorities of NetBSD, as mentioned in the [Style Guide](http://cvsweb.netbsd.org/bsdweb.cgi/src/share/misc/style?rev=HEAD&content-type=text/x-cvsweb-markup)
 The working of this Style rule shown below:

**Configuration:**
This revision introduces a new field under IncludeCategories named `SortPriority` which defines the priority of ordering the `#includes` and the `Priority` will define the categories for grouping the `#include blocks`.

Reviewers: cfe-commits, mgorny, christos, MyDeveloperDay

Reviewed By: MyDeveloperDay

Subscribers: lebedev.ri, rdwampler, christos, mgorny, krytarowski

Patch By: Manikishan

Tags: #clang, #clang-format

Differential Revision: https://reviews.llvm.org/D64695

llvm-svn: 372919
2019-09-25 20:33:01 +00:00
Yitzhak Mandelbaum 148fd9f9c1 [libTooling] Introduce the MatchConsumer abstraction
Summary:
This revision introduces a separate (small) library for the `MatchConsumer`
abstraction: computations over AST match results.  This abstraction is central
to the Transformer framework, and there deserves being defined explicitly.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67961

llvm-svn: 372870
2019-09-25 13:34:04 +00:00
Yitzhak Mandelbaum 0a81b4ed97 [libTooling] Add `access` and `ifBound` combinators to Stencil library.
Summary:
This revision add the `access` and `ifBound` combinators to the Stencil library:
* `access` -- constructs an idiomatic expression for accessing a member (a
  `MemberExpr`).
* `ifBound` -- chooses between two `StencilParts` based on the whether an id is
  bound in the match (corresponds to the combinator of the same name in
  RangeSelector).

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67633

llvm-svn: 372605
2019-09-23 13:21:42 +00:00
Yitzhak Mandelbaum 1588c08735 [libTooling] Introduce new library of source-code builders.
Summary:
Introduces facilities for easily building source-code strings, including
idiomatic use of parentheses and the address-of, dereference and member-access
operators (dot and arrow) and queries about need for parentheses.

Reviewers: gribozavr

Subscribers: mgorny, cfe-commits, ilya-biryukov

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67632

llvm-svn: 372595
2019-09-23 12:40:10 +00:00
Alex Lorenz 172e8a7a5d [clang-scan-deps] strip the --serialize-diagnostics argument
This ensures that clang-scan-deps won't write out diagnostics when
scanning dependencies.

llvm-svn: 372444
2019-09-21 00:17:26 +00:00
Yitzhak Mandelbaum eff88e42f7 [libTooling] Add `ifBound`, `elseBranch` RangeSelector combinators.
Summary:
Adds two new combinators and corresponding tests to the RangeSelector library.
* `ifBound` -- conditional evaluation of range-selectors, based on whether a
   given node id is bound in the match.
* `elseBranch` -- selects the source range of the else and its statement.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67621

llvm-svn: 372410
2019-09-20 17:11:03 +00:00
Alex Lorenz ee30b0ecc2 [clang-scan-deps] Fix for headers having the same name as a directory
Scan deps tool crashes when called on a C++ file, containing an include
that has the same name as a directory.
The tool crashes since it finds foo/dir and tries to read that as a file and fails.

Patch by: kousikk (Kousik Kumar)

Differential Revision: https://reviews.llvm.org/D67091

llvm-svn: 371903
2019-09-13 22:12:02 +00:00
Alex Lorenz 57a014d31f [clang-scan-deps] remove dots and dots dots from the reported file dependency paths
This resolves differences observed on LLVM + Clang when running the comparison between canonical
dependencies (full preprocessing, no file manager reused), and dependencies obtained
when the file manager was reused between the full preprocessing invocations.

llvm-svn: 371751
2019-09-12 18:03:24 +00:00
Alex Lorenz 428d92832c [clang-scan-deps] cast Result to ErrorOr<unique_ptr<vfs::File>> explicitly to avoid s390x-linux buildbot failure
llvm-svn: 371664
2019-09-11 21:00:13 +00:00
Alex Lorenz ca6e60971e [clang-scan-deps] add skip excluded conditional preprocessor block preprocessing optimization
This commit adds an optimization to clang-scan-deps and clang's preprocessor that skips excluded preprocessor
blocks by bumping the lexer pointer, and not lexing the tokens until reaching appropriate #else/#endif directive.
The skip positions and lexer offsets are computed when the file is minimized, directly from the minimized tokens.

On an 18-core iMacPro with macOS Catalina Beta I got 10-15% speed-up from this optimization when running clang-scan-deps on
the compilation database for a recent LLVM and Clang (3511 files).

Differential Revision: https://reviews.llvm.org/D67127

llvm-svn: 371656
2019-09-11 20:40:31 +00:00
Ilya Biryukov 433927595d [Driver] Use shared singleton instance of DriverOptTable
Summary:
This significantly reduces the time required to run clangd tests, by
~10%.

Should also have an effect on other tests that run command-line parsing
multiple times inside a single invocation.

Reviewers: gribozavr, sammccall

Reviewed By: sammccall

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D67163

llvm-svn: 370908
2019-09-04 14:26:28 +00:00
Alex Lorenz c4130cf132 [clang-scan-deps] NFC, remove outdated implementation comment
There's no need to purge symlinked entries in the FileManager,
as the new FileEntryRef API allows us to compute dependencies more
accurately when the FileManager is reused.

llvm-svn: 370493
2019-08-30 17:34:22 +00:00
Dmitri Gribenko b22804b354 [Tooling] Migrated APIs that take ownership of objects to unique_ptr
Subscribers: jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66960

llvm-svn: 370451
2019-08-30 09:29:34 +00:00
Alex Lorenz ca4216abde [clang-scan-deps] NFC, refactor the DependencyScanningWorker to use a consumer
to report the dependencies to the client

This will allow the scanner to report modular dependencies to the consumer.
This will also allow the scanner to accept regular cc1 clang invocations, e.g.
in an implementation of a libclang C API for clang-scan-deps, that I will add
follow-up patches for in the future.

llvm-svn: 370425
2019-08-30 01:25:57 +00:00
Alex Lorenz 3944c9638e [clang-scan-deps] reuse the file manager across invocations of
the dependency scanner on a single worker thread

This behavior can be controlled using the new `-reuse-filemanager` clang-scan-deps
option. By default the file manager is reused.

The added test/ClangScanDeps/symlink.cpp is able to pass with
the reused filemanager after the related FileEntryRef changes
landed earlier. The test test/ClangScanDeps/subframework_header_dir_symlink.m
still fails when the file manager is reused (I run the FileCheck with not to
make it PASS). I will address this in a follow-up patch that improves
the DirectoryEntry name modelling in the FileManager.

llvm-svn: 370420
2019-08-29 22:56:38 +00:00
Dmitri Gribenko 907452107d Changed FrontendActionFactory::create to return a std::unique_ptr
Subscribers: jkorous, arphaman, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66947

llvm-svn: 370379
2019-08-29 16:38:36 +00:00
Dmitri Gribenko 1fac68b0dc ArrayRef'ized CompilerInvocation::CreateFromArgs
Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66797

llvm-svn: 370122
2019-08-27 22:13:31 +00:00
Johan Vikstrom 6687fde07a [Syntax] Added function to get macro expansion tokens to TokenBuffer.
Summary:
Returns the first token in every mapping where the token is an identifier.
This API is required to be able to highlight macro expansions in clangd.

Reviewers: hokein, ilya-biryukov

Subscribers: kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66470

llvm-svn: 369385
2019-08-20 13:34:01 +00:00
David L. Jones 4ed5521cad [Tooling] Add a hack to work around issues with matcher binding in r368681.
The change in r368681 contains a (probably unintentional) behavioral change for
rewrite rules with a single matcher. Previously, the single matcher would not
need to be bound (`joinCaseMatchers` returned it directly), even though a final
DynTypeMatcher was created and bound by `buildMatcher`. With the new change, a
single matcher will be bound, in addition to the final binding (which is now in
`buildMatchers`, but happens roughly at the same point in the overall flow).

This patch simply duplicates the "final matcher" trick: it creates an extra
DynTypedMatcher for each rewrite rule case matcher, and unconditionally makes it
bindable. This is probably not the right long-term fix, but it does allow
existing code to continue to work with this interface.

Subscribers: cfe-commits, gribozavr, ymandel

Tags: #clang

Differential Revision: https://reviews.llvm.org/D66273

llvm-svn: 368958
2019-08-15 04:10:11 +00:00
Jonas Devlieghere 2b3d49b610 [Clang] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

Differential revision: https://reviews.llvm.org/D66259

llvm-svn: 368942
2019-08-14 23:04:18 +00:00
Yitzhak Mandelbaum 385a451844 [libTooling] Fix code to avoid unused-function warning after r368681.
llvm-svn: 368862
2019-08-14 15:20:06 +00:00
Shaurya Gupta 36ca1e6399 [Tooling] Added DeclStmtClass to ExtractionSemicolonPolicy
Since the DeclStmt range includes the semicolon, it doesn't need a
semicolon at the end during extraction

llvm-svn: 368850
2019-08-14 13:37:39 +00:00
Shaurya Gupta 52d0cfc91e [Refactor] Moving SourceExtraction header from lib to include
Summary:
- Moved the SourceExtraction header from lib to include so that it can be used in clangd.

Reviewers: arphaman

Subscribers: ilya-biryukov, dexonsmith, kadircet, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65878

llvm-svn: 368743
2019-08-13 20:21:00 +00:00
Yitzhak Mandelbaum 42b957aad7 [libTooling] In Transformer, generalize `applyFirst` to admit rules with incompatible matchers.
Summary:
This patch removes an (artificial) limitation of `applyFirst`, which requires
that all of the rules' matchers can be grouped together in a single `anyOf()`.
This change generalizes the code to group the matchers into separate `anyOf`s
based on compatibility. Correspondingly, `buildMatcher` is changed to
`buildMatchers`, to allow for returning a set of matchers rather than just one.

Reviewers: gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65877

llvm-svn: 368681
2019-08-13 12:31:29 +00:00
Shaurya Gupta 7137736e49 [Extract] Fixed SemicolonExtractionPolicy for SwitchStmt and SwitchCase
Reviewers: arphaman, sammccall

Subscribers: dexonsmith, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D65883

llvm-svn: 368267
2019-08-08 08:37:49 +00:00
Diego Astiazaran ba55970c15 [Tooling] Expose ExecutorConcurrency option.
D65628 requires a flag to specify the number of threads for a clang-doc step. It would be good to use ExecutorConcurrency after exposing it instead of creating a new one that has the same purpose.

Differential Revision: https://reviews.llvm.org/D65833

llvm-svn: 368196
2019-08-07 18:35:28 +00:00
Alex Lorenz e1f4c4aad2 [clang-scan-deps] Implementation of dependency scanner over minimized sources
This commit implements the fast dependency scanning mode in clang-scan-deps: the
preprocessing is done on files that are minimized using the dependency directives source minimizer.

A shared file system cache is used to ensure that the file system requests and source minimization
is performed only once. The cache assumes that the underlying filesystem won't change during the course
of the scan (or if it will, it will not affect the output), and it can't be evicted. This means that the
service and workers can be used for a single run of a dependency scanner, and can't be reused across multiple,
incremental runs. This is something that we'll most likely support in the future though.
Note that the driver still utilizes the underlying real filesystem.

This commit is also still missing the fast skipped PP block skipping optimization that I mentioned at EuroLLVM talk.
Additionally, the file manager is still not reused by the threads as well.

Differential Revision: https://reviews.llvm.org/D63907

llvm-svn: 368086
2019-08-06 20:43:25 +00:00
Ilya Biryukov bfbf6b6cab [Syntax] Do not add a node for 'eof' into the tree
Summary:
While useful as a sentinel value when iterating over tokens, having
'eof' in the tree, seems to do more harm than good.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: javed.absar, kristof.beyls, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64576

llvm-svn: 368062
2019-08-06 17:07:58 +00:00
Rainer Orth 09d890d728 Move LangStandard*, InputKind::Language to Basic
This patch is a prerequisite for using LangStandard from Driver in
https://reviews.llvm.org/D64793.

It moves LangStandard* and InputKind::Language to Basic.  It is mostly
mechanical, with only a few changes of note:

- enum Language has been changed into enum class Language : uint8_t to
  avoid a clash between OpenCL in enum Language and OpenCL in enum
  LangFeatures and not to increase the size of class InputKind.

- Now that getLangStandardForName, which is currently unused, also checks
  both canonical and alias names, I've introduced a helper getLangKind
  which factors out a code pattern already used 3 times.

The patch has been tested on x86_64-pc-solaris2.11, sparcv9-sun-solaris2.11,
and x86_64-pc-linux-gnu.

There's a companion patch for lldb which uses LangStandard.h
(https://reviews.llvm.org/D65717).

While polly includes isl which in turn uses InputKind::C, that part of the
code isn't even built inside the llvm tree.  I've posted a patch to allow
for both InputKind::C and Language::C upstream
(https://groups.google.com/forum/#!topic/isl-development/6oEvNWOSQFE).

Differential Revision: https://reviews.llvm.org/D65562

llvm-svn: 367864
2019-08-05 13:59:26 +00:00
Harlan Haskins 8d323d1506 [clang] Adopt new FileManager error-returning APIs
Update the callers of FileManager::getFile and FileManager::getDirectory to handle the new llvm::ErrorOr-returning methods.

Signed-off-by: Harlan Haskins <harlan@apple.com>
llvm-svn: 367616
2019-08-01 21:31:56 +00:00
Yitzhak Mandelbaum 3f1ab737e2 [LibTooling] Relax Transformer to allow rewriting macro expansions
Summary:
Currently, Transformer rejects any changes to source locations inside macro
expansions. This change relaxes that constraint to allow rewrites when the
entirety of the expansion is replaced, since that can be mapped to replacing the
entirety of the expansion range in the file source.  This change makes
Transformer consistent with the handling of edit ranges in `clang::edit::Commit`
(which is used, for example, for applying `FixItHint`s from diagnostics).

Reviewers: ilya-biryukov

Subscribers: gribozavr, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64518

llvm-svn: 366473
2019-07-18 17:44:54 +00:00
Yitzhak Mandelbaum 2e97a1e19e [LibTooling] Add function to translate and validate source range for editing
Summary:
Adds the function `getRangeForEdit` to validate that a given source range is
editable and, if needed, translate it into a range in the source file (for
example, if it's sourced in macro expansions).

Reviewers: ilya-biryukov

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64924

llvm-svn: 366469
2019-07-18 17:26:57 +00:00
Sam McCall 5d9d7c59ee Re-land [JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.
Use //net/dir like other test cases for windows compatibility

llvm-svn: 365975
2019-07-12 23:38:31 +00:00
Russell Gallop 614a78c15a Revert "[JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands."
New test is failing on Windows bot

This reverts commit 9c0391b36a.

llvm-svn: 365906
2019-07-12 15:15:56 +00:00
Sam McCall 9c0391b36a [JSONCompilationDatabase] Strip distcc/ccache/gomacc wrappers from parsed commands.
Summary:
It's common to use compiler wrappers by setting CC="gomacc clang++".
This results in both args appearing in compile_commands.json, and clang's driver
can't handle this.

This patch attempts to recognize this pattern (by looking for well-known
wrappers) and dropping argv0 in this case.

It conservatively ignores other cases for now:
 - wrappers with unknown names
 - wrappers that accept -flags
 - wrappers where the compiler to use is implied (usually cc or gcc)

This is done at the JSONCompilationDatabase level rather than somewhere more
fundamental, as (hopefully) this isn't a general conceptual problem, but a messy
aspect of the nature of the ecosystem around compile_commands.json.
i.e. compilation databases more tightly tied to the build system should not have
this problem.

Reviewers: phosek, klimek

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64297

llvm-svn: 365887
2019-07-12 10:11:40 +00:00
Ilya Biryukov 625a0f70a9 [Syntax] Add assertion to catch invalid tokens early. NFC
To help with identifiying root cause of a crash we are seeing.

llvm-svn: 365599
2019-07-10 08:24:42 +00:00
Ilya Biryukov 51dad4196e [Syntax] Move roles into a separate enum
To align with reviewer's suggestions.

llvm-svn: 365479
2019-07-09 13:31:43 +00:00
Ilya Biryukov 9b3f38f990 Reland r365355: [Syntax] Introduce syntax trees
With a fix to a PS4 buildbot crash.

llvm-svn: 365466
2019-07-09 11:32:13 +00:00
Simon Pilgrim 7e3b22758d Revert rL365355 : [Syntax] Introduce syntax trees
Summary:
A tooling-focused alternative to the AST. This commit focuses on the
memory-management strategy and the structure of the AST.

More to follow later:
  - Operations to mutate the syntax trees and corresponding textual
    replacements.
  - Mapping between clang AST nodes and syntax tree nodes.
  - More node types corresponding to the language constructs.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: llvm-commits, mgorny, cfe-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D61637
........
Fixes buildbots which were crashing on SyntaxTests.exe

llvm-svn: 365465
2019-07-09 11:26:35 +00:00
Bill Wendling b9816f674b Add parentheses to silence warnings.
llvm-svn: 365395
2019-07-08 22:01:07 +00:00
Ilya Biryukov b736969edd [Syntax] Introduce syntax trees
Summary:
A tooling-focused alternative to the AST. This commit focuses on the
memory-management strategy and the structure of the AST.

More to follow later:
  - Operations to mutate the syntax trees and corresponding textual
    replacements.
  - Mapping between clang AST nodes and syntax tree nodes.
  - More node types corresponding to the language constructs.

Reviewers: sammccall

Reviewed By: sammccall

Subscribers: llvm-commits, mgorny, cfe-commits

Tags: #clang, #llvm

Differential Revision: https://reviews.llvm.org/D61637

llvm-svn: 365355
2019-07-08 17:25:02 +00:00
Fangrui Song 7264a474b7 Change std::{lower,upper}_bound to llvm::{lower,upper}_bound or llvm::partition_point. NFC
llvm-svn: 365006
2019-07-03 08:13:17 +00:00
Yitzhak Mandelbaum 727bdcb237 [LibTooling] Extend `RewriteRule` with support for adding includes.
Summary:
This revision allows users to specify the insertion of an included directive (at
the top of the file being rewritten) as part of a rewrite rule.  These
directives are bundled with `RewriteRule` cases, so that different cases can
potentially result in different include actions.

Reviewers: ilya-biryukov, gribozavr

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D63892

llvm-svn: 364917
2019-07-02 13:11:04 +00:00
Kadir Cetinkaya 7d719b75ea [clang][ArgumentAdjusters] Do not add fsyntax-only if already exists
Reviewers: hokein

Subscribers: cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D64063

llvm-svn: 364904
2019-07-02 10:45:53 +00:00