Commit Graph

8200 Commits

Author SHA1 Message Date
Haojian Wu 263dcf452f [syntax] Introduce a TokenManager interface.
TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.

It decouples the syntax-tree from a particular token implementation (TokenBuffer
previously).  This enables us to use a different underlying token implementation
for the syntax Leaf node -- in clang pseudoparser, we want to produce a
syntax-tree with its own pseudo::Token rather than syntax::Token.

Differential Revision: https://reviews.llvm.org/D128411
2022-07-15 10:30:37 +02:00
Fangrui Song 3c849d0aef Modernize Optional::{getValueOr,hasValue} 2022-07-15 01:20:39 -07:00
Jonas Devlieghere 888673b6e3
Revert "[clang] Implement ElaboratedType sugaring for types written bare"
This reverts commit 7c51f02eff because it
stills breaks the LLDB tests. This was  re-landed without addressing the
issue or even agreement on how to address the issue. More details and
discussion in https://reviews.llvm.org/D112374.
2022-07-14 21:17:48 -07:00
Matheus Izvekov 7c51f02eff
[clang] Implement ElaboratedType sugaring for types written bare
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.

The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.

An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.

---

Troubleshooting list to deal with any breakage seen with this patch:

1) The most likely effect one would see by this patch is a change in how
   a type is printed. The type printer will, by design and default,
   print types as written. There are customization options there, but
   not that many, and they mainly apply to how to print a type that we
   somehow failed to track how it was written. This patch fixes a
   problem where we failed to distinguish between a type
   that was written without any elaborated-type qualifiers,
   such as a 'struct'/'class' tags and name spacifiers such as 'std::',
   and one that has been stripped of any 'metadata' that identifies such,
   the so called canonical types.
   Example:
   ```
   namespace foo {
     struct A {};
     A a;
   };
   ```
   If one were to print the type of `foo::a`, prior to this patch, this
   would result in `foo::A`. This is how the type printer would have,
   by default, printed the canonical type of A as well.
   As soon as you add any name qualifiers to A, the type printer would
   suddenly start accurately printing the type as written. This patch
   will make it print it accurately even when written without
   qualifiers, so we will just print `A` for the initial example, as
   the user did not really write that `foo::` namespace qualifier.

2) This patch could expose a bug in some AST matcher. Matching types
   is harder to get right when there is sugar involved. For example,
   if you want to match a type against being a pointer to some type A,
   then you have to account for getting a type that is sugar for a
   pointer to A, or being a pointer to sugar to A, or both! Usually
   you would get the second part wrong, and this would work for a
   very simple test where you don't use any name qualifiers, but
   you would discover is broken when you do. The usual fix is to
   either use the matcher which strips sugar, which is annoying
   to use as for example if you match an N level pointer, you have
   to put N+1 such matchers in there, beginning to end and between
   all those levels. But in a lot of cases, if the property you want
   to match is present in the canonical type, it's easier and faster
   to just match on that... This goes with what is said in 1), if
   you want to match against the name of a type, and you want
   the name string to be something stable, perhaps matching on
   the name of the canonical type is the better choice.

3) This patch could exposed a bug in how you get the source range of some
   TypeLoc. For some reason, a lot of code is using getLocalSourceRange(),
   which only looks at the given TypeLoc node. This patch introduces a new,
   and more common TypeLoc node which contains no source locations on itself.
   This is not an inovation here, and some other, more rare TypeLoc nodes could
   also have this property, but if you use getLocalSourceRange on them, it's not
   going to return any valid locations, because it doesn't have any. The right fix
   here is to always use getSourceRange() or getBeginLoc/getEndLoc which will dive
   into the inner TypeLoc to get the source range if it doesn't find it on the
   top level one. You can use getLocalSourceRange if you are really into
   micro-optimizations and you have some outside knowledge that the TypeLocs you are
   dealing with will always include some source location.

4) Exposed a bug somewhere in the use of the normal clang type class API, where you
   have some type, you want to see if that type is some particular kind, you try a
   `dyn_cast` such as `dyn_cast<TypedefType>` and that fails because now you have an
   ElaboratedType which has a TypeDefType inside of it, which is what you wanted to match.
   Again, like 2), this would usually have been tested poorly with some simple tests with
   no qualifications, and would have been broken had there been any other kind of type sugar,
   be it an ElaboratedType or a TemplateSpecializationType or a SubstTemplateParmType.
   The usual fix here is to use `getAs` instead of `dyn_cast`, which will look deeper
   into the type. Or use `getAsAdjusted` when dealing with TypeLocs.
   For some reason the API is inconsistent there and on TypeLocs getAs behaves like a dyn_cast.

5) It could be a bug in this patch perhaps.

Let me know if you need any help!

Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>

Differential Revision: https://reviews.llvm.org/D112374
2022-07-15 04:16:55 +02:00
Fangrui Song 52cb972537 [CommandLine] --help: print "-o <xxx>" instead of "-o=<xxx>"
Accepting -o= is a quirk of CommandLine. For --help, we should print the
conventional "-o <xxx>".
2022-07-14 01:28:28 -07:00
Balázs Kéri 0e95921bc3 [clang-tidy] Improve check cert-dcl58-cpp.
Detect template specializations that should be handled specially.
In some cases it is allowed to extend the `std` namespace with
template specializations.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D129353
2022-07-14 09:04:03 +02:00
Fangrui Song e690137dde [Support] Change compression::zlib::{compress,uncompress} to use uint8_t *
It's more natural to use uint8_t * (std::byte needs C++17 and llvm has
too much uint8_t *) and most callers use uint8_t * instead of char *.
The functions are recently moved into `llvm::compression::zlib::`, so
downstream projects need to make adaption anyway.
2022-07-13 16:26:54 -07:00
Jonas Devlieghere 3968936b92
Revert "[clang] Implement ElaboratedType sugaring for types written bare"
This reverts commit bdc6974f92 because it
breaks all the LLDB tests that import the std module.

  import-std-module/array.TestArrayFromStdModule.py
  import-std-module/deque-basic.TestDequeFromStdModule.py
  import-std-module/deque-dbg-info-content.TestDbgInfoContentDequeFromStdModule.py
  import-std-module/forward_list.TestForwardListFromStdModule.py
  import-std-module/forward_list-dbg-info-content.TestDbgInfoContentForwardListFromStdModule.py
  import-std-module/list.TestListFromStdModule.py
  import-std-module/list-dbg-info-content.TestDbgInfoContentListFromStdModule.py
  import-std-module/queue.TestQueueFromStdModule.py
  import-std-module/stack.TestStackFromStdModule.py
  import-std-module/vector.TestVectorFromStdModule.py
  import-std-module/vector-bool.TestVectorBoolFromStdModule.py
  import-std-module/vector-dbg-info-content.TestDbgInfoContentVectorFromStdModule.py
  import-std-module/vector-of-vectors.TestVectorOfVectorsFromStdModule.py

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/45301/
2022-07-13 09:20:30 -07:00
Sam McCall 6b50f2bbdb [clangd] Enable designator inlay hints by default.
Differential Revision: https://reviews.llvm.org/D129595
2022-07-13 16:31:45 +02:00
Kazu Hirata 53daa177f8 [clang, clang-tools-extra] Use has_value instead of hasValue (NFC) 2022-07-12 22:47:41 -07:00
Matheus Izvekov bdc6974f92
[clang] Implement ElaboratedType sugaring for types written bare
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.

The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.

An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.

Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>

Differential Revision: https://reviews.llvm.org/D112374
2022-07-13 02:10:09 +02:00
Sam McCall 93cd159ca9 [clangd] Remove `allCommitCharacters`
This was added in 2a095ff6f5, however it never worked with VSCode
due to bugs in vscode-languageclient
(https://github.com/microsoft/vscode-languageserver-node/issues/673).
Now that it does work, we can tell the interactions with text editing, with
snippets, and vscode's select-first-completion behavior are bad.

The spec is vague and clients could do something reasonable with the
current values. However they could clearly do something unreasonable
too, and over time behavior+spec tends to converge on VSCode's behavior.

This addresses https://github.com/clangd/vscode-clangd/pull/358
See also https://github.com/clangd/vscode-clangd/pull/358 which hotfixes
this on the client-side (as we can't apply this change retroactively to
clangd 12-14).

Differential Revision: https://reviews.llvm.org/D129579
2022-07-12 21:42:38 +02:00
Tom Praschan 9f57b65a27 [clangd] Include "final" when printing class declaration
Fixes https://github.com/clangd/clangd/issues/1184

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D128202
2022-07-11 12:20:23 +02:00
Tom Praschan cd85d9aeef Go-to-type on smart_ptr<Foo> now also shows Foo
Fixes clangd/clangd#1026

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D128826
2022-07-11 12:13:47 +02:00
Danny Mösch 963192228b [clang-tidy] Sort release notes entries alphabetically by check name 2022-07-09 15:45:19 +02:00
Danny Mösch 33e2129544 [clang-tidy] Initialize boolean variables with `false` in cppcoreguidelines-init-variables' fix-it
In case of a variable with a built-in boolean type, `false` is a better fit to default-initialize it.

Reviewed By: njames93

Differential Revision: https://reviews.llvm.org/D129420
2022-07-09 14:48:50 +02:00
Christian Kandeler b1fbc0519c [clangd] Support "usedAsMutableReference" in member initializations
That is, mark constructor parameters being used to initialize
non-const reference members.

Reviewed By: nridge

Differential Revision: https://reviews.llvm.org/D128977
2022-07-08 23:16:20 -04:00
Leonard Chan 474c873148 Revert "[llvm] cmake config groundwork to have ZSTD in LLVM"
This reverts commit f07caf20b9 which seems to break upstream https://lab.llvm.org/buildbot/#/builders/109/builds/42253.
2022-07-08 13:48:05 -07:00
Cole Kissane f07caf20b9 [llvm] cmake config groundwork to have ZSTD in LLVM
- added `FindZSTD.cmake`
- added a CMake option `LLVM_ENABLE_ZSTD` with behavior mirroring that of `LLVM_ENABLE_ZLIB`
- likewise added have_zstd to compiler-rt/test/lit.common.cfg.py, clang-tools-extra/clangd/test/lit.cfg.py, and several lit.site.cfg.py.in files mirroring have_zlib behavior

Reviewed By: leonardchan, MaskRay

Differential Revision: https://reviews.llvm.org/D128465
2022-07-08 11:46:52 -07:00
Cole Kissane ea61750c35 [NFC] Refactor llvm::zlib namespace
* Refactor compression namespaces across the project, making way for a possible
  introduction of alternatives to zlib compression.
  Changes are as follows:
  * Relocate the `llvm::zlib` namespace to `llvm::compression::zlib`.

Reviewed By: MaskRay, leonardchan, phosek

Differential Revision: https://reviews.llvm.org/D128953
2022-07-08 11:19:07 -07:00
Aaron Ballman fee77a2073 Disable clang-format entirely for test directories
See discussion here:

https://github.com/llvm/llvm-project/issues/55982

And the RFC here:
https://discourse.llvm.org/t/rfc-disable-clang-format-in-the-clang-test-tree/63498/2

We don't generally expect test files to be formatted according to the
style guide. Indeed, some tests may require specific formatting for the
purposes of the test.

When tests intentionally do not conform to the "correct" formatting,
this causes errors in the CI, which can drown out real errors and causes
people to stop trusting the CI over time.

From the history of the clang/test/.clang-format file, it looks as if
there have been attempts to make clang-format do a subset of formatting
that would be useful for tests. However, it looks as if it's hard to
make clang-format do exactly the right thing -- see the back-and-forth
between
13316a7
and
7b5bddf.

These changes disable the .clang-format file for clang/test, llvm/test,
and clang-tools-extra/test.

Fixes #55982
Differential Revision: https://reviews.llvm.org/D128706
2022-07-08 07:34:18 -04:00
Sam McCall b15127d056 [clangd] Disable flaky test 2022-07-07 19:17:09 +02:00
Sam McCall eb64dbd6e0 [clangd] Fix flaky throttler test
The production code doesn't depend on the relative destruction order of
the throttle request and the main request, but the test does.
2022-07-07 18:09:54 +02:00
Haojian Wu cd3aa338c7 [pseudo] NFC, fix the header guard for Language.h 2022-07-07 14:42:26 +02:00
serge-sans-paille 011d2bf864 [clang-tidy] Fix confusable identifier interaction with unavailable class def
Fix #56404
2022-07-07 08:02:29 +02:00
Tobias Ribizel a638648fef
[clangd] add inlay hints for std::forward-ed parameter packs
This adds special-case treatment for parameter packs in
make_unique-like functions to forward parameter names to inlay hints.
The parameter packs are being resolved recursively by traversing the
function body of forwarding functions looking for expressions matching
the (std::forwarded) parameters expanded from a pack.
The implementation checks whether parameters are being passed by
(rvalue) reference or value and adds reference inlay hints accordingly.
The traversal has a limited recursion stack depth, and recursive calls
like std::make_tuple are cut off to avoid hinting duplicate parameter
names.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D124690
2022-07-06 22:23:18 +02:00
Sam McCall e7fa272bc6 [clangd] Fix missing key function in PreambleThrottler 2022-07-06 18:27:09 +02:00
Sam McCall 511a7eef93 [clangd] Fix tests after ed0e20d5e8 2022-07-06 15:22:05 +02:00
Sam McCall 7d8e2742d9 [pseudo] Define recovery strategy as grammar extension.
Differential Revision: https://reviews.llvm.org/D129158
2022-07-06 15:03:38 +02:00
Sam McCall ed0e20d5e8 [clangd] Support external throttler for preamble builds
Building preambles is the most resource-intensive thing clangd does, driving
peak RAM and sustained CPU usage.

In a hosted environment where multiple clangd instances are packed into the same
container, it's useful to be able to limit the *aggregate* resource peaks.

Differential Revision: https://reviews.llvm.org/D129100
2022-07-06 14:58:24 +02:00
Dmitri Gribenko 40af95d80b [clang-tools-extra] Fix a link in ReleaseNotes.rst 2022-07-05 23:09:55 +02:00
Dmitri Gribenko 05130a6ba7 new clang-tidy checker for assignments within condition clause of if statement
new clang-tidy checker for assignments within the condition clause of an 'if' statement.

Reviewed By: gribozavr2

Differential Revision: https://reviews.llvm.org/D127114
2022-07-05 23:04:12 +02:00
Sam McCall 3121167488 [pseudo] Add error-recovery framework & brace-based recovery
The idea is:

- a parse failure is detected when all heads die when trying to shift the next token
- we can recover by choosing a nonterminal we're partway through parsing, and
  determining where it ends through nonlocal means (e.g. matching brackets)
- we can find candidates by walking up the stack from the (ex-)heads
- the token range is defined using heuristics attached to grammar rules
- the unparsed region is represented in the forest by an Opaque node

This patch has the core GLR functionality.
It does not allow recovery heuristics to be attached as extensions to
the grammar, but rather infers a brace-based heuristic.

Expected followups:

- make recovery heuristics grammar extensions (depends on D127448)
- add recovery to our grammar for bracketed constructs and sequence nodes
- change the structure of our augmented `_ := start` rules to eliminate some
  special-cases in glrParse.
- (if I can work out how): avoid some spurious recovery cases described in comments

(Previously mistakenly committed as a0f4c10ae2)

Differential Revision: https://reviews.llvm.org/D128486
2022-07-05 20:49:41 +02:00
Haojian Wu 9ab67cc8bf [pseudo] Implement guard extension.
- Extend the GLR parser to allow conditional reduction based on the
  guard functions;
- Implement two simple guards (contextual-override/final) for cxx.bnf;
- layering: clangPseudoCXX depends on clangPseudo (as the guard function need
  to access the TokenStream);

Differential Revision: https://reviews.llvm.org/D127448
2022-07-05 15:55:15 +02:00
Haojian Wu d263447311 [pseudo] Fix the build for the benchmark tool. 2022-07-05 15:42:41 +02:00
Haojian Wu 70c0d92930 [pseudo] Use the prebuilt cxx grammar for the lit tests, NFC.
Differential Revision: https://reviews.llvm.org/D129074
2022-07-05 15:17:18 +02:00
Sam McCall 9fbf1107cc [pseudo] Eliminate LRTable::Action. NFC
The last remaining uses are in tests/test builders.
Replace with a builder struct.

Differential Revision: https://reviews.llvm.org/D129093
2022-07-05 14:35:41 +02:00
serge-sans-paille 3897d3cf71 [clang-tidy] By-pass portability issues in confusable-identifiers test 2022-07-05 12:36:00 +02:00
serge-sans-paille 7a550212e8 [clang-tidy] Fix confusable identifiers interaction with DeclContext
Properly checks enclosing DeclContext, and add the related test case.
It would be great to be able to use Sema to check conflicting scopes, but that's
not something clang-tidy seems to be able to do :-/

Fix #56221

Differential Revision: https://reviews.llvm.org/D128715
2022-07-05 08:56:26 +02:00
Sam McCall b37dafd5dc [pseudo] Store shift and goto actions in a compact structure with faster lookup.
The actions table is very compact but the binary search to find the
correct action is relatively expensive.
A hashtable is faster but pretty large (64 bits per value, plus empty
slots, and lookup is constant time but not trivial due to collisions).

The structure in this patch uses 1.25 bits per entry (whether present or absent)
plus the size of the values, and lookup is trivial.

The Shift table is 119KB = 27KB values + 92KB keys.
The Goto table is 86KB = 30KB values + 57KB keys.
(Goto has a smaller keyspace as #nonterminals < #terminals, and more entries).

This patch improves glrParse speed by 28%: 4.69 => 5.99 MB/s
Overall the table grows by 60%: 142 => 228KB.

By comparison, DenseMap<unsigned, StateID> is "only" 16% faster (5.43 MB/s),
and results in a 285% larger table (547 KB) vs the baseline.

Differential Revision: https://reviews.llvm.org/D128485
2022-07-04 19:40:04 +02:00
Haojian Wu 5f0a054f89 [pseudo] Remove duplicated code in ClangPseudo.cpp
The code was added accidently during the rebase when landing fe66aebd.
2022-07-04 11:32:56 +02:00
Nicolas van Kempen f7a80c3d08 [clang-tidy] Properly forward clang-tidy output when running tests
When running tests, the check_clang_tidy script encodes the output
string, making it hard to read when debugging checks. This removes the
.encode() call.

Test Plan:
Making a new default check for testing (as of right now, it includes a
failing test):

[~/llvm-project/clang-tools-extra] python3 clang-tidy/add_new_check.py
bugprone example
<...>
Pre-changes:

[~/llvm-project/build] ninja check-clang-tools
<...>
------------------------ clang-tidy output -----------------------
b"1 warning
generated.\n/data/users/nvankempen/llvm-project/build/Debug/tools/clang/tools/extra/test/clang-tidy/checkers/Output/bugprone-example.cpp.tmp.cpp:4:6:
warning: function 'f' is insufficiently awesome [bugprone-example]\nvoid
f();\n
^\n/data/users/nvankempen/llvm-project/build/Debug/tools/clang/tools/extra/test/clang-tidy/checkers/Output/bugprone-example.cpp.tmp.cpp:4:6:
note: insert 'awesome'\nvoid f();\n     ^\n     awesome_\n"

------------------------------------------------------------------
<...>
Post-changes:

[~/llvm-project/build] ninja check-clang-tools
<...>
------------------------ clang-tidy output -----------------------
1 warning generated.
/data/users/nvankempen/llvm-project/build/Debug/tools/clang/tools/extra/test/clang-tidy/checkers/Output/bugprone-example.cpp.tmp.cpp:4:6:
warning: function 'f' is insufficiently awesome [bugprone-example]
void f();
     ^
/data/users/nvankempen/llvm-project/build/Debug/tools/clang/tools/extra/test/clang-tidy/checkers/Output/bugprone-example.cpp.tmp.cpp:4:6:
note: insert 'awesome'
void f();
     ^
     awesome_

------------------------------------------------------------------
<...>

Differential Revision: https://reviews.llvm.org/D127807
2022-07-03 16:30:17 -06:00
Ishaan Gandhi a65a3bffd3 [clang-tidy] Don't treat invalid branches as identical
The clang-tidy check bugprone-branch-clone has a false positive if some
symbols are undefined. This patch silences the warning when the two
sides of a branch are invalid.

Fixes #56057

Differential Revision: https://reviews.llvm.org/D128402
2022-07-03 15:50:33 -06:00
Argyrios Kyrtzidis 0d3a2b4c66 [Lex] Introduce `PPCallbacks::LexedFileChanged()` preprocessor callback
This is a preprocessor callback focused on the lexed file changing, without conflating effects of line number directives and other pragmas.
A client that only cares about what files the lexer processes, like dependency generation, can use this more straightforward
callback instead of `PPCallbacks::FileChanged()`. Clients that want the pragma directive effects as well can keep using `FileChanged()`.

A use case where `PPCallbacks::LexedFileChanged()` is particularly simpler to use than `FileChanged()` is in a situation
where a client wants to keep track of lexed file changes that include changes from/to the predefines buffer, where it becomes
unnecessary complicated trying to use `FileChanged()` while filtering out the pragma directives effects callbacks.

Also take the opportunity to provide information about the prior `FileID` the `Lexer` moved from, even when entering a new file.

Differential Revision: https://reviews.llvm.org/D128947
2022-07-01 14:22:31 -07:00
Haojian Wu bbcd8e5271 [pseudo] NFC, polish the fix of c998273499 2022-07-01 21:25:46 +02:00
Haojian Wu c998273499 [pseudo] Fix an out-of-bound issue in getReduceRules. 2022-07-01 20:16:06 +02:00
Sam McCall a322c104cb [pseudo] temporary fix for missing generated header after fe66aebd75
Better fix to be added by Haojian later!
2022-07-01 16:45:22 +02:00
David Goldman dc6c1f181b [clangd][ObjC] Fix ObjC method definition completion
D124637 improved filtering of method expressions, but not method
definitions. With this change, clangd will now filter ObjC method
definition completions based on their entire selector instead of
only the first selector fragment.

Differential Revision: https://reviews.llvm.org/D128821
2022-07-01 10:02:47 -04:00
Christian Kandeler ac511fd439 [clangd] Also mark output arguments of array subscript expressions
... with the "usedAsMutableReference" semantic token modifier.
It's quite unusual to declare the index parameter of a subscript
operator as a non-const reference type, but arguably that makes it even
more helpful to be aware of it when working with such code.

Reviewed By: nridge

Differential Revision: https://reviews.llvm.org/D128892
2022-07-01 04:45:20 -04:00
Haojian Wu fe66aebd75 [pseudo] Define a clangPseudoCLI library.
- define a common data structure Language which is a compiled result of the
  bnf grammar. It is defined in Language.h;
- creates a clangPseudoCLI lib which defines a grammar commandline flag and
  expose a function to get the Language. It supports --grammar=cxx,
  --grammmar=/path/to/file.bnf;
- use the clangPseudoCLI in clang-pseudo, fuzzer, and benchmark tools (
  simplify the code and use the prebuilt cxx grammar);

Split out from https://reviews.llvm.org/D127448.

Differential Revision: https://reviews.llvm.org/D128679
2022-07-01 08:31:34 +02:00
Sam McCall 9b6bb12b85 [pseudo] Add ForestNode descendants iterator, print ambiguous/opaque node stats.
Differential Revision: https://reviews.llvm.org/D128930
2022-06-30 21:20:55 +02:00
Sam McCall 8b04c331b5 [pseudo] Forest dump ascii art isn't broken by large indices 2022-06-30 16:53:51 +02:00
Fangrui Song 67854f9ed0 Use value_or instead of getValueOr. NFC 2022-06-29 21:55:02 -07:00
Christian Kandeler c09e533374 [clangd] Also mark output arguments of operator call expressions
There's no reason that arguments to e.g. lambda calls should be treated
differently than those to "real" functions.

Reviewed By: nridge

Differential Revision: https://reviews.llvm.org/D128329
2022-06-29 20:12:37 -04:00
Sam McCall bc5e7ced1c [pseudo] Fix bugs/inconsistencies in forest dump.
- when printing a shared node for the second time, don't print its children
  (This keeps output proportional to the size of the structure)
- when printing a shared node for the second time, print its type only, not rule
  (for consistency with above: don't dump details of nodes twice)
- don't abbreviate shared nodes, to ensure we can prune the tree there

Differential Revision: https://reviews.llvm.org/D128805
2022-06-29 22:56:26 +02:00
Sam Estep cf1f978d31 [clang][dataflow] Use NoopLattice in optional model
Followup to D128352. This patch pulls the `NoopLattice` class out from the `NoopAnalysis.h` test file into its own `NoopLattice.h` source file, and uses it to replace usage of `SourceLocationsLattice` in `UncheckedOptionalAccessModel`.

Reviewed By: ymandel, sgatev, gribozavr2, xazax.hun

Differential Revision: https://reviews.llvm.org/D128356
2022-06-29 20:10:42 +00:00
Sam Estep 2adaca532d [clang][dataflow] Use diagnosis API in optional checker
Followup to D127898. This patch updates `bugprone-unchecked-optional-access` to use the new `diagnoseCFG` function instead of just looking at the exit block.

A followup to this will update the optional model itself to use a noop lattice rather than redundantly computing the diagnostics in both phases of the analysis.

Reviewed By: ymandel, sgatev, gribozavr2, xazax.hun

Differential Revision: https://reviews.llvm.org/D128352
2022-06-29 19:50:36 +00:00
Sam Estep 2a33d12642 Revert "[clang][dataflow] Use diagnosis API in optional checker"
This reverts commit cafb8b4ff2.
2022-06-29 19:34:44 +00:00
Sam Estep cafb8b4ff2 [clang][dataflow] Use diagnosis API in optional checker
Followup to D127898. This patch updates `bugprone-unchecked-optional-access` to use the new `diagnoseCFG` function instead of just looking at the exit block.

A followup to this will update the optional model itself to use a noop lattice rather than redundantly computing the diagnostics in both phases of the analysis.

Reviewed By: ymandel, sgatev, gribozavr2, xazax.hun

Differential Revision: https://reviews.llvm.org/D128352
2022-06-29 19:20:09 +00:00
Corentin Jabot a9a60f20e6 [Clang] Rename StringLiteral::isAscii() => isOrdinary() [NFC]
"Ascii" StringLiteral instances are actually narrow strings
that are UTF-8 encoded and do not have an encoding prefix.
(UTF8 StringLiteral are also UTF-8 encoded strings, but with
the u8 prefix.

To avoid possible confusion both with actuall ASCII strings,
and with future works extending the set of literal encodings
supported by clang, this rename StringLiteral::isAscii() to
isOrdinary(), matching C++ standard terminology.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D128762
2022-06-29 18:28:51 +02:00
Haojian Wu 1ba7f5218c [pseudo] Update the cxx.bnf path in comments to reflect the new
location, NFC
2022-06-29 15:10:39 +02:00
Kadir Cetinkaya 333620d37a
[clangd] Support multiline semantic tokens
Per LSP, multiline tokens should be handled as if they end at the end
of the line starting the token (there's also a capability to enable them, but
that's an adventure for a different day).

Fixes https://github.com/clangd/clangd/issues/1145

Differential Revision: https://reviews.llvm.org/D127856
2022-06-29 13:49:03 +02:00
Sam McCall 743971faaf Revert "[pseudo] Add error-recovery framework & brace-based recovery"
This reverts commit a0f4c10ae2.
This commit hadn't been reviewed yet, and was unintentionally included
on another branch.
2022-06-28 21:11:09 +02:00
Sam McCall d25361c3af [pseudo] Move ellipsis into initializer-list-item. NFC
This makes the list formation a bit simpler.
2022-06-28 21:08:43 +02:00
Sam McCall a0f4c10ae2 [pseudo] Add error-recovery framework & brace-based recovery
The idea is:
 - a parse failure is detected when all heads die when trying to shift
   the next token
 - we can recover by choosing a nonterminal we're partway through parsing,
   and determining where it ends through nonlocal means (e.g. matching brackets)
 - we can find candidates by walking up the stack from the (ex-)heads
 - the token range is defined using heuristics attached to grammar rules
 - the unparsed region is represented in the forest by an Opaque node

This patch has the core GLR functionality.
It does not allow recovery heuristics to be attached as extensions to
the grammar, but rather infers a brace-based heuristic.

Expected followups:
 - make recovery heuristics grammar extensions (depends on D127448)
 - add recover to our grammar for bracketed constructs and sequence nodes
 - change the structure of our augmented `_ := start` rules to eliminate
   some special-cases in glrParse.
 - (if I can work out how): avoid some spurious recovery cases described
   in comments
 - grammar changes to eliminate the hard distinction between init-list
   and designated-init-list shown in the recovery-init-list.cpp testcase

Differential Revision: https://reviews.llvm.org/D128486
2022-06-28 21:08:43 +02:00
Sam McCall 3f028c02ba [pseudo] Grammar::parseBNF returns Grammar not unique_ptr. NFC 2022-06-28 16:34:21 +02:00
Sam McCall 241557fb06 [pseudo] Move cxx grammar into the cxx/ directory. NFC 2022-06-28 16:02:10 +02:00
Sam McCall aacefc817d [pseudo] Simplify/loosen the grammar around lambda captures.
Treat captures as a uniform list, rather than default-captures being special
snowflakes that may only appear at the start.

This accepts a larger set of (incorrect) code, and simplifies error-handling
by making this fit into the usual homogeneous-list pattern.

Differential Revision: https://reviews.llvm.org/D128708
2022-06-28 15:56:12 +02:00
Sam McCall 8cf28585a4 [pseudo] Allow mixed designated/undesignated init lists.
This isn't allowed by the standard grammar but is allowed in C, and clang/GCC
permit it as an extension.
It avoids the need to determine which type of list we have in error-recovery.

While here, also support array index designators `{ [4]=1 }` which are
also legal in C, and common extensions in C++.

Differential Revision: https://reviews.llvm.org/D128687
2022-06-28 15:45:41 +02:00
Sam McCall 85eaecbe8e [pseudo] Check follow-sets instead of tying reduce actions to lookahead tokens.
Previously, the action table stores a reduce action for each lookahead
token it should allow. These tokens are the followSet(action.rule.target).

In practice, the follow sets are large, so we spend a bunch of time binary
searching around all these essentially-duplicates to check whether our lookahead
token is there.
However the number of reduces for a given state is very small, so we're
much better off linear scanning over them and performing a fast check for each.

D128318 was an attempt at this, storing a bitmap for each reduce.
However it's even more compact just to use the follow sets directly, as
there are fewer nonterminals than (state, rule) pairs. It's also faster.

This specialized approach means unbundling Reduce from other actions in
LRTable, so it's no longer useful to support it in Action. I suspect
Action will soon go away, as we store each kind of action separately.

This improves glrParse speed by 42% (3.30 -> 4.69 MB/s).
It also reduces LR table size by 59% (343 -> 142kB).

Differential Revision: https://reviews.llvm.org/D128472
2022-06-28 00:36:16 +02:00
Aaron Ballman 9878e17624 Silence an "illegal conversion" diagnostic
MSVC was issuing "illegal conversion; more than one user-defined
conversion has been implicitly applied" as a warning on this code.
Explicitly calling .str() causes a StringRef to be materialized so
that a second user-defined conversion is not required.
2022-06-27 12:04:01 -04:00
Kazu Hirata 94460f5136 Don't use Optional::hasValue (NFC)
This patch replaces x.hasValue() with x where x is contextually
convertible to bool.
2022-06-26 19:54:41 -07:00
Joachim Priesner b2cb7e81f8 [clang-tidy] cppcoreguidelines-virtual-class-destructor: Fix crash when "virtual" keyword is expanded from a macro
Check llvm::Optional before dereferencing it.

Compute VirtualEndLoc differently to avoid an assertion failure
in clang::SourceManager::getFileIDLoaded:

Assertion `0 && "Invalid SLocOffset or bad function choice"' failed
2022-06-25 15:50:13 -06:00
Kazu Hirata 3b7c3a654c Revert "Don't use Optional::hasValue (NFC)"
This reverts commit aa8feeefd3.
2022-06-25 11:56:50 -07:00
Kazu Hirata aa8feeefd3 Don't use Optional::hasValue (NFC) 2022-06-25 11:55:57 -07:00
Kazu Hirata b8df4093e4 [clang, clang-tools-extra] Don't use Optional::{hasValue,getValue} (NFC) 2022-06-25 11:55:33 -07:00
Richard 5e97788a3e [clang-tidy] Update release notes (NFC)
- Sort changes to existing checks by check name
- Correct check link
2022-06-24 10:48:47 -06:00
Haojian Wu 9081d3d809 [clang-tidy] Make the cert/uppercase-literal-suffix-integer fully hermetic.
after the test-reorg commit (89a1d03e2b), the
cert/uppercase test starts to fail in our internal environment -- it accesses
a header file from "../readability", which is not friendly to a hermetic test environment.

This change makes the test fully hermetic, and does some cleanup on the
uppercase header (I think it is better to move it the share
Inputs/Header directory, and rename it)

Differential Revision: https://reviews.llvm.org/D128511
2022-06-24 14:18:02 +02:00
Nathan James fbf611ed2a
[clang-tidy] Extend spelling for CheckOptions
The current way to specify CheckOptions is pretty verbose and unintuitive.
Given that the options are a dictionary it makes much more sense to treat them as such in the config files.
Example:
```
CheckOptions: {SomeCheck.Option: true, SomeCheck.OtherOption: 'ignore'}
# Or
CheckOptions:
  SomeCheck.Option: true
  SomeCheck.OtherOption: 'ignore'
```

This change will still handle the old syntax with no issue, ensuring we don't screw up users current config files.

The only observable differences are support for the new syntax and `-dump=config` will emit using the new syntax.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D128337
2022-06-23 19:59:31 +01:00
Sam McCall 768216cac0 [pseudo] Handle no-reductions-available on the fastpath. NFC
This is a ~2% speedup.
2022-06-23 20:34:11 +02:00
Nathan James 5ca68d5845
[clang-tidy] Add `-verify-config` command line argument
Adds a `-verify-config` command line argument, that when specified will verify the Checks and CheckOptions fields in the config files:
 - A warning will be raised for any check that doesn't correspond to a registered check, a suggestion will also be emitted for close misses.
 - A warning will be raised for any check glob(containing *) that doesn't match any registered check.
 - A warning will be raised for any CheckOption that isn't read by any registered check, a suggestion will also be emitted for close misses.

This can be useful if debuging why a certain check isn't enabled, or the options are being handled as you expect them to be.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D127446
2022-06-23 19:23:09 +01:00
Sam McCall 466eae6aa3 [pseudo] Store last node popped in the queue, not its parent(s). NFC
We have to walk up to the last node to find the start token, but no need
to go even one node further.

This is one node fewer to store, but more importantly if the last node
happens to have multiple parents we avoid storing the sequence multiple times.

This saves ~5% on glrParse.
Based on a comment by hokein@ on https://reviews.llvm.org/D128307
2022-06-23 20:10:20 +02:00
Sam McCall 7aff663b2a [pseudo] Store reduction sequences by pointer in heaps, instead of by value.
Copying sequences around as the heap resized is significantly expensive.

This speeds up glrParse by ~35% (2.4 => 3.25 MB/s)

Differential Revision: https://reviews.llvm.org/D128307
2022-06-23 19:41:11 +02:00
Sam McCall 3e610f2cdc [pseudo] Turn glrReduce into a class, reuse storage across calls.
This is a ~5% speedup, we no longer have to allocate the priority queues and
other collections for each reduction step where we use them.

It's also IMO easier to understand the structure of a class with methods vs a
function with nested lambdas.

Differential Revision: https://reviews.llvm.org/D128301
2022-06-23 19:27:47 +02:00
Sam McCall f9710d1908 [pseudo] Add a fast-path to GLR reduce when both pop and push are trivial
In general we split a reduce into pop/push, so concurrently-available reductions
can run in the correct order. The data structures for this are expensive.

When only one reduction is possible at a time, we need not do this: we can pop
and immediately push instead.
Strictly this is correct whenever we yield one concurrent PushSpec.

This patch recognizes a trivial but common subset of these cases:
 - there must be no pending pushes and only one head available to pop
 - the head must have only one reduction rule
 - the reduction path must be a straight line (no multiple parents)

On my machine this speeds up by 2.12 -> 2.30 MB/s = 8%

Differential Revision: https://reviews.llvm.org/D128299
2022-06-23 18:21:59 +02:00
Sam McCall b70ee9d984 Reland "[pseudo] Track heads as GSS nodes, rather than as "pending actions"."
This reverts commit 2c80b53198.

Fixes LRTable::buildForTest to create states that are referenced but
have no actions.
2022-06-23 18:21:44 +02:00
Sam McCall 2c80b53198 Revert "[pseudo] Track heads as GSS nodes, rather than as "pending actions"."
This reverts commit e3ec054dfd.

Tests fail in asserts mode: https://lab.llvm.org/buildbot/#/builders/109/builds/41217
2022-06-23 18:16:38 +02:00
Sam McCall e3ec054dfd [pseudo] Track heads as GSS nodes, rather than as "pending actions".
IMO this model is simpler to understand (borrowed from the LR0 patch D127357).
It also makes error recovery easier to implement, as we have a simple list of
head nodes lying around to recover from when needed.
(It's not quite as nice as LR0 in this respect though).

It's slightly slower (2.24 -> 2.12 MB/S on my machine = 5%) but nothing close
to as bad as LR0.
However
 - I think we'd have to eat a litle performance loss otherwise to implement
   error recovery.
 - this frees up some complexity budget for optimizations like fastpath push/pop
   (this + fastpath is already faster than head)
 - I haven't changed the data structure here and it's now pretty dumb, we can
   make it faster

Differential Revision: https://reviews.llvm.org/D128297
2022-06-23 17:26:42 +02:00
Sam McCall 6b187fdf3b [pseudo] Add xfail tests for a simple-declaration/function-definition ambiguity
I expect to eliminate this ambiguity at the grammar level by use of guards,
because it interferes with brace-based error recvoery.

Differential Revision: https://reviews.llvm.org/D127400
2022-06-23 15:52:22 +02:00
Aaron Ballman e36535f99c Fix sphinx build for clang-tools-extra
I think it doesn't like the non-ASCII characters in the block, so using
a text block to disable syntax highlighting.

This should fix:
https://lab.llvm.org/buildbot/#/builders/115/builds/29888
2022-06-23 07:22:00 -04:00
Nathan James 165d69337a
[clang-tidy][docs] Fix a couple of missed cases from 6e566bc552
A few of the checks had documentation URLs that weren't updated in the aforementioned commit.
2022-06-22 21:37:16 +01:00
Nathan James b1cc59fd3a
[clang-tidy][docs] Reorganise release notes 2022-06-22 21:21:24 +01:00
Richard 89a1d03e2b [clang-tidy] Organize test files into subdirectories by module (NFC)
Eliminate clutter by reorganizing the Lit test files for clang-tidy:
- Move checkers/<module>-* to checkers/<module>/*.
- Move module specific inputs from Inputs to <module>/Inputs.  Remove
  any module prefix from the file or subdirectory name as they are no
  longer needed.
- Introduce a Lit substitution %clang_tidy_headers for the system
  headers in checkers/Inputs/Headers and use this throughout.  This
  avoids referencing system headers through a relative path to the
  parent directory and makes it clear that these fake system headers are
  shared among all modules.
- Update add_new_check.py to follow the above conventions when creating
  the boiler plate test files for a new check.
- Update Contributing.rst to describe per-module Inputs directory and
  fix link to test source code.

Differential Revision: https://reviews.llvm.org/D128072
2022-06-22 12:13:34 -06:00
Richard b967a97550 [clang-tidy] Fix documentation (NFC)
The documentation files were reorganized into subdirectories, but a new
check was added concurrently and wasn't rebased correctly before
submitting.  Sort the new clang-tidy checks by check name and fix the
indentation of bugprone-unchecked-optional-access.
2022-06-22 10:49:00 -06:00
Nico Weber 721875db2b Reland "[gn build] (manually) port b94db7ed7e (Confusables.inc)"
b94db7ed7e relanded in c3574ef739.

This relands commit 180bae08a0, rebased across the new version of
commit c3574ef739, and rebased across 10f7255d32.
2022-06-22 10:38:14 -04:00
serge-sans-paille c3574ef739 [clang-tidy] Confusable identifiers detection
Detect identifiers that are confusable using a variant of Unicode definition

        http://www.unicode.org/reports/tr39/#Confusable_Detection

and have conflicting scopes.

This a recommit (with portability and feature fixes) of b94db7ed7e

Differential Revision: https://reviews.llvm.org/D112916
2022-06-22 16:17:20 +02:00
Aaron Ballman bb297024fa Don't treat invalid parameters as being unused
The misc-unused-parameters check would trigger false positive warnings
about the parameter being unused when the parameter declaration was
invalid. No longer issue the warning in that case on the assumption
that most parameters are used in practice, so the extra diagnostic is
most likely a false positive.

Fixes #56152
2022-06-22 08:56:38 -04:00
Balazs Benics ae76b2f455 [clang-tidy][docs] Fix wrong sphinx link after d9afb8c3e8
There was a copy-paste mistake at the embedded link:
  `clang-tidy/checks/cppcoreguidelines-virtual-class-destructor`
  ->
  `clang-tidy/checks/cppcoreguidelines/virtual-class-destructor`

Sphinx error:
/home/zbebnal/git/llvm-project/clang-tools-extra/docs/ReleaseNotes.rst:168:unknown document: clang-tidy/checks/cppcoreguidelines-virtual-class-destructor

Build bot: https://lab.llvm.org/buildbot#builders/115/builds/29805

Differential Revision: https://reviews.llvm.org/D126891
2022-06-21 11:42:09 +02:00
Balazs Benics d9afb8c3e8 [clang-tidy] cppcoreguidelines-virtual-class-destructor should ignore final classes
The `cppcoreguidelines-virtual-class-destructor` supposed to enforce
http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c35-a-base-class-destructor-should-be-either-public-and-virtual-or-protected-and-non-virtual

Quote:
> A **base** class destructor should be either public and virtual, or
> protected and non-virtual
[emphasis mine]

However, this check still rules the following case:

  class MostDerived final : public Base {
  public:
    MostDerived() = default;
    ~MostDerived() = default;
    void func() final;
  };

Even though `MostDerived` class is marked `final`, thus it should not be
considered as a **base** class. Consequently, the rule is satisfied, yet
the check still flags this code.

In this patch, I'm proposing to ignore `final` classes since they cannot
be //base// classes.

Reviewed By: whisperity

Differential Revision:  https://reviews.llvm.org/D126891
2022-06-21 11:02:18 +02:00
Kazu Hirata ed8fceaa09 Don't use Optional::getValue (NFC) 2022-06-20 23:35:53 -07:00