Commit Graph

4991 Commits

Author SHA1 Message Date
Marek Kurdej fdee512048 [clang-format] Add test for SpacesInLineCommentPrefix. NFC.
Fixes https://github.com/llvm/llvm-project/issues/52649.
This was already fixed in commit e967d97a35.
2022-02-16 13:54:55 +01:00
Björn Schäpers b786a4aefe [clang-format] Extend SpaceBeforeParens for requires
We can now configure the space between requires and the following paren,
seperate for clauses and expressions.

Differential Revision: https://reviews.llvm.org/D113369
2022-02-15 21:37:36 +01:00
Björn Schäpers bcd1e4612f [clang-format] Further improve support for requires expressions
Detect requires expressions in more unusable contexts. This is far from
perfect, but currently we have no good metric to decide between a
requires expression and a trailing requires clause.

Differential Revision: https://reviews.llvm.org/D119138
2022-02-15 21:37:35 +01:00
Marek Kurdej e21db15be8 [clang-format] Honour PointerAlignment in statements with initializers.
Fixes https://github.com/llvm/llvm-project/issues/53843.

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119814
2022-02-15 18:06:32 +01:00
Eric Li d1e3235f60 [libTooling] Change Tranformer's consumer to take multiple changes
Previously, Transformer would invoke the consumer once per file modified per
match, in addition to any errors encountered. The consumer is not aware of which
AtomicChanges come from any particular match. It is unclear which sets of edits
may be related or whether an error invalidates any previously emitted changes.

Modify the signature of the consumer to accept a set of changes. This keeps
related changes (i.e. all edits from a single match) together, and clarifies
that errors don't produce partial changes.

Reviewed By: ymandel

Differential Revision: https://reviews.llvm.org/D119745
2022-02-15 16:34:36 +00:00
Jan Svoboda d8298f04a9 [clang][lex][minimizer] Avoid treating path separators as comments
The minimizer strips out single-line comments (introduced by `//`). This sequence of characters can also appear in `#include` or `#import` directives where they play the role of path separators. We already avoid stripping this character sequence for `#include` but not for `#import` (which has the same semantics). This patch makes it so `#import <A//A.h>` is not affected by minimization. Previously, we would incorrectly reduce it into `#import <A`.

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D119226
2022-02-15 09:49:19 +01:00
Jan Svoboda fd2dff17c5 [clang][lex][minimizer] Ensure whitespace between squashed lines
The minimizer tries to squash multi-line macro definitions into single line. For that to work, contents of each line need to be separated by a space. Since we always strip leading whitespace on lines of a macro definition, the code currently tries to preserve exactly one space that appeared before the backslash.

This means the following code:

```
#define FOO(BAR) \
  #BAR           \
  baz
```

gets minimized into:

```
#define FOO(BAR) #BAR baz
```

However, if there are no spaces before the backslash on line 2:

```
#define FOO(BAR) \
  #BAR\
  baz
```

no space can be preserved, leading to (most likely) malformed macro definition:

```
#define FOO(BAR) #BARbaz
```

This patch makes sure we always put exactly one space at the end of line ending with a backslash.

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D119231
2022-02-15 09:49:03 +01:00
Alex Lorenz 00cd6c0420 [Preprocessor] Reduce the memory overhead of `#define` directives (Recommit)
Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.

The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.

For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.

The recommit fixes the LLDB build failure.

Differential Revision: https://reviews.llvm.org/D117348
2022-02-14 09:27:44 -08:00
Marek Kurdej c72fdad71b [clang-format] Reformat. NFC. 2022-02-14 14:05:05 +01:00
Marek Kurdej e967d97a35 [clang-format] Fix SpacesInLineCommentPrefix deleting tokens.
Fixes https://github.com/llvm/llvm-project/issues/53799.

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119680
2022-02-14 09:53:16 +01:00
Marek Kurdej e01f624adb [clang-format] Fix PointerAlignment within lambdas in a multi-variable declaration statement.
Fixes https://github.com/llvm/llvm-project/issues/43115.

Also, handle while loops with initializers (C++20) the same way as for loops.

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119648
2022-02-14 09:41:24 +01:00
Balázs Kéri 83028ad934 [clang][AST][ASTImporter] Set record to complete during import of its members.
At import of a member it may require that the record is already set to complete.
(For example 'computeDependence' at create of some Expr nodes.)
The record at this time may not be completely imported, the result of layout
calculations can be incorrect, but at least no crash occurs this way.

A good solution would be if fields of every encountered record are imported
before other members of all records. This is much more difficult to implement.

Differential Revision: https://reviews.llvm.org/D116155
2022-02-14 08:27:44 +01:00
Marek Kurdej 09559bc59a Avoid a vulgarism. NFC. 2022-02-13 22:01:06 +01:00
Marek Kurdej 25282bd6c4 [clang-format] Handle PointerAlignment in `if` and `switch` statements with initializers (C++17) the same way as in `for` loops.
Reviewed By: MyDeveloperDay, owenpan

Differential Revision: https://reviews.llvm.org/D119650
2022-02-13 21:36:58 +01:00
Marek Kurdej 9cb9445979 [clang-format] Correctly format loops and `if` statements even if preceded with comments.
Fixes https://github.com/llvm/llvm-project/issues/53758.

Braces in loops and in `if` statements with leading (block) comments were formatted according to `BraceWrapping.AfterFunction` and not `AllowShortBlocksOnASingleLine`/`AllowShortLoopsOnASingleLine`/`AllowShortIfStatementsOnASingleLine`.

Previously, the code:
```
while (true) {
  f();
}
/*comment*/ while (true) {
  f();
}
```

was incorrectly formatted to:
```
while (true) {
  f();
}
/*comment*/ while (true) { f(); }
```

when using config:
```
BasedOnStyle: LLVM
BreakBeforeBraces: Custom
BraceWrapping:
  AfterFunction: false
AllowShortBlocksOnASingleLine: false
AllowShortLoopsOnASingleLine: false
```

and it was (correctly but by chance) formatted to:
```
while (true) {
  f();
}
/*comment*/ while (true) {
  f();
}
```

when using enabling brace wrapping after functions:
```
BasedOnStyle: LLVM
BreakBeforeBraces: Custom
BraceWrapping:
  AfterFunction: true
AllowShortBlocksOnASingleLine: false
AllowShortLoopsOnASingleLine: false
```

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119649
2022-02-13 21:22:17 +01:00
Alex Lorenz 3f05192c4c Revert "[Preprocessor] Reduce the memory overhead of `#define` directives"
This reverts commit 0d9b91524e.

This change broke LLDB's build. I will need to recommit after fixing LLDB.
2022-02-11 15:53:16 -08:00
Alex Lorenz 0d9b91524e [Preprocessor] Reduce the memory overhead of `#define` directives
Recently we observed high memory pressure caused by clang during some parallel builds.
We discovered that we have several projects that have a large number of #define directives
in their TUs (on the order of millions), which caused huge memory consumption in clang due
to a lot of allocations for MacroInfo. We would like to reduce the memory overhead of
clang for a single #define to reduce the memory overhead for these files, to allow us to
reduce the memory pressure on the system during highly parallel builds. This change achieves
that by removing the SmallVector in MacroInfo and instead storing the tokens in an array
allocated using the bump pointer allocator, after all tokens are lexed.

The added unit test with 1000000 #define directives illustrates the problem. Prior to this
change, on arm64 macOS, clang's PP bump pointer allocator allocated 272007616 bytes, and
used roughly 272 bytes per #define. After this change, clang's PP bump pointer allocator
allocates 120002016 bytes, and uses only roughly 120 bytes per #define.

For an example test file that we have internally with 7.8 million #define directives, this
change produces the following improvement on arm64 macOS: Persistent allocation footprint for
this test case file as it's being compiled to LLVM IR went down 22% from 5.28 GB to 4.07 GB
and the total allocations went down 14% from 8.26 GB to 7.05 GB. Furthermore, this change
reduced the total number of allocations made by the system for this clang invocation from
1454853 to 133663, an order of magnitude improvement.

Differential Revision: https://reviews.llvm.org/D117348
2022-02-11 15:01:10 -08:00
Björn Schäpers 9aab0db13f [clang-format] Improve require and concept handling
- Added an option where to put the requires clauses.
- Renamed IndentRequires to IndentRequiresClause.
- Changed BreakBeforeConceptDeclaration from bool to an enum.

Fixes https://llvm.org/PR32165, and https://llvm.org/PR52401.

Differential Revision: https://reviews.llvm.org/D113319
2022-02-11 22:42:37 +01:00
Marek Kurdej fd16eeea9d [clang-format] Assert default style instead of commenting. NFC. 2022-02-11 12:01:25 +01:00
Marek Kurdej a218706cba [clang-format] Add tests for spacing between ref-qualifier and `noexcept`. NFC.
Cf. https://github.com/llvm/llvm-project/issues/44542.
Cf. ae1b7859cb.
2022-02-11 10:50:05 +01:00
Balazs Benics abc873694f [analyzer] Restrict CallDescription fuzzy builtin matching
`CallDescriptions` for builtin functions relaxes the match rules
somewhat, so that the `CallDescription` will match for calls that have
some prefix or suffix. This was achieved by doing a `StringRef::contains()`.
However, this is somewhat problematic for builtins that are substrings
of each other.

Consider the following:

`CallDescription{ builtin, "memcpy"}` will match for
`__builtin_wmemcpy()` calls, which is unfortunate.

This patch addresses/works around the issue by checking if the
characters around the function's name are not part of the 'name'
semantically. In other words, to accept a match for `"memcpy"` the call
should not have alphanumeric (`[a-zA-Z]`) characters around the 'match'.

So, `CallDescription{ builtin, "memcpy"}` will not match on:

 - `__builtin_wmemcpy: there is a `w` alphanumeric character before the match.
 - `__builtin_memcpyFOoBar_inline`: there is a `F` character after the match.
 - `__builtin_memcpyX_inline`: there is an `X` character after the match.

But it will still match for:
 - `memcpy`: exact match
 - `__builtin_memcpy`: there is an _ before the match
 - `__builtin_memcpy_inline`: there is an _ after the match
 - `memcpy_inline_builtinFooBar`: there is an _ after the match

Reviewed By: NoQ

Differential Revision: https://reviews.llvm.org/D118388
2022-02-11 10:45:18 +01:00
Marek Kurdej 6c7e6fc7b6 [clang-format] Do not remove required spaces when aligning tokens.
Fixes https://github.com/llvm/llvm-project/issues/44292.
Fixes https://github.com/llvm/llvm-project/issues/45874.

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119419
2022-02-10 19:15:27 +01:00
Ivan Murashko 71d7c8d870 [clangd] Crash in __memcmp_avx2_movbe
There is a clangd crash at `__memcmp_avx2_movbe`. Short problem description is below.

The method `HeaderIncludes::addExistingInclude` stores `Include` objects by reference at 2 places: `ExistingIncludes` (primary storage) and `IncludesByPriority` (pointer to the object's location at ExistingIncludes). `ExistingIncludes` is a map where value is a `SmallVector`. A new element is inserted by `push_back`. The operation might do resize. As result pointers stored at `IncludesByPriority` might become invalid.

Typical stack trace
```
    frame #0: 0x00007f11460dcd94 libc.so.6`__memcmp_avx2_movbe + 308
    frame #1: 0x00000000004782b8 clangd`llvm::StringRef::compareMemory(Lhs="
\"t2.h\"", Rhs="", Length=6) at StringRef.h:76:22
    frame #2: 0x0000000000701253 clangd`llvm::StringRef::compare(this=0x0000
7f10de7d8610, RHS=(Data = "", Length = 7166742329480737377)) const at String
Ref.h:206:34
  * frame #3: 0x00000000007603ab clangd`llvm::operator<(llvm::StringRef, llv
m::StringRef)(LHS=(Data = "\"t2.h\"", Length = 6), RHS=(Data = "", Length =
7166742329480737377)) at StringRef.h:907:23
    frame #4: 0x0000000002d0ad9f clangd`clang::tooling::HeaderIncludes::inse
rt(this=0x00007f10de7fb1a0, IncludeName=(Data = "t2.h\"", Length = 4), IsAng
led=false) const at HeaderIncludes.cpp:365:22
    frame #5: 0x00000000012ebfdd clangd`clang::clangd::IncludeInserter::inse
rt(this=0x00007f10de7fb148, VerbatimHeader=(Data = "\"t2.h\"", Length = 6))
const at Headers.cpp:262:70
```

A unit test test for the crash was created (`HeaderIncludesTest.RepeatedIncludes`). The proposed solution is to use std::list instead of llvm::SmallVector

Test Plan
```
./tools/clang/unittests/Tooling/ToolingTests --gtest_filter=HeaderIncludesTest.RepeatedIncludes
```

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D118755
2022-02-10 09:40:44 -08:00
Marek Kurdej a7b5e5b413 [clang-format] Fix formatting of macro definitions with a leading comment.
Fixes https://github.com/llvm/llvm-project/issues/43206.

Reviewed By: HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D118924
2022-02-09 22:39:59 +01:00
Marek Kurdej a77c67f939 [clang-format] Fix formatting of the array form of delete.
Fixes https://github.com/llvm/llvm-project/issues/53576.

There was an inconsistency in formatting of delete expressions.

Before:
```
delete (void*)a;
delete[](void*) a;
```

After this patch:
```
delete (void*)a;
delete[] (void*)a;
```

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119117
2022-02-09 22:36:13 +01:00
Marek Kurdej e329b5866f [clang-format] Honour "// clang-format off" when using QualifierOrder.
Fixes https://github.com/llvm/llvm-project/issues/53643.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D119218
2022-02-09 22:15:20 +01:00
Haojian Wu f1984b1433 [pseudo] Implement LRGraph
LRGraph is the key component of the clang pseudo parser, it is a
deterministic handle-finding finite-state machine, which is used to
generated the LR parsing table.

Separate from https://reviews.llvm.org/D118196.

Differential Revision: https://reviews.llvm.org/D119172
2022-02-09 11:20:07 +01:00
Kirill Bobyrev 46a6f5ae14 [clangd] NFC: Move stdlib headers handling to Clang
This will allow moving the IncludeCleaner library essentials to Clang
and decoupling them from the majority of clangd.

The patch itself just moves the code, it doesn't change existing
functionality.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D119130
2022-02-09 11:05:39 +01:00
Haojian Wu fe932a88e9 [pseudo] Add first and follow set computation in Grammar.
These will be used when building parsing table for LR parsers.

Separate from https://reviews.llvm.org/D118196.

Differential Revision: https://reviews.llvm.org/D118990
2022-02-09 09:16:27 +01:00
ksyx a70549ae43 [clang-format] Fix DefSeparator empty line issues
- Add or remove empty lines surrounding union blocks.
- Fixes https://github.com/llvm/llvm-project/issues/53229, in which
  keywords like class and struct in a line ending with left brace or
  whose next line is left brace only, will be falsely recognized as
  definition line, causing extra empty lines inserted surrounding blocks
  with no need to be formatted.

Reviewed By: MyDeveloperDay, curdeius, HazardyKnusperkeks, owenpan
Differential Revision: https://reviews.llvm.org/D119067
2022-02-07 14:23:21 +00:00
Kim Gräsman d7ddad408f Reformat CastExpr unittest suite; NFC
In preparation for adding new tests. No functional change.
2022-02-07 09:21:41 -05:00
Kadir Cetinkaya f59787084e
[clang][Lexer] Fix tests after ff77071a4d 2022-02-07 14:06:32 +01:00
Paul Robinson 0d54457f8a [IntrospectionTest] Replace "return" with "GTEST_SKIP"
If a test simply returns, it gets mis-reported as a pass; being
reported as SKIPPED is correct.

Found by the Rotten Green Tests project.
2022-02-04 12:35:44 -08:00
Owen Pan 35f7dd601d [clang-format][NFC] Fix a bug in setting type FunctionLBrace
The l_brace token in a macro definition should not be set to
TT_FunctionLBrace.

This patch could have fixed #42087.

Differential Revision: https://reviews.llvm.org/D118969
2022-02-04 11:36:30 -08:00
David Goldman 9385ece95a [HeaderSearch] Track framework name in LookupFile
Previously, the Framework name was only set if the file
came from a header mapped framework; now we'll always
set the framework name if the file is in a framework.

Differential Revision: https://reviews.llvm.org/D117830
2022-02-04 13:32:39 -05:00
Sam McCall cc8ed7b5aa [Format] Also test rvalue-qualified functions 2022-02-04 12:17:25 +01:00
Sam McCall acc3ce945c [Format] Don't derive pointers right based on space before method ref-qualifiers
The second space in `void foo() &` is always produced by clang-format,
and isn't evidence of any particular style.

Before this patch, it was considered evidence of PAS_Right, because
there is a space before a pointerlike ampersand.

This caused the following code to have "unstable" pointer alignment:
  void a() &;
  void b() &;
  int *x;
PAS_Left, Derive=false would produce 'int* x' with other lines unchanged.
But subsequent formatting with Derive=true would produce 'int *x' again.

Differential Revision: https://reviews.llvm.org/D118921
2022-02-04 12:13:58 +01:00
Haojian Wu b94f09524e [pseudo] NFC, clangSyntaxPsuedo => clangToolingSyntaxPseudo
To be consistent with existing name pattern.
2022-02-04 09:57:20 +01:00
Haojian Wu 2189960e65 [pseudo] Rename Tests.cpp => Test.cpp
To be consistent with other files in clang unittest directory.
2022-02-04 09:48:14 +01:00
ksyx 88e4e6be16 [clang-format] Use wider comment prefix space rule
This commit changes the condition of requiring comment to start with
alphanumeric characters to make no change only for a certain set of
characters, currently horizontal whitespace and punctuation characters,
to support wider set of leading characters unrelated to documentation
generation directives.

Reviewed By: HazardyKnusperkeks
Differential Revision: https://reviews.llvm.org/D118869
2022-02-03 21:49:10 +00:00
mydeveloperday 23fc20e06c [clang-format] regression from clang-format v13
https://github.com/llvm/llvm-project/issues/53567

The following source

```
namespace A {

template <int N> struct Foo<char[N]> {
  void foo() { std::cout << "Bar"; }
}; // namespace A
```

is incorrectly formatted as:

```
namespace A {

template <int N> struct Foo<char[N]>{void foo(){std::cout << "Bar";
}
}
; // namespace A
```

This looks to be caused by 5c2e7c9ca0

Reviewed By: curdeius

Differential Revision: https://reviews.llvm.org/D118911
2022-02-03 18:37:43 +00:00
Marek Kurdej ca0d97072e [clang-format] Avoid merging macro definitions.
Fixes https://github.com/llvm/llvm-project/issues/42087.

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D118879
2022-02-03 18:54:46 +01:00
Marek Kurdej 529aa4b011 [clang-format] Avoid adding space after the name of a function-like macro when the name is a keyword.
Fixes https://github.com/llvm/llvm-project/issues/31086.

Before the code:
```
#define if(x)
```

was erroneously formatted to:
```
#define if (x)
```

Reviewed By: HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D118844
2022-02-03 18:45:51 +01:00
Owen Pan eaef54f213 [clang-format] Revert a feature in RemoveBracesLLVM
Revert the handling of a single-statement block that gets wrapped.

See issue #53543.

Differential Revision: https://reviews.llvm.org/D118873
2022-02-03 02:56:09 -08:00
Haojian Wu 20e05b9f0e [syntax][pseudo] Add Grammar for the clang pseudo-parser
This patch introduces the Grammar class, which is a critial piece for constructing
a tabled-based parser.

As the first patch, the scope is limited to:
  - define base types (symbol, rules) of modeling the grammar
  - construct Grammar by parsing the BNF file (annotations are excluded for now)

Differential Revision: https://reviews.llvm.org/D114790
2022-02-03 11:28:27 +01:00
Marek Kurdej bc40b76b5b [clang-format] Correctly parse C99 digraphs: "<:", ":>", "<%", "%>", "%:", "%:%:".
Fixes https://github.com/llvm/llvm-project/issues/31592.

This commits enables lexing of digraphs in C++11 and onwards.
Enabling them in C++03 is error-prone, as it would unconditionally treat sequences like "<:" as digraphs, even if they are followed by a single colon, e.g. "<::" would be treated as "[:" instead of "<" followed by "::". Lexing in C++11 doesn't have this problem as it looks ahead the following token.
The relevant excerpt from Lexer::LexTokenInternal:
```
        // C++0x [lex.pptoken]p3:
        //  Otherwise, if the next three characters are <:: and the subsequent
        //  character is neither : nor >, the < is treated as a preprocessor
        //  token by itself and not as the first character of the alternative
        //  token <:.
```

Also, note that both clang and gcc turn on digraphs by default (-fdigraphs), so clang-format should match this behaviour.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D118706
2022-02-02 10:25:24 +01:00
Stanislav Gatev 6b8800dfb5 [clang][dataflow] Enable comparison of distinct values in Environment
Make specializations of `DataflowAnalysis` extendable with domain-specific
logic for comparing distinct values when comparing environments.

This includes a breaking change to the `runDataflowAnalysis` interface
as the return type is now `llvm::Expected<...>`.

This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: ymandel, xazax.hun

Differential Revision: https://reviews.llvm.org/D118596
2022-02-01 15:25:59 +00:00
Marek Kurdej fd33cca762 [clang-format] Fix AlignConsecutiveAssignments breaking lambda formatting.
Fixes https://github.com/llvm/llvm-project/issues/52772.

This patch fixes the formatting of the code:
```
auto aaaaaaaaaaaaaaaaaaaaa = {};
auto b                     = g([] {
  return;
});
```
which should be left as is, but before this patch was formatted to:
```
auto aaaaaaaaaaaaaaaaaaaaa = {};
auto b                     = g([] {
  return;
                    });
```

Reviewed By: MyDeveloperDay, HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D115972
2022-02-01 09:17:59 +01:00
Marek Kurdej 95bf0a9ebd [clang-format] Don't break block comments when sorting includes.
Fixes https://github.com/llvm/llvm-project/issues/34626.

Before, the include sorter would break the code:
```
#include <stdio.h>
#include <stdint.h> /* long
                       comment */
```
and change it into:
```
#include <stdint.h> /* long
#include <stdio.h>
                       comment */
```

This commit handles only the most basic case of a single block comment on an include line, but does not try to handle all the possible edge cases with multiple comments.

Reviewed By: HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D118627
2022-02-01 08:51:10 +01:00
Kadir Cetinkaya ff77071a4d
[clang][Lexer] Make raw and normal lexer behave the same for line comments
Normally there are heruistics in lexer to treat `//*` specially in
language modes that don't have line comments (to emit `/`). Unfortunately this
only applied to the first occurence of a line comment inside the file, as the
subsequent line comments were treated as if language had support for them.

This unfortunately only holds in normal lexing mode, as in raw mode all
occurences of line comments received this treatment, which created discrepancies
when comparing expanded and spelled tokens.

The proper fix would be to just make sure we treat all the line comments with a
subsequent `*` the same way, but it would imply breaking some code that's
accepted by clang today. So instead we introduce the same bug into raw lexing
mode.

Fixes https://github.com/clangd/clangd/issues/1003.

Differential Revision: https://reviews.llvm.org/D118471
2022-01-31 16:15:16 +01:00
Marek Kurdej 438f0e1f00 [clang-format] Use EXPECT_EQ instead of setting style to a default value. NFC. 2022-01-31 09:06:00 +01:00
Philip Sigillito d1aed486ef [clang-format] Handle C variables with name that matches c++ access specifier
Reviewed By: MyDeveloperDay, curdeius, HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D117416
2022-01-30 20:56:50 +01:00
Stanislav Gatev 56cc697323 [clang][dataflow] Merge distinct pointer values in Environment::join
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: ymandel, xazax.hun

Differential Revision: https://reviews.llvm.org/D118480
2022-01-29 16:33:15 +00:00
Marek Kurdej 64df51624f [clang-format] Fix misaligned trailing comments in the presence of an empty block comment.
Fixes https://github.com/llvm/llvm-project/issues/53441.

Expected code:
```
/**/   //
int a; //
```

was before misformatted to:
```
/**/     //
int a; //
```

Because the "remaining length" (after the starting `/*`) of an empty block comment `/**/` was computed to be 0 instead of 2.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D118475
2022-01-28 22:28:48 +01:00
Marek Kurdej f4d5195d2f [clang-format] Move irrelevant code from getRangeLength to getRemainingLength. NFC. 2022-01-28 12:01:02 +01:00
Martin Probst 03c59765b3 clang-format: [JS] sort import aliases. Users can define aliases for long symbols using import aliases:
import X = A.B.C;

Previously, these were unhandled and would terminate import sorting.
With this change, aliases sort as their own group, coming last after all
other imports.

Aliases are not sorted within their group, as they may reference each
other, so order is significant.

This reverts commit f750c3d95a. It fixes
the msan issue by not parsing past the end of the line when handling
import aliases.

Differential Revision: https://reviews.llvm.org/D118446
2022-01-28 11:51:28 +01:00
Vitaly Buka f750c3d95a Revert "clang-format: [JS] sort import aliases."
Triggers MSAN report.

This reverts commit c6d5efb5d9.
2022-01-27 21:16:53 -08:00
Marek Kurdej 36622c4e1a [clang-format] Fix AllowShortFunctionsOnASingleLine: InlineOnly with wrapping after record.
Fixes https://github.com/llvm/llvm-project/issues/53430.

Initially, I had a quick and dirty approach, but it led to a myriad of special cases handling comments (that may add unwrapped lines).
So I added TT_RecordLBrace type annotations and it seems like a much nicer solution.
I think that in the future it will allow us to clean up some convoluted code that detects records.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D118337
2022-01-27 18:06:31 +01:00
Martin Probst c6d5efb5d9 clang-format: [JS] sort import aliases.
Users can define aliases for long symbols using import aliases:

    import X = A.B.C;

Previously, these were unhandled and would terminate import sorting.
With this change, aliases sort as their own group, coming last after all
other imports.

Aliases are not sorted within their group, as they may reference each
other, so order is significant.

Revision URI: https://reviews.llvm.org/D118361
2022-01-27 16:16:37 +01:00
Jim Lin ad39b5bc59 [NFC] Remove duplicate include 2022-01-27 13:56:13 +08:00
Yitzhak Mandelbaum 3595189217 [clang][dataflow] Allow clients to disable built-in transfer functions.
These built-in functions build the (sophisticated) model of the code's
memory. This model isn't used by all analyses, so we provide for disabling it to
avoid incurring the costs associated with its construction.

Differential Revision: https://reviews.llvm.org/D118178
2022-01-26 17:24:59 +00:00
Benjamin Kramer f15014ff54 Revert "Rename llvm::array_lengthof into llvm::size to match std::size from C++17"
This reverts commit ef82063207.

- It conflicts with the existing llvm::size in STLExtras, which will now
  never be called.
- Calling it without llvm:: breaks C++17 compat
2022-01-26 16:55:53 +01:00
Stanislav Gatev 75c22b382f [clang][dataflow] Add a transfer function for CXXBoolLiteralExpr
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D118236
2022-01-26 15:33:00 +00:00
serge-sans-paille ef82063207 Rename llvm::array_lengthof into llvm::size to match std::size from C++17
As a conquence move llvm::array_lengthof from STLExtras.h to
STLForwardCompat.h (which is included by STLExtras.h so no build
breakage expected).
2022-01-26 16:17:45 +01:00
Marek Kurdej 93948c5299 [clang-format] Correctly format lambdas with variadic template parameters.
Fixes https://github.com/llvm/llvm-project/issues/53405.

Reviewed By: MyDeveloperDay, owenpan

Differential Revision: https://reviews.llvm.org/D118220
2022-01-26 16:10:52 +01:00
Kadir Cetinkaya b777d354f6
[clang][DeclPrinter] Fix printing for noexcept expressions
We are already building into the final result, no need to append it
again.

Fixes https://github.com/clangd/vscode-clangd/issues/290.

Differential Revision: https://reviews.llvm.org/D118245
2022-01-26 16:04:24 +01:00
Sam McCall 33c3ef2fbe [CodeCompletion][clangd] Clean __uglified parameter names in completion & hover
Underscore-uglified identifiers are used in standard library implementations to
guard against collisions with macros, and they hurt readability considerably.
(Consider `push_back(Tp_ &&__value)` vs `push_back(Tp value)`.
When we're describing an interface, the exact names of parameters are not
critical so we can drop these prefixes.

This patch adds a new PrintingPolicy flag that can applies this stripping
when recursively printing pieces of AST.
We set it in code completion/signature help, and in clangd's hover display.
All three features also do a bit of manual poking at names, so fix up those too.

Fixes https://github.com/clangd/clangd/issues/736

Differential Revision: https://reviews.llvm.org/D116387
2022-01-26 15:51:17 +01:00
Stanislav Gatev d3597ec0aa [clang][dataflow] Enable merging distinct values in Environment::join
Make specializations of `DataflowAnalysis` extendable with domain-specific
logic for merging distinct values when joining environments. This could be
a strict lattice join or a more general widening operation.

This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D118038
2022-01-26 11:40:51 +00:00
Stanislav Gatev 188d28f73c [clang][dataflow] Assign aggregate storage locations to union stmts
This patch ensures that the dataflow analysis framework does not crash
when it encounters access to members of union types.

This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D118226
2022-01-26 10:36:49 +00:00
Marek Kurdej 72e29caf03 [clang-format] Fix regression in parsing pointers to arrays.
Fixes https://github.com/llvm/llvm-project/issues/53293.

After commit 5c2e7c9, the code:
```
template <> struct S : Template<int (*)[]> {};
```
was misformatted as:
```
template <> struct S : Template<int (*)[]>{};
```

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D118106
2022-01-26 09:27:38 +01:00
Yitzhak Mandelbaum 0944c196c5 [libTooling] Adds more support for constructing object access expressions.
This patch adds a `buildAccess` function, which constructs a string with the
proper operator to use based on the expression's form and type. It also adds two
predicates related to smart pointers, which are needed by `buildAccess` but are
also of general value.

We deprecate `buildDot` and `buildArrow` in favor of the more general
`buildAccess`. These will be removed in a future patch.

Differential Revision: https://reviews.llvm.org/D116377
2022-01-25 19:43:36 +00:00
Stanislav Gatev 64ba462b6e [clang][dataflow] Add a transfer function for InitListExpr
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D118119
2022-01-25 16:28:15 +00:00
Marek Kurdej 50999e82e8 [clang-format] Space between attribute closing parenthesis and qualified type colon.
Fixes https://github.com/llvm/llvm-project/issues/35711.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks, owenpan

Differential Revision: https://reviews.llvm.org/D117894
2022-01-24 18:09:20 +01:00
ksyx 5e5efd8a91 [clang-format] Fix SeparateDefinitionBlocks issues
- Fixes https://github.com/llvm/llvm-project/issues/53227 that wrongly
  indents multiline comments
- Fixes wrong detection of single-line opening braces when used along
  with those only opening scopes, causing crashes due to duplicated
  replacements on the same token:
    void foo()
    {
      {
        int x;
      }
    }
- Fixes wrong recognition of first line of definition when the line
  starts with block comment, causing crashes due to duplicated
  replacements on the same token for this leads toward skipping the line
  starting with inline block comment:
    /*
      Some descriptions about function
    */
    /*inline*/ void bar() {
    }
- Fixes wrong recognition of enum when used as a type name rather than
  starting definition block, causing crashes due to duplicated
  replacements on the same token since both actions for enum and for
  definition blocks were taken place:
    void foobar(const enum EnumType e) {
    }
- Change to use function keyword for JavaScript instead of comparing
  strings
- Resolves formatting conflict with options EmptyLineAfterAccessModifier
  and EmptyLineBeforeAccessModifier (prompts with --dry-run (-n) or
  --output-replacement-xml but no observable change)
- Recognize long (len>=5) uppercased name taking a single line as return
  type and fix the problem of adding newline below it, with adding new
  token type FunctionLikeOrFreestandingMacro and marking tokens in
  UnwrappedLineParser:
    void
    afunc(int x) {
      return;
    }
    TYPENAME
    func(int x, int y) {
      // ...
    }
- Remove redundant and repeated initialization
- Do no change to newlines before EOF

Reviewed By: MyDeveloperDay, curdeius, HazardyKnusperkeks
Differential Revision: https://reviews.llvm.org/D117520
2022-01-24 14:23:20 +00:00
Evgeny Shulgin 589a939072 Add `isConstinit` matcher
Support C++20 constinit variables for AST Matchers.
2022-01-24 08:35:42 -05:00
Jan Svoboda 5daeada330 [clang][deps] Ensure filesystem cache consistency
The minimizing filesystem used by the dependency scanner isn't great when it comes to the consistency of its caches. There are two problems that can be exposed by a filesystem that changes during dependency scan:
1. In-memory cache entries for original and minimized files are distinct, populated at different times using separate stat/open syscalls. This means that when a file is read with minimization disabled, its contents might be inconsistent when the same file is read with minimization enabled at later point (and vice versa).
2. In-memory cache entries are indexed by filename. This is problematic for symlinks, where the contents of the symlink might be inconsistent with contents of the original file (for the same reason as in problem 1).

This patch ensures consistency by always stating/reading a file exactly once. The original contents are always cached and minimized contents are derived from that on demand. The cache entries are now indexed by their `UniqueID` ensuring consistency for symlinks too. Moreover, the stat/read syscalls are now issued outside of critical section.

Depends on D115935.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D114966
2022-01-21 13:04:25 +01:00
Evgeny Shulgin b80db150cd Add `isConsteval` matcher
Support C++20 consteval functions and C++2b if consteval for AST Matchers.
2022-01-20 13:35:10 -05:00
Stanislav Gatev c95cb4de1b [clang][dataflow] Intersect ExprToLoc when joining environments
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D117754
2022-01-20 14:30:17 +00:00
Marek Kurdej 69ecd2484f [clang-format] Indicate source location on test failure. NFC. 2022-01-20 14:10:59 +01:00
Marek Kurdej 794b1eebe7 [clang-format] Add tests for aligning `operator=` with `=delete`. NFC.
Also, add test case from https://github.com/llvm/llvm-project/issues/33044.
This was actually fixed in 480a1fab72, but there were no tests for delete.
2022-01-20 09:39:08 +01:00
Jino Park 560eb2277b [clang-format] Fix bug in parsing `operator<` with template
Fixes https://github.com/llvm/llvm-project/issues/44601.

This patch handles a bug when parsing a below example code :

```
template <class> class S;

template <class T> bool operator<(S<T> const &x, S<T> const &y) {
  return x.i < y.i;
}

template <class T> class S {
  int i = 42;
  friend bool operator< <>(S const &, S const &);
};

int main() { return S<int>{} < S<int>{}; }
```
which parse `< <>` as `<< >`, not `< <>` in terms of tokens as discussed in discord.

1. Add a condition in `tryMergeLessLess()` considering `operator` keyword and `>`
2. Force to leave a whitespace between `tok::less` and a template opener
3. Add unit test

Reviewed By: MyDeveloperDay, curdeius

Differential Revision: https://reviews.llvm.org/D117398
2022-01-20 08:59:04 +01:00
Stanislav Gatev 8e53ae3d37 [clang][dataflow] Add a transfer function for conditional operator
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D117667
2022-01-19 16:25:05 +00:00
Balazs Benics 881b6a009f [analyzer][NFC] Re-enable skipped SValTests by relaxing expectations
Some tests were skipped in D114454 to resolve test failures on some
platforms, where the pointers have different bitwidth than expected.
This patch re-enables these tests, by relaxing the requirements on the
types of the SVal.

The issue:
There is no way to reconstruct the type of the `SVal` perfectly
accurately, since there could be multiple types having the required
bitwidth and signedness.
Consider platforms where `int` and `long` have the same bitwidth.
Additionally, we need to be careful about casting a pointer to an
integral representation, because we don't know what smallest integral
type can represent that.

To workaround these issues, I propose enforcing a type that has the
same signedness and bitwidth as the expected type, instead of perfect
equality.

In the `GetLocAsIntType` test, in case of pointer-to-integral casts
I'm using the widest standard integral type (long long) to make sure
that the pointer can be represented by the type without losing
precision. This won't affect the test in any meaningful way, since the
type of the `lvalue` remained the same.

In one case, I had to replace `getUIntPtrType()` with `UnsignedLongTy`
because on some platforms `getUIntPtrType()` is different then `long
int`.

In this patch, I also enforce that the tests must compile without
errors, to prevent narrowing conversions in the future.

Reviewed By: stevewan

Differential Revision: https://reviews.llvm.org/D115349
2022-01-19 15:16:18 +01:00
Haojian Wu 6c78703e3a [AST] Fix the incorrect auto-keyword loc for constrained auto type loc.
E.g.  `Concept auto Func();`

The nameLoc for the constained auto type loc pointed to the concept name
loc, it should be the auto token loc. This patch fixes it, and remove
a relevant hack in clang-tidy check.

Reviewed By: sammccall

Differential Revision: https://reviews.llvm.org/D117009
2022-01-19 14:18:38 +01:00
Stanislav Gatev acd4b03590 Revert "Revert "[clang][dataflow] Add a test to justify skipping past references in UO_Deref""
This reverts commit a0262043bb.

Add the -fno-delayed-template-parsing arg to fix the failing test on Windows.
2022-01-19 10:00:01 +00:00
Elliott Maguire 480a1fab72 [clang-format] Fix incorrect alignment of operator= overloads.
Fixes https://github.com/llvm/llvm-project/issues/31568.

Added a check for operator keyword tokens.

Reviewed By: MyDeveloperDay, curdeius, owenpan, HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D117421
2022-01-19 10:18:47 +01:00
Stanislav Gatev a0262043bb Revert "[clang][dataflow] Add a test to justify skipping past references in UO_Deref"
This reverts commit 68226e572f.
2022-01-19 06:46:37 +00:00
Stanislav Gatev 68226e572f [clang][dataflow] Add a test to justify skipping past references in UO_Deref
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D117567
2022-01-18 21:27:43 +00:00
Jan Svoboda 105c913156 [clang][lex] NFC: Simplify calls to `LookupFile`
The `{HeaderSearch,Preprocessor}::LookupFile()` functions take an out-parameter `const DirectoryLookup *&`. Most callers end up creating a `const DirectoryLookup *` variable that's otherwise unused.

This patch changes the out-parameter from reference to a pointer, making it possible to simply pass `nullptr` to the function without the ceremony.

Reviewed By: ahoppen

Differential Revision: https://reviews.llvm.org/D117312
2022-01-18 16:02:18 +01:00
Stanislav Gatev 59e031ff90 [clang][dataflow] Add transfer function for addrof
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: xazax.hun

Differential Revision: https://reviews.llvm.org/D117496
2022-01-18 11:23:08 +00:00
Marek Kurdej 1e512f022a [clang-format] Treat ForEachMacros as loops
TT_ForEachMacro should be considered in rules AllowShortBlocksOnASingleLine
and AllowShortLoopsOnASingleLine.
Fixes https://github.com/llvm/llvm-project/issues/45432.

Reviewed By: MyDeveloperDay

Differential Revision: https://reviews.llvm.org/D94955
2022-01-17 17:11:06 +01:00
Yitzhak Mandelbaum 875117ae7a [clang][dataflow] Remove TestingSupport's dependency on gtest
Users outside of the clang repo may use different googletest versions. So, it's
better not to depend on llvm's googletest. This patch removes the dependency by
having `checkDataflow` return an `llvm::Error` instead of calling googletest's
`FAIL` or `ASSERT...` macros.

Differential Revision: https://reviews.llvm.org/D117304
2022-01-17 15:33:23 +00:00
Haojian Wu ab3f100bec Reland (2) "[AST] Add RParen loc for decltype AutoTypeloc.""
The patch was reverted because it caused a crash during PCH build -- we
missed to update the RParenLoc in TreeTransform<Derived>::TransformAutoType.

This relands 55d96ac and 37ec65e with a test and fix.
2022-01-17 11:33:11 +01:00
Cameron Mulhern 966f24e5a6 [clang-format] Add a BlockIndent option to AlignAfterOpenBracket
This style is similar to AlwaysBreak, but places closing brackets on new lines.

For example, if you have a multiline parameter list, clang-format currently only supports breaking per-parameter, but places the closing bracket on the line of the last parameter.

Function(
    param1,
    param2,
    param3);

A style supported by other code styling tools (e.g. rustfmt) is to allow the closing brackets to be placed on their own line, aiding the user in being able to quickly infer the bounds of the block of code.

Function(
    param1,
    param2,
    param3
);

For prior work on a similar feature, see: https://reviews.llvm.org/D33029.

Note: This currently only supports block indentation for closing parentheses.

Differential Revision: https://reviews.llvm.org/D109557
2022-01-17 09:03:23 +01:00
Stanislav Gatev 37e6496c80 [clang][dataflow] Add transfer functions for bind temporary and static cast
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Differential Revision: https://reviews.llvm.org/D117339
2022-01-16 17:41:02 +00:00
Owen Pan 533fbae8d8 [clang-format] Add experimental option to remove LLVM braces
See the style examples at:
https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

Differential Revision: https://reviews.llvm.org/D116316
2022-01-14 15:10:17 -08:00
Marek Kurdej 7af11989be [clang-format] Fix short functions being considered as inline inside an indented namespace.
Fixes https://github.com/llvm/llvm-project/issues/24784.

With config:
```
AllowShortFunctionsOnASingleLine: Inline
NamespaceIndentation: All
```

The code:
```
namespace Test
{
    void f()
    {
        return;
    }
}
```
was incorrectly formatted to:
```
namespace Test
{
    void f() { return; }
}
```

since the function `f` was considered being inside a class/struct/record.
That's because the check was simplistic and only checked for a non-zero indentation level of the line starting `f`.

Reviewed By: MyDeveloperDay, HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D117142
2022-01-14 21:57:02 +01:00
Marek Kurdej 6ea3d9efc5 [clang-format] Fix CompactNamespaces corner case when AllowShortLambdasOnASingleLine/BraceWrapping.BeforeLambdaBody are set
In clang-format 12, `CompactNamespaces` misformatted the code when `AllowShortLambdasOnASingleLine` is set to false and `BraceWrapping.BeforeLambdaBody` is true.

Input:
```
namespace out {
namespace in {
}
} // namespace out::in
```

Expected output:
```
namespace out { namespace in {
}} // namespace out::in
```

Output from v12:
```
namespace out {
namespace in {
}
} // namespace out::in
```

Config triggering the issue:
```
---
AllowShortLambdasOnASingleLine: None
BraceWrapping:
  BeforeLambdaBody :    true
BreakBeforeBraces: Custom
CompactNamespaces: true
...
```

Seems there's a corner case when `AllowShortLambdasOnASingleLine` is false, and `BraceWrapping.BeforeLambdaBody` is true, that causes CompactNamespaces to stop working.
The cause was a misannotation of `{` opening brace after `namespace` as a lambda opening brace.
The regression was probably introduced with [this commit](fa0118e6e5).

Originally contributed by Ahmed Mahdy (@aybassiouny). Thank you!

Reviewed By: Wawha, HazardyKnusperkeks

Differential Revision: https://reviews.llvm.org/D99031
2022-01-14 21:47:16 +01:00
Stanislav Gatev 7d941d6d21 [clang][dataflow] Add transfer functions for constructors
This is part of the implementation of the dataflow analysis framework.
See "[RFC] A dataflow analysis framework for Clang AST" on cfe-dev.

Reviewed-by: ymandel, xazax.hun

Differential Revision: https://reviews.llvm.org/D117218
2022-01-14 14:58:01 +00:00
Marek Kurdej 6a4957cbdf [clang-format] Add missing test for loops formatting. NFC.
The case with an inner while loop wasn't tested before. Same for outer loop with a ForeachMacro.
2022-01-14 14:43:15 +01:00