[clang] Emit SARIF Diagnostics: Create clang::SarifDocumentWriter interface
Create an interface for writing SARIF documents from within clang:
The primary intent of this change is to introduce the interface
clang::SarifDocumentWriter, which allows incrementally adding
diagnostic data to a JSON backed document. The proposed interface is
not yet connected to the compiler internals, which will be covered in
future work. As such this change will not change the input/output
interface of clang.
This change also introduces the clang::FullSourceRange type that is
modeled after clang::SourceRange + clang::FullSourceLoc, this is useful
for packaging a pair of clang::SourceLocation objects with their
corresponding SourceManagers.
Previous discussions:
RFC for this change: https://lists.llvm.org/pipermail/cfe-dev/2021-March/067907.htmlhttps://lists.llvm.org/pipermail/cfe-dev/2021-July/068480.html
SARIF Standard (2.1.0):
https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/sarif-v2.1.0-os.html
Differential Revision: https://reviews.llvm.org/D109701
Some code [0] consider that trailing arrays are flexible, whatever their size.
Support for these legacy code has been introduced in
f8f6324983 but it prevents evaluation of
__builtin_object_size and __builtin_dynamic_object_size in some legit cases.
Introduce -fstrict-flex-arrays=<n> to have stricter conformance when it is
desirable.
n = 0: current behavior, any trailing array member is a flexible array. The default.
n = 1: any trailing array member of undefined, 0 or 1 size is a flexible array member
n = 2: any trailing array member of undefined or 0 size is a flexible array member
This takes into account two specificities of clang: array bounds as macro id
disqualify FAM, as well as non standard layout.
Similar patch for gcc discuss here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101836
[0] https://docs.freebsd.org/en/books/developers-handbook/sockets/#sockets-essential-functions
When an issue exists in the main file (caller) instead of an included file
(callee), using a `src` pattern applying to the included file may be
inappropriate if it's the caller's responsibility. Add `mainfile` prefix to check
the main filename.
For the example below, the issue may reside in a.c (foo should not be called
with a misaligned pointer or foo should switch to an unaligned load), but with
`src` we can only apply to the innocent callee a.h. With this patch we can use
the more appropriate `mainfile:a.c`.
```
//--- a.h
// internal linkage
static inline int load(int *x) { return *x; }
//--- a.c, -fsanitize=alignment
#include "a.h"
int foo(void *x) { return load(x); }
```
See the updated clang/docs/SanitizerSpecialCaseList.rst for a caveat due
to C++ vague linkage functions.
Reviewed By: #sanitizers, kstoimenov, vitalybuka
Differential Revision: https://reviews.llvm.org/D129832
Summary: Introduce a new function 'clang_analyzer_value'. It emits a report that in turn prints a RangeSet or APSInt associated with SVal. If there is no associated value, prints "n/a".
The time profiler traces the stages during the clang compile
process. Each compiling stage of a single source file
corresponds to a separately .json file which holds its
time tracing data. However, the .json files are stored in the
same path/directory as its corresponding stage's '-o' option.
For example, if we compile the "demo.cc" to "demo.o" with option
"-o /tmp/demo.o", the time trace data file path is "/tmp/demo.json".
A typical c++ project can contain multiple source files in different
path, but all the json files' paths can be a mess.
The option "-ftime-trace=<value>" allows you to specify where the json
files should be stored. This allows the users to place the time trace
data files of interest in the desired location for further data analysis.
Usage:
- clang/clang++ -ftime-trace ...
- clang/clang++ -ftime-trace=the-directory-you-want ...
- clang/clang++ -ftime-trace=the-directory-you-want/ ...
- clang/clang++ -ftime-trace=the-full-file-path-you-want ...
Differential Revision: https://reviews.llvm.org/D128048
TokenManager defines Token interfaces for the clang syntax-tree. This is the level
of abstraction that the syntax-tree should use to operate on Tokens.
It decouples the syntax-tree from a particular token implementation (TokenBuffer
previously). This enables us to use a different underlying token implementation
for the syntax Leaf node -- in clang pseudoparser, we want to produce a
syntax-tree with its own pseudo::Token rather than syntax::Token.
Differential Revision: https://reviews.llvm.org/D128411
This reverts commit 7c51f02eff because it
stills breaks the LLDB tests. This was re-landed without addressing the
issue or even agreement on how to address the issue. More details and
discussion in https://reviews.llvm.org/D112374.
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
---
Troubleshooting list to deal with any breakage seen with this patch:
1) The most likely effect one would see by this patch is a change in how
a type is printed. The type printer will, by design and default,
print types as written. There are customization options there, but
not that many, and they mainly apply to how to print a type that we
somehow failed to track how it was written. This patch fixes a
problem where we failed to distinguish between a type
that was written without any elaborated-type qualifiers,
such as a 'struct'/'class' tags and name spacifiers such as 'std::',
and one that has been stripped of any 'metadata' that identifies such,
the so called canonical types.
Example:
```
namespace foo {
struct A {};
A a;
};
```
If one were to print the type of `foo::a`, prior to this patch, this
would result in `foo::A`. This is how the type printer would have,
by default, printed the canonical type of A as well.
As soon as you add any name qualifiers to A, the type printer would
suddenly start accurately printing the type as written. This patch
will make it print it accurately even when written without
qualifiers, so we will just print `A` for the initial example, as
the user did not really write that `foo::` namespace qualifier.
2) This patch could expose a bug in some AST matcher. Matching types
is harder to get right when there is sugar involved. For example,
if you want to match a type against being a pointer to some type A,
then you have to account for getting a type that is sugar for a
pointer to A, or being a pointer to sugar to A, or both! Usually
you would get the second part wrong, and this would work for a
very simple test where you don't use any name qualifiers, but
you would discover is broken when you do. The usual fix is to
either use the matcher which strips sugar, which is annoying
to use as for example if you match an N level pointer, you have
to put N+1 such matchers in there, beginning to end and between
all those levels. But in a lot of cases, if the property you want
to match is present in the canonical type, it's easier and faster
to just match on that... This goes with what is said in 1), if
you want to match against the name of a type, and you want
the name string to be something stable, perhaps matching on
the name of the canonical type is the better choice.
3) This patch could exposed a bug in how you get the source range of some
TypeLoc. For some reason, a lot of code is using getLocalSourceRange(),
which only looks at the given TypeLoc node. This patch introduces a new,
and more common TypeLoc node which contains no source locations on itself.
This is not an inovation here, and some other, more rare TypeLoc nodes could
also have this property, but if you use getLocalSourceRange on them, it's not
going to return any valid locations, because it doesn't have any. The right fix
here is to always use getSourceRange() or getBeginLoc/getEndLoc which will dive
into the inner TypeLoc to get the source range if it doesn't find it on the
top level one. You can use getLocalSourceRange if you are really into
micro-optimizations and you have some outside knowledge that the TypeLocs you are
dealing with will always include some source location.
4) Exposed a bug somewhere in the use of the normal clang type class API, where you
have some type, you want to see if that type is some particular kind, you try a
`dyn_cast` such as `dyn_cast<TypedefType>` and that fails because now you have an
ElaboratedType which has a TypeDefType inside of it, which is what you wanted to match.
Again, like 2), this would usually have been tested poorly with some simple tests with
no qualifications, and would have been broken had there been any other kind of type sugar,
be it an ElaboratedType or a TemplateSpecializationType or a SubstTemplateParmType.
The usual fix here is to use `getAs` instead of `dyn_cast`, which will look deeper
into the type. Or use `getAsAdjusted` when dealing with TypeLocs.
For some reason the API is inconsistent there and on TypeLocs getAs behaves like a dyn_cast.
5) It could be a bug in this patch perhaps.
Let me know if you need any help!
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
Introducing the support for evaluating the constructor
of every element in an array. The idea is to record the
index of the current array member being constructed and
create a loop during the analysis. We looping over the
same CXXConstructExpr as many times as many elements
the array has.
Differential Revision: https://reviews.llvm.org/D127973
Add two options, `-fprofile-function-groups=N` and `-fprofile-selected-function-group=i` used to partition functions into `N` groups and only instrument the functions in group `i`. Similar options were added to xray in https://reviews.llvm.org/D87953 and the goal is the same; to reduce instrumented size overhead by spreading the overhead across multiple builds. Raw profiles from different groups can be added like normal using the `llvm-profdata merge` command.
Reviewed By: ianlevesque
Differential Revision: https://reviews.llvm.org/D129594
This reverts commit b7e77ff25f.
Reason: Broke sanitizer builds bots + libcxx. 'static assertion
expression is not an integral constant expression'. More details
available in the Phabricator review: https://reviews.llvm.org/D129048
This patch rewords the static assert diagnostic output. Failing a
_Static_assert in C should not report that static_assert failed. This
changes the wording to be more like GCC and uses "static assertion"
when possible instead of hard coding the name. This also changes some
instances of 'static_assert' to instead be based on the token in the
source code.
Differential Revision: https://reviews.llvm.org/D129048
WG21 approved delimited escape sequences and named escape
sequences.
Adjust the extension warnings accordingly, and update
the release notes.
Reviewed By: aaron.ballman
Differential Revision: https://reviews.llvm.org/D129664
The new driver primarily allows us to support RDC-mode compilations with
proper linking. This is not needed for non-RDC mode compilation, but we
still would like the new driver to be able to handle this mode so we can
transition away from the old driver in the future. This patch adds the
necessary code to support creating a fatbinary for CUDA code generation
as well as removing old assumptions and errors about RDC-mode with the
new driver.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D129655
Add `pcm-info` to the `target module dump` subcommands.
This dump command shows information about clang .pcm files. This command
effectively runs `clang -module-file-info` and produces identical output.
The .pcm file format is tightly coupled to the clang version. The clang
embedded in lldb is not guaranteed to match the version of the clang executable
available on the local system.
There have been times when I've needed to view the details about a .pcm file
produced by lldb's embedded clang, but because the clang executable was a
slightly different version, the `-module-file-info` invocation failed. With
this command, users can inspect .pcm files generated by lldb too.
Differential Revision: https://reviews.llvm.org/D129456
Follow-up to 6626f6fec3, this fixes the handling of -MT
* If no targets are provided, we need to invent one since cc1 expects
the driver to have handled it. The default is to use -o, quoting as
necessary for a make target.
* Fix the splitting for empty string, which was incorrectly treated as
{""} instead of {}.
* Add a way to test this behaviour in clang-scan-deps.
Differential Revision: https://reviews.llvm.org/D129607
Previous warning went on whenever a struct with a struct member with alignment => 16
was declared. This led to too many false positives and led to diagnostic lit failures
due to it being emitted too frequently. Only emit the warning when such a struct and
that struct contains a member that has an alignment of 16 bytes is passed to a caller
function since this is where the potential binary compatibility issue with XL 16.1.0
and older exists.
Reviewed By: sfertile, aaron.ballman
Differential Revision: https://reviews.llvm.org/D118350
This reverts commit bdc6974f92 because it
breaks all the LLDB tests that import the std module.
import-std-module/array.TestArrayFromStdModule.py
import-std-module/deque-basic.TestDequeFromStdModule.py
import-std-module/deque-dbg-info-content.TestDbgInfoContentDequeFromStdModule.py
import-std-module/forward_list.TestForwardListFromStdModule.py
import-std-module/forward_list-dbg-info-content.TestDbgInfoContentForwardListFromStdModule.py
import-std-module/list.TestListFromStdModule.py
import-std-module/list-dbg-info-content.TestDbgInfoContentListFromStdModule.py
import-std-module/queue.TestQueueFromStdModule.py
import-std-module/stack.TestStackFromStdModule.py
import-std-module/vector.TestVectorFromStdModule.py
import-std-module/vector-bool.TestVectorBoolFromStdModule.py
import-std-module/vector-dbg-info-content.TestDbgInfoContentVectorFromStdModule.py
import-std-module/vector-of-vectors.TestVectorOfVectorsFromStdModule.py
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/45301/
Previously, `Status` was named after the enum type `Status` which caused the enum to be hidden by the non-type declaration of the `Status` field. This patch fixes this issue by using different names for the field and type.
Differential Revision: https://reviews.llvm.org/D129568
Introduce an off-by default `-Winvalid-utf8` warning
that detects invalid UTF-8 code units sequences in comments.
Invalid UTF-8 in other places is already diagnosed,
as that cannot appear in identifiers and other grammar constructs.
The warning is off by default as its likely to be somewhat disruptive
otherwise.
This warning allows clang to conform to the yet-to be approved WG21
"P2295R5 Support for UTF-8 as a portable source file encoding"
paper.
Reviewed By: aaron.ballman, #clang-language-wg
Differential Revision: https://reviews.llvm.org/D128059
Without this patch, clang will not wrap in an ElaboratedType node types written
without a keyword and nested name qualifier, which goes against the intent that
we should produce an AST which retains enough details to recover how things are
written.
The lack of this sugar is incompatible with the intent of the type printer
default policy, which is to print types as written, but to fall back and print
them fully qualified when they are desugared.
An ElaboratedTypeLoc without keyword / NNS uses no storage by itself, but still
requires pointer alignment due to pre-existing bug in the TypeLoc buffer
handling.
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Differential Revision: https://reviews.llvm.org/D112374
This reverts commit cc309721d2 because it
breaks the following tests on GreenDragon:
TestDataFormatterObjCCF.py
TestDataFormatterObjCExpr.py
TestDataFormatterObjCKVO.py
TestDataFormatterObjCNSBundle.py
TestDataFormatterObjCNSData.py
TestDataFormatterObjCNSError.py
TestDataFormatterObjCNSNumber.py
TestDataFormatterObjCNSURL.py
TestDataFormatterObjCPlain.py
TestDataFormatterObjNSException.py
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/45288/
There were two assertions in DefaultArgStorage::setInherited previously.
It requires the DefaultArgument is either empty or an argument value. It
would crash if it has a pointer refers to the previous declaration or
contains a chain to the previous declaration.
But there are edge cases could hit them actually. One is
InheritDefaultArguments.cppm that I found recently. Another one is pr31469.cpp,
which was created fives years ago.
This patch tries to fix the two failures by handling full cases in
DefaultArgStorage::setInherited.
This is guaranteed to not introduce any breaking change since it lives
in the path we wouldn't touch before. And the added assertions for
sameness should keep the correctness.
Reviewed By: v.g.vassilev
Differential Revision: https://reviews.llvm.org/D128974
parameters.
The current implementation to judge the similarity of TypeConstraint in
ASTContext::isSameTemplateParameter is problematic, it couldn't handle
the following case:
```C++
template <__integer_like _Tp, C<_Tp> Sentinel>
constexpr _Tp operator()(_Tp &&__t, Sentinel &&last) const {
return __t;
}
```
When we see 2 such declarations from different modules, we would judge
their similarity by `ASTContext::isSame*` methods. But problems come for
the TypeConstraint. Originally, we would profile each argument one by
one. But it is not right. Since the profiling result of `_Tp` would
refer to two different template type declarations. So it would get
different results. It is right since the `_Tp` in different modules
refers to different declarations indeed. So the same declaration in
different modules would meet incorrect our-checking results.
It is not the thing we want. We want to know if the TypeConstraint have
the same expression.
Reviewer: vsapsai, ilya-biryukov
Differential Revision: https://reviews.llvm.org/D129068
When building modules, override secondary outputs (dependency file,
dependency targets, serialized diagnostic file) in addition to the pcm
file path. This avoids inheriting per-TU command-line options that
cause non-determinism in the results (non-deterministic command-line for
the module build, non-determinism in which TU's .diag and .d files will
contain the module outputs). In clang-scan-deps we infer whether to
generate dependency or serialized diagnostic files based on an original
command-line. In a real build system this should be modeled explicitly.
Differential Revision: https://reviews.llvm.org/D129389
This was promised 5 years ago in https://reviews.llvm.org/D32796,
let's do it.
Both flags are still accepted. No behavior change except for which
form shows up in --help output and in dumps of internal state
(such as with RC_DEBUG_OPTIONS).
Differential Revision: https://reviews.llvm.org/D129226
Introduce an off-by default `-Winvalid-utf8` warning
that detects invalid UTF-8 code units sequences in comments.
Invalid UTF-8 in other places is already diagnosed,
as that cannot appear in identifiers and other grammar constructs.
The warning is off by default as its likely to be somewhat disruptive
otherwise.
This warning allows clang to conform to the yet-to be approved WG21
"P2295R5 Support for UTF-8 as a portable source file encoding"
paper.
Reviewed By: aaron.ballman, #clang-language-wg
Differential Revision: https://reviews.llvm.org/D128059
Stanislav Mekhanoshin