BindingDecl was added recently but the related DecompositionDecl is needed
to make C++17 structured bindings importable.
Import of BindingDecl was changed to avoid infinite import loop.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D105354
This reverts commit 20176bc7dd as some
versions of GCC do not seem to handle the new code very well. They
complain about:
/tmp/ccqUQZyw.s: Assembler messages:
/tmp/ccqUQZyw.s:1151: Error: symbol `_ZNSt14_Function_base13_Base_managerIN5clangUlPKNS1_4StmtEE2_EE10_M_managerERSt9_Any_dataRKS7_St18_Manager_operation' is already defined
/tmp/ccqUQZyw.s:11963: Error: symbol `_ZNSt17_Function_handlerIFbPKN5clang4StmtEENS0_UlS3_E2_EE9_M_invokeERKSt9_Any_dataOS3_' is already defined
This seems like it is some GCC issue, but multiple buildbots (and my
local machine) are all failing because of it.
Original commit message:
[clang-repl] Implement partial translation units and error recovery.
https://reviews.llvm.org/D96033 contained a discussion regarding efficient
modeling of error recovery. @rjmccall has outlined the key ideas:
Conceptually, we can split the translation unit into a sequence of partial
translation units (PTUs). Every declaration will be associated with a unique PTU
that owns it.
The first key insight here is that the owning PTU isn't always the "active"
(most recent) PTU, and it isn't always the PTU that the declaration
"comes from". A new declaration (that isn't a redeclaration or specialization of
anything) does belong to the active PTU. A template specialization, however,
belongs to the most recent PTU of all the declarations in its signature - mostly
that means that it can be pulled into a more recent PTU by its template
arguments.
The second key insight is that processing a PTU might extend an earlier PTU.
Rolling back the later PTU shouldn't throw that extension away. For example, if
the second PTU defines a template, and the third PTU requires that template to
be instantiated at float, that template specialization is still part of the
second PTU. Similarly, if the fifth PTU uses an inline function belonging to the
fourth, that definition still belongs to the fourth. When we go to emit code in
a new PTU, we map each declaration we have to emit back to its owning PTU and
emit it in a new module for just the extensions to that PTU. We keep track of
all the modules we've emitted for a PTU so that we can unload them all if we
decide to roll it back.
Most declarations/definitions will only refer to entities from the same or
earlier PTUs. However, it is possible (primarily by defining a
previously-declared entity, but also through templates or ADL) for an entity
that belongs to one PTU to refer to something from a later PTU. We will have to
keep track of this and prevent unwinding to later PTU when we recognize it.
Fortunately, this should be very rare; and crucially, we don't have to do the
bookkeeping for this if we've only got one PTU, e.g. in normal compilation.
Otherwise, PTUs after the first just need to record enough metadata to be able
to revert any changes they've made to declarations belonging to earlier PTUs,
e.g. to redeclaration chains or template specialization lists.
It should even eventually be possible for PTUs to provide their own slab
allocators which can be thrown away as part of rolling back the PTU. We can
maintain a notion of the active allocator and allocate things like Stmt/Expr
nodes in it, temporarily changing it to the appropriate PTU whenever we go to do
something like instantiate a function template. More care will be required when
allocating declarations and types, though.
We would want the PTU to be efficiently recoverable from a Decl; I'm not sure
how best to do that. An easy option that would cover most declarations would be
to make multiple TranslationUnitDecls and parent the declarations appropriately,
but I don't think that's good enough for things like member function templates,
since an instantiation of that would still be parented by its original class.
Maybe we can work this into the DC chain somehow, like how lexical DCs are.
We add a different kind of translation unit `TU_Incremental` which is a
complete translation unit that we might nonetheless incrementally extend later.
Because it is complete (and we might want to generate code for it), we do
perform template instantiation, but because it might be extended later, we don't
warn if it declares or uses undefined internal-linkage symbols.
This patch teaches clang-repl how to recover from errors by disconnecting the
most recent PTU and update the primary PTU lookup tables. For instance:
```./clang-repl
clang-repl> int i = 12; error;
In file included from <<< inputs >>>:1:
input_line_0:1:13: error: C++ requires a type specifier for all declarations
int i = 12; error;
^
error: Parsing failed.
clang-repl> int i = 13; extern "C" int printf(const char*,...);
clang-repl> auto r1 = printf("i=%d\n", i);
i=13
clang-repl> quit
```
Differential revision: https://reviews.llvm.org/D104918
This reverts commit 6775fc6ffa.
It also reverts "[lldb] Fix compilation by adjusting to the new ASTContext signature."
This reverts commit 03a3f86071.
We see some failures on the lldb infrastructure, these changes might play a role
in it. Let's revert it now and see if the bots will become green.
Ref: https://reviews.llvm.org/D104918
https://reviews.llvm.org/D96033 contained a discussion regarding efficient
modeling of error recovery. @rjmccall has outlined the key ideas:
Conceptually, we can split the translation unit into a sequence of partial
translation units (PTUs). Every declaration will be associated with a unique PTU
that owns it.
The first key insight here is that the owning PTU isn't always the "active"
(most recent) PTU, and it isn't always the PTU that the declaration
"comes from". A new declaration (that isn't a redeclaration or specialization of
anything) does belong to the active PTU. A template specialization, however,
belongs to the most recent PTU of all the declarations in its signature - mostly
that means that it can be pulled into a more recent PTU by its template
arguments.
The second key insight is that processing a PTU might extend an earlier PTU.
Rolling back the later PTU shouldn't throw that extension away. For example, if
the second PTU defines a template, and the third PTU requires that template to
be instantiated at float, that template specialization is still part of the
second PTU. Similarly, if the fifth PTU uses an inline function belonging to the
fourth, that definition still belongs to the fourth. When we go to emit code in
a new PTU, we map each declaration we have to emit back to its owning PTU and
emit it in a new module for just the extensions to that PTU. We keep track of
all the modules we've emitted for a PTU so that we can unload them all if we
decide to roll it back.
Most declarations/definitions will only refer to entities from the same or
earlier PTUs. However, it is possible (primarily by defining a
previously-declared entity, but also through templates or ADL) for an entity
that belongs to one PTU to refer to something from a later PTU. We will have to
keep track of this and prevent unwinding to later PTU when we recognize it.
Fortunately, this should be very rare; and crucially, we don't have to do the
bookkeeping for this if we've only got one PTU, e.g. in normal compilation.
Otherwise, PTUs after the first just need to record enough metadata to be able
to revert any changes they've made to declarations belonging to earlier PTUs,
e.g. to redeclaration chains or template specialization lists.
It should even eventually be possible for PTUs to provide their own slab
allocators which can be thrown away as part of rolling back the PTU. We can
maintain a notion of the active allocator and allocate things like Stmt/Expr
nodes in it, temporarily changing it to the appropriate PTU whenever we go to do
something like instantiate a function template. More care will be required when
allocating declarations and types, though.
We would want the PTU to be efficiently recoverable from a Decl; I'm not sure
how best to do that. An easy option that would cover most declarations would be
to make multiple TranslationUnitDecls and parent the declarations appropriately,
but I don't think that's good enough for things like member function templates,
since an instantiation of that would still be parented by its original class.
Maybe we can work this into the DC chain somehow, like how lexical DCs are.
We add a different kind of translation unit `TU_Incremental` which is a
complete translation unit that we might nonetheless incrementally extend later.
Because it is complete (and we might want to generate code for it), we do
perform template instantiation, but because it might be extended later, we don't
warn if it declares or uses undefined internal-linkage symbols.
This patch teaches clang-repl how to recover from errors by disconnecting the
most recent PTU and update the primary PTU lookup tables. For instance:
```./clang-repl
clang-repl> int i = 12; error;
In file included from <<< inputs >>>:1:
input_line_0:1:13: error: C++ requires a type specifier for all declarations
int i = 12; error;
^
error: Parsing failed.
clang-repl> int i = 13; extern "C" int printf(const char*,...);
clang-repl> auto r1 = printf("i=%d\n", i);
i=13
clang-repl> quit
```
Differential revision: https://reviews.llvm.org/D104918
Template parameters are created in ASTImporter with the translation unit as DeclContext.
The DeclContext is later updated (by the create function of template classes).
ASTImporterLookupTable was not updated after these changes of the DC. The patch
adds update of the DeclContext in ASTImporterLookupTable.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D103792
Given `int foo, bar;`, TraverseAST reveals this tree:
TranslationUnitDecl
- foo
- bar
Before this patch, with the TraversalScope set to {foo}, TraverseAST yields:
foo
After this patch it yields:
TranslationUnitDecl
- foo
Also, TraverseDecl(TranslationUnitDecl) now respects the traversal scope.
---
The main effect of this today is that clang-tidy checks that match the
translationUnitDecl(), either in order to traverse it or check
parentage, should work.
Differential Revision: https://reviews.llvm.org/D104071
This implements the 'using enum maybe-qualified-enum-tag ;' part of
1099. It introduces a new 'UsingEnumDecl', subclassed from
'BaseUsingDecl'. Much of the diff is the boilerplate needed to get the
new class set up.
There is one case where we accept ill-formed, but I believe this is
merely an extended case of an existing bug, so consider it
orthogonal. AFAICT in class-scope the c++20 rule is that no 2 using
decls can bring in the same target decl ([namespace.udecl]/8). But we
already accept:
struct A { enum { a }; };
struct B : A { using A::a; };
struct C : B { using A::a;
using B::a; }; // same enumerator
this patch permits mixtures of 'using enum Bob;' and 'using Bob::member;' in the same way.
Differential Revision: https://reviews.llvm.org/D102241
This diff adds testcase for the issue fixed in https://reviews.llvm.org/D77468
but regression test was not added in the diff. On Clang 9 it caused
crash in cland during code completion.
Test Plan: check-clang-unit
Differential Revision: https://reviews.llvm.org/D103722
ParmVarDecl is created with translation unit as the parent DeclContext
and later moved to the correct DeclContext. ASTImporterLookupTable
should be updated at this move.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D103231
In the case of TypedefDecls we set the DeclContext after we imported it.
It turns out, it could lead to null pointer dereferences during the
cleanup part of a failed import.
This patch demonstrates this issue and fixes it by checking if the
DeclContext is available or not.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D102640
Reverts parts of https://reviews.llvm.org/D17183, but keeps the
resetDataLayout() API and adds an assert that checks that datalayout string and
user label prefix are in sync.
Approach 1 in https://reviews.llvm.org/D17183#2653279
Reduces number of TUs build for 'clang-format' from 689 to 575.
I also implemented approach 2 in D100764. If someone feels motivated
to make us use DataLayout more, it's easy to revert this change here
and go with D100764 instead. I don't plan on doing more work in this
area though, so I prefer going with the smaller, more self-consistent change.
Differential Revision: https://reviews.llvm.org/D100776
(PR49478)
As ArrayType::ArrayType mentioned in clang/lib/AST/Type.cpp, a
DependentSizedArrayType might not have size expression because it it
used as the type of a dependent array of unknown bound with a dependent
braced initializer.
Thus, I add a check when mangling array of that type.
This should fix https://bugs.llvm.org/show_bug.cgi?id=49478
Reviewed By: Richard Smith - zygoloid
Differential Revision: https://reviews.llvm.org/D99407
All three cases were imported correctly.
For BlockDecls, correctly means that we don't support importing them, thus an
error is the expected behaviour.
- BlockDecls were not yet covered. I know that they are not imported but the
test at least documents it.
- Default values for ParmVarDecls were also uncovered.
- Importing bitfield FieldDecls were imported correctly.
Reviewed By: martong, shafik
Differential Revision: https://reviews.llvm.org/D99576
We do the import of the member enum specialization similarly to as we do
with member CXXRecordDecl specialization.
Differential Revision: https://reviews.llvm.org/D99421
This moves the two tests we have for importing Objective-C nodes to their own
file. The motivation is that this means I can add more Objective-C tests without
making the compilation time of ASTImporterTest even longer. Also it seems nice
to separate the Apple-specific stuff from the ASTImporter test.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D99162
Update ASTImporter to import value of FieldDecl::getCapturedVLAType.
Reviewed By: shafik, martong
Differential Revision: https://reviews.llvm.org/D99062
Objective-C apparently allows name conflicts between instance and class
properties, so this is valid code:
```
@protocol DupProp
@property (class, readonly) int prop;
@property (readonly) int prop;
@end
```
The ASTImporter however isn't aware of this and will consider the two properties
as if they are the same property because it just compares their name and types.
This causes that when importing both properties we only end up with one property
(whatever is imported first from what I can see).
Beside generating a different AST this also leads to a bunch of asserts and
crashes as we still correctly import the two different getters for both
properties (the import code for methods does the correct check where it
differentiated between instance and class methods). As one of the setters will
not have its associated ObjCPropertyDecl imported, any call to
`ObjCMethodDecl::findPropertyDecl` will just lead to an assert or crash.
Fixes rdar://74322659
Reviewed By: shafik, kastiglione
Differential Revision: https://reviews.llvm.org/D99077
It is possible that imported `SourceLocExpr` can cause not expected behavior (if `__builtin_LINE()` is used together with `__LINE__` for example) but still it may be worth to import these because some projects use it.
Reviewed By: teemperor
Differential Revision: https://reviews.llvm.org/D98876
After the import, we did not copy the `TSCSpec`.
This commit resolves that.
Reviewed By: balazske
Differential Revision: https://reviews.llvm.org/D98707
The idiom:
```
DeclContext::lookup_result R = DeclContext::lookup(Name);
for (auto *D : R) {...}
```
is not safe when in the loop body we trigger deserialization from an AST file.
The deserialization can insert new declarations in the StoredDeclsList whose
underlying type is a vector. When the vector decides to reallocate its storage
the pointer we hold becomes invalid.
This patch replaces a SmallVector with an singly-linked list. The current
approach stores a SmallVector<NamedDecl*, 4> which is around 8 pointers.
The linked list is 3, 5, or 7. We do better in terms of memory usage for small
cases (and worse in terms of locality -- the linked list entries won't be near
each other, but will be near their corresponding declarations, and we were going
to fetch those memory pages anyway). For larger cases: the vector uses a
doubling strategy for reallocation, so will generally be between half-full and
full. Let's say it's 75% full on average, so there's N * 4/3 + 4 pointers' worth
of space allocated currently and will be 2N pointers with the linked list. So we
break even when there are N=6 entries and slightly lose in terms of memory usage
after that. We suspect that's still a win on average.
Thanks to @rsmith!
Differential revision: https://reviews.llvm.org/D91524
ASTContext were only passed to the StmtPrinter in some places, while it
is always available in DeclPrinter. The context is used by StmtPrinter to better
print statements in some cases, like printing constants as written.
Differential Revision: https://reviews.llvm.org/D97043
This change enables the builtin function declarations
in clang driver by default using the Tablegen solution
along with the implicit include of 'opencl-c-base.h'
header.
A new flag '-cl-no-stdinc' disabling all default
declarations and header includes is added. If any other
mechanisms were used to include the declarations (e.g.
with -Xclang -finclude-default-header) and the new default
approach is not sufficient the, `-cl-no-stdinc` flag has
to be used with clang to activate the old behavior.
Tags: #clang
Differential Revision: https://reviews.llvm.org/D96515
The assertion can happen if ASTImporter imports a CXXRecordDecl in a template
and then imports another redeclaration of this declaration, while the first import is in progress.
The process of first import did not set the "described template" yet
and the second import finds the first declaration at setting the injected types.
Setting the injected type requires in the assertion that the described template is set.
The exact assertion was:
clang/lib/AST/ASTContext.cpp:4411:
clang::QualType clang::ASTContext::getInjectedClassNameType(clang::CXXRecordDecl*, clang::QualType) const:
Assertion `NeedsInjectedClassNameType(Decl)' failed.
Reviewed By: shafik
Differential Revision: https://reviews.llvm.org/D94067
This allows ASTs to be merged when they contain GenericSelectionExpr
nodes (this is _Generic from C11). This is needed, for example, for
CTU analysis of C code that makes use of _Generic, like the Linux
kernel.
The node is already supported in the AST, but it didn't have a matcher
in ASTMatchers. So, this change adds the matcher and adds support to
ASTImporter. Additionally, this change adds support for structural
equivalence of _Generic in the AST.
Reviewed By: martong, aaron.ballman
Differential Revision: https://reviews.llvm.org/D92600
The import of a typedefs with an attribute uses clang::Decl::setAttrs().
But that needs the ASTContext which we can get only from the
TranslationUnitDecl. But we can get the TUDecl only thourgh the
DeclContext, which is not set by the time of the setAttrs call.
Fix: import the attributes only after the DC is surely imported.
Btw, having the attribute import initiated from GetImportedOrCreateDecl was
fundamentally flawed. Now that is implicitly fixed.
Differential Revision: https://reviews.llvm.org/D92962
CXXDeductionGuideDecl with a local typedef has its own copy of the
TypedefDecl with the CXXDeductionGuideDecl as the DeclContext of that
TypedefDecl.
```
template <typename T> struct A {
typedef T U;
A(U, T);
};
A a{(int)0, (int)0};
```
Related discussion on cfe-dev:
http://lists.llvm.org/pipermail/cfe-dev/2020-November/067252.html
Without this fix, when we import the CXXDeductionGuideDecl (via
VisitFunctionDecl) then before creating the Decl we must import the
FunctionType. However, the first parameter's type is the afore mentioned
local typedef. So, we then start importing the TypedefDecl whose
DeclContext is the CXXDeductionGuideDecl itself. The infinite loop is
formed.
```
#0 clang::ASTNodeImporter::VisitCXXDeductionGuideDecl(clang::CXXDeductionGuideDecl*) clang/lib/AST/ASTImporter.cpp:3543:0
#1 clang::declvisitor::Base<std::add_pointer, clang::ASTNodeImporter, llvm::Expected<clang::Decl*> >::Visit(clang::Decl*) /home/egbomrt/WORK/llvm5/build/debug/tools/clang/include/clang/AST/DeclNodes.inc:405:0
#2 clang::ASTImporter::ImportImpl(clang::Decl*) clang/lib/AST/ASTImporter.cpp:8038:0
#3 clang::ASTImporter::Import(clang::Decl*) clang/lib/AST/ASTImporter.cpp:8200:0
#4 clang::ASTImporter::ImportContext(clang::DeclContext*) clang/lib/AST/ASTImporter.cpp:8297:0
#5 clang::ASTNodeImporter::ImportDeclContext(clang::Decl*, clang::DeclContext*&, clang::DeclContext*&) clang/lib/AST/ASTImporter.cpp:1852:0
#6 clang::ASTNodeImporter::ImportDeclParts(clang::NamedDecl*, clang::DeclContext*&, clang::DeclContext*&, clang::DeclarationName&, clang::NamedDecl*&, clang::SourceLocation&) clang/lib/AST/ASTImporter.cpp:1628:0
#7 clang::ASTNodeImporter::VisitTypedefNameDecl(clang::TypedefNameDecl*, bool) clang/lib/AST/ASTImporter.cpp:2419:0
#8 clang::ASTNodeImporter::VisitTypedefDecl(clang::TypedefDecl*) clang/lib/AST/ASTImporter.cpp:2500:0
#9 clang::declvisitor::Base<std::add_pointer, clang::ASTNodeImporter, llvm::Expected<clang::Decl*> >::Visit(clang::Decl*) /home/egbomrt/WORK/llvm5/build/debug/tools/clang/include/clang/AST/DeclNodes.inc:315:0
#10 clang::ASTImporter::ImportImpl(clang::Decl*) clang/lib/AST/ASTImporter.cpp:8038:0
#11 clang::ASTImporter::Import(clang::Decl*) clang/lib/AST/ASTImporter.cpp:8200:0
#12 llvm::Expected<clang::TypedefNameDecl*> clang::ASTNodeImporter::import<clang::TypedefNameDecl>(clang::TypedefNameDecl*) clang/lib/AST/ASTImporter.cpp:165:0
#13 clang::ASTNodeImporter::VisitTypedefType(clang::TypedefType const*) clang/lib/AST/ASTImporter.cpp:1304:0
#14 clang::TypeVisitor<clang::ASTNodeImporter, llvm::Expected<clang::QualType> >::Visit(clang::Type const*) /home/egbomrt/WORK/llvm5/build/debug/tools/clang/include/clang/AST/TypeNodes.inc:74:0
#15 clang::ASTImporter::Import(clang::QualType) clang/lib/AST/ASTImporter.cpp:8071:0
#16 llvm::Expected<clang::QualType> clang::ASTNodeImporter::import<clang::QualType>(clang::QualType const&) clang/lib/AST/ASTImporter.cpp:179:0
#17 clang::ASTNodeImporter::VisitFunctionProtoType(clang::FunctionProtoType const*) clang/lib/AST/ASTImporter.cpp:1244:0
#18 clang::TypeVisitor<clang::ASTNodeImporter, llvm::Expected<clang::QualType> >::Visit(clang::Type const*) /home/egbomrt/WORK/llvm5/build/debug/tools/clang/include/clang/AST/TypeNodes.inc:47:0
#19 clang::ASTImporter::Import(clang::QualType) clang/lib/AST/ASTImporter.cpp:8071:0
#20 llvm::Expected<clang::QualType> clang::ASTNodeImporter::import<clang::QualType>(clang::QualType const&) clang/lib/AST/ASTImporter.cpp:179:0
#21 clang::QualType clang::ASTNodeImporter::importChecked<clang::QualType>(llvm::Error&, clang::QualType const&) clang/lib/AST/ASTImporter.cpp:198:0
#22 clang::ASTNodeImporter::VisitFunctionDecl(clang::FunctionDecl*) clang/lib/AST/ASTImporter.cpp:3313:0
#23 clang::ASTNodeImporter::VisitCXXDeductionGuideDecl(clang::CXXDeductionGuideDecl*) clang/lib/AST/ASTImporter.cpp:3543:0
```
The fix is to first create the TypedefDecl and only then start to import
the DeclContext.
Basically, we could do this during the import of all other Decls (not
just for typedefs). But it seems, there is only one another AST
construct that has a similar cycle: a struct defined as a function
parameter:
```
int struct_in_proto(struct data_t{int a;int b;} *d);
```
In that case, however, we had decided to return simply with an error
back then because that seemed to be a very rare construct.
Differential Revision: https://reviews.llvm.org/D92209
CXXDeductionGuideDecl is a FunctionDecl, but its constructor should be called
appropriately, at least to set the kind variable properly.
Differential Revision: https://reviews.llvm.org/D92109
The test case isn't using the AST matchers for all checks as there doesn't seem to be support for
matching NonTypeTemplateParmDecl default arguments. Otherwise this is simply importing the
default arguments.
Reviewed By: martong
Differential Revision: https://reviews.llvm.org/D92106
Balazs Benics