Do import the definition of objects from a foreign translation unit if that's type is const and trivial.
Differential Revision: https://reviews.llvm.org/D122805
This error was found when analyzing MySQL with CTU enabled.
When there are space characters in the lookup name, the current
delimiter searching strategy will make the file path wrongly parsed.
And when two lookup names have the same prefix before their first space
characters, a 'multiple definitions' error will be wrongly reported.
e.g. The lookup names for the two lambda exprs in the test case are
`c:@S@G@F@G#@Sa@F@operator int (*)(char)#1` and
`c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1` respectively. And their
prefixes are both `c:@S@G@F@G#@Sa@F@operator` when using the first space
character as the delimiter.
Solving the problem by adding a length for the lookup name, making the
index items in the format of `<USR-Length>:<USR File> <Path>`.
---
In the test case of this patch, we found that it will trigger a "triple
mismatch" warning when using `clang -cc1` to analyze the source file
with CTU using the on-demand-parsing strategy in Darwin systems. And
this problem is also encountered in D75665, which is the patch
introducing the on-demand parsing strategy.
We temporarily bypass this problem by using the loading-ast-file
strategy.
Refer to the [discourse topic](https://discourse.llvm.org/t/60762) for
more details.
Differential Revision: https://reviews.llvm.org/D102669
This error was found when analyzing MySQL with CTU enabled.
When there are space characters in the lookup name, the current
delimiter searching strategy will make the file path wrongly parsed.
And when two lookup names have the same prefix before their first space
characters, a 'multiple definitions' error will be wrongly reported.
e.g. The lookup names for the two lambda exprs in the test case are
`c:@S@G@F@G#@Sa@F@operator int (*)(char)#1` and
`c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1` respectively. And their
prefixes are both `c:@S@G@F@G#@Sa@F@operator` when using the first space
character as the delimiter.
Solving the problem by adding a length for the lookup name, making the
index items in the format of `USR-Length:USR File-Path`.
Reviewed By: steakhal
Differential Revision: https://reviews.llvm.org/D102669
C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible with C++23 (& a follow-up change I've made
will make this ambiguous/invalid even in <C++23 so we don't regress
this & it generally improves the code anyway)
..during on-demand parsing of CTU"
During CTU, the *on-demand parsing* will read and parse the invocation
list to know how to compile the file being imported. However, it seems
that the invocation list will be parsed again if a previous parsing
has failed.
Then, parse again and fail again. This patch tries to overcome the
problem by storing the error code during the first parsing, and
re-create the stored error during the later parsings.
Reland without test.
Reviewed By: steakhal
Patch By: OikawaKirie!
Differential Revision: https://reviews.llvm.org/D101763
During CTU, the *on-demand parsing* will read and parse the invocation
list to know how to compile the file being imported. However, it seems
that the invocation list will be parsed again if a previous parsing
has failed.
Then, parse again and fail again. This patch tries to overcome the
problem by storing the error code during the first parsing, and
re-create the stored error during the later parsings.
Reviewed By: steakhal
Patch By: OikawaKirie!
Differential Revision: https://reviews.llvm.org/D101763
Removes `CrossTranslationUnitContext::getImportedFromSourceLocation`
Removes the corresponding unit-test segment.
Introduces the `CrossTranslationUnitContext::getMacroExpansionContextForSourceLocation`
which will return the macro expansion context for an imported TU. Also adds a
few implementation FIXME notes where applicable, since this feature is
not implemented yet. This fact is also noted as Doxygen comments.
Uplifts a few CTU LIT test to match the current **incomplete** behavior.
It is a regression to some extent since now we don't expand any
macros in imported TUs. At least we don't crash anymore.
Note that the introduced function is already covered by LIT tests.
Eg.: Analysis/plist-macros-with-expansion-ctu.c
Reviewed By: balazske, Szelethus
Differential Revision: https://reviews.llvm.org/D94673
Summary:
The default CTUImportThreshold (8) seems to be too conservative with C projects.
We increase this value to 24 and we introduce another threshold for C++ source
files (defaulted to 8) because their AST is way more compilcated than C source
files.
Differential Revision: https://reviews.llvm.org/D83475
Summary:
Parent map of ASTContext is built once. If this happens and later
the TU is modified by getCrossTUDefinition the parent map does not
contain the newly imported objects and has to be re-created.
Invalidation of the parent map is added to the CrossTranslationUnitContext.
It could be added to ASTImporter as well but for now this task remains the
responsibility of the user of ASTImporter. Reason for this is mostly that
ASTImporter calls itself recursively.
Reviewers: gamesh411, martong
Reviewed By: gamesh411
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, martong, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82568
Summary:
Add an option to enable on-demand parsing of needed ASTs during CTU analysis.
Two options are introduced. CTUOnDemandParsing enables the feature, and
CTUOnDemandParsingDatabase specifies the path to a compilation database, which
has all the necessary information to generate the ASTs.
Reviewers: martong, balazske, Szelethus, xazax.hun
Subscribers: ormris, mgorny, whisperity, xazax.hun, baloghadamsoftware, szepet, rnkovacs, a.sidorin, mikhail.ramalho, Szelethus, donat.nagy, dkrupp, Charusso, steakhal, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D75665
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Summary:
Number of loaded ASTs is to be incremented only if the AST was really loaded
but not if it was returned from cache. At the same place the message about
a loaded AST is displayed.
Reviewers: martong, gamesh411
Reviewed By: martong
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66054
llvm-svn: 368545
Summary:
When searching for a declaration to be loaded the "lookup name" for every
other Decl is computed. If the USR can not be determined here should be
not an assert, instead skip this Decl.
Reviewers: martong
Reviewed By: martong
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65445
llvm-svn: 368020
It seems because of the recent refactorings this variable has become unused
and now we get this warning in the build logs:
In file included from llvm/clang/lib/CrossTU/CrossTranslationUnit.cpp:12:
llvm/clang/include/clang/CrossTU/CrossTranslationUnit.h:200:21: warning: private field 'CI' is not used [-Wunused-private-field]
CompilerInstance &CI;
^
I'll remove them for now to get the builds back to green.
llvm-svn: 367840
Summary:
Refactor loadExternalAST method of CrossTranslationUnitContext in order to
reduce maintenance burden and so that features are easier to add in the future.
Reviewers: martong
Subscribers: rnkovacs, dkrupp, Szelethus, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64753
llvm-svn: 367829
Summary:
A new function will be added to get the original SourceLocation
for a SourceLocation that was imported as result of getCrossTUDefinition.
The returned SourceLocation is in the context of the (original)
SourceManager for the original source file. Additionally the
ASTUnit object for that source file is returned. This is needed
to get a SourceManager to operate on with the returned source location.
The new function works if multiple different source files are loaded
with the same CrossTU context.
Reviewers: martong, shafik
Reviewed By: martong
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65064
llvm-svn: 366884
Reason: the commit breaks layering by adding a dependency on ASTUnit
(which is inside clangFrontend) from the ASTImporter (which is inside
clangAST).
llvm-svn: 366453
Summary:
A new function will be added to get the original SourceLocation
for a SourceLocation that was imported as result of getCrossTUDefinition.
The returned SourceLocation is in the context of the (original)
SourceManager for the original source file. Additionally the
ASTUnit object for that source file is returned. This is needed
to get a SourceManager to operate on with the returned source location.
The new function works if multiple different source files are loaded
with the same CrossTU context.
This patch can be treated as part of a bigger change that is needed to
improve macro expansion handliong at plist generation.
Reviewers: martong, shafik, a_sidorin, xazax.hun
Reviewed By: martong
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D64554
llvm-svn: 366449
Summary:
During CTU analysis of complex projects, the loaded AST-contents of
imported TUs can grow bigger than available system memory. This option
introduces a threshold on the number of TUs to be imported for a single
TU in order to prevent such cases.
Differential Revision: https://reviews.llvm.org/D59798
llvm-svn: 365314
Summary:
Now we store the errors for the Decls in the "to" context too. For
that, however, we have to put these errors in a shared state (among all
the ASTImporter objects which handle the same "to" context but different
"from" contexts).
After a series of imports from different "from" TUs we have a "to" context
which may have erroneous nodes in it. (Remember, the AST is immutable so
there is no way to delete a node once we had created it and we realized
the error later.) All these erroneous nodes are marked in
ASTImporterSharedState::ImportErrors. Clients of the ASTImporter may
use this as an input. E.g. the static analyzer engine may not try to
analyze a function if that is marked as erroneous (it can be queried via
ASTImporterSharedState::getImportDeclErrorIfAny()).
Reviewers: a_sidorin, a.sidorin, shafik
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D62376
llvm-svn: 364785
Summary:
This is the final phase of the refactoring towards using llvm::Expected
and llvm::Error in the ASTImporter API.
This involves the following:
- remove old Import functions which returned with a pointer,
- use the Import_New functions (which return with Err or Expected) everywhere
and handle their return value
- rename Import_New functions to Import
This affects both Clang and LLDB.
Reviewers: shafik, teemperor, aprantl, a_sidorin, balazske, a.sidorin
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits, lldb-commits
Tags: #clang, #lldb
Differential Revision: https://reviews.llvm.org/D61438
llvm-svn: 360760
Summary:
The existing CTU mechanism imports `FunctionDecl`s where the definition is available in another TU. This patch extends that to VarDecls, to bind more constants.
- Add VarDecl importing functionality to CrossTranslationUnitContext
- Import Decls while traversing them in AnalysisConsumer
- Add VarDecls to CTU external mappings generator
- Name changes from "external function map" to "external definition map"
Reviewers: NoQ, dcoughlin, xazax.hun, george.karpenkov, martong
Reviewed By: xazax.hun
Subscribers: Charusso, baloghadamsoftware, mikhail.ramalho, Szelethus, donat.nagy, dkrupp, george.karpenkov, mgorny, whisperity, szepet, rnkovacs, a.sidorin, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D46421
llvm-svn: 358968
Summary:
If CPP dialects are different then return with error.
Consider this STL code:
template<typename _Alloc>
struct __alloc_traits
#if __cplusplus >= 201103L
: std::allocator_traits<_Alloc>
#endif
{ // ...
};
This class template would create ODR errors during merging the two units,
since in one translation unit the class template has a base class, however
in the other unit it has none.
Reviewers: xazax.hun, a_sidorin, r.stahl
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D57906
llvm-svn: 355096
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Summary: This is just changing naming and documentation to be general about external definitions that can be imported for cross translation unit analysis. There is at least a plan to add VarDecls: D46421
Reviewers: NoQ, xazax.hun, martong, a.sidorin, george.karpenkov, serge-sans-paille
Reviewed By: xazax.hun, martong
Subscribers: mgorny, whisperity, baloghadamsoftware, szepet, rnkovacs, mikhail.ramalho, Szelethus, donat.nagy, dkrupp, cfe-commits
Differential Revision: https://reviews.llvm.org/D56441
llvm-svn: 350852
Summary:
In loadExternalAST we return with either an error or with a valid
ASTUnit pointer which should not be a nullptr.
This prevents in the call site any superfluous check for being a nullptr.
Reviewers: xazax.hun, a_sidorin, Szelethus, balazske
Subscribers: rnkovacs, dkrupp, gamesh411, cfe-commits
Differential Revision: https://reviews.llvm.org/D55280
llvm-svn: 350521
Summary:
There are certain cases when normal C/C++ lookup (localUncachedLookup)
does not find AST nodes. E.g.:
Example 1:
template <class T>
struct X {
friend void foo(); // this is never found in the DC of the TU.
};
Example 2:
// The fwd decl to Foo is not found in the lookupPtr of the DC of the
// translation unit decl.
struct A { struct Foo *p; };
In these cases we create a new node instead of returning with the old one.
To fix it we create a new lookup table which holds every node and we are
not interested in any C++ specific visibility considerations.
Simply, we must know if there is an existing Decl in a given DC.
Reviewers: a_sidorin, a.sidorin
Subscribers: mgorny, rnkovacs, dkrupp, Szelethus, cfe-commits
Differential Revision: https://reviews.llvm.org/D53708
llvm-svn: 349351
Move some diagnostics around between Diagnostic*Kinds.td files. Diagnostics
used in multiple places were moved to DiagnosticCommonKinds.td. Diagnostics
listed in the wrong place (ie, Sema diagnostics listed in
DiagnosticsParseKinds.td) were moved to the correct places. One diagnostic
split into two so that the diagnostic string is in the .td file instead of in
code. Cleaned up the diagnostic includes after all the changes.
llvm-svn: 349125
Summary:
We introduce a strict policy for C++ CTU. It can work across TUs only if
the C++ dialects are the same. We neither allow C vs C++ CTU. We do this
because the same constructs might be represented with different properties in
the corresponding AST nodes or even the nodes might be completely different (a
struct will be RecordDecl in C, but it will be a CXXRectordDecl in C++, thus it
may cause certain assertions during cast operations).
Reviewers: xazax.hun, a_sidorin
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Differential Revision: https://reviews.llvm.org/D55134
llvm-svn: 348610
Summary:
Adding some more CTU list tests. E.g. to check if a construct is unsupported.
We also slightly modify the handling of the return value of the `Import`
function from ASTImporter.
Reviewers: xazax.hun, balazske, a_sidorin
Subscribers: rnkovacs, dkrupp, Szelethus, gamesh411, cfe-commits
Differential Revision: https://reviews.llvm.org/D55131
llvm-svn: 348605
Gabor Marton