maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/lib/Lex/PreprocessingRecord.cpp

191 lines
6.4 KiB
C++
Raw Normal View History

Introduce the notion of a "preprocessing record", which keeps track of the macro definitions and macro instantiations that are found during preprocessing. Preprocessing records are *not* generated by default; rather, we provide a PPCallbacks subclass that hooks into the existing callback mechanism to record this activity. The only client of preprocessing records is CIndex, which keeps track of macro definitions and instantations so that they can be exposed via cursors. At present, only token annotation uses these facilities, and only for macro instantiations; both will change in the near future. However, with this change, token annotation properly annotates macro instantiations that do not produce any tokens and instantiations of macros that are later undef'd, improving our consistency. Preprocessing directives that are not macro definitions are still handled by clang_annotateTokens() via re-lexing, so that we don't have to track every preprocessing directive in the preprocessing record. Performance impact of preprocessing records is still TBD, although it is limited to CIndex and therefore out of the path of the main compiler. llvm-svn: 98836
2010-03-19 01:52:52 +08:00
//===--- PreprocessingRecord.cpp - Record of Preprocessing ------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This file implements the PreprocessingRecord class, which maintains a record
// of what occurred during preprocessing, and its helpers.
//
//===----------------------------------------------------------------------===//
#include "clang/Lex/PreprocessingRecord.h"
#include "clang/Lex/MacroInfo.h"
#include "clang/Lex/Token.h"
Extend the preprocessing record and libclang with support for inclusion directives, keeping track of every #include, #import, etc. in the translation unit. We keep track of the source location and kind of the inclusion, how the file name was spelled, and the underlying file to which the inclusion resolved. llvm-svn: 116952
2010-10-21 06:00:55 +08:00
#include "clang/Basic/IdentifierTable.h"
#include "llvm/Support/ErrorHandling.h"
Introduce the notion of a "preprocessing record", which keeps track of the macro definitions and macro instantiations that are found during preprocessing. Preprocessing records are *not* generated by default; rather, we provide a PPCallbacks subclass that hooks into the existing callback mechanism to record this activity. The only client of preprocessing records is CIndex, which keeps track of macro definitions and instantations so that they can be exposed via cursors. At present, only token annotation uses these facilities, and only for macro instantiations; both will change in the near future. However, with this change, token annotation properly annotates macro instantiations that do not produce any tokens and instantiations of macros that are later undef'd, improving our consistency. Preprocessing directives that are not macro definitions are still handled by clang_annotateTokens() via re-lexing, so that we don't have to track every preprocessing directive in the preprocessing record. Performance impact of preprocessing records is still TBD, although it is limited to CIndex and therefore out of the path of the main compiler. llvm-svn: 98836
2010-03-19 01:52:52 +08:00
using namespace clang;
Implement serialization and lazy deserialization of the preprocessing record (which includes all macro instantiations and definitions). As with all lay deserialization, this introduces a new external source (here, an external preprocessing record source) that loads all of the preprocessed entities prior to iterating over the entities. The preprocessing record is an optional part of the precompiled header that is disabled by default (enabled with -detailed-preprocessing-record). When the preprocessor given to the PCH writer has a preprocessing record, that record is written into the PCH file. When the PCH reader is given a PCH file that contains a preprocessing record, it will be lazily loaded (which, effectively, implicitly adds -detailed-preprocessing-record). This is the first case where we have sections of the precompiled header that are added/removed based on a compilation flag, which is unfortunate. However, this data consumes ~550k in the PCH file for Cocoa.h (out of ~9.9MB), and there is a non-trivial cost to gathering this detailed preprocessing information, so it's too expensive to turn on by default. In the future, we should investigate a better encoding of this information. llvm-svn: 99002
2010-03-20 05:51:54 +08:00
ExternalPreprocessingRecordSource::~ExternalPreprocessingRecordSource() { }
Plug a leak in the preprocessing record's handling of inclusion directives. We had a std::string in an object that was allocated via a BumpPtrAllocator. llvm-svn: 117912
2010-11-01 23:03:47 +08:00
InclusionDirective::InclusionDirective(PreprocessingRecord &PPRec,
InclusionKind Kind,
llvm::StringRef FileName,
bool InQuotes, const FileEntry *File,
SourceRange Range)
: PreprocessingDirective(InclusionDirectiveKind, Range),
InQuotes(InQuotes), Kind(Kind), File(File)
{
char *Memory
= (char*)PPRec.Allocate(FileName.size() + 1, llvm::alignOf<char>());
memcpy(Memory, FileName.data(), FileName.size());
Memory[FileName.size()] = 0;
this->FileName = llvm::StringRef(Memory, FileName.size());
}
Implement serialization and lazy deserialization of the preprocessing record (which includes all macro instantiations and definitions). As with all lay deserialization, this introduces a new external source (here, an external preprocessing record source) that loads all of the preprocessed entities prior to iterating over the entities. The preprocessing record is an optional part of the precompiled header that is disabled by default (enabled with -detailed-preprocessing-record). When the preprocessor given to the PCH writer has a preprocessing record, that record is written into the PCH file. When the PCH reader is given a PCH file that contains a preprocessing record, it will be lazily loaded (which, effectively, implicitly adds -detailed-preprocessing-record). This is the first case where we have sections of the precompiled header that are added/removed based on a compilation flag, which is unfortunate. However, this data consumes ~550k in the PCH file for Cocoa.h (out of ~9.9MB), and there is a non-trivial cost to gathering this detailed preprocessing information, so it's too expensive to turn on by default. In the future, we should investigate a better encoding of this information. llvm-svn: 99002
2010-03-20 05:51:54 +08:00
void PreprocessingRecord::MaybeLoadPreallocatedEntities() const {
if (!ExternalSource || LoadedPreallocatedEntities)
return;
LoadedPreallocatedEntities = true;
ExternalSource->ReadPreprocessedEntities();
}
Introduce a new libclang parsing flag, CXTranslationUnit_NestedMacroInstantiations, which indicates whether we want to see "nested" macro instantiations (e.g., those that occur inside other macro instantiations) within the detailed preprocessing record. Many clients (e.g., those that only care about visible tokens) don't care about this information, and in code that uses preprocessor metaprogramming, this information can have a very high cost. Addresses <rdar://problem/9389320>. llvm-svn: 130990
2011-05-07 00:33:08 +08:00
PreprocessingRecord::PreprocessingRecord(bool IncludeNestedMacroInstantiations)
: IncludeNestedMacroInstantiations(IncludeNestedMacroInstantiations),
ExternalSource(0), NumPreallocatedEntities(0),
Implement serialization and lazy deserialization of the preprocessing record (which includes all macro instantiations and definitions). As with all lay deserialization, this introduces a new external source (here, an external preprocessing record source) that loads all of the preprocessed entities prior to iterating over the entities. The preprocessing record is an optional part of the precompiled header that is disabled by default (enabled with -detailed-preprocessing-record). When the preprocessor given to the PCH writer has a preprocessing record, that record is written into the PCH file. When the PCH reader is given a PCH file that contains a preprocessing record, it will be lazily loaded (which, effectively, implicitly adds -detailed-preprocessing-record). This is the first case where we have sections of the precompiled header that are added/removed based on a compilation flag, which is unfortunate. However, this data consumes ~550k in the PCH file for Cocoa.h (out of ~9.9MB), and there is a non-trivial cost to gathering this detailed preprocessing information, so it's too expensive to turn on by default. In the future, we should investigate a better encoding of this information. llvm-svn: 99002
2010-03-20 05:51:54 +08:00
LoadedPreallocatedEntities(false)
{
}
PreprocessingRecord::iterator
PreprocessingRecord::begin(bool OnlyLocalEntities) {
if (OnlyLocalEntities)
return PreprocessedEntities.begin() + NumPreallocatedEntities;
MaybeLoadPreallocatedEntities();
return PreprocessedEntities.begin();
}
PreprocessingRecord::iterator PreprocessingRecord::end(bool OnlyLocalEntities) {
if (!OnlyLocalEntities)
MaybeLoadPreallocatedEntities();
return PreprocessedEntities.end();
}
PreprocessingRecord::const_iterator
PreprocessingRecord::begin(bool OnlyLocalEntities) const {
if (OnlyLocalEntities)
return PreprocessedEntities.begin() + NumPreallocatedEntities;
MaybeLoadPreallocatedEntities();
return PreprocessedEntities.begin();
}
PreprocessingRecord::const_iterator
PreprocessingRecord::end(bool OnlyLocalEntities) const {
if (!OnlyLocalEntities)
MaybeLoadPreallocatedEntities();
return PreprocessedEntities.end();
}
Introduce the notion of a "preprocessing record", which keeps track of the macro definitions and macro instantiations that are found during preprocessing. Preprocessing records are *not* generated by default; rather, we provide a PPCallbacks subclass that hooks into the existing callback mechanism to record this activity. The only client of preprocessing records is CIndex, which keeps track of macro definitions and instantations so that they can be exposed via cursors. At present, only token annotation uses these facilities, and only for macro instantiations; both will change in the near future. However, with this change, token annotation properly annotates macro instantiations that do not produce any tokens and instantiations of macros that are later undef'd, improving our consistency. Preprocessing directives that are not macro definitions are still handled by clang_annotateTokens() via re-lexing, so that we don't have to track every preprocessing directive in the preprocessing record. Performance impact of preprocessing records is still TBD, although it is limited to CIndex and therefore out of the path of the main compiler. llvm-svn: 98836
2010-03-19 01:52:52 +08:00
void PreprocessingRecord::addPreprocessedEntity(PreprocessedEntity *Entity) {
PreprocessedEntities.push_back(Entity);
}
Implement serialization and lazy deserialization of the preprocessing record (which includes all macro instantiations and definitions). As with all lay deserialization, this introduces a new external source (here, an external preprocessing record source) that loads all of the preprocessed entities prior to iterating over the entities. The preprocessing record is an optional part of the precompiled header that is disabled by default (enabled with -detailed-preprocessing-record). When the preprocessor given to the PCH writer has a preprocessing record, that record is written into the PCH file. When the PCH reader is given a PCH file that contains a preprocessing record, it will be lazily loaded (which, effectively, implicitly adds -detailed-preprocessing-record). This is the first case where we have sections of the precompiled header that are added/removed based on a compilation flag, which is unfortunate. However, this data consumes ~550k in the PCH file for Cocoa.h (out of ~9.9MB), and there is a non-trivial cost to gathering this detailed preprocessing information, so it's too expensive to turn on by default. In the future, we should investigate a better encoding of this information. llvm-svn: 99002
2010-03-20 05:51:54 +08:00
void PreprocessingRecord::SetExternalSource(
ExternalPreprocessingRecordSource &Source,
unsigned NumPreallocatedEntities) {
assert(!ExternalSource &&
"Preprocessing record already has an external source");
ExternalSource = &Source;
this->NumPreallocatedEntities = NumPreallocatedEntities;
PreprocessedEntities.insert(PreprocessedEntities.begin(),
NumPreallocatedEntities, 0);
}
void PreprocessingRecord::SetPreallocatedEntity(unsigned Index,
PreprocessedEntity *Entity) {
assert(Index < NumPreallocatedEntities &&"Out-of-bounds preallocated entity");
PreprocessedEntities[Index] = Entity;
}
void PreprocessingRecord::RegisterMacroDefinition(MacroInfo *Macro,
MacroDefinition *MD) {
MacroDefinitions[Macro] = MD;
}
Robustify PreprocessingRecord slightly, by only creating macro instantiations when we have the corresponding macro definition and by removing macro definition information from our table when the macro is undefined. llvm-svn: 99004
2010-03-20 05:58:23 +08:00
MacroDefinition *PreprocessingRecord::findMacroDefinition(const MacroInfo *MI) {
Make the preprocessing record a PPCallbacks subclass itself, eliminating the extra PopulatePreprocessingRecord object. This will become useful once we start writing the preprocessing record to precompiled headers. llvm-svn: 98966
2010-03-20 01:12:43 +08:00
llvm::DenseMap<const MacroInfo *, MacroDefinition *>::iterator Pos
= MacroDefinitions.find(MI);
if (Pos == MacroDefinitions.end())
return 0;
return Pos->second;
}
void PreprocessingRecord::MacroExpands(const Token &Id, const MacroInfo* MI) {
Introduce a new libclang parsing flag, CXTranslationUnit_NestedMacroInstantiations, which indicates whether we want to see "nested" macro instantiations (e.g., those that occur inside other macro instantiations) within the detailed preprocessing record. Many clients (e.g., those that only care about visible tokens) don't care about this information, and in code that uses preprocessor metaprogramming, this information can have a very high cost. Addresses <rdar://problem/9389320>. llvm-svn: 130990
2011-05-07 00:33:08 +08:00
if (!IncludeNestedMacroInstantiations && Id.getLocation().isMacroID())
return;
Robustify PreprocessingRecord slightly, by only creating macro instantiations when we have the corresponding macro definition and by removing macro definition information from our table when the macro is undefined. llvm-svn: 99004
2010-03-20 05:58:23 +08:00
if (MacroDefinition *Def = findMacroDefinition(MI))
PreprocessedEntities.push_back(
Make the preprocessing record a PPCallbacks subclass itself, eliminating the extra PopulatePreprocessingRecord object. This will become useful once we start writing the preprocessing record to precompiled headers. llvm-svn: 98966
2010-03-20 01:12:43 +08:00
new (*this) MacroInstantiation(Id.getIdentifierInfo(),
Id.getLocation(),
Robustify PreprocessingRecord slightly, by only creating macro instantiations when we have the corresponding macro definition and by removing macro definition information from our table when the macro is undefined. llvm-svn: 99004
2010-03-20 05:58:23 +08:00
Def));
Make the preprocessing record a PPCallbacks subclass itself, eliminating the extra PopulatePreprocessingRecord object. This will become useful once we start writing the preprocessing record to precompiled headers. llvm-svn: 98966
2010-03-20 01:12:43 +08:00
}
Several PPCallbacks take an SourceLocation + IdentifierInfo, rather than a Token that holds the same information all in one easy-to-use package. There's no technical reason to prefer the former -- the information comes from a Token originally -- and it's clumsier to use, so I've changed the code to use tokens everywhere. Approved by clattner llvm-svn: 119845
2010-11-20 05:33:15 +08:00
void PreprocessingRecord::MacroDefined(const Token &Id,
Make the preprocessing record a PPCallbacks subclass itself, eliminating the extra PopulatePreprocessingRecord object. This will become useful once we start writing the preprocessing record to precompiled headers. llvm-svn: 98966
2010-03-20 01:12:43 +08:00
const MacroInfo *MI) {
SourceRange R(MI->getDefinitionLoc(), MI->getDefinitionEndLoc());
MacroDefinition *Def
Several PPCallbacks take an SourceLocation + IdentifierInfo, rather than a Token that holds the same information all in one easy-to-use package. There's no technical reason to prefer the former -- the information comes from a Token originally -- and it's clumsier to use, so I've changed the code to use tokens everywhere. Approved by clattner llvm-svn: 119845
2010-11-20 05:33:15 +08:00
= new (*this) MacroDefinition(Id.getIdentifierInfo(),
MI->getDefinitionLoc(),
R);
Make the preprocessing record a PPCallbacks subclass itself, eliminating the extra PopulatePreprocessingRecord object. This will become useful once we start writing the preprocessing record to precompiled headers. llvm-svn: 98966
2010-03-20 01:12:43 +08:00
MacroDefinitions[MI] = Def;
PreprocessedEntities.push_back(Def);
}
Implement serialization and lazy deserialization of the preprocessing record (which includes all macro instantiations and definitions). As with all lay deserialization, this introduces a new external source (here, an external preprocessing record source) that loads all of the preprocessed entities prior to iterating over the entities. The preprocessing record is an optional part of the precompiled header that is disabled by default (enabled with -detailed-preprocessing-record). When the preprocessor given to the PCH writer has a preprocessing record, that record is written into the PCH file. When the PCH reader is given a PCH file that contains a preprocessing record, it will be lazily loaded (which, effectively, implicitly adds -detailed-preprocessing-record). This is the first case where we have sections of the precompiled header that are added/removed based on a compilation flag, which is unfortunate. However, this data consumes ~550k in the PCH file for Cocoa.h (out of ~9.9MB), and there is a non-trivial cost to gathering this detailed preprocessing information, so it's too expensive to turn on by default. In the future, we should investigate a better encoding of this information. llvm-svn: 99002
2010-03-20 05:51:54 +08:00
Several PPCallbacks take an SourceLocation + IdentifierInfo, rather than a Token that holds the same information all in one easy-to-use package. There's no technical reason to prefer the former -- the information comes from a Token originally -- and it's clumsier to use, so I've changed the code to use tokens everywhere. Approved by clattner llvm-svn: 119845
2010-11-20 05:33:15 +08:00
void PreprocessingRecord::MacroUndefined(const Token &Id,
Robustify PreprocessingRecord slightly, by only creating macro instantiations when we have the corresponding macro definition and by removing macro definition information from our table when the macro is undefined. llvm-svn: 99004
2010-03-20 05:58:23 +08:00
const MacroInfo *MI) {
llvm::DenseMap<const MacroInfo *, MacroDefinition *>::iterator Pos
= MacroDefinitions.find(MI);
if (Pos != MacroDefinitions.end())
MacroDefinitions.erase(Pos);
}
Add a 'RawPath' parameter to the PPCallbacks interface. This allows clients to observe the exact path through which an #included file was located. This is very useful when trying to record and replay inclusion operations without it beind influenced by the aggressive caching done inside the FileManager to avoid redundant system calls and filesystem operations. The work to compute and return this is only done in the presence of callbacks, so it should have no effect on normal compilation. Patch by Manuel Klimek. llvm-svn: 127742
2011-03-17 02:34:36 +08:00
void PreprocessingRecord::InclusionDirective(
SourceLocation HashLoc,
const clang::Token &IncludeTok,
llvm::StringRef FileName,
bool IsAngled,
const FileEntry *File,
clang::SourceLocation EndLoc,
To be able to replay compilations we need to accurately remodel how includes get resolved, especially when they are found relatively to another include file. We also try to get it working for framework includes, but that part of the code is untested, as I don't have a code base that uses it. llvm-svn: 130246
2011-04-27 05:50:03 +08:00
llvm::StringRef SearchPath,
llvm::StringRef RelativePath) {
Extend the preprocessing record and libclang with support for inclusion directives, keeping track of every #include, #import, etc. in the translation unit. We keep track of the source location and kind of the inclusion, how the file name was spelled, and the underlying file to which the inclusion resolved. llvm-svn: 116952
2010-10-21 06:00:55 +08:00
InclusionDirective::InclusionKind Kind = InclusionDirective::Include;
switch (IncludeTok.getIdentifierInfo()->getPPKeywordID()) {
case tok::pp_include:
Kind = InclusionDirective::Include;
break;
case tok::pp_import:
Kind = InclusionDirective::Import;
break;
case tok::pp_include_next:
Kind = InclusionDirective::IncludeNext;
break;
case tok::pp___include_macros:
Kind = InclusionDirective::IncludeMacros;
break;
default:
llvm_unreachable("Unknown include directive kind");
return;
}
clang::InclusionDirective *ID
Plug a leak in the preprocessing record's handling of inclusion directives. We had a std::string in an object that was allocated via a BumpPtrAllocator. llvm-svn: 117912
2010-11-01 23:03:47 +08:00
= new (*this) clang::InclusionDirective(*this, Kind, FileName, !IsAngled,
File, SourceRange(HashLoc, EndLoc));
Extend the preprocessing record and libclang with support for inclusion directives, keeping track of every #include, #import, etc. in the translation unit. We keep track of the source location and kind of the inclusion, how the file name was spelled, and the underlying file to which the inclusion resolved. llvm-svn: 116952
2010-10-21 06:00:55 +08:00
PreprocessedEntities.push_back(ID);
}