llvm-project/clang/lib/Sema
Alexander Kornienko 8b3f623582 Keep history of macro definitions and #undefs
Summary:
Summary: Keep history of macro definitions and #undefs with corresponding source locations, so that we can later find out all macros active in a specified source location. We don't save the history in PCH (no need currently). Memory overhead is about sizeof(void*)*3*<number of macro definitions and #undefs>+<in-memory size of all #undef'd macros>

I've run a test on a file composed of 109 .h files from boost 1.49 on x86-64 linux.
Stats before this patch:
*** Preprocessor Stats:
73222 directives found:
  19171 #define.
  4345 #undef.
  #include/#include_next/#import:
    5233 source files entered.
    27 max include stack depth
  19210 #if/#ifndef/#ifdef.
  2384 #else/#elif.
  6891 #endif.
  408 #pragma.
14466 #if/#ifndef#ifdef regions skipped
80023/451669/1270 obj/fn/builtin macros expanded, 85724 on the fast path.
127145 token paste (##) operations performed, 11008 on the fast path.

Preprocessor Memory: 5874615B total
  BumpPtr: 4399104
  Macro Expanded Tokens: 417768
  Predefines Buffer: 8135
  Macros: 1048576
  #pragma push_macro Info: 0
  Poison Reasons: 1024
  Comment Handlers: 8

Stats with this patch:
...
Preprocessor Memory: 7541687B total
  BumpPtr: 6066176
  Macro Expanded Tokens: 417768
  Predefines Buffer: 8135
  Macros: 1048576
  #pragma push_macro Info: 0
  Poison Reasons: 1024
  Comment Handlers: 8

In my test increase in memory usage is about 1.7Mb, which is ~28% of initial preprocessor's memory usage and about 0.8% of clang's total VMM allocation.

As for CPU overhead, it should only be noticeable when iterating over all macros, and should mostly consist of couple extra dereferences and one comparison per macro + skipping of #undef'd macros. It's less trivial to measure, though, as the preprocessor consumes a very small fraction of compilation time.


Reviewers: doug.gregor, klimek, rsmith, djasper

Reviewed By: doug.gregor

CC: cfe-commits, chandlerc

Differential Revision: http://llvm-reviews.chandlerc.com/D28

llvm-svn: 162810
2012-08-29 00:20:03 +00:00
..
AnalysisBasedWarnings.cpp [ms-inline asm] Remove old cruft now that MS-style asms their own code path. 2012-08-20 17:11:53 +00:00
AttributeList.cpp Add support for "type safety" attributes that allow checking that 'void *' 2012-08-17 00:08:38 +00:00
CMakeLists.txt [ms-inline asm] Extract AsmStmt handling into a separate file, so as to not 2012-08-17 21:19:40 +00:00
CodeCompleteConsumer.cpp Add a new libclang completion API to get brief documentation comment that is 2012-07-02 17:35:10 +00:00
DeclSpec.cpp Splitting the duplicated decl spec extension warning into two: one is an ExtWarn and the other a vanilla warning. This addresses PR13705, where const char const * wouldn't warn unless -pedantic was specified under the right conditions. 2012-08-28 20:55:40 +00:00
DelayedDiagnostic.cpp objc: When issue diagnostic about deprecated method, also 2012-03-02 21:50:02 +00:00
IdentifierResolver.cpp Revert Decl's iterators back to pointer value_type rather than reference value_type 2012-06-06 20:45:41 +00:00
JumpDiagnostics.cpp Unify naming of LangOptions variable/get function across the Clang stack (Lex to AST). 2012-03-11 07:00:24 +00:00
Makefile BUILD_ARCHIVE is the default for libraries, no need to set it. 2010-07-18 00:14:47 +00:00
Scope.cpp Reapply r151638 and r151641. 2012-02-29 10:24:19 +00:00
Sema.cpp Now that ASTMultiPtr is nothing more than a array reference, make it a MutableArrayRef. 2012-08-23 23:38:35 +00:00
SemaAccess.cpp When computing the effective context for access control, 2012-08-24 22:54:02 +00:00
SemaAttr.cpp Unify naming of LangOptions variable/get function across the Clang stack (Lex to AST). 2012-03-11 07:00:24 +00:00
SemaCXXScopeSpec.cpp Move Sema::RequireCompleteType() and Sema::RequireCompleteExprType() 2012-05-04 16:32:21 +00:00
SemaCast.cpp Now that ASTMultiPtr is nothing more than a array reference, make it a MutableArrayRef. 2012-08-23 23:38:35 +00:00
SemaChecking.cpp Warn about suspicious implicit conversions from floating point to bool 2012-08-28 15:44:30 +00:00
SemaCodeComplete.cpp Keep history of macro definitions and #undefs 2012-08-29 00:20:03 +00:00
SemaConsumer.cpp Unweaken vtables as per http://llvm.org/docs/CodingStandards.html#ll_virtual_anch 2011-12-20 02:48:34 +00:00
SemaDecl.cpp CUDA: give static storage class to __shared__ and __constant__ 2012-08-28 20:37:50 +00:00
SemaDeclAttr.cpp objective-C arc: ns_returns_retained is a type attribute in ARC, 2012-08-28 22:26:21 +00:00
SemaDeclCXX.cpp Push ArrayRef through the Expr hierarchy. 2012-08-24 11:54:20 +00:00
SemaDeclObjC.cpp objective-C: Do not warn if align attribute on method 2012-08-24 23:50:13 +00:00
SemaExceptionSpec.cpp Store SourceManager pointer on PrintingPolicy in the case where we're dumping, 2012-08-16 03:56:14 +00:00
SemaExpr.cpp Push ArrayRef through the Expr hierarchy. 2012-08-24 11:54:20 +00:00
SemaExprCXX.cpp Push ArrayRef through the Expr hierarchy. 2012-08-24 11:54:20 +00:00
SemaExprMember.cpp Rip out remnants of move semantic emulation and smart pointers in Sema. 2012-08-23 21:35:17 +00:00
SemaExprObjC.cpp Push ArrayRef through the Expr hierarchy. 2012-08-24 11:54:20 +00:00
SemaFixItUtils.cpp Drop the ASTContext.h include from Stmt.h and fix up transitive users. 2012-07-04 17:04:04 +00:00
SemaInit.cpp Push ArrayRef through the Expr hierarchy. 2012-08-24 11:54:20 +00:00
SemaLambda.cpp Provide isConst/Volatile on CXXMethodDecl. 2012-08-10 00:55:35 +00:00
SemaLookup.cpp Remove RequireCompleteType call which was made redundant by r162586. 2012-08-24 23:51:39 +00:00
SemaObjCProperty.cpp objective-C: When checking for valid overriden property 2012-08-24 20:10:53 +00:00
SemaOverload.cpp Instantiate class template specializations during ADL. 2012-08-24 20:38:34 +00:00
SemaPseudoObject.cpp Push ArrayRef through the Expr hierarchy. 2012-08-24 11:54:20 +00:00
SemaStmt.cpp Now that ASTMultiPtr is nothing more than a array reference, make it a MutableArrayRef. 2012-08-23 23:38:35 +00:00
SemaStmtAsm.cpp Revert r162424 now that the scoping issue has been fixed in r162444. 2012-08-28 22:08:58 +00:00
SemaStmtAttr.cpp Inline storage of attributes in AttributedStmt. 2012-07-09 10:04:07 +00:00
SemaTemplate.cpp Now that ASTMultiPtr is nothing more than a array reference, make it a MutableArrayRef. 2012-08-23 23:38:35 +00:00
SemaTemplateDeduction.cpp Fix a bunch of -Wdocumentation warnings. 2012-08-23 17:58:28 +00:00
SemaTemplateInstantiate.cpp Rip out remnants of move semantic emulation and smart pointers in Sema. 2012-08-23 21:35:17 +00:00
SemaTemplateInstantiateDecl.cpp Remove ASTOwningVector, it doesn't own anything and provides no value over SmallVector. 2012-08-23 22:51:59 +00:00
SemaTemplateVariadic.cpp PR12057: Allow variadic template pack expansions to cross lambda boundaries. 2012-07-25 03:56:55 +00:00
SemaType.cpp Fix undefined behavior: don't call ObjCInterfaceDecl::isArcWeakrefUnavailable 2012-08-23 04:53:18 +00:00
TargetAttributesSema.cpp Reapply r158700 and fixup patches, minus one hunk that slipped through and 2012-06-19 23:57:03 +00:00
TargetAttributesSema.h Remove Sema.h's dependency on DeclCXX.h. 2010-08-25 07:42:41 +00:00
TreeTransform.h Fix for assertion when solving unresolved templates. 2012-08-28 04:13:54 +00:00
TypeLocBuilder.h Remove more unnecessary #include <llvm/ADT/SmallVector.h> 2011-06-20 06:11:46 +00:00