[PCH] When pre-validating the headers from the PCH, only validate non-system headers.

Stat'ing all the headers from the PCH to make sure they are up-to-date takes significant time.
In a particular source file (whose PCH file included Cocoa.h) from total -fsyntax-only time
12% was just stat calls. Change pre-validation to only check non-system headers.

There are some notable disadvantages:

-If a system header, that is not include-guarded, changes after the PCH was created, we will not
find it in the header info table and we will #import it, effectively #importing it twice, thus
we will emit some error due to a multiple definition and after that the "header was modified" error will likely
be emitted, for example something like:

NSDictionary.h:12:1: error: duplicate interface definition for class 'NSDictionary'
@interface NSDictionary : NSObject <NSCopying, NSMutableCopying, NSSecureCoding, NSFastEnumeration>
^
NSDictionary.h:12:12: note: previous definition is here
@interface NSDictionary : NSObject <NSCopying, NSMutableCopying, NSSecureCoding, NSFastEnumeration>
           ^
fatal error: file 'NSDictionary.h' has been modified since the precompiled header was built

Though we get the "header was modified" error, this is a bit confusing.

-Theoretically it is possible that such a system header will cause no errors but it will just cause an
unfortunate semantic change, though I find this rather unlikely.

The advantages:

-Reduces compilation time when using a huge PCH like the Cocoa ones
-System headers change very infrequent and when they do, users/build systems should be able to know that
re-building from scratch is needed.

Addresses rdar://13056262

llvm-svn: 176567

clang/lib/Serialization/ASTReader.cpp

@@ -1680,10 +1680,12 @@ ASTReader::ReadControlBlock(ModuleFile &F,
       Error("malformed block record in AST file");
       return Failure;
     case llvm::BitstreamEntry::EndBlock:
-      // Validate all of the input files.
+      // Validate all of the non-system input files.
       if (!DisableValidation) {
         bool Complain = (ClientLoadCapabilities & ARR_OutOfDate) == 0;
-        for (unsigned I = 0, N = Record[0]; I < N; ++I) {
+        // All user input files reside at the index range [0, Record[1]).
+        // Record is the one from INPUT_FILE_OFFSETS.
+        for (unsigned I = 0, N = Record[1]; I < N; ++I) {
           InputFile IF = getInputFile(F, I+1, Complain);
           if (!IF.getFile() || IF.isOutOfDate())
             return OutOfDate;

clang/lib/Serialization/ASTWriter.cpp

@@ -1232,8 +1232,9 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, StringRef isysroot) {
   IFAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob)); // File name
   unsigned IFAbbrevCode = Stream.EmitAbbrev(IFAbbrev);
 
-  // Write out all of the input files.
-  std::vector<uint32_t> InputFileOffsets;
+  // Get all ContentCache objects for files, sorted by whether the file is a
+  // system one or not. System files go at the back, users files at the front.
+  std::deque<const SrcMgr::ContentCache *> SortedFiles;
   for (unsigned I = 1, N = SourceMgr.local_sloc_entry_size(); I != N; ++I) {
     // Get this source location entry.
     const SrcMgr::SLocEntry *SLoc = &SourceMgr.getLocalSLocEntry(I);
@@ -1246,6 +1247,19 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, StringRef isysroot) {
     if (!Cache->OrigEntry)
       continue;
 
+    if (Cache->IsSystemFile)
+      SortedFiles.push_back(Cache);
+    else
+      SortedFiles.push_front(Cache);
+  }
+
+  unsigned UserFilesNum = 0;
+  // Write out all of the input files.
+  std::vector<uint32_t> InputFileOffsets;
+  for (std::deque<const SrcMgr::ContentCache *>::iterator
+         I = SortedFiles.begin(), E = SortedFiles.end(); I != E; ++I) {
+    const SrcMgr::ContentCache *Cache = *I;
+
     uint32_t &InputFileID = InputFileIDs[Cache->OrigEntry];
     if (InputFileID != 0)
       continue; // already recorded this file.
@@ -1255,6 +1269,9 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, StringRef isysroot) {
 
     InputFileID = InputFileOffsets.size();
 
+    if (!Cache->IsSystemFile)
+      ++UserFilesNum;
+
     Record.clear();
     Record.push_back(INPUT_FILE);
     Record.push_back(InputFileOffsets.size());
@@ -1290,6 +1307,8 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, StringRef isysroot) {
   BitCodeAbbrev *OffsetsAbbrev = new BitCodeAbbrev();
   OffsetsAbbrev->Add(BitCodeAbbrevOp(INPUT_FILE_OFFSETS));
   OffsetsAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6)); // # input files
+  OffsetsAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::VBR, 6)); // # non-system
+                                                                //   input files
   OffsetsAbbrev->Add(BitCodeAbbrevOp(BitCodeAbbrevOp::Blob));   // Array
   unsigned OffsetsAbbrevCode = Stream.EmitAbbrev(OffsetsAbbrev);
 
@@ -1297,6 +1316,7 @@ void ASTWriter::WriteInputFiles(SourceManager &SourceMgr, StringRef isysroot) {
   Record.clear();
   Record.push_back(INPUT_FILE_OFFSETS);
   Record.push_back(InputFileOffsets.size());
+  Record.push_back(UserFilesNum);
   Stream.EmitRecordWithBlob(OffsetsAbbrevCode, Record, data(InputFileOffsets));
 }