Commit Graph

16 Commits

Author SHA1 Message Date
Douglas Gregor 09b6989ef0 Implement two related optimizations that make de-serialization of
AST/PCH files more lazy:
  - Don't preload all of the file source-location entries when reading
  the AST file. Instead, load them lazily, when needed.
  - Only look up header-search information (whether a header was already
  #import'd, how many times it's been included, etc.) when it's needed
  by the preprocessor, rather than pre-populating it.

Previously, we would pre-load all of the file source-location entries,
which also populated the header-search information structure. This was
a relatively minor performance issue, since we would end up stat()'ing
all of the headers stored within a AST/PCH file when the AST/PCH file
was loaded. In the normal PCH use case, the stat()s were cached, so
the cost--of preloading ~860 source-location entries in the Cocoa.h
case---was relatively low.

However, the recent optimization that replaced stat+open with
open+fstat turned this into a major problem, since the preloading of
source-location entries would now end up opening those files. Worse,
those files wouldn't be closed until the file manager was destroyed,
so just opening a Cocoa.h PCH file would hold on to ~860 file
descriptors, and it was easy to blow through the process's limit on
the number of open file descriptors.

By eliminating the preloading of these files, we neither open nor stat
the headers stored in the PCH/AST file until they're actually needed
for something. Concretely, we went from

*** HeaderSearch Stats:
835 files tracked.
  364 #import/#pragma once files.
  823 included exactly once.
  6 max times a file is included.
  3 #include/#include_next/#import.
    0 #includes skipped due to the multi-include optimization.
1 framework lookups.
0 subframework lookups.

*** Source Manager Stats:
835 files mapped, 3 mem buffers mapped.
37460 SLocEntry's allocated, 11215575B of Sloc address space used.
62 bytes of files mapped, 0 files with line #'s computed.

with a trivial program that uses a chained PCH including a Cocoa PCH
to

*** HeaderSearch Stats:
4 files tracked.
  1 #import/#pragma once files.
  3 included exactly once.
  2 max times a file is included.
  3 #include/#include_next/#import.
    0 #includes skipped due to the multi-include optimization.
1 framework lookups.
0 subframework lookups.

*** Source Manager Stats:
3 files mapped, 3 mem buffers mapped.
37460 SLocEntry's allocated, 11215575B of Sloc address space used.
62 bytes of files mapped, 0 files with line #'s computed.

for the same program.

llvm-svn: 125286
2011-02-10 17:09:37 +00:00
Michael J. Spencer f6efe58d45 Replace all uses of PathV1::exists with PathV2::fs::exists.
llvm-svn: 123150
2011-01-10 02:34:13 +00:00
Michael J. Spencer f28df4cdba Replace all uses of PathV1::isAbsolute with PathV2::is_{absolute,relative}.
llvm-svn: 122087
2010-12-17 21:22:22 +00:00
Michael J. Spencer 8aaf49959c Merge System into Support.
llvm-svn: 120297
2010-11-29 18:12:39 +00:00
Chris Lattner 5159f6162e now the FileManager has a FileSystemOpts ivar, stop threading
FileSystemOpts through a ton of apis, simplifying a lot of code.
This also fixes a latent bug in ASTUnit where it would invoke
methods on FileManager without creating one in some code paths
in cindextext.

llvm-svn: 120010
2010-11-23 08:35:12 +00:00
Chris Lattner 8afa6deb1d remove old compatibility APIs, use StringRef versions instead.
llvm-svn: 119935
2010-11-21 09:55:08 +00:00
Argyrios Kyrtzidis 71731d6b05 Implement -working-directory.
When -working-directory is passed in command line, file paths are resolved relative to the specified directory.
This helps both when using libclang (where we can't require the user to actually change the working directory)
and to help reproduce test cases when the reproduction work comes along.

--FileSystemOptions is introduced which controls how file system operations are performed (currently it just contains
 the working directory value if set).
--FileSystemOptions are passed around to various interfaces that perform file operations.
--Opening & reading the content of files should be done only through FileManager. This is useful in general since
 file operations will be abstracted in the future for the reproduction mechanism.

FileSystemOptions is independent of FileManager so that we can have multiple translation units sharing the same
FileManager but with different FileSystemOptions.

Addresses rdar://8583824.

llvm-svn: 118203
2010-11-03 22:45:23 +00:00
Douglas Gregor 618e64a23b Revert r110440, the fix for PR4897. Chris claims to have a better way.
llvm-svn: 110544
2010-08-08 07:49:23 +00:00
Douglas Gregor d26129a98a Fix the #include search path when reading from stdin, from Jon Simons!
Fixes PR4897.

llvm-svn: 110440
2010-08-06 12:06:13 +00:00
Chris Lattner d081f8c851 stringref'ize a bunch of filename handling logic. Much
nicer than passing around two const char*'s.

llvm-svn: 93094
2010-01-10 01:35:12 +00:00
Mike Stump 11289f4280 Remove tabs, and whitespace cleanups.
llvm-svn: 81346
2009-09-09 15:08:12 +00:00
Chris Lattner 6b8f78eb03 use the new Path::isAbsolute function, fixing a fixme. Patch by Gregory Curfman!
llvm-svn: 73370
2009-06-15 04:25:18 +00:00
Douglas Gregor 99734e7669 Lazily load the controlling macros for all of the headers known in the
PCH file. In the Cocoa-prefixed "Hello, World" benchmark, this takes
us from reading 503 identifiers down to 37 and from 470 macros down to
4. It also results in an 8% performance improvement.

llvm-svn: 70094
2009-04-25 23:30:02 +00:00
Steve Naroff 3fa455a1aa Add PCH support for #import.
llvm-svn: 69987
2009-04-24 20:03:17 +00:00
Chris Lattner c25d8a7e30 improve compatibility with GCC 4.4, patch by Michel Salim (PR3697)
llvm-svn: 65884
2009-03-02 22:20:04 +00:00
Chris Lattner 7a51313d8a Make a major restructuring of the clang tree: introduce a top-level
lib dir and move all the libraries into it.  This follows the main
llvm tree, and allows the libraries to be built in parallel.  The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in.  This speeds
up parallel builds, particularly incremental ones.

llvm-svn: 48402
2008-03-15 23:59:48 +00:00