Commit Graph

74 Commits

Author SHA1 Message Date
Ben Langmuir 090610d37a Initial implementation of virtual file system
This adds the minimum virtual file system support to start migrating
FileManager onto the VFS.

Originally discussed here:
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-February/035188.html

Differential Revision: http://llvm-reviews.chandlerc.com/D2745

llvm-svn: 201618
2014-02-19 00:10:30 +00:00
Dmitri Gribenko f430da4de6 Add an option to allow Clang verify source files for a module only once during
the build

When Clang loads the module, it verifies the user source files that the module
was built from.  If any file was changed, the module is rebuilt.  There are two
problems with this:
1. correctness: we don't verify system files (there are too many of them, and
   stat'ing all of them would take a lot of time);
2. performance: the same module file is verified again and again during a
   single build.

This change allows the build system to optimize source file verification.  The
idea is based on the fact that while the project is being built, the source
files don't change.  This allows us to verify the module only once during a
single build session.  The build system passes a flag,
-fbuild-session-timestamp=, to inform Clang of the time when the build started.
The build system also requests to enable this feature by passing
-fmodules-validate-once-per-build-session.  If these flags are not passed, the
behavior is not changed.  When Clang verifies the module the first time, it
writes out a timestamp file.  Then, when Clang loads the module the second
time, it finds a timestamp file, so it can compare the verification timestamp
of the module with the time when the build started.  If the verification
timestamp is too old, the module is verified again, and the timestamp file is
updated.

llvm-svn: 201224
2014-02-12 10:33:14 +00:00
Eli Friedman c27d0d5ef2 Add a bit more info to modules fatal error.
Just a minor tweak to make it easier to track down the cause of fatal errors
with modules.

llvm-svn: 190108
2013-09-05 23:50:58 +00:00
Benjamin Kramer 45025c0cf1 This wasn't headers, just missing namespaces.
/me bows head in shame.

llvm-svn: 189172
2013-08-24 13:22:59 +00:00
Benjamin Kramer af04f98c11 Add missing includes.
llvm-svn: 189171
2013-08-24 13:16:22 +00:00
Robert Wilhelm 25284cc95b Use pop_back_val() instead of both back() and pop_back().
No functionality change intended.

llvm-svn: 189112
2013-08-23 16:11:15 +00:00
Rafael Espindola 552c169ed3 Include Path.h instead of PathV2.h.
I am about to move PathV2.h to Path.h.

llvm-svn: 183795
2013-06-11 22:15:02 +00:00
Douglas Gregor bf7fc9c542 <rdar://problem/13509689> Introduce -module-file-info option that provides information about a particular module file.
This option can be useful for end users who want to know why they
ended up with a ton of different variants of the "std" module in their
module cache. This problem should go away over time, as we reduce the
need for module variants, but it will never go away entirely.

llvm-svn: 178148
2013-03-27 16:47:18 +00:00
Douglas Gregor 603cd869f7 <rdar://problem/13479539> Simplify ModuleManager/GlobalModuleIndex interaction to eliminate a pile of extraneous stats().
The refactoring in r177367 introduced a serious performance bug where
the "lazy" resolution of module file names in the global module index
to actual module file entries in the module manager would perform
repeated negative stats(). The new interaction requires the module
manager to inform the global module index when a module file has been
loaded, eliminating the extraneous stat()s and a bunch of bookkeeping
on both sides.

llvm-svn: 177750
2013-03-22 18:50:14 +00:00
Benjamin Kramer 3d0235e16e Remove unused variable.
llvm-svn: 177657
2013-03-21 19:47:38 +00:00
Douglas Gregor 7029ce1a0c <rdar://problem/13363214> Eliminate race condition between module rebuild and the global module index.
The global module index was querying the file manager for each of the
module files it knows about at load time, to prune out any out-of-date
information. The file manager would then cache the results of the
stat() falls used to find that module file.

Later, the same translation unit could end up trying to import one of the
module files that had previously been ignored by the module cache, but
after some other Clang instance rebuilt the module file to bring it
up-to-date. The stale stat() results in the file manager would
trigger a second rebuild of the already-up-to-date module, causing
failures down the line.

The global module index now lazily resolves its module file references
to actual AST reader module files only after the module file has been
loaded, eliminating the stat-caching race. Moreover, the AST reader
can communicate to its caller that a module file is missing (rather
than simply being out-of-date), allowing us to simplify the
module-loading logic and allowing the compiler to recover if a
dependent module file ends up getting deleted.

llvm-svn: 177367
2013-03-19 00:28:20 +00:00
Douglas Gregor dadd85dc0c Never cache the result of a module file lookup.
llvm-svn: 174744
2013-02-08 21:27:45 +00:00
Douglas Gregor e97cd90a67 Eliminate memory allocation from most invocations of
ModuleManager::visit() by keeping a free list of the two data
structures used to store state (a preallocated stack and a visitation
number vector). Improves -fsyntax-only performance for my modules test
case by 2.8%. Modules has pulled ahead by almost 10% with the global
module index.

llvm-svn: 173692
2013-01-28 16:46:33 +00:00
Douglas Gregor 7211ac15bb Improve coordination between the module manager and the global module
index, optimizing the operation that skips lookup in modules where we
know the identifier will not be found. This makes the global module
index optimization actually useful, providing an 8.5% speedup over
modules without the global module index for -fsyntax-only.

llvm-svn: 173529
2013-01-25 23:32:03 +00:00
Douglas Gregor e41d7feaf5 Optimize ModuleManager::visit() by precomputing the visitation order
and limiting ourselves to two memory allocations. 10% speedup in
-fsyntax-only time for modules.

With this change, we can actually see some performance different from
the global module index, but it's still about 1%.

llvm-svn: 173512
2013-01-25 22:25:23 +00:00
Douglas Gregor e060e57bf7 Implement the reader of the global module index and wire it into the
AST reader.

The global module index tracks all of the identifiers known to a set
of module files. Lookup of those identifiers looks first in the global
module index, which returns the set of module files in which that
identifier can be found. The AST reader only needs to look into those
module files and any module files not known to the global index (e.g.,
because they were (re)built after the global index), reducing the
number of on-disk hash tables to visit. For an example source I'm
looking at, we go from 237844 total identifier lookups into on-disk
hash tables down to 126817.

Unfortunately, this does not translate into a performance advantage.
At best, it's a wash once the global module index has been built, but
that's ignore the cost of building the global module index (which
is itself fairly large). Profiles show that the global module index
code is far less efficient than it should be; optimizing it might give
enough of an advantage to justify its continued inclusion.

llvm-svn: 173405
2013-01-25 01:03:03 +00:00
Douglas Gregor bdb259d223 Give ModuleFiles an index, so that we can use indexed vectors rather
than DenseMaps and SmallPtrSets for module-visitation data. ~2.6%
speedup for modules.

llvm-svn: 173081
2013-01-21 20:07:12 +00:00
Douglas Gregor 6fb03aeae1 Actually keep track of the source locations at which particular module
files are loaded.

llvm-svn: 169027
2012-11-30 19:28:05 +00:00
Douglas Gregor 188dbef26d When loading a module fails because it is out of date, rebuild that
module in place. <rdar://problem/10138913>

llvm-svn: 167539
2012-11-07 17:46:15 +00:00
Argyrios Kyrtzidis aedf7144d1 Set the file entry for a Module* that was created during deserialization
of a module file.

llvm-svn: 165086
2012-10-03 01:58:42 +00:00
Douglas Gregor 4fc9f3e819 Optimize unqualified/global name lookup in modules by introducing a
generational scheme for identifiers that avoids searching the hash
tables of a given module more than once for a given
identifier. Previously, loading any new module invalidated all of the
previous lookup results for all identifiers, causing us to perform the
lookups repeatedly.

llvm-svn: 148412
2012-01-18 20:56:22 +00:00
Douglas Gregor de3ef502a9 Promote ModuleMap::Module to a namespace-scope class in the Basic
library, since modules cut across all of the libraries. Rename
serialization::Module to serialization::ModuleFile to side-step the
annoying naming conflict. Prune a bunch of ModuleMap.h includes that
are no longer needed (most files only needed the Module type).

llvm-svn: 145538
2011-11-30 23:21:26 +00:00
Douglas Gregor 9d7c1a2a18 Add support for viewing the module graph via Graphviz, for debugging
purposes.

llvm-svn: 141697
2011-10-11 19:27:55 +00:00
Douglas Gregor d44252ec03 Factor the Module and ModuleManager classes out into separate headers
and .cpp files, since ASTReader.cpp was getting way too large. No
functionality change.

llvm-svn: 138582
2011-08-25 20:47:51 +00:00