Commit Graph

39 Commits

Author SHA1 Message Date
Rafael Espindola c080917ec2 Replace llvm::error_code with std::error_code.
llvm-svn: 210780
2014-06-12 14:02:15 +00:00
Ben Langmuir 4f05478c97 Invalidate the file system cache entries for files that may rebuild
This reapplies r209910 with a fix for the assertion failures hit on the
buildbots.

original commit message:
I thought we could get away without this, but it means that the
FileEntry objects actually refer to the wrong files, since pcms are not
updated inplace, they are atomically renamed into place after compiling
a module.

So we are close to the original behaviour of invalidating the cache for
all modules being removed, but now we should only invalidate the ones
that depend on whichever module failed to load.

Unfortunately I haven't come up with a new test that didn't require
a race between parallel invocations of clang.

<rdar://problem/17038180>

llvm-svn: 209922
2014-05-30 21:20:54 +00:00
Ben Langmuir e45b4e328f Revert "Invalidate the file system cache entries for files that may rebuild"
This reverts commit r209910, which is breaking some of the bots.

llvm-svn: 209911
2014-05-30 17:54:59 +00:00
Ben Langmuir 04b9c70da6 Invalidate the file system cache entries for files that may rebuild
I thought we could get away without this, but it means that the
FileEntry objects actually refer to the wrong files, since pcms are not
updated inplace, they are atomically renamed into place after compiling
a module.

So we are close to the original behaviour of invalidating the cache for
all modules being removed, but now we should only invalidate the ones
that depend on whichever module failed to load.

Unfortunately I haven't come up with a new test that didn't require
a race between parallel invocations of clang.

<rdar://problem/17038180>

llvm-svn: 209910
2014-05-30 17:42:15 +00:00
Craig Topper a13603a247 [C++11] Use 'nullptr'. Serialization edition.
llvm-svn: 209392
2014-05-22 05:54:18 +00:00
Ben Langmuir 5988590363 Speculative fix for Windows buildbot after r209138
It appears that Windows doesn't like renaming over open files, which we
do in clearOutputFiles. The file being compiled should be safe to
removed, but this isn't very satisfying - we don't want to manually
manage the lifetime of files we cannot prove have no references.

llvm-svn: 209195
2014-05-20 05:55:04 +00:00
Ben Langmuir caea13185e Don't refresh stat() info for pcm files
Follow-up fix for 209138.  Actually, since we already have this file
open, we don't want to refresh the stat() info, since that might be
newer than what we have open (bad!).

llvm-svn: 209143
2014-05-19 17:04:28 +00:00
Ben Langmuir ca39214f4e Fix use-after-free and spurious error during module load
FileManager::invalidateCache is not safe to call when there may be
existing references to the file. What module load failure needs is
to refresh so stale stat() info isn't stored.

This may be the last user of invalidateCache; I'll take a look and
remove it if possible in a future commit.

This caused a use-after-free error as well as a spurious error message
that a module was "found in both 'X.pcm' and 'X.pcm'" in some cases.

llvm-svn: 209138
2014-05-19 16:13:45 +00:00
Ben Langmuir 027731d7b5 Fix a use-after-free bug I recently introduced in lookupModuleFile
llvm-svn: 207932
2014-05-04 05:20:54 +00:00
Ben Langmuir 05f82ba252 Avoid a potential race between stat() and open() of ASTFile
We need to open an ASTFile while checking its expected size and
modification time, or another clang instance can modify the file between
the stat() and the open().

llvm-svn: 207735
2014-05-01 03:33:36 +00:00
Ben Langmuir beee15e721 Allow multiple modules with the same name to coexist in the module cache
To differentiate between two modules with the same name, we will
consider the path the module map file that they are defined by* part of
the ‘key’ for looking up the precompiled module (pcm file).
Specifically, this patch renames the precompiled module (pcm) files from
  cache-path/<module hash>/Foo.pcm
to
  cache-path/<module hash>/Foo-<hash of module map path>.pcm

In addition, I’ve taught the ASTReader to re-resolve the names of
imported modules during module loading so that if the header search
context changes between when a module was originally built and when it
is loaded we can rebuild it if necessary.  For example, if module A
imports module B

first time:
clang -I /path/to/A -I /path/to/B ...

second time:
clang -I /path/to/A -I /different/path/to/B ...

will now rebuild A as expected.

* in the case of inferred modules, we use the module map file that
allowed the inference, not the __inferred_module.map file, since the
inferred file path is the same for every inferred module.

llvm-svn: 206201
2014-04-14 18:00:01 +00:00
Chandler Carruth b55d0226b7 [C++11] Remove a now unnecessary use of std::function for a remove_if
predicate. The wrapper used by SetVector was erroneously requiring an
adaptable predicate. It has been fixed and we really don't want to
require an indirect call for every predicate evaluation.

llvm-svn: 202744
2014-03-03 19:36:27 +00:00
Benjamin Kramer bbdd7640e8 [C++11] Replace verbose functors with succinct lambdas
No functionality change.

llvm-svn: 202590
2014-03-01 14:48:57 +00:00
Ben Langmuir c8130a74f4 Recommit virtual file system
Previously reverted in r201755 due to causing an assertion failure.

I've removed the offending assertion, and taught the CompilerInstance to
create a default virtual file system inside createFileManager. In the
future, we should be able to reach into the CompilerInvocation to
customize this behaviour without breaking clients that don't care.

llvm-svn: 201818
2014-02-20 21:59:23 +00:00
Juergen Ributzka a32575e4f6 Reverting the virtual file system implementation, because it triggers an assertion
in our internal build bots.

This reverts commits 201618, 201635, 201636, 201639, 201685, 201691, and 201696.

llvm-svn: 201755
2014-02-20 05:24:58 +00:00
Ben Langmuir 090610d37a Initial implementation of virtual file system
This adds the minimum virtual file system support to start migrating
FileManager onto the VFS.

Originally discussed here:
http://lists.cs.uiuc.edu/pipermail/cfe-dev/2014-February/035188.html

Differential Revision: http://llvm-reviews.chandlerc.com/D2745

llvm-svn: 201618
2014-02-19 00:10:30 +00:00
Dmitri Gribenko f430da4de6 Add an option to allow Clang verify source files for a module only once during
the build

When Clang loads the module, it verifies the user source files that the module
was built from.  If any file was changed, the module is rebuilt.  There are two
problems with this:
1. correctness: we don't verify system files (there are too many of them, and
   stat'ing all of them would take a lot of time);
2. performance: the same module file is verified again and again during a
   single build.

This change allows the build system to optimize source file verification.  The
idea is based on the fact that while the project is being built, the source
files don't change.  This allows us to verify the module only once during a
single build session.  The build system passes a flag,
-fbuild-session-timestamp=, to inform Clang of the time when the build started.
The build system also requests to enable this feature by passing
-fmodules-validate-once-per-build-session.  If these flags are not passed, the
behavior is not changed.  When Clang verifies the module the first time, it
writes out a timestamp file.  Then, when Clang loads the module the second
time, it finds a timestamp file, so it can compare the verification timestamp
of the module with the time when the build started.  If the verification
timestamp is too old, the module is verified again, and the timestamp file is
updated.

llvm-svn: 201224
2014-02-12 10:33:14 +00:00
Eli Friedman c27d0d5ef2 Add a bit more info to modules fatal error.
Just a minor tweak to make it easier to track down the cause of fatal errors
with modules.

llvm-svn: 190108
2013-09-05 23:50:58 +00:00
Benjamin Kramer 45025c0cf1 This wasn't headers, just missing namespaces.
/me bows head in shame.

llvm-svn: 189172
2013-08-24 13:22:59 +00:00
Benjamin Kramer af04f98c11 Add missing includes.
llvm-svn: 189171
2013-08-24 13:16:22 +00:00
Robert Wilhelm 25284cc95b Use pop_back_val() instead of both back() and pop_back().
No functionality change intended.

llvm-svn: 189112
2013-08-23 16:11:15 +00:00
Rafael Espindola 552c169ed3 Include Path.h instead of PathV2.h.
I am about to move PathV2.h to Path.h.

llvm-svn: 183795
2013-06-11 22:15:02 +00:00
Douglas Gregor bf7fc9c542 <rdar://problem/13509689> Introduce -module-file-info option that provides information about a particular module file.
This option can be useful for end users who want to know why they
ended up with a ton of different variants of the "std" module in their
module cache. This problem should go away over time, as we reduce the
need for module variants, but it will never go away entirely.

llvm-svn: 178148
2013-03-27 16:47:18 +00:00
Douglas Gregor 603cd869f7 <rdar://problem/13479539> Simplify ModuleManager/GlobalModuleIndex interaction to eliminate a pile of extraneous stats().
The refactoring in r177367 introduced a serious performance bug where
the "lazy" resolution of module file names in the global module index
to actual module file entries in the module manager would perform
repeated negative stats(). The new interaction requires the module
manager to inform the global module index when a module file has been
loaded, eliminating the extraneous stat()s and a bunch of bookkeeping
on both sides.

llvm-svn: 177750
2013-03-22 18:50:14 +00:00
Benjamin Kramer 3d0235e16e Remove unused variable.
llvm-svn: 177657
2013-03-21 19:47:38 +00:00
Douglas Gregor 7029ce1a0c <rdar://problem/13363214> Eliminate race condition between module rebuild and the global module index.
The global module index was querying the file manager for each of the
module files it knows about at load time, to prune out any out-of-date
information. The file manager would then cache the results of the
stat() falls used to find that module file.

Later, the same translation unit could end up trying to import one of the
module files that had previously been ignored by the module cache, but
after some other Clang instance rebuilt the module file to bring it
up-to-date. The stale stat() results in the file manager would
trigger a second rebuild of the already-up-to-date module, causing
failures down the line.

The global module index now lazily resolves its module file references
to actual AST reader module files only after the module file has been
loaded, eliminating the stat-caching race. Moreover, the AST reader
can communicate to its caller that a module file is missing (rather
than simply being out-of-date), allowing us to simplify the
module-loading logic and allowing the compiler to recover if a
dependent module file ends up getting deleted.

llvm-svn: 177367
2013-03-19 00:28:20 +00:00
Douglas Gregor dadd85dc0c Never cache the result of a module file lookup.
llvm-svn: 174744
2013-02-08 21:27:45 +00:00
Douglas Gregor e97cd90a67 Eliminate memory allocation from most invocations of
ModuleManager::visit() by keeping a free list of the two data
structures used to store state (a preallocated stack and a visitation
number vector). Improves -fsyntax-only performance for my modules test
case by 2.8%. Modules has pulled ahead by almost 10% with the global
module index.

llvm-svn: 173692
2013-01-28 16:46:33 +00:00
Douglas Gregor 7211ac15bb Improve coordination between the module manager and the global module
index, optimizing the operation that skips lookup in modules where we
know the identifier will not be found. This makes the global module
index optimization actually useful, providing an 8.5% speedup over
modules without the global module index for -fsyntax-only.

llvm-svn: 173529
2013-01-25 23:32:03 +00:00
Douglas Gregor e41d7feaf5 Optimize ModuleManager::visit() by precomputing the visitation order
and limiting ourselves to two memory allocations. 10% speedup in
-fsyntax-only time for modules.

With this change, we can actually see some performance different from
the global module index, but it's still about 1%.

llvm-svn: 173512
2013-01-25 22:25:23 +00:00
Douglas Gregor e060e57bf7 Implement the reader of the global module index and wire it into the
AST reader.

The global module index tracks all of the identifiers known to a set
of module files. Lookup of those identifiers looks first in the global
module index, which returns the set of module files in which that
identifier can be found. The AST reader only needs to look into those
module files and any module files not known to the global index (e.g.,
because they were (re)built after the global index), reducing the
number of on-disk hash tables to visit. For an example source I'm
looking at, we go from 237844 total identifier lookups into on-disk
hash tables down to 126817.

Unfortunately, this does not translate into a performance advantage.
At best, it's a wash once the global module index has been built, but
that's ignore the cost of building the global module index (which
is itself fairly large). Profiles show that the global module index
code is far less efficient than it should be; optimizing it might give
enough of an advantage to justify its continued inclusion.

llvm-svn: 173405
2013-01-25 01:03:03 +00:00
Douglas Gregor bdb259d223 Give ModuleFiles an index, so that we can use indexed vectors rather
than DenseMaps and SmallPtrSets for module-visitation data. ~2.6%
speedup for modules.

llvm-svn: 173081
2013-01-21 20:07:12 +00:00
Douglas Gregor 6fb03aeae1 Actually keep track of the source locations at which particular module
files are loaded.

llvm-svn: 169027
2012-11-30 19:28:05 +00:00
Douglas Gregor 188dbef26d When loading a module fails because it is out of date, rebuild that
module in place. <rdar://problem/10138913>

llvm-svn: 167539
2012-11-07 17:46:15 +00:00
Argyrios Kyrtzidis aedf7144d1 Set the file entry for a Module* that was created during deserialization
of a module file.

llvm-svn: 165086
2012-10-03 01:58:42 +00:00
Douglas Gregor 4fc9f3e819 Optimize unqualified/global name lookup in modules by introducing a
generational scheme for identifiers that avoids searching the hash
tables of a given module more than once for a given
identifier. Previously, loading any new module invalidated all of the
previous lookup results for all identifiers, causing us to perform the
lookups repeatedly.

llvm-svn: 148412
2012-01-18 20:56:22 +00:00
Douglas Gregor de3ef502a9 Promote ModuleMap::Module to a namespace-scope class in the Basic
library, since modules cut across all of the libraries. Rename
serialization::Module to serialization::ModuleFile to side-step the
annoying naming conflict. Prune a bunch of ModuleMap.h includes that
are no longer needed (most files only needed the Module type).

llvm-svn: 145538
2011-11-30 23:21:26 +00:00
Douglas Gregor 9d7c1a2a18 Add support for viewing the module graph via Graphviz, for debugging
purposes.

llvm-svn: 141697
2011-10-11 19:27:55 +00:00
Douglas Gregor d44252ec03 Factor the Module and ModuleManager classes out into separate headers
and .cpp files, since ASTReader.cpp was getting way too large. No
functionality change.

llvm-svn: 138582
2011-08-25 20:47:51 +00:00