When mangling the module map path into a .pcm file name, also mangle the
IsSystem bit, which can also depend on the header search paths. For
example, the user may change from -I to -isystem. This can affect
diagnostics in the importing TU.
llvm-svn: 228966
components. These sometimes get synthetically added, and we don't want -Ifoo
and -I./foo to be treated fundamentally differently here.
llvm-svn: 224055
Original commit message:
[modules] Add experimental -fmodule-map-file-home-is-cwd flag to -cc1.
For files named by -fmodule-map-file=, and files found by 'extern module'
directives, this flag specifies that we should resolve filenames relative to
the current working directory rather than relative to the directory in which
the module map file resides. This is aimed at fixing path handling, in
particular for relative -I paths, when building modules that represent
components of the current project (rather than libraries installed on the
current system, which the current project has as dependencies, where we'd
typically expect the module map files to be looked up implicitly).
llvm-svn: 223913
For files named by -fmodule-map-file=, and files found by 'extern module'
directives, this flag specifies that we should resolve filenames relative to
the current working directory rather than relative to the directory in which
the module map file resides. This is aimed at fixing path handling, in
particular for relative -I paths, when building modules that represent
components of the current project (rather than libraries installed on the
current system, which the current project has as dependencies, where we'd
typically expect the module map files to be looked up implicitly).
llvm-svn: 223753
rather than trying to extract this information from the FileEntry after the
fact.
This has a number of beneficial effects. For instance, diagnostic messages for
failed module builds give a path relative to the "module root" rather than an
absolute file path, and the contents of the module includes file is no longer
dependent on what files the including TU happened to inspect prior to
triggering the module build.
llvm-svn: 223095
According to the gcc docs, -include uses the current working directory
for the lookup instead of the main source file.
This patch gets rid of NormalizeIncludePath (which relied on an
implementation detail of FileManager / FileEntry for the include path
logic to work), and instead hands the correct lookup information down to
LookupFile.
This will allow us to change the FileEntry's behavior regarding its Name
caching.
llvm-svn: 215433
class Module. It's almost always going to be the same as
getContainingModule() for top-level modules, so just add a map to cover
the remaining cases. This lets us do less bookkeeping to keep the
ModuleMap fields up to date.
llvm-svn: 215268
The compilation pipeline doesn't actually need to know about the high-level
concept of diagnostic mappings, and hiding the final computed level presents
several simplifications and other potential benefits.
The only exceptions are opportunistic checks to see whether expensive code
paths can be avoided for diagnostics that are guaranteed to be ignored at a
certain SourceLocation.
This commit formalizes that invariant by introducing and using
DiagnosticsEngine::isIgnored() in place of individual level checks throughout
lex, parse and sema.
llvm-svn: 211005
When using the VFS, we want the virtual header location when searching
for a framework module, since that will be the one in the correct
directory structure for the module.
I'll add a regression test once I finish reducing the larger one I have.
llvm-svn: 208901
To differentiate between two modules with the same name, we will
consider the path the module map file that they are defined by* part of
the ‘key’ for looking up the precompiled module (pcm file).
Specifically, this patch renames the precompiled module (pcm) files from
cache-path/<module hash>/Foo.pcm
to
cache-path/<module hash>/Foo-<hash of module map path>.pcm
In addition, I’ve taught the ASTReader to re-resolve the names of
imported modules during module loading so that if the header search
context changes between when a module was originally built and when it
is loaded we can rebuild it if necessary. For example, if module A
imports module B
first time:
clang -I /path/to/A -I /path/to/B ...
second time:
clang -I /path/to/A -I /different/path/to/B ...
will now rebuild A as expected.
* in the case of inferred modules, we use the module map file that
allowed the inference, not the __inferred_module.map file, since the
inferred file path is the same for every inferred module.
llvm-svn: 206201
We were 'allowing' the following import
@import Sub;
where Sub is a subframework of Foo and we had a -F path inside
Foo.framework/Frameworks and no module map file for Sub. This would
later hit assertion failures in debug builds.
Now we should correctly diagnose this as a module not found error.
llvm-svn: 204368
This name, while more verbose, plays more nicely with tools that use
file extensions to determine file types. The existing spelling
'module.map' will continue to work, but the new spelling will take
precedence.
In frameworks, this new filename will only go in a new 'Modules'
sub-directory.
Similarly, add a module.private.modulemap corresponding to
module_private.map.
llvm-svn: 204261
When building an AST file, we don't want to output HeaderFileInfo
structures for files that are not actually used as headers in the
current context. This can lead to assuming that unrelated files have
include counts of 0, defeating multiple-include prevention.
This is accomplished by adding an IsValid bit to the HFI.
llvm-svn: 203813
This makes Clang and LLVM -Wmsvc-include clean.
I believe the correct behavior here is to avoid updating the cache when
we find the header via MSVC's search rules.
Differential Revision: http://llvm-reviews.chandlerc.com/D2733
llvm-svn: 201615
continue header lookup using the framework include as filename.
This allows us to conveniently treat
#import "Foo.h"
as an implicit module import if we can resolve "Foo/Foo.h" as such.
rdar://16042979
llvm-svn: 201419
encodes the canonical rules for LLVM's style. I noticed this had drifted
quite a bit when cleaning up LLVM, so wanted to clean up Clang as well.
llvm-svn: 198686
Instead, mark the module as unavailable so that clang errors as soon as
someone tries to build this module.
This works towards the long-term goal of not stat'ing the header files at all
while reading the module map and instead read them only when the module is
being built (there is a corresponding FIXME in parseHeaderDecl()). However, it
seems non-trivial to get there and this unblock us and moves us into the right
direction.
Also changed the implementation to reuse the same DiagnosticsEngine.
llvm-svn: 197485
Instead, mark the module as unavailable so that clang errors as soon as
someone tries to build this module.
A better long-term strategy might be to not stat the header files at all
while reading the module map and instead read them only when the module
is being built (there is a corresponding FIXME in parseHeaderDecl()).
However, it seems non-trivial to get there and this would be a temporary
solution to unblock us.
Also changed the implementation to reuse the same DiagnosticsEngine as
otherwise warnings can't be enabled or disabled with command-line flags.
llvm-svn: 197388
This allows using virtual file mappings on the original SourceManager to
map in virtual module.map files. Without this patch, the ModuleMap
search will find a module.map file (as the FileEntry exists in the
FileManager), but will be unable to get the content from the
SourceManager (as ModuleMap previously created its own SourceManager).
Two problems needed to be fixed which this patch exposed:
1. Storing the inferred module map
When writing out a module, the ASTWriter stores the names of the files
in the main source manager; when loading the AST again, the ASTReader
errs out if such a file is found missing, unless it is overridden.
Previously CompilerInstance's compileModule method would store the
inferred module map to a temporary file; the problem with this approach
is that now that the module map is handled by the main source manager,
the ASTWriter stores the name of the temporary module map as source to
the compilation; later, when the module is loaded, the temporary file
has already been deleted, which leads to a compilation error. This patch
changes the inferred module map to instead inject a virtual file into
the source manager. This both saves some disk IO, and works with how the
ASTWriter/ASTReader handle overridden source files.
2. Changing test input in test/Modules/Inputs/*
Now that the module map file is handled by the main source manager, the
VerifyDiagnosticConsumer will not ignore diagnostics created while
parsing the module map file. The module test test/Modules/renamed.m uses
-I test/Modules/Inputs and triggers recursive loading of all module maps
in test/Modules/Inputs, some of which had conflicting names, thus
leading errors while parsing the module maps. Those diagnostics already
occur on trunk, but before this patch they would not break the test, as
they were ignored by the VerifyDiagnosticConsumer. This patch thus
changes the module maps that have been recently introduced which broke
the invariant of compatible modules maps in test/Modules/Inputs.
llvm-svn: 193314
This patch changes two things:
a) Allow a header to be part of multiple modules. The reasoning is that
in existing codebases that have a module-like build system, the same
headers might be used in several build targets. Simple reasons might be
that they defined different classes that are declared in the same
header. Supporting a header as a part of multiple modules will make the
transistion easier for those cases. A later step in clang can then
determine whether the two modules are actually compatible and can be
merged and error out appropriately. The later check is similar to what
needs to be done for template specializations anyway.
b) Allow modules to be stored in a directory tree separate from the
headers they describe.
Review: http://llvm-reviews.chandlerc.com/D1951
llvm-svn: 193151
With this option, arbitrarily named module map files can be specified
to be loaded as required for headers in the respective (sub)directories.
This, together with the extern module declaration allows for specifying
module maps in a modular fashion without the need for files called
"module.map".
Among other things, this allows a directory to contain two modules that
are completely independent of one another.
Review: http://llvm-reviews.chandlerc.com/D1697.
llvm-svn: 191284
Review: http://llvm-reviews.chandlerc.com/D1546.
I have picked up this patch form Lawrence
(http://llvm-reviews.chandlerc.com/D1063) and did a few changes.
From the original change description (updated as appropriate):
This patch adds a check that ensures that modules only use modules they
have so declared. To this end, it adds a statement on intended module
use to the module.map grammar:
use module-id
A module can then only use headers from other modules if it 'uses' them.
This enforcement is off by default, but may be turned on with the new
option -fmodules-decluse.
When enforcing the module semantics, we also need to consider a source
file part of a module. This is achieved with a compiler option
-fmodule-name=<module-id>.
The compiler at present only applies restrictions to the module directly
being built.
llvm-svn: 191283
After r180934 we may initiate module map parsing for modules not related to the module what we are building,
make sure we ignore the header file info of headers from such modules.
First part of rdar://13840148
llvm-svn: 181489
In a module-enabled Cocoa PCH file, we spend a lot of time stat'ing the headers
in order to associate the FileEntries with their modules and support implicit
module import.
Use a more lazy scheme by enhancing HeaderInfoTable to store extra info about
the module that a header belongs to, and associate it with its module only when
there is a request for loading the header info for a particular file.
Part of rdar://13391765
llvm-svn: 176976
factor the realpath calls into FileManager::getCanonicalName() so we
can cache the results of this epically slow operation. 5% speedup on
my modules test, and realpath drops out of the profile.
llvm-svn: 173542
uncovered.
This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.
I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.
llvm-svn: 169237
allowing a module map to be placed one level above the '.framework'
directories to specify that all .frameworks within that directory can
be inferred as framework modules. One can also specifically exclude
frameworks known not to work.
This makes explicit (and more restricted) behavior modules have had
"forever", where *any* .framework was assumed to be able to be built
as a module. That's not necessarily true, so we white-list directories
(with exclusions) when those directories have been audited.
llvm-svn: 167482
top-level frameworks can actually be symlinked over to embedded
frameworks, and accessed via the top-level framework's headers. In
this case, we need to determine that the framework was *actually* an
embedded framework, so we can load the appropriate top-level module.
llvm-svn: 164620
current directory, propagate the framework and in-index-header-map
from the including header's information down to the included header's
information. Fixes <rdar://problem/11261291>.
As with everything header-map related, we can't really test this in
isolation within Clang, so it's tested elsewhere.
llvm-svn: 161759
* Escaped # and < characters in Doxygen comments as needed;
* Removed a Doxygen comment in HeaderSearch.cpp that was redundant with
the corresponding comment in the header file.
llvm-svn: 158776
override whether headers are system headers by checking for prefixes of the
header name specified in the #include directive.
This allows warnings to be disabled for third-party code which is found in
specific subdirectories of include paths.
llvm-svn: 158418
- Developers of system frameworks need a way for their framework to be treated as a "system framework" during development. Otherwise, they are unable to properly test how their framework behaves when installed because of the semantic changes (in warning behavior) applied to system frameworks.
llvm-svn: 154105
for getting the name of the module file, unifying the code for
searching for a module with a given name (into lookupModule()) and
separating out the mapping to a module file (into
getModuleFileName()). No functionality change.
llvm-svn: 149197
single attribute ("system") that allows us to mark a module as being a
"system" module. Each of the headers that makes up a system module is
considered to be a system header, so that we (for example) suppress
warnings there.
If a module is being inferred for a framework, and that framework
directory is within a system frameworks directory, infer it as a
system framework.
llvm-svn: 149143
framework is actually a subframework within a top-level framework. If
so, only infer a module for the top-level framework and then dig out
the appropriate submodule.
This helps us cope with an amusing subframeworks anti-pattern, where
one uses -F <framework>/Frameworks to get direct include access to the
subframeworks of a framework (which otherwise would not be
permitted).
llvm-svn: 148148
features needed for a particular module to be available. This allows
mixed-language modules, where certain headers only work under some
language variants (e.g., in C++, std.tuple might only be available in
C++11 mode).
llvm-svn: 147387
part of HeaderSearch. This function just normalizes filenames for use
inside of a synthetic include directive, but it is used in both the
Frontend and Serialization libraries so it needs a common home.
llvm-svn: 146227
when we load a module map (module.map) from a directory, also load a
private module map (module_private.map) for that directory, if
present. That private module map can inject a new submodule that
captures private headers.
llvm-svn: 146012
most specific (sub)module based on the actual file we find, rather
than always importing the top-level module. This means
that #include'ing <Foo/Blah.h> should give us the submodule Foo.Blah.
llvm-svn: 145942
library, since modules cut across all of the libraries. Rename
serialization::Module to serialization::ModuleFile to side-step the
annoying naming conflict. Prune a bunch of ModuleMap.h includes that
are no longer needed (most files only needed the Module type).
llvm-svn: 145538
return the module itself (in the module map) rather than returning the
umbrella header used to build the module. While doing this, make sure
that we're inferring modules for frameworks to build that module.
llvm-svn: 145310
into a module. This module can either be loaded from a module map in
the framework directory (which isn't quite working yet) or inferred
from an umbrella header (which does work, and replaces the existing
hack).
llvm-svn: 144877
the module is described in one of the module maps in a search path or
in a subdirectory off the search path that has the same name as the
module we're looking for.
llvm-svn: 144433
map, so long as they have an umbrella header. This makes it possible
to introduce a module map + umbrella header for a given set of
headers, to turn it into a module.
There are two major deficiencies here: first, we don't go hunting for
module map files when we just see a module import (so we won't know
about the modules described therein). Second, we don't yet have a way
to build modules that don't have umbrella headers, or have incomplete
umbrella headers.
llvm-svn: 144424
the corresponding (top-level) modules. This isn't actually useful yet,
because we don't yet have a way to build modules out of module maps.
llvm-svn: 144410
Module map files provide a way to map between headers and modules, so
that we can layer a module system on top of existing headers without
changing those headers at all.
This commit introduces the module map file parser and the module map
that it generates, and wires up the module map file parser so that
we'll automatically find module map files as part of header
search. Note that we don't yet use the information stored in the
module map.
llvm-svn: 144402
the AST reader), merge that header file information with whatever
header file information we already have. Otherwise, we might forget
something we already knew (e.g., that the header was #import'd already).
llvm-svn: 139979
target triple to separate modules built under different
conditions. The hash is used to create a subdirectory in the module
cache path where other invocations of the compiler (with the same
version, language options, etc.) can find the precompiled modules.
llvm-svn: 139662
but there is a corresponding umbrella header in a framework, build the
module on-the-fly so it can be immediately loaded at the import
statement. This is very much proof-of-concept code, with details to be
fleshed out over time.
llvm-svn: 139558
where the compiler will look for module files. Eliminates the
egregious hack where we looked into the header search paths for
modules.
llvm-svn: 139538
which determines whether a particular file is actually a header that
is intended to be guarded from multiple inclusions within the same
translation unit.
llvm-svn: 130808
includes get resolved, especially when they are found relatively to
another include file. We also try to get it working for framework
includes, but that part of the code is untested, as I don't have a code
base that uses it.
llvm-svn: 130246
Add 'openFile' bool to FileManager::getFile to specify whether we want to have the file opened or not, have it
false by default, and enable it only in HeaderSearch.cpp where the open+fstat optimization matters.
Fixes rdar://9139899.
llvm-svn: 127748
clients to observe the exact path through which an #included file was
located. This is very useful when trying to record and replay inclusion
operations without it beind influenced by the aggressive caching done
inside the FileManager to avoid redundant system calls and filesystem
operations.
The work to compute and return this is only done in the presence of
callbacks, so it should have no effect on normal compilation.
Patch by Manuel Klimek.
llvm-svn: 127742
AST/PCH files more lazy:
- Don't preload all of the file source-location entries when reading
the AST file. Instead, load them lazily, when needed.
- Only look up header-search information (whether a header was already
#import'd, how many times it's been included, etc.) when it's needed
by the preprocessor, rather than pre-populating it.
Previously, we would pre-load all of the file source-location entries,
which also populated the header-search information structure. This was
a relatively minor performance issue, since we would end up stat()'ing
all of the headers stored within a AST/PCH file when the AST/PCH file
was loaded. In the normal PCH use case, the stat()s were cached, so
the cost--of preloading ~860 source-location entries in the Cocoa.h
case---was relatively low.
However, the recent optimization that replaced stat+open with
open+fstat turned this into a major problem, since the preloading of
source-location entries would now end up opening those files. Worse,
those files wouldn't be closed until the file manager was destroyed,
so just opening a Cocoa.h PCH file would hold on to ~860 file
descriptors, and it was easy to blow through the process's limit on
the number of open file descriptors.
By eliminating the preloading of these files, we neither open nor stat
the headers stored in the PCH/AST file until they're actually needed
for something. Concretely, we went from
*** HeaderSearch Stats:
835 files tracked.
364 #import/#pragma once files.
823 included exactly once.
6 max times a file is included.
3 #include/#include_next/#import.
0 #includes skipped due to the multi-include optimization.
1 framework lookups.
0 subframework lookups.
*** Source Manager Stats:
835 files mapped, 3 mem buffers mapped.
37460 SLocEntry's allocated, 11215575B of Sloc address space used.
62 bytes of files mapped, 0 files with line #'s computed.
with a trivial program that uses a chained PCH including a Cocoa PCH
to
*** HeaderSearch Stats:
4 files tracked.
1 #import/#pragma once files.
3 included exactly once.
2 max times a file is included.
3 #include/#include_next/#import.
0 #includes skipped due to the multi-include optimization.
1 framework lookups.
0 subframework lookups.
*** Source Manager Stats:
3 files mapped, 3 mem buffers mapped.
37460 SLocEntry's allocated, 11215575B of Sloc address space used.
62 bytes of files mapped, 0 files with line #'s computed.
for the same program.
llvm-svn: 125286
FileSystemOpts through a ton of apis, simplifying a lot of code.
This also fixes a latent bug in ASTUnit where it would invoke
methods on FileManager without creating one in some code paths
in cindextext.
llvm-svn: 120010
When -working-directory is passed in command line, file paths are resolved relative to the specified directory.
This helps both when using libclang (where we can't require the user to actually change the working directory)
and to help reproduce test cases when the reproduction work comes along.
--FileSystemOptions is introduced which controls how file system operations are performed (currently it just contains
the working directory value if set).
--FileSystemOptions are passed around to various interfaces that perform file operations.
--Opening & reading the content of files should be done only through FileManager. This is useful in general since
file operations will be abstracted in the future for the reproduction mechanism.
FileSystemOptions is independent of FileManager so that we can have multiple translation units sharing the same
FileManager but with different FileSystemOptions.
Addresses rdar://8583824.
llvm-svn: 118203
PCH file. In the Cocoa-prefixed "Hello, World" benchmark, this takes
us from reading 503 identifiers down to 37 and from 470 macros down to
4. It also results in an 8% performance improvement.
llvm-svn: 70094
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.
llvm-svn: 48402
Ben Langmuir