The refactoring in r177367 introduced a serious performance bug where
the "lazy" resolution of module file names in the global module index
to actual module file entries in the module manager would perform
repeated negative stats(). The new interaction requires the module
manager to inform the global module index when a module file has been
loaded, eliminating the extraneous stat()s and a bunch of bookkeeping
on both sides.
llvm-svn: 177750
Configuration macros are macros that are intended to alter how a
module works, such that we need to build different module variants
for different values of these macros. A module can declare its
configuration macros, in which case we will complain if the definition
of a configation macro on the command line (or lack thereof) differs
from the current preprocessor state at the point where the module is
imported. This should eliminate some surprises when enabling modules,
because "#define CONFIG_MACRO ..." followed by "#include
<module/header.h>" would silently ignore the CONFIG_MACRO setting. At
least it will no longer be silent about it.
Configuration macros are eventually intended to help reduce the number
of module variants that need to be built. When the list of
configuration macros for a module is exhaustive, we only need to
consider the settings for those macros when building/finding the
module, which can help isolate modules for various project-specific -D
flags that should never affect how modules are build (but currently do).
llvm-svn: 177466
The global module index was querying the file manager for each of the
module files it knows about at load time, to prune out any out-of-date
information. The file manager would then cache the results of the
stat() falls used to find that module file.
Later, the same translation unit could end up trying to import one of the
module files that had previously been ignored by the module cache, but
after some other Clang instance rebuilt the module file to bring it
up-to-date. The stale stat() results in the file manager would
trigger a second rebuild of the already-up-to-date module, causing
failures down the line.
The global module index now lazily resolves its module file references
to actual AST reader module files only after the module file has been
loaded, eliminating the stat-caching race. Moreover, the AST reader
can communicate to its caller that a module file is missing (rather
than simply being out-of-date), allowing us to simplify the
module-loading logic and allowing the compiler to recover if a
dependent module file ends up getting deleted.
llvm-svn: 177367
In a module-enabled Cocoa PCH file, we spend a lot of time stat'ing the headers
in order to associate the FileEntries with their modules and support implicit
module import.
Use a more lazy scheme by enhancing HeaderInfoTable to store extra info about
the module that a header belongs to, and associate it with its module only when
there is a request for loading the header info for a particular file.
Part of rdar://13391765
llvm-svn: 176976
This allows resolving top-header filenames of modules to FileEntries when
we need them, not eagerly.
Note that that this breaks ABI for libclang functions
clang_Module_getTopLevelHeader / clang_Module_getNumTopLevelHeaders
but this is fine because they are experimental and not widely used yet.
llvm-svn: 176975
It passes to the visitor, that the caller provides, CXCursor_InclusionDirective cursors for
all the include directives in a particular file.
llvm-svn: 176682
Stat'ing all the headers from the PCH to make sure they are up-to-date takes significant time.
In a particular source file (whose PCH file included Cocoa.h) from total -fsyntax-only time
12% was just stat calls. Change pre-validation to only check non-system headers.
There are some notable disadvantages:
-If a system header, that is not include-guarded, changes after the PCH was created, we will not
find it in the header info table and we will #import it, effectively #importing it twice, thus
we will emit some error due to a multiple definition and after that the "header was modified" error will likely
be emitted, for example something like:
NSDictionary.h:12:1: error: duplicate interface definition for class 'NSDictionary'
@interface NSDictionary : NSObject <NSCopying, NSMutableCopying, NSSecureCoding, NSFastEnumeration>
^
NSDictionary.h:12:12: note: previous definition is here
@interface NSDictionary : NSObject <NSCopying, NSMutableCopying, NSSecureCoding, NSFastEnumeration>
^
fatal error: file 'NSDictionary.h' has been modified since the precompiled header was built
Though we get the "header was modified" error, this is a bit confusing.
-Theoretically it is possible that such a system header will cause no errors but it will just cause an
unfortunate semantic change, though I find this rather unlikely.
The advantages:
-Reduces compilation time when using a huge PCH like the Cocoa ones
-System headers change very infrequent and when they do, users/build systems should be able to know that
re-building from scratch is needed.
Addresses rdar://13056262
llvm-svn: 176567
Previously the hash would be the filename portion of the path, which could be
different for a filename with different case or a symbolic link with a different
name completely.
This did not actually create any issue so far because by validating all headers
in the PCH we created uniqued FileEntries based on inodes, so an #include of
a symbolic link (refering to a file from the PCH) would end up with a FileEntry
with filename same as the one recorded in the PCH.
llvm-svn: 176566
llvm::sys::fs::equivalent() does 2 stat calls every time it's called. Use FileManager::getFile() to take advantage
of the stat caching that FileManager is providing.
llvm-svn: 176450
Previously we would return null for an out-of-date file. This inhibited ASTReader::ReadSLocEntry
from creating a FileID to recover gracefully in such a case.
llvm-svn: 176332
its index in the preprocessed entities vector.
This is because the order of the entities in the vector can change in some (uncommon) cases.
llvm-svn: 175907
Add an ability to specify custom documentation block comment commands via a new
class CommentOptions. The intention is that this class will hold future
customizations for comment parsing, including defining documentation comments
with specific numbers of parameters, etc.
CommentOptions instance is a member of LangOptions.
CommentOptions is controlled by a new command-line parameter
-fcomment-block-commands=Foo,Bar,Baz.
llvm-svn: 175892
for the data specific to a macro definition (e.g. what the tokens are), and
MacroDirective class which encapsulates the changes to the "macro namespace"
(e.g. the location where the macro name became active, the location where it was undefined, etc.)
(A MacroDirective always points to a MacroInfo object.)
Usually a macro definition (MacroInfo) is where a macro name becomes active (MacroDirective) but
splitting the concepts allows us to better model the effect of modules to the macro namespace
(also as a bonus it allows better modeling of push_macro/pop_macro #pragmas).
Modules can have their own macro history, separate from the local (current translation unit)
macro history; MacroDirectives will be used to model the macro history (changes to macro namespace).
For example, if "@import A;" imports macro FOO, there will be a new local MacroDirective created
to indicate that "FOO" became active at the import location. Module "A" itself will contain another
MacroDirective in its macro history (at the point of the definition of FOO) and both MacroDirectives
will point to the same MacroInfo object.
Introducing the separation of macro concepts is the first part towards better modeling of module macros.
llvm-svn: 175585
This commit introduces a set of related changes to ensure that the
declaration that shows up in the identifier chain after deserializing
declarations with a given identifier is, in fact, the most recent
declaration. The primary change involves waiting until after we
deserialize and wire up redeclaration chains before updating the
identifier chains. There is a minor optimization in here to avoid
recursively deserializing names as part of looking to see whether
top-level declarations for a given name exist.
A related change that became suddenly more urgent is to property
record a merged declaration when an entity first declared in the
current translation unit is later deserialized from a module (that had
not been loaded at the time of the original declaration). Since we key
off the canonical declaration (which is parsed, not from an AST file)
for emitted redeclarations, we simply record this as a merged
declaration during AST writing and let the readers merge them.
Re-fixes <rdar://problem/13189985>, presumably for good this time.
llvm-svn: 175447
until recursive loading is finished.
Otherwise we may end up with a template trying to deserialize a template
parameter that is in the process of getting loaded.
rdar://13135282
llvm-svn: 175329
These two related tweaks to keep the information associated with a
given identifier correct when the identifier has been given some
top-level information (say, a top-level declaration) and more
information is then loaded from a module. The first ensures that an
identifier that was "interesting" before being loaded from an AST is
considered to be different from its on-disk counterpart. Otherwise, we
lose such changes when writing the current translation unit as a
module.
Second, teach the code that injects AST-loaded names into the
identifier chain for name lookup to keep the most recent declaration,
so that we don't end up confusing our declaration chains by having a
different declaration in there.
llvm-svn: 174895
if it found any decls, rather than returning a list of found decls. This
removes a returning-ArrayRef-to-deleted-storage bug from
MultiplexExternalSemaSource (in code not exercised by any of the clang
binaries), reduces the work required in the found-no-decls case with PCH, and
importantly removes the need for DeclContext::lookup to be reentrant.
No functionality change intended!
llvm-svn: 174576
This can happen when one abuses precompiled headers by passing more -D
options when using a precompiled hedaer than when it was built. This
is intentionally permitted by precompiled headers (and is exploited by
some build environments), but causes problems for modules.
First part of <rdar://problem/13165109>, detecting when something when
horribly wrong.
llvm-svn: 174554
Different modules may have different views of the various "special"
types in the AST, such as the redefinition type for "id". Merge those
types rather than only considering the redefinition types for the
first AST file loaded.
llvm-svn: 174234
- The only group where it makes sense for the "ExternC" bit is System, so this
simplifies having to have the extra isCXXAware (or ImplicitExternC, depending
on what code you talk to) bit caried around.
llvm-svn: 173859
index, optimizing the operation that skips lookup in modules where we
know the identifier will not be found. This makes the global module
index optimization actually useful, providing an 8.5% speedup over
modules without the global module index for -fsyntax-only.
llvm-svn: 173529
AST reader.
The global module index tracks all of the identifiers known to a set
of module files. Lookup of those identifiers looks first in the global
module index, which returns the set of module files in which that
identifier can be found. The AST reader only needs to look into those
module files and any module files not known to the global index (e.g.,
because they were (re)built after the global index), reducing the
number of on-disk hash tables to visit. For an example source I'm
looking at, we go from 237844 total identifier lookups into on-disk
hash tables down to 126817.
Unfortunately, this does not translate into a performance advantage.
At best, it's a wash once the global module index has been built, but
that's ignore the cost of building the global module index (which
is itself fairly large). Profiles show that the global module index
code is far less efficient than it should be; optimizing it might give
enough of an advantage to justify its continued inclusion.
llvm-svn: 173405
The global module index is a "global" index for all of the module
files within a particular subdirectory in the module cache, which
keeps track of all of the "interesting" identifiers and selectors
known in each of the module files. One can perform a fast lookup in
the index to determine which module files will have more information
about entities with a particular name/selector. This information can
help eliminate redundant lookups into module files (a serious
performance problem) and help with creating auto-import/auto-include
Fix-Its.
The global module index is created or updated at the end of a
translation unit that has triggered a (re)build of a module by
scraping all of the .pcm files out of the module cache subdirectory,
so it catches everything. As with module rebuilds, we use the file
system's atomicity to synchronize.
llvm-svn: 173301
identifiers into two parts: the part that involves dealing with the
key (which can be re-used) and the ASTReader-specific part that
creates the IdentifierInfos. While I'm at it, StringRef'ify this code,
which was using pair<const char*, unsigned>. No functionality change.
llvm-svn: 173283
This change also makes the serialisation store the required semantics,
fixing an issue where PPC128 was always assumed when re-reading a
128-bit value.
llvm-svn: 173139
in a StringRef to bind to them forces them to be unpacked into the Record as individual
bytes. This is wasteful, but not likely to be measurable in this instance.
llvm-svn: 173066
forming the identifier, e.g., as part of a selector or a declaration
name, don't actually deserialize any information about the
identifier. Instead, simply mark it "out-of-date" and we'll load the
the information on demand. 2% speedup on the modules testcase I'm
looking at; should also help PCH.
llvm-svn: 173056
DeclContext. When the DeclContext is of a kind that can only be
defined once and never updated, we limit the search to the module file
that conatins the lookup table. Provides a 15% speedup in one
modules-heavy source file.
llvm-svn: 173050
Makes sure that a deserialized macro is only added to the preprocessor macro definitions only once.
Unfortunately I couldn't get a reduced test case.
rdar://13016031
llvm-svn: 172843
Previously we would serialize the macro redefinitions as a list, part of
the identifier, and try to chain them together across modules individually
without having the info that they were already chained at definition time.
Change this by serializing the macro redefinition chain and then try
to synthesize the chain parts across modules. This allows us to correctly
pinpoint when 2 different definitions are ambiguous because they came from
unrelated modules.
Fixes bogus "ambiguous expansion of macro" warning when a macro in a PCH
is redefined without undef'ing it first.
rdar://13016031
llvm-svn: 172620
metadata for linking against the libraries/frameworks for imported
modules.
The module map language is extended with a new "link" directive that
specifies what library or framework to link against when a module is
imported, e.g.,
link "clangAST"
or
link framework "MyFramework"
Importing the corresponding module (or any of its submodules) will
eventually link against the named library/framework.
For now, I've added some placeholder global metadata that encodes the
imported libraries/frameworks, so that we can test that this
information gets through to the IR. The format of the data is still
under discussion.
llvm-svn: 172437
which a particular declaration resides. Use this information to
customize the "definition of 'blah' must be imported from another
module" diagnostic with the module the user actually has to
import. Additionally, recover by importing that module, so we don't
complain about other names in that module.
Still TODO: coming up with decent Fix-Its for these cases, and expand
this recovery approach for other name lookup failures.
llvm-svn: 172290
modules when getting the decls for a namespace or translation unit.
Otherwise the code-completion results will not be complete.
rdar://12889089
llvm-svn: 170596
don't crash when loading a PCH with the older format.
The introduction of the control block broke compatibility with PCHs from
older versions. This patch allows loading (and rejecting) PCHs from an older
version and allows newer PCHs to be rejected from older clang versions as well.
rdar://12821386
llvm-svn: 170150
uncovered.
This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.
I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.
llvm-svn: 169237
PreprocessingRecord and into its own class, PPConditionalDirectiveRecord.
Decoupling allows a client to use the functionality of PPConditionalDirectiveRecord
without needing a PreprocessingRecord.
llvm-svn: 169229
module, provide a module import stack similar to what we would get for
an include stack, e.g.,
In module 'DependsOnModule' imported from build-fail-notes.m:4:
In module 'Module' imported from DependsOnModule.framework/Headers/DependsOnModule.h:1:
Inputs/Module.framework/Headers/Module.h:15:12: note: previous definition is here
@interface Module
<rdar://problem/12696425>
llvm-svn: 169042
allocated using the allocator associated with an ASTContext.
Use this inside CXXRecordDecl::DefinitionData instead of an UnresolvedSet to
avoid a potential memory leak.
rdar://12761275
llvm-svn: 168771
the related comma pasting extension.
In certain cases, we used to get two diagnostics for what is essentially one
extension. This change suppresses the first diagnostic in certain cases
where we know we're going to print the second diagnostic. The
diagnostic is redundant, and it can't be suppressed in the definition
of the macro because it points at the use of the macro, so we want to
avoid printing it if possible.
The implementation works by detecting constructs which look like comma
pasting at the time of the definition of the macro; this information
is then used when the macro is used. (We can't actually detect
whether we're using the comma pasting extension until the macro is
actually used, but we can detecting constructs which will be comma
pasting if the varargs argument is elided.)
<rdar://problem/12292192>
llvm-svn: 167907
The stat cache became essentially useless ever since we started
validating all file entries in the PCH.
But the motivating reason for removing it now is that it also affected
correctness in this situation:
-You have a header without include guards (using "#pragma once" or #import)
-When creating the PCH:
-The same header is referenced in an #include with different filename cases.
-In the PCH, of course, we record only one file entry for the header file
-But we cache in the PCH file the stat info for both filename cases
-Then the source files are updated and the header file is updated in a way that
its size and modification time are the same but its inode changes
-When using the PCH:
-We validate the headers, we check that header file and we create a file entry with its current inode
-There's another #include with a filename with different case than the previously created file entry
-In order to get its stat info we go through the cached stat info of the PCH and we receive the old inode
-because of the different inodes, we think they are different files so we go ahead and include its contents.
Removing the stat cache will potentially break clients that are attempting to use the stat cache
as a way of avoiding having the actual input files available. If that use case is important, patches are welcome
to bring it back in a way that will actually work correctly (i.e., emit a PCH that is self-contained, coping with
literal strings, line/column computations, etc.).
This fixes rdar://5502805
llvm-svn: 167172
diagnostic states; make sure the ASTReader sets the diagnostic state
properly instead of always recreating it.
Fixes rdar://12581618 & http://llvm.org/PR14181
llvm-svn: 166987
the macros that are #define'd or #undef'd on the command line. This
checking happens much earlier than the current macro-definition
checking and is far cleaner, because it does a direct comparison
rather than a diff of the predefines buffers. Moreover, it allows us
to use the result of this check to skip over PCH files within a
directory that have non-matching -D's or -U's on the command
line. Finally, it improves the diagnostics a bit for mismatches,
fixing <rdar://problem/8612222>.
The old predefines-buffer diff'ing will go away in a subsequent commit.
llvm-svn: 166641
Argyrios Kyrtzidis