id-expression 'x' will compute the type based on the assumption that
'x' will be captured, even if it isn't captured, per C++11
[expr.prim.lambda]p18. There are two related refactors that go into
implementing this:
1) Split out the check that determines whether we should capture a
particular variable reference, along with the computation of the
type of the field, from the actual act of capturing the
variable.
2) Always compute the result of decltype() within Sema, rather than
AST, because the decltype() computation is now context-sensitive.
llvm-svn: 150347
to pretty-print such function types better, and to fix a case where we were not
instantiating templates in lexical order. In passing, move the Variadic bit from
Type's bitfields to FunctionProtoType to get the Type bitfields down to 32 bits.
Also ensure that we always substitute the return type of a function when
substituting explicitly-specified arguments, since that can cause us to bail
out with a SFINAE error before we hit a hard error in parameter substitution.
llvm-svn: 150241
We were passing a decl to the consumer after all pending deserializations were finished
but this was not enough; due to processing by the consumer we may end up into yet another
deserialization process but the way FinishedDeserializing() was setup we would not ensure
that everything was fully deserialized before returning to the consumer.
Separate ASTReader::FinishedDeserializing() into two semantic actions.
The first is ensuring that a deserialization process ends up will fully deserialized decls/types even
if the process is started by the consumer.
The second is pushing "interesting" decls to the consumer; we make sure that we don't re-enter this
section recursively be checking a variable.
llvm-svn: 150160
The new info is propagated to TSTLoc on template instantiation, getting rid of 3 FIXMEs in TreeTransform.h and another one Parser.cpp.
Simplified code in TypeSpecLocFiller visitor methods for DTSTLoc and DependentNameTypeLoc by removing what now seems to be dead code (adding corresponding assertions).
llvm-svn: 149923
single attribute ("system") that allows us to mark a module as being a
"system" module. Each of the headers that makes up a system module is
considered to be a system header, so that we (for example) suppress
warnings there.
If a module is being inferred for a framework, and that framework
directory is within a system frameworks directory, infer it as a
system framework.
llvm-svn: 149143
the direct serialization of the linked-list structure. Instead, use a
scheme similar to how we handle redeclarations, with redeclaration
lists on the side. This addresses several issues:
- In cases involving mixing and matching of many categories across
many modules, the linked-list structure would not be consistent
across different modules, and categories would get lost.
- If a module is loaded after the class definition and its other
categories have already been loaded, we wouldn't see any categories
in the newly-loaded module.
llvm-svn: 149112
return pre-built lists. Instead, it feeds the methods it deserializes
to Sema so that Sema can unique them, which keeps the chains shorter.
llvm-svn: 148889
when it actually has changed (and not, e.g., when we've simply attached a
deserialized macro definition). Good for ~1.5% reduction in module
file size, mostly in the identifier table.
llvm-svn: 148808
generational scheme for identifiers that avoids searching the hash
tables of a given module more than once for a given
identifier. Previously, loading any new module invalidated all of the
previous lookup results for all identifiers, causing us to perform the
lookups repeatedly.
llvm-svn: 148412
corresponding to TagType and ObjCInterfaceType. Previously, we would
serialize the definition (if available) or the canonical declaration
(if no definition was available). However, this can end up forcing the
deserialization of the definition even through we might not want to
yet.
Instead, always serialize the canonical declaration reference in the
TagType/ObjCInterfaceType entry, and as part of loading a pending
definition, update the "decl" pointer within the type node to point at
the definition. This is more robust in hard-to-isolate cases
where the *Type gets built and filled in before we see the definition.
llvm-svn: 148323
chains, again. The prior implementation was very linked-list oriented, and
the list-splicing logic was both fairly convoluted (when loading from
multiple modules) and failed to preserve a reasonable ordering for the
redeclaration chains.
This new implementation uses a simpler strategy, where we store the
ordered redeclaration chains in an array-like structure (indexed based
on the first declaration), and use that ordering to add individual
deserialized declarations to the end of the existing chain. That way,
the chain mimics the ordering from its modules, and a bug somewhere is
far less likely to result in a broken linked list.
llvm-svn: 148222
Redeclarable<RedeclarableTemplateDecl>, eliminating a bunch of
redeclaration-chain logic both in RedeclarableTemplateDecl and
especially in its (de-)serialization.
As part of this, eliminate the RedeclarableTemplate<> class template,
which was an abstraction that didn't actually save anything.
llvm-svn: 148181
modules. Teach name lookup into namespaces to search in each of the
merged DeclContexts as well as the (now-primary) DeclContext. This
supports the common case where two different modules put something
into the same namespace.
llvm-svn: 147778
is hidden from name lookup. The previous hack of tweaking the
ModulePrivate bit when loading a declaration from a hidden submodule
was brittle.
Note that we now have 34 bits in Decl. I'll fix that next.
llvm-svn: 147658
in the module map. This provides a bit more predictability for the
user, as well as eliminating the need to sort the submodules when
serializing them.
llvm-svn: 147564
for Objective-C protocols, including:
- Using the first declaration as the canonical declaration
- Using the definition as the primary DeclContext
- Making sure that all declarations have a pointer to the definition
data, and that we know which declaration is the definition
- Serialization support for redeclaration chains and for adding
definitions to already-serialized declarations.
However, note that we're not taking advantage of much of this code
yet, because we're still re-using ObjCProtocolDecls.
llvm-svn: 147410
features needed for a particular module to be available. This allows
mixed-language modules, where certain headers only work under some
language variants (e.g., in C++, std.tuple might only be available in
C++11 mode).
llvm-svn: 147387
set of (previously-canonical) declaration IDs to the module file, so
that future AST reader instances that load the module know which
declarations are merged. This is important in the fairly tricky case
where a declaration of an entity, e.g.,
@class X;
occurs before the import of a module that also declares that
entity. We merge the declarations, and record the fact that the
declaration of X loaded from the module was merged into the (now
canonical) declaration of X that we parsed.
llvm-svn: 147181
declaration of that same class that either came from some other module
or occurred in the translation unit loading the module. In this case,
we need to merge the two redeclaration chains immediately so that all
such declarations have the same canonical declaration in the resulting
AST (even though they don't in the module files we've imported).
Focusing on Objective-C classes until I'm happy with the design, then
I'll both (1) extend this notion to other kinds of declarations, and
(2) optimize away this extra checking when we're not dealing with
modules. For now, doing this checking for PCH files/preambles gives us
better testing coverage.
llvm-svn: 147123
hitting a submodule that was never actually created, e.g., because
that header wasn't parsed. In such cases, complain (because the
module's umbrella headers don't cover everything) and fall back to
including the header.
Later, we'll add a warning at module-build time to catch all such
cases. However, this fallback is important to eliminate assertions in
the ASTWriter when this happens.
llvm-svn: 146933
with a definition pointer (e.g., C++ and Objective-C classes), zip
through the redeclaration chain to make sure that all of the
declarations point to the definition data.
As part of this, realized again why the first redeclaration of an
entity in a file is important, and brought back that idea.
llvm-svn: 146886
imported modules that don't introduce any new entities of a particular
kind. Allow these entries to be replaced with entries for another
loaded module.
In the included test case, selectors exhibit this behavior.
llvm-svn: 146870
chains. The previous implementation relied heavily on the declaration
chain being stored as a (circular) linked list on disk, as it is in
memory. However, when deserializing from multiple modules, the
different chains could get mixed up, leading to broken declaration chains.
The new solution keeps track of the first and last declarations in the
chain for each module file. When we load a declaration, we search all
of the module files for redeclarations of that declaration, then
splice together all of the lists into a coherent whole (along with any
redeclarations that were actually parsed).
As a drive-by fix, (de-)serialize the redeclaration chains of
TypedefNameDecls, which had somehow gotten missed previously. Add a
test of this serialization.
This new scheme creates a redeclaration table that is fairly large in
the PCH file (on the order of 400k for Cocoa.h's 12MB PCH file). The
table is mmap'd in and searched via a binary search, but it's still
quite large. A future tweak will eliminate entries for declarations
that have no redeclarations anywhere, and should
drastically reduce the size of this table.
llvm-svn: 146841
including deserializing their bodies, so that any other declarations that
get referenced in the body will be fully deserialized by the time we pass them to the consumer.
Could not reduce to a test case unfortunately. rdar://10587158.
llvm-svn: 146817
part of HeaderSearch. This function just normalizes filenames for use
inside of a synthetic include directive, but it is used in both the
Frontend and Serialization libraries so it needs a common home.
llvm-svn: 146227
diagnostics. Conflating them was highly confusing and makes it harder to
establish a firm layering separation between these two libraries.
llvm-svn: 146207
umbrella headers in the sense that all of the headers within that
directory (and eventually its subdirectories) are considered to be
part of the module with that umbrella directory. However, unlike
umbrella headers, which are expected to include all of the headers
within their subdirectories, Clang will automatically include all of
the headers it finds in the named subdirectory.
The intent here is to allow a module map to trivially turn a
subdirectory into a module, where the module's structure can mimic the
directory structure.
llvm-svn: 146165
header to also support umbrella directories. The umbrella directory
for an umbrella header is the directory in which the umbrella header
resides.
No functionality change yet, but it's coming.
llvm-svn: 146158
to re-export anything that it imports. This opt-in feature makes a
module behave more like a header, because it can be used to re-export
the transitive closure of a (sub)module's dependencies.
llvm-svn: 145811
"main" files that import modules. When loading any of these kinds of
AST files, we make the modules that were imported visible into the
translation unit that loaded the PCH file or preamble.
llvm-svn: 145737
precompiled header. Previously, we were trying to gather predefines
buffers from all kinds of AST files (which doesn't make sense) and
were performing some validation when AST files were loaded as main
files.
With these tweaks, using PCH files that import modules no longer fails
immediately (due to mismatched predefines buffers). However, module
visibility is lost, so this feature does not yet work.
llvm-svn: 145709
only the macro definitions from visible (sub)modules will actually be
visible. This provides the same behavior for macros that r145640
provided for declarations.
llvm-svn: 145683
(sub)module, all of the names may be hidden, just the macro names may
be exposed (for example, after the preprocessor has seen the import of
the module but the parser has not), or all of the names may be
exposed. Importing a module makes its names, and the names in any of
its non-explicit submodules, visible to name lookup (transitively).
This commit only introduces the notion of name visible and marks
modules and submodules as visible when they are imported. The actual
name-hiding logic in the AST reader will follow (along with test cases).
llvm-svn: 145586
a standard global/local scheme, so that submodule definitions will
eventually be able to refer to submodules in other top-level
modules. We'll need this functionality soonish.
llvm-svn: 145549
library, since modules cut across all of the libraries. Rename
serialization::Module to serialization::ModuleFile to side-step the
annoying naming conflict. Prune a bunch of ModuleMap.h includes that
are no longer needed (most files only needed the Module type).
llvm-svn: 145538
we may end up having added more pending stuff to do, so go in a loop until everything
is cleared out.
This fixes the error in rdar://10278815 which has a certain David Lynch-esque quality..
error: unknown type name 'BOOL'; did you mean 'BOOL'?
llvm-svn: 145536
submodules. This information will eventually be used for name hiding
when dealing with submodules. For now, we only use it to ensure that
the module "key" returned when loading a module will always be a
module (rather than occasionally being a FileEntry).
llvm-svn: 145497
inside an objc container that "contains" other file-level declarations.
When getting the array of file-level declarations that overlap with a file region,
we failed to report that the region overlaps with an objc container, if
the container had other file-level declarations declared lexically inside it.
Fix this by marking such declarations as "isTopLevelDeclInObjCContainer" in the AST
and handling them appropriately.
llvm-svn: 145109
file in the source manager. This allows us to properly create and use
modules described by module map files without umbrella headers (or
with incompletely umbrella headers). More generally, we can actually
build a PCH file that makes use of file -> buffer remappings, which
could be useful in libclang in the future.
llvm-svn: 144830
In certain cases ASTReader would call the normal DiagnosticsEngine API to initialize
the state of diagnostic pragmas but DiagnosticsEngine would try to compare source locations
leading to crash because the main FileID was not yet initialized.
Yet another case of the ASTReader trying to use the normal APIs and inadvertently breaking
invariants. Fix this by having the ASTReader set up the internal state directly.
llvm-svn: 144153
that it retains source location information for the type. Aside from
general goodness (being able to walk the types described in that
information), we now have a proper representation for dependent
delegating constructors. Fixes PR10457 (for real).
llvm-svn: 143410
Introduce a FILE_SORTED_DECLS [de]serialization record that contains
a file sorted array of file-level DeclIDs in a PCH/Module.
The rationale is to allow "targeted" deserialization of decls inside
a range of a source file.
Cocoa PCH increased by 0.8%
Difference of creation time for Cocoa PCH is below the noise level.
llvm-svn: 143238
of decl bit offsets.
This allows us to easily get at the location of a decl without deserializing it.
It increases size of Cocoa PCH by only 0.2%.
llvm-svn: 143123
AST file more lazy, so that we don't eagerly load that information for
all known identifiers each time a new AST file is loaded. The eager
reloading made some sense in the context of precompiled headers, since
very few identifiers were defined before PCH load time. With modules,
however, a huge amount of code can get parsed before we see an
@import, so laziness becomes important here.
The approach taken to make this information lazy is fairly simple:
when we load a new AST file, we mark all of the existing identifiers
as being out-of-date. Whenever we want to access information that may
come from an AST (e.g., whether the identifier has a macro definition,
or what top-level declarations have that name), we check the
out-of-date bit and, if it's set, ask the AST reader to update the
IdentifierInfo from the AST files. The update is a merge, and we now
take care to merge declarations before/after imports with declarations
from multiple imports.
The results of this optimization are fairly dramatic. On a small
application that brings in 14 non-trivial modules, this takes modules
from being > 3x slower than a "perfect" PCH file down to 30% slower
for a full rebuild. A partial rebuild (where the PCH file or modules
can be re-used) is down to 7% slower. Making the PCH file just a
little imperfect (e.g., adding two smallish modules used by a bunch of
.m files that aren't in the PCH file) tips the scales in favor of the
modules approach, with 24% faster partial rebuilds.
This is just a first step; the lazy scheme could possibly be improved
by adding versioning, so we don't search into modules we already
searched. Moreover, we'll need similar lazy schemes for all of the
other lookup data structures, such as DeclContexts.
llvm-svn: 143100
expressions: expressions which refer to a logical rather
than a physical l-value, where the logical object is
actually accessed via custom getter/setter code.
A subsequent patch will generalize the AST for these
so that arbitrary "implementing" sub-expressions can
be provided.
Right now the only client is ObjC properties, but
this should be generalizable to similar language
features, e.g. Managed C++'s __property methods.
llvm-svn: 142914
preprocessed entities that are #included in the range that we are interested.
This is useful when we are interested in preprocessed entities of a specific file, e.g
when we are annotating tokens. There is also an optimization where we cache the last
result of PreprocessingRecord::getPreprocessedEntitiesInRange and we re-use it if
there is a call with the same range as before.
rdar://10313365
llvm-svn: 142887
PreprocessingRecord's getPreprocessedEntitiesInRange.
Also remove all the stuff that were added in ASTUnit that are unnecessary now
that we do a binary search for preprocessed entities and deserialize only
what is necessary.
llvm-svn: 140063
check whether the requested location points inside the precompiled preamble,
in which case the returned source location will be a "loaded" one.
llvm-svn: 140060
which will do a binary search and return a pair of iterators
for preprocessed entities in the given source range.
Source ranges of preprocessed entities are stored twice currently in
the PCH/Module file but this will be fixed in a subsequent commit.
llvm-svn: 140058
don't call ReadSLocEntryRecord() directly because the entry may have
already been loaded in which case calling ReadSLocEntryRecord()
directly would trigger an assertion in SourceManager.
llvm-svn: 140052
-Use an array of offsets for all preprocessed entities
-Get rid of the separate array of offsets for just macro definitions;
for references to macro definitions use an index inside the preprocessed
entities array.
-Deserialize each preprocessed entity lazily, at first request; not in bulk.
Paves the way for binary searching of preprocessed entities that will offer
efficiency and will simplify things on the libclang side a lot.
llvm-svn: 139809
to the consumer without being fully deserialized).
The regression was on compiling boost.python and it was too difficult to get a reduced
test case unfortunately.
Also modify the logic of how objc methods are getting passed to the consumer;
codegen depended on receiving objc methods before the implementation decl.
Since the interesting objc methods are ones with a body and such methods only
exist inside an ObjCImplDecl, deserialize and pass to consumer all the methods
of ObCImplDecl when we see one.
Fixes http://llvm.org/PR10922 & rdar://10117105.
llvm-svn: 139644
language options. Use that .def file to declare the LangOptions class
and initialize all of its members, eliminating a source of annoying
initialization bugs.
AST serialization changes are next up.
llvm-svn: 139605
modifying directly for the preamble.
This avoids an awful, hard to find, bug where "PreprocessorOpts.DisablePCHValidation = true"
would be persistent for subsequent reparses of the translation unit which would result
in defines, present in command-line but not in the PCH, being ignored.
Fixes rdar://9615399.
llvm-svn: 139512
ASTContext reference. Remove all of the extra checking and logic that
was used to cope with a NULL ASTContext. No effective functionality
change.
llvm-svn: 139413
identifier, also make them visible in the translation unit," which
isn't needed now that John's eliminated the AST dependency in blocks
CodeGen.
llvm-svn: 139408
'id' that can be used (only!) via a contextual keyword as the result
type of an Objective-C message send. 'instancetype' then gives the
method a related result type, which we have already been inferring for
a variety of methods (new, alloc, init, self, retain). Addresses
<rdar://problem/9267640>.
llvm-svn: 139275
Preprocessor, eliminating the constructor that was used by ASTUnit
(which didn't provide an ASTContext or Prepreprocessor). Ensuring that
both objects are non-NULL will simplify module loading (but none of
that is done yet).
llvm-svn: 138986
include guards don't show up as macro definitions in every translation
unit that imports a module. Macro definitions can, however, be
exported with the intentionally-ugly #__export_macro__
directive. Implement this feature by not even bothering to serialize
non-exported macros to a module, because clients of that module need
not (should not) know that these macros even exist.
llvm-svn: 138943
The initial incentive was to fix a crash when PCH chaining categories
to an interface, but the fix was done in the "modules way" that I hear
is popular with the kids these days.
Each module stores the local chain of categories and we combine them
when the interface is loaded. We also warn if non-dependent modules
introduce duplicate named categories.
llvm-svn: 138926
to "when loading a particular module" validation, since it was only
validating local information anyway. This shouldn't change anything.
llvm-svn: 138583
given selector, rather than walking the chain backwards. Teach its
visitor how to merge multiple result sets into a single result set,
combining the results of selector lookup in several different modules
into a single result set.
llvm-svn: 138556
table when serializing an AST file. This was a holdover from the days
before chained PCH, and is a complete waste of time and storage
now. It's a good thing it's useless, because I have no idea how I
would have implemented MaterializeVisibleDecls efficiently in the
presence of modules.
llvm-svn: 138496
which supports both pre-order and post-order traversal via a visitor
mechanism. Use this depth-first search with a post-order traversal to
give predictable ordering semantics when walking all of the lexical
declarations in the translation unit.
Eventually, module imports will occur in the source code rather than
at the beginning, and we'll have to revisit this walk.
llvm-svn: 138490
module DAG-based lookup scheme. This required some reshuffling, so
that each module stores its own mapping from DeclContexts to their
lexical and visible sets for those DeclContexts (rather than one big
"chain").
Overall, this allows simple qualified name lookup into the translation
unit to gather results from multiple modules, with the lookup results
in module B shadowing the lookup results in module A when B imports A.
Walking all of the lexical declarations in a module DAG is still a
mess; we'll end up walking the loaded module list backwards, which
works fine for chained PCH but doesn't make sense in a DAG. I'll
tackle this issue as a separate commit.
llvm-svn: 138463
Currently getMacroArgExpandedLocation is very inefficient and for the case
of a location pointing at the main file it will end up checking almost all of
the SLocEntries. Make it faster:
-Use a map of macro argument chunks to their expanded source location. The map
is for a single source file, it's stored in the file's ContentCache and lazily
computed, like the source lines cache.
-In SLocEntry's FileInfo add an 'unsigned NumCreatedFIDs' field that keeps track
of the number of FileIDs (files and macros) that were created during preprocessing
of that particular file SLocEntry. This is useful when computing the macro argument
map in skipping included files while scanning for macro arg FileIDs that lexed from
a specific source file. Due to padding, the new field does not increase the size
of SLocEntry.
llvm-svn: 138225
different modules) more robust. It already handled (simple) merges of
the set of declarations attached to that identifier, so add a test
case that shows us getting two different declarations for the same
identifier (one struct, one function) from different modules, and are
able to use both of them.
llvm-svn: 138189
modules (those that no other module depends on) and performs a search
over all of the modules, visiting a new module only when all of the
modules that depend on it have already been visited. The visitor can
abort the search for all modules that a module depends on, which
allows us to minimize the number of lookups necessary when performing
a search.
Switch identifier lookup from a linear walk over the set of modules to
this module visitation operation. The behavior is the same for simple
PCH and chained PCH, but provides the proper search order for
modules. Verified with printf debugging, since we don't have enough in
place to actually test this.
llvm-svn: 138187
has already been loaded before allocating a new Module structure. If
the module has already been loaded (uniquing based on file name), then
just return the existing module rather than trying to load it again.
This allows us to load a DAG of modules. Introduce a simple test case
that forms a diamond-shaped module graph, and illustrates that a
source file importing the bottom of the diamond can see declarations
in all four of the modules that make up the diamond.
Note that this version moves the file-opening logic into the module
manager, rather than splitting it between the module manager and the
AST reader. More importantly, it properly handles the
weird-but-possibly-useful case of loading an AST file from "-".
llvm-svn: 138030
Teach ModuleManager::addModule() to check whether a particular module
has already been loaded before allocating a new Module structure. If
the module has already been loaded (uniquing based on file name), then
just return the existing module rather than trying to load it again.
This allows us to load a DAG of modules. Introduce a simple test case
that forms a diamond-shaped module graph, and illustrates that a
source file importing the bottom of the diamond can see declarations
in all four of the modules that make up the diamond.
llvm-svn: 137971
has already been loaded before allocating a new Module structure. If
the module has already been loaded (uniquing based on file name), then
just return the existing module rather than trying to load it again.
This allows us to load a DAG of modules. Introduce a simple test case
that forms a diamond-shaped module graph, and illustrates that a
source file importing the bottom of the diamond can see declarations
in all four of the modules that make up the diamond.
llvm-svn: 137925
-import-module) vs. loaded because some other module depends on
them. As part of doing this, pass down the module that caused a module
to be loaded directly, rather than assuming that we're loading a
chain. Finally, write out all of the directly-loaded modules when
serializing an AST file (using the new IMPORTS record), so that an AST
file can depend on more than one other AST file, all of which will be
loaded when that AST file is loaded. This allows us to form and load a
tree of modules, but we can't yet load a DAG of modules.
llvm-svn: 137923
all AST files have a normal METADATA record that has the same form
regardless of whether we refer to a chained PCH or any other kind of
AST file.
Introduce the IMPORTS record, which describes all of the AST files
that are imported by this AST file, and how (as a module, a PCH file,
etc.). Currently, we emit at most one entry to this record, to support
chained PCH.
llvm-svn: 137869
type over into the AST context, then make that declaration a
predefined declaration in the AST format. This ensures that different
AST files will at least agree on the (global) declaration ID for 'id',
and eliminates one of the "special" types in the AST file format.
llvm-svn: 137429
declaration that never actually gets serialized. Instead, serialize
the various kinds of update records (lexical decls, visible decls, the
addition of an anonymous namespace) for the translation unit, even if
we're not chaining. This way, we won't have to deal with multiple
loaded translation unit declarations.
llvm-svn: 137395
either "special" type has already been initialized. Previously, we did
this check based on just the first special type (__builtin_va_list),
but now we have some NULL special type entries to content with.
llvm-svn: 137373
enumerations from the ASTContext into CodeGen, so that we don't need
to serialize it to AST files. This appears to be the last of the
low-hanging fruit for SpecialTypes.
llvm-svn: 137124
layout of a constant NSString from the ASTContext over to CodeGen,
since this is solely CodeGen's responsibility. Eliminates one of the
unnecessary "special" types that we serialize.
llvm-svn: 137121
the last of the ID/offset/index mappings that I know
of. Unfortunately, the "gap" method of testing doesn't work here due
to the way the preprocessing record performs iteration. We'll do more
testing once multi-AST loading is possible.
llvm-svn: 136902
IDs will never cross module boundaries, since they're tied to the
CXXDefinitionData, so just use a local mapping throughout. Eliminate
the global -> local tables and supporting data.
llvm-svn: 136847
AST file, along with an enumeration naming those predefined
declarations. No functionality change, but this will make it easier to
introduce new predefined declarations, when/if we need them.
llvm-svn: 136781
reader, to allow AST files to be loaded with their declarations
remapped to different ID numbers. Fix a number of places where we were
either failing to map local declaration IDs into global declaration
IDs or where interpreting the local declaration IDs within the wrong
module.
I've tested this via the usual "random gaps" method. It works well
except for the preamble tests, because our handling of the precompiled
preamble requires declaration and preprocessed entity to be stable
when parsing code and then loading that back into memory. This
property will hold in general, but my randomized testing naturally
breaks this property to get more coverage. In the future, I expect
that the precompiled preamble logic won't need this property.
I am very unhappy with the current handling of the translation unit,
which is a rather egregious hack. We're going to have to do something
very different here for loading multiple AST files, because we don't
want to have to cope with merging two translation units. Likely, we'll
just handle translation units entirely via "update" records, and
predefine a single, fixed declaration ID for the translation
unit. That will come later.
llvm-svn: 136779
by eliminating the type ID from constructor, destructor, and
conversion function names. There are several reasons for this change:
- A given type (say, int*) isn't guaranteed to have a single, unique
type ID within a chain of PCH files. Hence, we could end up hashing
based on the wrong type ID, causing name lookup to fail.
- The mapping from types back to type IDs required one DenseMap
entry for every type that was ever deserialized, which was an
unacceptable cost to support just the name lookup of constructors,
destructors, and conversion functions. Plus, this mapping could
never actually work with chained or multiple PCH, based on the first
bullet.
Once we have eliminated the type from the hash function, these
problems go away, as does my horrible "reverse type remap" hack, which
was doomed from the start (see bullet #1 above) and far too
complicated.
However, note that removing the type from the hash function means that
all constructors, destructors, and conversion functions have the same
hash key, so I've updated the caller to double-check that the
declarations found have the appropriate name.
llvm-svn: 136708
reader. This scheme permits an AST file to be loaded with its type IDs
shifted anywhere in the type ID space.
At present, the type indices are still allocated in the same boring
way they always have been, just by adding up the number of types in
each PCH file within the chain. However, I've done testing with this
patch by randomly sliding the base indices at load time, to ensure
that remapping is occurring as expected. I may eventually formalize
this in some testing flag, but loading multiple (non-chained) AST
files at once will eventually exercise the same code.
There is one known problem with this patch, which involves name lookup
of operator names (e.g., "x.operator int*()") in cases where multiple
PCH files in the chain. The hash function itself depends on having a
stable type ID, which doesn't happen with chained PCH and *certainly*
doesn't happen when sliding type IDs around. We'll need another
approach. I'll tackle that next.
llvm-svn: 136693
reader statistics), to show the local-to-global mappings. The only
such mapping we have (at least, for now) is for source location
offsets.
llvm-svn: 136687
were (Module*, Offset) with equivalent maps whose value type is just a
Module*. The offsets have moved into corresponding "Base" fields
within the Module itself, where they will also be helpful for
local->global translation (eventually).
llvm-svn: 136441
point, ASTReader::InitializeSema() has very little interesting work,
*except* issues stemming from preloaded declarations. That's something
we'll still need to cope with.
llvm-svn: 136378
completely broken deserialization mapping code we had for VTableUses,
which would have broken horribly as soon as our local-to-global ID
mapping became interesting.
llvm-svn: 136371
we could turn this into an on-disk hash table so we don't load the
whole thing the first time we need it. However, it tends to be very,
very small (i.e., empty) for most precompiled headers, so it isn't all
that interesting.
llvm-svn: 136352
- Added LazyVector::erase() to support this use case.
- Factored out the LazyDecl-of-Decls to RecordData translation in
the ASTWriter. There is still a pile of code duplication here to
eliminate.
llvm-svn: 136270
contents are lazily loaded on demand from an external source (e.g., an
ExternalASTSource or ExternalSemaSource). The "loaded" entities are
kept separate from the "local" entities, so that the two can grow
independently.
Switch Sema::TentativeDefinitions from a normal vector that is eagerly
populated by the ASTReader into one of these LazyVectors, making the
ASTReader a bit more like me (i.e., lazy).
llvm-svn: 136262
so that we have one, simple way to map from global bit offsets to
local bit offsets. Eliminates a number of loops over the chain, and
generalizes for more interesting bit remappings.
Also, as an amusing oddity, we were computing global bit offsets
*backwards* for preprocessed entities (e.g., the directly included PCH
file in the chain would start at offset zero, rather than the original
PCH that occurs first in translation unit). Even more amusingly, it
made precompiled preambles work, because we were forgetting to adjust
the local bit offset to a global bit offset when storing preprocessed
entity offsets in the ASTUnit. Two wrongs made a right, and now
they're both right.
llvm-svn: 135750
type IDs into a single place, and make sure that all of the callers
use the appropriate functions to do the mapping. Since the mapping is
still the identity function, this is essentially a no-op.
llvm-svn: 135733
such that every declaration ID loaded from an AST file will go through
a central local -> global mapping function. At present, this change
does nothing, since the local -> global mapping function is the
identity function.
This is the mechanical part of the refactoring; a follow-up patch will
address a few remaining areas where it's not obvious whether we're
dealing with local or global IDs.
llvm-svn: 135711
entities generated directly by the preprocessor from those loaded from
the external source (e.g., the ASTReader). By separating these two
sets of entities into different vectors, we allow both to grow
independently, and eliminate the need for preallocating all of the
loaded preprocessing entities. This is similar to the way the recent
SourceManager refactoring treats FileIDs and the source location
address space.
As part of this, switch over to building a continuous range map to
track preprocessing entities.
llvm-svn: 135646
the AST reader down to the AST file + local ID, rather than walking
the PCH chain. More cleanup/generalization, although there is more
work to do for preprocessed entities. In particular, the
"preallocation" scheme for preprocessed entities is not going to work
well with late loading of PCH files, and it's likely we'll have to do
something akin to the SourceManager's negative/positive loading.
llvm-svn: 135556
reader down to the AST file + local ID, rather than walking the PCH
chain. No functionality change; this is generalization and cleanup.
llvm-svn: 135554
AST reader down to the AST file + local ID, rather than walking the
PCH chain. No functionality change; this is generalization and cleanup.
llvm-svn: 135551
AST reader down to the AST file + local ID within that file, rather
than lamely walking the PCH chain. There's no actual functionality
change now, but this is cleaner and more general.
llvm-svn: 135548
source locations from source locations loaded from an AST/PCH file.
Previously, loading an AST/PCH file involved carefully pre-allocating
space at the beginning of the source manager for the source locations
and FileIDs that correspond to the prefix, and then appending the
source locations/FileIDs used for parsing the remaining translation
unit. This design forced us into loading PCH files early, as a prefix,
whic has become a rather significant limitation.
This patch splits the SourceManager space into two parts: for source
location "addresses", the lower values (growing upward) are used to
describe parsed code, while upper values (growing downward) are used
for source locations loaded from AST/PCH files. Similarly, positive
FileIDs are used to describe parsed code while negative FileIDs are
used to file/macro locations loaded from AST/PCH files. As a result,
we can load PCH/AST files even during parsing, making various
improvemnts in the future possible, e.g., teaching #include <foo.h> to
look for and load <foo.h.gch> if it happens to be already available.
This patch was originally written by Sebastian Redl, then brought
forward to the modern age by Jonathan Turner, and finally
polished/finished by me to be committed.
llvm-svn: 135484
to allow clients to specify that they've already (correctly) loaded
declarations, and that no further action is needed.
Also, make sure that we clear the "has external lexical declarations"
bit before calling FindExternalLexicalDecls(), to avoid infinite
recursion.
llvm-svn: 135306
variants to 'expand'. This changed a couple of public APIs, including
one public type "MacroInstantiation" which is now "MacroExpansion". The
rest of the codebase was updated to reflect this, especially the
libclang code. Two of the C++ (and thus easily changed) libclang APIs
were updated as well because they pertained directly to the old
MacroInstantiation class.
No functionality changed.
llvm-svn: 135139
for a template template parameter.
Uses to follow.
I've also made the uniquing of SubstTemplateTemplateParmPacks
use a ContextualFoldingSet as a minor space efficiency.
llvm-svn: 134137
vector<int>
to
std::vector<int>
Patch by Kaelyn Uhrain, with minor tweaks + PCH support from me. Fixes
PR5776/<rdar://problem/8652971>.
Thanks Kaelyn!
llvm-svn: 134007
Language-design credit goes to a lot of people, but I particularly want
to single out Blaine Garst and Patrick Beard for their contributions.
Compiler implementation credit goes to Argyrios, Doug, Fariborz, and myself,
in no particular order.
llvm-svn: 133103
Related result types apply Cocoa conventions to the type of message
sends and property accesses to Objective-C methods that are known to
always return objects whose type is the same as the type of the
receiving class (or a subclass thereof), such as +alloc and
-init. This tightens up static type safety for Objective-C, so that we
now diagnose mistakes like this:
t.m:4:10: warning: incompatible pointer types initializing 'NSSet *'
with an
expression of type 'NSArray *' [-Wincompatible-pointer-types]
NSSet *array = [[NSArray alloc] init];
^ ~~~~~~~~~~~~~~~~~~~~~~
/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h:72:1:
note:
instance method 'init' is assumed to return an instance of its
receiver
type ('NSArray *')
- (id)init;
^
It also means that we get decent type inference when writing code in
Objective-C++0x:
auto array = [[NSMutableArray alloc] initWithObjects:@"one", @"two",nil];
// ^ now infers NSMutableArray* rather than id
llvm-svn: 132868
in ASTReader::validateFileEntries().
This avoids going through all source location entries and fixes the performance regression.
Many thanks to Doug for the hint!
(rdar://9530587)
llvm-svn: 132481
a file was modified since the time the PCH was created.
The parser is not fit to deal with stale PCHs, too many invariants do not hold up. rdar://9530587.
llvm-svn: 132389
type that turns one type into another. This is used as the basis to
implement __underlying_type properly - with TypeSourceInfo and proper
behavior in the face of templates.
llvm-svn: 132017
build a precompiled header. Use this information to eliminate the call
to SourceManager::getLocation() while loading a precompiled preamble,
since SourceManager::getLocation() itself causes unwanted
deserialization.
Fixed <rdar://problem/9399352>.
llvm-svn: 131021
CXTranslationUnit_NestedMacroInstantiations, which indicates whether
we want to see "nested" macro instantiations (e.g., those that occur
inside other macro instantiations) within the detailed preprocessing
record. Many clients (e.g., those that only care about visible tokens)
don't care about this information, and in code that uses preprocessor
metaprogramming, this information can have a very high cost.
Addresses <rdar://problem/9389320>.
llvm-svn: 130990
Increase robustness of the delegating constructor cycle detection
mechanism. No more infinite loops on invalid or logic errors leading to
false results. Ensure that this is maintained correctly accross
serialization.
llvm-svn: 130887
which determines whether a particular file is actually a header that
is intended to be guarded from multiple inclusions within the same
translation unit.
llvm-svn: 130808
accompanying fixes to make it work today.
The core of this patch is to provide a link from a TemplateTypeParmType
back to the TemplateTypeParmDecl node which declared it. This in turn
provides much more precise information about the type, where it came
from, and how it functions for AST consumers.
To make the patch work almost a year after its first attempt, it needed
serialization support, and it now retains the old getName() interface.
Finally, it requires us to not attempt to instantiate the type in an
unsupported friend decl -- specifically those coming from template
friend decls but which refer to a specific type through a dependent
name.
A cleaner representation of the last item would be to build
FriendTemplateDecl nodes for these, storing their template parameters
etc, and to perform proper instantation of them like any other template
declaration. They can still be flagged as unsupported for the purpose of
access checking, etc.
This passed an asserts-enabled bootstrap for me, and the reduced test
case mentioned in the original review thread no longer causes issues,
likely fixed at somewhere amidst the 24k revisions that have elapsed.
llvm-svn: 130628
member function, i.e. something of the form 'x.f' where 'f' is a non-static
member function. Diagnose this in the general case. Some of the new diagnostics
are probably worse than the old ones, but we now get this right much more
universally, and there's certainly room for improvement in the diagnostics.
llvm-svn: 130239
language options, and warn when reading an AST with a different value
for the bit.
There doesn't appear to be a good way to test this (commenting out
similar other language options doesn't break anything) but if folks have
suggestions on tests I'm happy to add them.
llvm-svn: 130071
during deserialization from a precompiled header, and update all of
its callers to note when this problem occurs and recover (more)
gracefully. Fixes <rdar://problem/9119249>.
llvm-svn: 129839
The idea is that you can create a VarDecl with an unknown type, or a
FunctionDecl with an unknown return type, and it will still be valid to
access that object as long as you explicitly cast it at every use. I'm
still going back and forth about how I want to test this effectively, but
I wanted to go ahead and provide a skeletal implementation for the LLDB
folks' benefit and because it also improves some diagnostic goodness for
placeholder expressions.
llvm-svn: 129065
which versions of an OS provide a certain facility. For example,
void foo()
__attribute__((availability(macosx,introduced=10.2,deprecated=10.4,obsoleted=10.6)));
says that the function "foo" was introduced in 10.2, deprecated in
10.4, and completely obsoleted in 10.6. This attribute ties in with
the deployment targets (e.g., -mmacosx-version-min=10.1 specifies that
we want to deploy back to Mac OS X 10.1). There are several concrete
behaviors that this attribute enables, as illustrated with the
function foo() above:
- If we choose a deployment target >= Mac OS X 10.4, uses of "foo"
will result in a deprecation warning, as if we had placed
attribute((deprecated)) on it (but with a better diagnostic)
- If we choose a deployment target >= Mac OS X 10.6, uses of "foo"
will result in an "unavailable" warning (in C)/error (in C++), as
if we had placed attribute((unavailable)) on it
- If we choose a deployment target prior to 10.2, foo() is
weak-imported (if it is a kind of entity that can be weak
imported), as if we had placed the weak_import attribute on it.
Naturally, there can be multiple availability attributes on a
declaration, for different platforms; only the current platform
matters when checking availability attributes.
The only platforms this attribute currently works for are "ios" and
"macosx", since we already have -mxxxx-version-min flags for them and we
have experience there with macro tricks translating down to the
deprecated/unavailable/weak_import attributes. The end goal is to open
this up to other platforms, and even extension to other "platforms"
that are really libraries (say, through a #pragma clang
define_system), but that hasn't yet been designed and we may want to
shake out more issues with this narrower problem first.
Addresses <rdar://problem/6690412>.
As a drive-by bug-fix, if an entity is both deprecated and
unavailable, we only emit the "unavailable" diagnostic.
llvm-svn: 128127
Change the interface to expose the new information and deal with the enormous fallout.
Introduce the new ExceptionSpecificationType value EST_DynamicNone to more easily deal with empty throw specifications.
Update the tests for noexcept and fix the various bugs uncovered, such as lack of tentative parsing support.
llvm-svn: 127537
template arguments. I believe that this is the last place in the AST
where we were storing a source range for a nested-name-specifier
rather than a proper nested-name-specifier location structure. (Yay!)
There is still a lot of cleanup to do in the TreeTransform, which
doesn't take advantage of nested-name-specifiers with source-location
information everywhere it could.
llvm-svn: 126844
template specialization types. There are still a few rough edges to
clean up with some of the parser actions dropping
nested-name-specifiers too early.
llvm-svn: 126776
nested-name-speciciers within elaborated type names, e.g.,
enum clang::NestedNameSpecifier::SpecifierKind
Fixes in this iteration include:
(1) Compute the type-source range properly for a dependent template
specialization type that starts with "template template-id ::", as
in a member access expression
dep->template f<T>::f()
This is a latent bug I triggered with this change (because now we're
checking the computed source ranges for dependent template
specialization types). But the real problem was...
(2) Make sure to set the qualifier range on a dependent template
specialization type appropriately. This will go away once we push
nested-name-specifier locations into dependent template
specialization types, but it was the source of the
valgrind errors on the buildbots.
llvm-svn: 126765
information for qualifier type names throughout the parser to address
several problems.
The commit message from r126737:
Push nested-name-specifier source location information into elaborated
name types, e.g., "enum clang::NestedNameSpecifier::SpecifierKind".
Aside from the normal changes, this also required some tweaks to the
parser. Essentially, when we're looking at a type name (via
getTypeName()) specifically for the purpose of creating an annotation
token, we pass down the flag that asks for full type-source location
information to be stored within the returned type. That way, we retain
source-location information involving nested-name-specifiers rather
than trying to reconstruct that information later, long after it's
been lost in the parser.
With this change, test/Index/recursive-cxx-member-calls.cpp is showing
much improved results again, since that code has lots of
nested-name-specifiers.
llvm-svn: 126748
name types, e.g., "enum clang::NestedNameSpecifier::SpecifierKind".
Aside from the normal changes, this also required some tweaks to the
parser. Essentially, when we're looking at a type name (via
getTypeName()) specifically for the purpose of creating an annotation
token, we pass down the flag that asks for full type-source location
information to be stored within the returned type. That way, we retain
source-location information involving nested-name-specifiers rather
than trying to reconstruct that information later, long after it's
been lost in the parser.
With this change, test/Index/recursive-cxx-member-calls.cpp is showing
much improved results again, since that code has lots of
nested-name-specifiers.
llvm-svn: 126737
DependentNameTypeLoc. Teach the recursive AST visitor and libclang how to
walk DependentNameTypeLoc nodes.
Also, teach libclang about TypedefDecl source ranges, so that we get
those. The massive churn in test/Index/recursive-cxx-member-calls.cpp
is a good thing: we're annotating a lot more of this test correctly
now.
llvm-svn: 126729
source-location information into a NestedNameSpecifierLocBuilder
class, which lives within the AST library and centralize all knowledge
of the format of nested-name-specifier location information here.
No functionality change.
llvm-svn: 126716
UnresolvedUsingValueDecl to use NestedNameSpecifierLoc rather than the
extremely-lossy NestedNameSpecifier/SourceRange pair it used to use,
improving source-location information.
Various infrastructure updates to support NestedNameSpecifierLoc:
- AST/PCH (de-)serialization
- Recursive AST visitor
- libclang traversal (including the first tests of this
functionality)
llvm-svn: 126459
way it keeps track of namespaces. Previously, we would map from the
namespace alias to its underlying namespace when building a
nested-name-specifier, losing source information in the process.
llvm-svn: 126358
LabelDecl and LabelStmt. There is a 1-1 correspondence between the
two, but this simplifies a bunch of code by itself. This is because
labels are the only place where we previously had references to random
other statements, causing grief for AST serialization and other stuff.
This does cause one regression (attr(unused) doesn't silence unused
label warnings) which I'll address next.
This does fix some minor bugs:
1. "The only valid attribute " diagnostic was capitalized.
2. Various diagnostics printed as ''labelname'' instead of 'labelname'
3. This reduces duplication of label checking between functions and blocks.
Review appreciated, particularly for the cindex and template bits.
llvm-svn: 125733
Store in PCH the directory that the PCH was originally created in.
If a header file is not found at the path that we expect it to be and the PCH file
was moved from its original location, try to resolve the file by assuming that
header+PCH were moved together and the header is in the same place relative to the PCH.
llvm-svn: 125576
Temporarily set the first (canonical) declaration as the previous one, which is the one that
matters, and mark the real previous DeclID to be loaded & attached later on.
Fixes rdar://8956193.
llvm-svn: 125434
AST/PCH files more lazy:
- Don't preload all of the file source-location entries when reading
the AST file. Instead, load them lazily, when needed.
- Only look up header-search information (whether a header was already
#import'd, how many times it's been included, etc.) when it's needed
by the preprocessor, rather than pre-populating it.
Previously, we would pre-load all of the file source-location entries,
which also populated the header-search information structure. This was
a relatively minor performance issue, since we would end up stat()'ing
all of the headers stored within a AST/PCH file when the AST/PCH file
was loaded. In the normal PCH use case, the stat()s were cached, so
the cost--of preloading ~860 source-location entries in the Cocoa.h
case---was relatively low.
However, the recent optimization that replaced stat+open with
open+fstat turned this into a major problem, since the preloading of
source-location entries would now end up opening those files. Worse,
those files wouldn't be closed until the file manager was destroyed,
so just opening a Cocoa.h PCH file would hold on to ~860 file
descriptors, and it was easy to blow through the process's limit on
the number of open file descriptors.
By eliminating the preloading of these files, we neither open nor stat
the headers stored in the PCH/AST file until they're actually needed
for something. Concretely, we went from
*** HeaderSearch Stats:
835 files tracked.
364 #import/#pragma once files.
823 included exactly once.
6 max times a file is included.
3 #include/#include_next/#import.
0 #includes skipped due to the multi-include optimization.
1 framework lookups.
0 subframework lookups.
*** Source Manager Stats:
835 files mapped, 3 mem buffers mapped.
37460 SLocEntry's allocated, 11215575B of Sloc address space used.
62 bytes of files mapped, 0 files with line #'s computed.
with a trivial program that uses a chained PCH including a Cocoa PCH
to
*** HeaderSearch Stats:
4 files tracked.
1 #import/#pragma once files.
3 included exactly once.
2 max times a file is included.
3 #include/#include_next/#import.
0 #includes skipped due to the multi-include optimization.
1 framework lookups.
0 subframework lookups.
*** Source Manager Stats:
3 files mapped, 3 mem buffers mapped.
37460 SLocEntry's allocated, 11215575B of Sloc address space used.
62 bytes of files mapped, 0 files with line #'s computed.
for the same program.
llvm-svn: 125286
record away from the core processor record. The tangling of these two
data structures led to some inefficiencies (e.g., deserializing all
of the detailed preprocessing record when we didn't need it, such as
while performing code completion) along with some unnecessary
ugliness.
llvm-svn: 125117
overridden via remapping. Thus, when we create a "virtual" file in the
file manager, we still stat() the real file that lives behind it so
that we can provide proper uniquing based on inodes. This helps keep
the file manager much more consistent.
To take advantage of this when reparsing files in libclang, we disable
the use of the stat() cache when reparsing or performing code
completion, since the stat() cache is very likely to be out of date in
this use case.
llvm-svn: 124971
FileManager.cpp: Allow virtual files in nonexistent directories.
FileManager.cpp: Close FileDescriptor for virtual files that correspond to actual files.
FileManager.cpp: Enable virtual files to be created even for files that were flagged as NON_EXISTENT_FILE, e.g. by a prior (unsuccessful) addFile().
ASTReader.cpp: Read a PCH even if the original source files cannot be found.
Add a test for reading a PCH of a file that has been removed and diagnostics referencing that file.
llvm-svn: 124374
- Add ref-qualifiers to the type system; they are part of the
canonical type. Print & profile ref-qualifiers
- Translate the ref-qualifier from the Declarator chunk for
functions to the function type.
- Diagnose mis-uses of ref-qualifiers w.r.t. static member
functions, free functions, constructors, destructors, etc.
- Add serialization and deserialization of ref-qualifiers.
llvm-svn: 124281
template template parameter pack that cannot be fully expanded because
its enclosing pack expansion could not be expanded. This form of
TemplateName plays the same role as SubstTemplateTypeParmPackType and
SubstNonTypeTemplateParmPackExpr do for template type parameter packs
and non-type template parameter packs, respectively.
We should now handle these multi-level pack expansion substitutions
anywhere. The largest remaining gap in our variadic-templates support
is that we cannot cope with non-type template parameter packs whose
type is a pack expansion.
llvm-svn: 123521
expansion, when it is known due to the substitution of an out
parameter pack. This allows us to properly handle substitution into
pack expansions that involve multiple parameter packs at different
template parameter levels, even when this substitution happens one
level at a time (as with partial specializations of member class
templates and the signatures of member function templates).
Note that the diagnostic we provide when there is an arity mismatch
between an outer parameter pack and an inner parameter pack in this
case isn't as clear as the normal diagnostic for an arity
mismatch. However, this doesn't matter because these cases are very,
very rare and (even then) only typically occur in a SFINAE context.
The other kinds of pack expansions (expression, template, etc.) still
need to support optional tracking of the number of expansions, and we
need the moral equivalent of SubstTemplateTypeParmPackType for
substituted argument packs of template template and non-type template
parameters.
llvm-svn: 123448
Douglas Gregor