Reduce memory footprint of AST Reader/Writer:
1. Adjust internal data containers' element type.
2. Switch to set for deduplication of deferred diags.
Differential Revision: https://reviews.llvm.org/D101793
The idiom:
```
DeclContext::lookup_result R = DeclContext::lookup(Name);
for (auto *D : R) {...}
```
is not safe when in the loop body we trigger deserialization from an AST file.
The deserialization can insert new declarations in the StoredDeclsList whose
underlying type is a vector. When the vector decides to reallocate its storage
the pointer we hold becomes invalid.
This patch replaces a SmallVector with an singly-linked list. The current
approach stores a SmallVector<NamedDecl*, 4> which is around 8 pointers.
The linked list is 3, 5, or 7. We do better in terms of memory usage for small
cases (and worse in terms of locality -- the linked list entries won't be near
each other, but will be near their corresponding declarations, and we were going
to fetch those memory pages anyway). For larger cases: the vector uses a
doubling strategy for reallocation, so will generally be between half-full and
full. Let's say it's 75% full on average, so there's N * 4/3 + 4 pointers' worth
of space allocated currently and will be 2N pointers with the linked list. So we
break even when there are N=6 entries and slightly lose in terms of memory usage
after that. We suspect that's still a win on average.
Thanks to @rsmith!
Differential revision: https://reviews.llvm.org/D91524
I was trying to pick this up a bit when reviewing D48426 (& perhaps D69778) - in any case, looks like D48426 added a module level flag that might not be needed.
The D48426 implementation worked by setting a module level flag, then code generating contents from the PCH a special case in ASTContext::DeclMustBeEmitted would be used to delay emitting the definition of these functions if they came from a Module with this flag.
This strategy is similar to the one initially implemented for modular codegen that was removed in D29901 in favor of the modular decls list and a bit on each decl to specify whether it's homed to a module.
One major difference between PCH object support and modular code generation, other than the specific list of decls that are homed, is the compilation model: MSVC PCH modules are built into the object file for some other source file (when compiling that source file /Yc is specified to say "this compilation is where the PCH is homed"), whereas modular code generation invokes a separate compilation for the PCH alone. So the current modular code generation test of to decide if a decl should be emitted "is the module where this decl is serialized the current main file" has to be extended (as Lubos did in D69778) to also test the command line flag -building-pch-with-obj.
Otherwise the whole thing is basically streamlined down to the modular code generation path.
This even offers one extra material improvement compared to the existing divergent implementation: Homed functions are not emitted into object files that use the pch. Instead at -O0 they are not emitted into the IR at all, and at -O1 they are emitted using available_externally (existing functionality implemented for modular code generation). The pch-codegen test has been updated to reflect this new behavior.
[If possible: I'd love it if we could not have the extra MSVC-style way of accessing dllexport-pch-homing, and just do it the modular codegen way, but I understand that it might be a limitation of existing build systems. @hans / @thakis: Do either of you know if it'd be practical to move to something more similar to .pcm handling, where the pch itself is passed to the compilation, rather than homed as a side effect of compiling some other source file?]
Reviewers: llunak, hans
Differential Revision: https://reviews.llvm.org/D83652
Summary:
We currently have some very basic LLVM-style RTTI support in the ExternalASTSource class hierarchy
based on the `SemaSource` bool( to discriminate it form the ExternalSemaSource). As ExternalASTSource
is supposed to be subclassed we should have extendable LLVM-style RTTI in this class hierarchy to make life easier
for projects building on top of Clang.
Most notably the current RTTI implementation forces LLDB to implement RTTI for its
own ExternalASTSource class (ClangExternalASTSourceCommon) by keeping a global set of
ExternalASTSources that are known to be ClangExternalASTSourceCommon. Projects
using Clang currently have to dosimilar workarounds to get RTTI support for their subclasses.
This patch turns this into full-fledged LLVM-style RTTI based on a static `ID` variable similar to
other LLVM class hierarchies. Also removes the friend declaration from ExternalASTSource to
its child class that was only used to grant access to the `SemaSource` member.
Reviewers: aprantl, dblaikie, rjmccall
Reviewed By: aprantl
Subscribers: riccibruno, labath, lhames, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D71397
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
With MSVC, PCH files are created along with an object file that needs to
be linked into the final library or executable. That object file
contains the code generated when building the headers. In particular, it
will include definitions of inline dllexport functions, and because they
are emitted in this object file, other files using the PCH do not need
to emit them. See the bug for an example.
This patch makes clang-cl match MSVC's behaviour in this regard, causing
significant compile-time savings when building dlls using precompiled
headers.
For example, in a 64-bit optimized shared library build of Chromium with
PCH, it reduces the binary size and compile time of
stroke_opacity_custom.obj from 9315564 bytes to 3659629 bytes and 14.6
to 6.63 s. The wall-clock time of building blink_core.dll goes from
38m41s to 22m33s. ("user" time goes from 1979m to 1142m).
Differential Revision: https://reviews.llvm.org/D48426
llvm-svn: 335466
This is similar to the LLVM change https://reviews.llvm.org/D46290.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
Differential Revision: https://reviews.llvm.org/D46320
llvm-svn: 331834
Summary:
The MultiplexExternalSemaSource doesn't correctly overload the `getModule` function,
causing the multiplexer to not forward this call as intended.
Reviewers: v.g.vassilev
Reviewed By: v.g.vassilev
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D39416
llvm-svn: 323122
Matching the function-homing support for modular codegen. Any type
implicitly (implicit template specializations) or explicitly defined in
a module is attached to that module's object file and omitted elsewhere
(only a declaration used if necessary for references).
llvm-svn: 299987
Some decls are created not where they are written, but in other module
files/users (implicit special members and function template implicit
specializations). To correctly identify them, use a bit next to the definition
to track the modular codegen property.
Discussed whether the module file bit could be omitted in favor of
reconstituting from the modular codegen decls list - best guess today is that
the efficiency improvement of not having to deserialize the whole list whenever
any function is queried by a module user is worth it for the small size
increase of this redundant (list + bit-on-def) representation.
Reviewers: rsmith
Differential Revision: https://reviews.llvm.org/D29901
llvm-svn: 299982
First pass at generating weak definitions of inline functions from module files
(& skipping (-O0) or emitting available_externally (optimizations)
definitions where those modules are used).
External functions defined in modules are emitted into the modular
object file as well (this may turn an existing ODR violation (if that
module were imported into multiple translations) into valid/linkable
code).
Internal symbols (static functions, for example) are not correctly
supported yet. The symbol will be produced, internal, in the modular
object - unreferenceable from the users.
Reviewers: rsmith
Differential Revision: https://reviews.llvm.org/D28845
llvm-svn: 293456
Summary:
This is possible now that MapVector supports move-only values.
Depends on D25404.
Reviewers: timshen
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D25405
llvm-svn: 283766
a selector, the entry should be complete, containing everything introduced by
that module and all modules it imports.
Before writing out the method pool of a module, we sync up the out of date
selectors by pulling in methods for the selectors, from all modules it imports.
In ReadMethodPool, after pulling in the method pool entry for module A, this
lets us skip the modules that module A imports.
rdar://problem/25900131
llvm-svn: 268091
non-deterministic diagnostics (and non-deterministic PCH files). Check these
when building a module rather than serializing it; it's not reasonable for a
module's use to be satisfied by a definition in the user of the module.
llvm-svn: 264466
Summary: It breaks the build for the ASTMatchers
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D13893
llvm-svn: 250827
useless return value. Switch to using it directly when completing the
redeclaration chain for an anonymous declaration, and reduce the set of
declarations that we load in the process to just those of the right kind.
llvm-svn: 244161
Emit warning when operand to `delete` is allocated with `new[]` or
operand to `delete[]` is allocated with `new`.
rev 2 update:
`getNewExprFromInitListOrExpr` should return `dyn_cast_or_null`
instead of `dyn_cast`, since `E` might be null.
Reviewers: rtrieu, jordan_rose, rsmith
Subscribers: majnemer, cfe-commits
Differential Revision: http://reviews.llvm.org/D4661
llvm-svn: 237608
This reverts commit 742dc9b6c9686ab52860b7da39c3a126d8a97fbc.
This is generating multiple segfaults in our internal builds.
Test case coming up shortly.
llvm-svn: 237391
Emit warning when operand to `delete` is allocated with `new[]` or
operand to `delete[]` is allocated with `new`.
Reviewers: rtrieu, jordan_rose, rsmith
Subscribers: majnemer, cfe-commits
Differential Revision: http://reviews.llvm.org/D4661
llvm-svn: 237368
Clang was inserting these into a dense map. While it never iterated the
dense map during normal compilation, it did when emitting a module. Fix
this by using a standard MapVector to preserve the order in which we
encounter the late parsed templates.
I suspect this still isn't ideal, as we don't seem to remove things from
this map even when we mark the templates as no longer late parsed. But
I don't know enough about this particular extension to craft a nice,
subtle test case covering this. I've managed to get the stress test to
at least do some late parsing and demonstrate the core problem here.
This patch fixes the test and provides deterministic behavior which is
a strict improvement over the prior state.
I've cleaned up some of the code here as well to be explicit about
inserting when that is what is actually going on.
llvm-svn: 233264
Previously we'd deserialize the list of mem-initializers for a constructor when
we deserialized the declaration of the constructor. That could trigger a
significant amount of unnecessary work (pulling in all base classes
recursively, for a start) and was causing problems for the modules buildbot due
to cyclic deserializations. We now deserialize these on demand.
This creates a certain amount of duplication with the handling of
CXXBaseSpecifiers; I'll look into reducing that next.
llvm-svn: 233052
of extern "C" declarations. This is simpler and vastly more efficient for
modules builds (we no longer need to load *all* extern "C" declarations to
determine if we have a redeclaration).
No functionality change intended.
llvm-svn: 231538
dynamic classes in the translation unit and check whether each one's key
function is defined when we got to the end of the TU (and when we got to the
end of each module). This is really terrible for modules performance, since it
causes unnecessary deserialization of every dynamic class in every compilation.
We now use a much simpler (and, in a modules build, vastly more efficient)
system: when we see an out-of-line definition of a virtual function, we check
whether that function was in fact its class's key function. (If so, we need to
emit the vtable.)
llvm-svn: 230830
The warning warns on TypedefNameDecls -- typedefs and C++11 using aliases --
that are !isReferenced(). Since the isReferenced() bit on TypedefNameDecls
wasn't used for anything before this warning it wasn't always set correctly,
so this patch also adds a few missing MarkAnyDeclReferenced() calls in
various places for TypedefNameDecls.
This is made a bit complicated due to local typedefs possibly being used only
after their local scope has closed. Consider:
template <class T>
void template_fun(T t) {
typename T::Foo s3foo; // YYY
(void)s3foo;
}
void template_fun_user() {
struct Local {
typedef int Foo; // XXX
} p;
template_fun(p);
}
Here the typedef in XXX is only used at end-of-translation unit, when YYY in
template_fun() gets instantiated. To handle this, typedefs that are unused when
their scope exits are added to a set of potentially unused typedefs, and that
set gets checked at end-of-TU. Typedefs that are still unused at that point then
get warned on. There's also serialization code for this set, so that the
warning works with precompiled headers and modules. For modules, the warning
is emitted when the module is built, for precompiled headers each time the
header gets used.
Finally, consider a function using C++14 auto return types to return a local
type defined in a header:
auto f() {
struct S { typedef int a; };
return S();
}
Here, the typedef escapes its local scope and could be used by only some
translation units including the header. To not warn on this, add a
RecursiveASTVisitor that marks all delcs on local types returned from auto
functions as referenced. (Except if it's a function with internal linkage, or
the decls are private and the local type has no friends -- in these cases, it
_is_ safe to warn.)
Several of the included testcases (most of the interesting ones) were provided
by Richard Smith.
(gcc's spelling -Wunused-local-typedefs is supported as an alias for this
warning.)
llvm-svn: 217298
ensure that querying the first declaration for its most recent declaration
checks for redeclarations from the imported module.
This works as follows:
* The 'most recent' pointer on a canonical declaration grows a pointer to the
external AST source and a generation number (space- and time-optimized for
the case where there is no external source).
* Each time the 'most recent' pointer is queried, if it has an external source,
we check whether it's up to date, and update it if not.
* The ancillary data stored on the canonical declaration is allocated lazily
to avoid filling it in for declarations that end up being non-canonical.
We'll still perform a redundant (ASTContext) allocation if someone asks for
the most recent declaration from a decl before setPreviousDecl is called,
but such cases are probably all bugs, and are now easy to find.
Some finessing is still in order here -- in particular, we use a very general
mechanism for handling the DefinitionData pointer on CXXRecordData, and a more
targeted approach would be more compact.
Also, the MayHaveOutOfDateDef mechanism should now be expunged, since it was
addressing only a corner of the full problem space here. That's not covered
by this patch.
Early performance benchmarks show that this makes no measurable difference to
Clang performance without modules enabled (and fixes a major correctness issue
with modules enabled). I'll revert if a full performance comparison shows any
problems.
llvm-svn: 209046
if it found any decls, rather than returning a list of found decls. This
removes a returning-ArrayRef-to-deleted-storage bug from
MultiplexExternalSemaSource (in code not exercised by any of the clang
binaries), reduces the work required in the found-no-decls case with PCH, and
importantly removes the need for DeclContext::lookup to be reentrant.
No functionality change intended!
llvm-svn: 174576
This does limit these typedefs to being sequences, but no current usage
requires them to be contiguous (we could expand this to a more general
iterator pair range concept at some point).
Also, it'd be nice if SmallVector were constructible directly from an ArrayRef
but this is a bit tricky since ArrayRef depends on SmallVectorBaseImpl for the
inverse conversion. (& generalizing over all range-like things, while nice,
would require some nontrivial SFINAE I haven't thought about yet)
llvm-svn: 170482
uncovered.
This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.
I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.
llvm-svn: 169237
Wei Wang