Fix the current parsing of subframeworks in modulemaps to lookup for
headers based on whether they are frameworks.
rdar://problem/30563982
llvm-svn: 298391
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298278
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
This fixes lookup mismatches that could happen when the module cache
path contained a '/./' component.
<rdar://problem/30413458>
Differential Revision: https://reviews.llvm.org/D30915
llvm-svn: 297790
Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.
- Move definition ASTFileSignature to Basic/Module.h so Module and
ASTSourceDescriptor can use it.
- Change the signature from uint64_t to std::array<uint32_t,5>.
- Stop using (saving/reading) the size and modification time of PCM
files when there is a valid SIGNATURE.
- Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
and other records that shouldn't affect the hash. Because implicit
modules reuses the same file for multiple levels of -Werror, this
includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.
This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system. This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).
Design and original patch by Manman Ren!
llvm-svn: 297655
Printing typedefs or type aliases using clang_getTypeSpelling() is missing the
namespace they are defined in. This is in contrast to other types that always
yield the full typename including namespaces.
Patch by Michael Reiher!
Differential Revision: https://reviews.llvm.org/D29944
llvm-svn: 297465
Now print diagnostics for static, virtual, inline, volatile, and const
differences in methods. Also use DeclarationName instead of IdentifierInfo
for additional robustness in diagnostic printing.
llvm-svn: 296932
The goal of this is to fix a bug in modules where we'd merge
FunctionDecls that differed in their pass_object_size attributes. Since
we can overload on the presence of pass_object_size attributes, this
behavior is incorrect.
We don't represent `N` in `pass_object_size(N)` as part of
ExtParameterInfo, since it's an error to overload solely on the value of
N. This means that we have a bug if we have two modules that declare
functions that differ only in their pass_object_size attrs, like so:
// In module A, from a.h
void foo(char *__attribute__((pass_object_size(0))));
// In module B, from b.h
void foo(char *__attribute__((pass_object_size(1))));
// In module C, in main.c
#include "a.h"
#include "b.h"
At the moment, we'll merge the foo decls, when we should instead emit a
diagnostic about an invalid overload. We seem to have similar (silent)
behavior if we overload only on the return type of `foo` instead; I'll
try to find a good place to put a FIXME (or I'll just file a bug) soon.
This patch also fixes a bug where we'd not output the proper extended
parameter info for declarations with pass_object_size attrs.
llvm-svn: 296076
Fields will now have their types added to the hash, allowing for detection of
mismatched field types. This detection allows the existing ODR checking to
produce the correct message.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 295931
IdentifierInfo is hashed based on the stored string. FieldDecl versus other
Decl is now detected, as well as differently named fields.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 295911
Add support for static_cast in classes. Add pointer-independent profiling for
Stmt's, sharing most of the logic with Stmt::Profile. This is the first of the
deep sub-Decl diffing for error messages.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 295890
Add the basics for the ODRHash class, which will only process Decl's from
a whitelist, which currently only has AccessSpecDecl. Different access
specifiers in merged classes can now be detected.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 295800
A slightly weaker form of ODR checking than previous attempts, but hopefully
won't break the modules build bot. Future work will be needed to catch all
cases.
When objects are imported for modules, there is a chance that a name collision
will cause an ODR violation. Previously, only a small number of such
violations were detected. This patch provides a stronger check based on
AST nodes.
The information needed to uniquely identify an object is taken from the AST and
put into a one-dimensional byte stream. This stream is then hashed to give
a value to represent the object, which is stored with the other object data
in the module.
When modules are loaded, and Decl's are merged, the hash values of the two
Decl's are compared. Only Decl's with matched hash values will be merged.
Mismatch hashes will generate a module error, and if possible, point to the
first difference between the two objects.
The transform from AST to byte stream is a modified depth first algorithm.
Due to references between some AST nodes, a pure depth first algorithm could
generate loops. For Stmt nodes, a straight depth first processing occurs.
For Type and Decl nodes, they are replaced with an index number and only on
first visit will these nodes be processed. As an optimization, boolean
values are saved and stored together in reverse order at the end of the
byte stream to lower the ammount of data that needs to be hashed.
Compile time impact was measured at 1.5-2.0% during module building, and
negligible during builds without module building.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 295421
Recommit r293585 that was reverted in r293611 with new fixes. The previous
issue was determined to be an overly aggressive AST visitor from forward
declared objects. The visitor will now only deeply visit certain Decl's and
only do a shallow information extraction from all other Decl's.
When objects are imported for modules, there is a chance that a name collision
will cause an ODR violation. Previously, only a small number of such
violations were detected. This patch provides a stronger check based on
AST nodes.
The information needed to uniquely identify an object is taken from the AST and
put into a one-dimensional byte stream. This stream is then hashed to give
a value to represent the object, which is stored with the other object data
in the module.
When modules are loaded, and Decl's are merged, the hash values of the two
Decl's are compared. Only Decl's with matched hash values will be merged.
Mismatch hashes will generate a module error, and if possible, point to the
first difference between the two objects.
The transform from AST to byte stream is a modified depth first algorithm.
Due to references between some AST nodes, a pure depth first algorithm could
generate loops. For Stmt nodes, a straight depth first processing occurs.
For Type and Decl nodes, they are replaced with an index number and only on
first visit will these nodes be processed. As an optimization, boolean
values are saved and stored together in reverse order at the end of the
byte stream to lower the ammount of data that needs to be hashed.
Compile time impact was measured at 1.5-2.0% during module building, and
negligible during builds without module building.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 295284
Two functions that differ only in their enable_if attributes are
considered overloads, so we should check for those when we're trying to
figure out if two functions are mergeable.
We need to do the same thing for pass_object_size, as well. Looks like
that'll be a bit less trivial, since we sometimes do these merging
checks before we have pass_object_size attributes available (see the
merge checks in ASTDeclReader::VisitFunctionDecl that happen before we
read parameters, and merge checks in calls to ReadDeclAs<>()).
llvm-svn: 295252
Following up on r291465 after a regression in r276159. When we use
-fmodule-name=X while building a PCH, modular headers in X will be
textually included and the compiler knows that we are not building
module X, so don't serialize such headers in the PCH as being part of a
module, because at this point they are not.
This was causing subtle bugs and malformed AST crashes, for instance,
when using the PCH in subsequent compiler invocation with -fmodules, the
HFI for a modular header would map to the PCH, which would force a
module load of and unexistent module ID.
rdar://problem/30171164
llvm-svn: 294361
We're seeing what we believe are false positives. (It's hard to tell with the
available diagnostics, and I'm not sure how to reduce them yet).
I'll send Richard reproduction details offline.
djasper/chandlerc suggested this should be a warning for now, to make rolling it
out feasible.
llvm-svn: 293611
When objects are imported for modules, there is a chance that a name collision
will cause an ODR violation. Previously, only a small number of such
violations were detected. This patch provides a stronger check based on
AST nodes.
The information needed to uniquely identify an object is taked from the AST and
put into a one-dimensional byte stream. This stream is then hashed to give
a value to represent the object, which is stored with the other object data
in the module.
When modules are loaded, and Decl's are merged, the hash values of the two
Decl's are compared. Only Decl's with matched hash values will be merged.
Mismatch hashes will generate a module error, and if possible, point to the
first difference between the two objects.
The transform from AST to byte stream is a modified depth first algorithm.
Due to references between some AST nodes, a pure depth first algorithm could
generate loops. For Stmt nodes, a straight depth first processing occurs.
For Type and Decl nodes, they are replaced with an index number and only on
first visit will these nodes be processed. As an optimization, boolean
values are saved and stored together in reverse order at the end of the
byte stream to lower the ammount of data that needs to be hashed.
Compile time impact was measured at 1.5-2.0% during module building, and
negligible during builds without module building.
Differential Revision: https://reviews.llvm.org/D21675
llvm-svn: 293585
First pass at generating weak definitions of inline functions from module files
(& skipping (-O0) or emitting available_externally (optimizations)
definitions where those modules are used).
External functions defined in modules are emitted into the modular
object file as well (this may turn an existing ODR violation (if that
module were imported into multiple translations) into valid/linkable
code).
Internal symbols (static functions, for example) are not correctly
supported yet. The symbol will be produced, internal, in the modular
object - unreferenceable from the users.
Reviewers: rsmith
Differential Revision: https://reviews.llvm.org/D28845
llvm-svn: 293456
another declaration, ensure we actually serialize / deserialize that
declaration.
Before this patch, if another copy of the typedef were merged with the parsed
version, we would emit type information referring to the merged version and
consequently emit nothing about the parsed anonymous struct. This resulted in
us losing information, particularly the visible merged module set for the
parsed definition. Force that information to be emitted and to be loaded when
the typedef is used.
llvm-svn: 293219
Vassil Vassilev