when saving a module timestamp file
This commit doesn't include a test as it requires a test that reproduces
a file write/close error that couldn't really be constructed artificially.
rdar://31860650
Differential Revision: https://reviews.llvm.org/D33357
llvm-svn: 304538
This patch adds support for a `header` declaration in a module map to specify
certain `stat` information (currently, size and mtime) about that header file.
This has two purposes:
- It removes the need to eagerly `stat` every file referenced by a module map.
Instead, we track a list of unresolved header files with each size / mtime
(actually, for simplicity, we track submodules with such headers), and when
attempting to look up a header file based on a `FileEntry`, we check if there
are any unresolved header directives with that `FileEntry`'s size / mtime and
perform deferred `stat`s if so.
- It permits a preprocessed module to be compiled without the original files
being present on disk. The only reason we used to need those files was to get
the `stat` information in order to do header -> module lookups when using the
module. If we're provided with the `stat` information in the preprocessed
module, we can avoid requiring the files to exist.
Unlike most `header` directives, if a `header` directive with `stat`
information has no corresponding on-disk file the enclosing module is *not*
marked unavailable (so that behavior is consistent regardless of whether we've
resolved a header directive, and so that preprocessed modules don't get marked
unavailable). We could actually do this for all `header` directives: the only
reason we mark the module unavailable if headers are missing is to give a
diagnostic slightly earlier (rather than waiting until we actually try to build
the module / load and validate its .pcm file).
Differential Revision: https://reviews.llvm.org/D33703
llvm-svn: 304515
This patch makes it an error to have a mismatch between the enabled
sanitizers in a CU, and in any module being imported into the CU. Only
mismatches between non-modular sanitizers are treated as errors.
This patch also includes non-modular sanitizers in module hashes, in
order to ensure module rebuilds occur when -fsanitize=X is toggled on
and off for non-modular sanitizers, and to cut down on module rebuilds
when the option is toggled for modular sanitizers.
This fixes a longstanding issue with implicit modules and sanitizers,
which Duncan originally diagnosed.
When building with implicit modules it's possible to hit a scenario
where modules are built without -fsanitize=address, and are subsequently
imported into CUs with -fsanitize=address enabled. This causes strange
failures at runtime. The case Duncan found affects libcxx, since its
vector implementation behaves differently when ASan is enabled.
Implicit module builds should "just work" when -fsanitize=X is toggled
on and off across multiple compiler invocations, which is what this
patch does.
Differential Revision: https://reviews.llvm.org/D32724
llvm-svn: 304463
to the original module map.
Also use the path and name of the original module map when emitting that
information into the .pcm file. The upshot of this is that the produced .pcm
file will track information for headers in their original locations (where the
module was preprocessed), not relative to whatever directory the preprocessed
module map was in when it was built.
llvm-svn: 304346
Previously, a preamble only included #if blocks (and friends like
ifdef) if there was a corresponding #endif before any declaration or
definition. The problem is that any header file that uses include guards
will not have a preamble generated, which can make code-completion very
slow.
To prevent errors about unbalanced preprocessor conditionals in the
preamble, and unbalanced preprocessor conditionals after a preamble
containing unfinished conditionals, the conditional stack is stored
in the pch file.
This fixes PR26045.
Differential Revision: http://reviews.llvm.org/D15994
llvm-svn: 304207
As discussed in D30793, we have some unsafe calls to isConsumerInterestedIn().
This patch implements Richard's suggestion (from the inline comment) that we
should track if we just deserialized an declaration. If we just deserialized,
we can skip the unsafe call because we know it's interesting. If we didn't just
deserialize the declaration, calling isConsumerInterestedIn() should be safe.
We tried to create a test case for this but we were not successful.
Patch by Raphael Isemann (D32499)!
llvm-svn: 303432
inferring based on the current module at the point of creation.
This should result in no functional change except when building a preprocessed
module (or more generally when using #pragma clang module begin/end to switch
module in the middle of a file), in which case it allows us to correctly track
the owning module for declarations. We can't map from FileID to module in the
preprocessed module case, since all modules would have the same FileID.
There are still a couple of remaining places that try to infer a module from a
source location; I'll clean those up in follow-up changes.
llvm-svn: 303322
I think this is a false positive in GCC's warning, but nonetheless, we
should try to be warning-free. Smaller reproducer (reproduces with GCC
6.3):
https://godbolt.org/g/cJuO2z
llvm-svn: 302003
The intent for an explicit module build is that the diagnostics produced within
the module are those that were configured when the module was built, not those
that are enabled within a user of the module. This includes diagnostics that
don't actually show up until the module is used (for instance, diagnostics
produced during template instantiation and weird cases like -Wpadded).
We serialized and restored the diagnostic state for individual warning groups,
but previously did not track the state for flags like -Werror and -Weverything,
which are implemented as separate bits rather than as part of the diagnostics
mapping information.
llvm-svn: 301992
If a file has no diagnostic pragmas, we build its diagnostic state lazily, but
in this case we never set up the root state to be the diagnostic state in which
the module was originally built, so the diagnostic flags for files in the
module with no diagnostic pragmas were incorrectly based on the user of the
module rather than the diagnostic state when the module was built.
llvm-svn: 301846
https://reviews.llvm.org/D32237
This patch prepares sema with additional fields to support all those composite and combined constructs of OpenMP that include pragma 'distribute' and 'for', such as 'distribute parallel for'. It also extends the regression tests for 'distribute parallel for' and adds a new one.
llvm-svn: 300802
The modules side of r299226, which serializes #pragma pack state,
doesn't work well.
The main purpose was to make -include and -include-pch match semantics
(the PCH side). We also started serializing #pragma pack in PCMs, in
the hopes of making modules and non-modules builds more consistent. But
consider:
$ cat a.h
$ cat b.h
#pragma pack(push, 2)
$ cat module.modulemap
module M {
module a { header "a.h" }
module b { header "b.h" }
}
$ cat t.cpp
#include "a.h"
#pragma pack(show)
As of r299226, the #pragma pack(show) gives "2", even though we've only
included "a.h".
- With -fmodules-local-submodule-visibility, this is clearly wrong. We
should get the default state (8 on x86_64).
- Without -fmodules-local-submodule-visibility, this kind of matches how
other things work (as if include-the-whole-module), but it's still
really terrible, and it doesn't actually make modules and non-modules
builds more consistent.
This commit disables the serialization for modules, essentially a
partial revert of r299226.
Going forward:
1. Having this #pragma pack stuff escape is terrible design (or, more
often, a horrible bug). We should prioritize adding warnings (maybe
-Werror by default?).
2. If we eventually reintroduce this for modules, it should only apply
to -fmodules-local-submodule-visibility, and it should be tracked on
a per-submodule basis.
llvm-svn: 300380
This patch implements the suggestion in D29753 that calling DeclMustBeEmitted in
the middle of deserialization should be avoided and that the actual check should
be deferred until it's safe to do so.
This patch fixes a crash when accessing the invalid redecl chains while trying
to evaluate the value of a const VarDecl that contains a function call.
Patch by Raphael Isemann (D30793)!
llvm-svn: 300110
This allows using and testing these two features separately. (noteably,
debug info is, so far as I know, always a win (basically). But function
modular codegen is currently a loss for highly optimized code - where
most of the linkonce_odr definitions are optimized away, so providing
weak_odr definitions is only overhead)
llvm-svn: 300104
r293123 started serializing diagnostic pragma state for modules. This
makes the serialization work properly for implicit modules.
An implicit module build (using Clang's internal build system) uses the
same PCM file location for different `-Werror` levels.
E.g., if a TU has `-Werror=format` and tries to load a PCM built without
`-Werror=format`, a new PCM will be built in its place (and the new PCM
should have the same signature, since r297655). In the other direction,
if a TU does not have `-Werror=format` and tries to load a PCM built
with `-Werror=format`, it should "just work".
The idea is to evolve the PCM toward the strictest -Werror flags that
anyone tries.
r293123 started serializing the diagnostic pragma state for each PCM.
Since this encodes the -Werror settings at module-build time, it breaks
the implicit build model.
This commit filters the diagnostic state in order to simulate the
current compilation's diagnostic settings. Firstly, it ignores the
module's serialized first diagnostic state, replacing it with the state
from this compilation's command-line. Secondly, if a pragma warning was
upgraded to error/fatal when generating the PCM (e.g., due to `-Werror`
on the command-line), it checks whether it should still be upgraded in
its current context.
llvm-svn: 300025
Emit the final diagnostic state last to match source order. This also
prepares for a follow-up commit for implicit modules.
There's no real functionaliy change, just a slightly different AST file
format.
llvm-svn: 300024
r299989 fixes the underlying issue by waiting long enough to late parsed
arguments to be processed before doing an calculating the hash.
r298742
[ODRHash] Add error messages for mismatched parameters in methods.
r298754
[ODRHash] Add support for array and decayed types.
llvm-svn: 300001
Calculating the hash in Sema::ActOnTagFinishDefinition could happen before
all sub-Decls were parsed or processed, which would produce the wrong hash
value. Change to calculating the hash on the first use and storing the value
instead. Also, avoid using the macros that were only for Boolean fields and
use an explicit checker during the DefintionData merge. No functional change,
but was this blocking other ODRHash patches.
llvm-svn: 299989
Matching the function-homing support for modular codegen. Any type
implicitly (implicit template specializations) or explicitly defined in
a module is attached to that module's object file and omitted elsewhere
(only a declaration used if necessary for references).
llvm-svn: 299987
Some decls are created not where they are written, but in other module
files/users (implicit special members and function template implicit
specializations). To correctly identify them, use a bit next to the definition
to track the modular codegen property.
Discussed whether the module file bit could be omitted in favor of
reconstituting from the modular codegen decls list - best guess today is that
the efficiency improvement of not having to deserialize the whole list whenever
any function is queried by a module user is worth it for the small size
increase of this redundant (list + bit-on-def) representation.
Reviewers: rsmith
Differential Revision: https://reviews.llvm.org/D29901
llvm-svn: 299982
This patch serializes the state of #pragma pack. It preserves the state of the
pragma from a PCH/from modules in a file that uses that PCH/those modules.
rdar://21359084
Differential Revision: https://reviews.llvm.org/D31241
llvm-svn: 299226
Sema holds the current FPOptions which is adjusted by 'pragma STDC
FP_CONTRACT'. This then gets propagated into expression nodes as they are
built.
This encapsulates FPOptions so that this propagation happens opaquely rather
than directly with the fp_contractable on/off bit. This allows controlled
transitioning of fp_contractable to a ternary value (off, on, fast). It will
also allow adding more fast-math flags later.
This is toward moving fp-contraction=fast from an LLVM TargetOption to a
FastMathFlag in order to fix PR25721.
Differential Revision: https://reviews.llvm.org/D31166
llvm-svn: 298877
This restores behavior pre-r230064 since after PCMCache work (r298278)
we don't reload PCMs from disk within the same compiler invocation.
Testcases from r230064 are still left around since they still guarantee
the correct behavior we're expecting.
rdar://problem/19889777
llvm-svn: 298464
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298278
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
This enhances the AST to keep track of locations of the names in those ObjC property attributes, and reports them for indexing.
Patch by Nathan Hawes!
https://reviews.llvm.org/D30907
llvm-svn: 297972
Since bitcode uses VBR encoding, large numbers are more expensive than
small ones. Instead of emitting a UINT_MAX sentinel after each sequence
of state-change pairs, emit the size of the sequence as a prefix.
This should have no functionality change besides saving bits from the
encoding.
llvm-svn: 297770
Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.
- Move definition ASTFileSignature to Basic/Module.h so Module and
ASTSourceDescriptor can use it.
- Change the signature from uint64_t to std::array<uint32_t,5>.
- Stop using (saving/reading) the size and modification time of PCM
files when there is a valid SIGNATURE.
- Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
and other records that shouldn't affect the hash. Because implicit
modules reuses the same file for multiple levels of -Werror, this
includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.
This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system. This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).
Design and original patch by Manman Ren!
llvm-svn: 297655
Summary:
The changes contained in this patch are:
1. Defines a new AST node `CoawaitDependentExpr` for representing co_await expressions while the promise type is still dependent.
2. Correctly detect and transform the 'co_await' operand to `p.await_transform(<expr>)` when possible.
3. Change the initial/final suspend points to build during the initial parse, so they have the correct operator co_await lookup results.
4. Fix transformation of the CoroutineBodyStmt so that it doesn't re-build the final/initial suspends.
@rsmith: This change is a little big, but it's not trivial for me to split it up. Please let me know if you would prefer this submitted as multiple patches.
Reviewers: rsmith, GorNishanov
Reviewed By: rsmith
Subscribers: ABataev, rsmith, mehdi_amini, cfe-commits
Differential Revision: https://reviews.llvm.org/D26057
llvm-svn: 297093
Benjamin Kramer