This patch adds support for a `header` declaration in a module map to specify
certain `stat` information (currently, size and mtime) about that header file.
This has two purposes:
- It removes the need to eagerly `stat` every file referenced by a module map.
Instead, we track a list of unresolved header files with each size / mtime
(actually, for simplicity, we track submodules with such headers), and when
attempting to look up a header file based on a `FileEntry`, we check if there
are any unresolved header directives with that `FileEntry`'s size / mtime and
perform deferred `stat`s if so.
- It permits a preprocessed module to be compiled without the original files
being present on disk. The only reason we used to need those files was to get
the `stat` information in order to do header -> module lookups when using the
module. If we're provided with the `stat` information in the preprocessed
module, we can avoid requiring the files to exist.
Unlike most `header` directives, if a `header` directive with `stat`
information has no corresponding on-disk file the enclosing module is *not*
marked unavailable (so that behavior is consistent regardless of whether we've
resolved a header directive, and so that preprocessed modules don't get marked
unavailable). We could actually do this for all `header` directives: the only
reason we mark the module unavailable if headers are missing is to give a
diagnostic slightly earlier (rather than waiting until we actually try to build
the module / load and validate its .pcm file).
Differential Revision: https://reviews.llvm.org/D33703
llvm-svn: 304515
to the original module map.
Also use the path and name of the original module map when emitting that
information into the .pcm file. The upshot of this is that the produced .pcm
file will track information for headers in their original locations (where the
module was preprocessed), not relative to whatever directory the preprocessed
module map was in when it was built.
llvm-svn: 304346
Previously, a preamble only included #if blocks (and friends like
ifdef) if there was a corresponding #endif before any declaration or
definition. The problem is that any header file that uses include guards
will not have a preamble generated, which can make code-completion very
slow.
To prevent errors about unbalanced preprocessor conditionals in the
preamble, and unbalanced preprocessor conditionals after a preamble
containing unfinished conditionals, the conditional stack is stored
in the pch file.
This fixes PR26045.
Differential Revision: http://reviews.llvm.org/D15994
llvm-svn: 304207
and it has an include guard, produce callbacks for a module import, not for a
skipped non-modular header.
Fixes -E output when preprocessing a module to list these cases as a module
import, rather than suppressing the #include and losing the import side effect.
llvm-svn: 304183
This is an initial commit to allow using it with constant expressions, a follow-up commit will enable full support for it in ObjC methods.
llvm-svn: 303712
C++14 added user-defined literal support for complex numbers so that you can
write something like "complex<double> val = 2i". However, there is an existing
GNU extension supporting this syntax and interpreting the result as a _Complex
type.
This changes parsing so that such literals are interpreted in terms of C++14's
operators if an overload is present but otherwise falls back to the original
GNU extension.
llvm-svn: 303694
This allows #line directives to appear in system headers that have code
that clang would normally warn on. This is compatible with GCC, which is
easy to test by running `gcc -E`.
Fixes PR30752
llvm-svn: 303582
remove the mechanism for doing so.
This mechanism was incorrect in the presence of preprocessed modules (and
#pragma clang module begin/end).
llvm-svn: 303469
When reaching the end of a module, we used to convert its macros to
ModuleMacros but also leave them in the MacroDirective chain for the
identifier. This meant that every lookup of such a macro would find two
(identical) definitions. It also made it difficult to determine the correct
owner for a macro when reaching the end of a module: the most recent
MacroDirective in the chain could be from an #included submodule rather than
the current module.
Simplify this: whenever we convert a MacroDirective to a ModuleMacro when
leaving a module, clear out the MacroDirective chain for that identifier, and
just rely on the ModuleMacro to provide the macro definition information.
(We don't want to do this for local submodule visibility mode, because in that
mode we maintain a distinct MacroDirective chain for each submodule, and we
need to keep around the prior MacroDirective in case we re-enter the submodule
-- for instance, if its header is #included more than once in a module build,
we need the include guard directive to stick around. But the problem doesn't
arise in this case for the same reason: each submodule has its own
MacroDirective chain, so the macros don't leak out of submodules in the first
place.)
This reinstates r302932, reverted in r302947, with a fix for a bug that
resulted in us sometimes losing macro definitions due to failing to clear out
the overridden module macro list when promoting a directive to a module macro.
llvm-svn: 303468
retrieving the identifer info for an Objective-C keyword
This commit fixes an assertion that's triggered in getIdentifier when the token
is an annotation token.
rdar://32225463
llvm-svn: 303246
When reaching the end of a module, we used to convert its macros to
ModuleMacros but also leave them in the MacroDirective chain for the
identifier. This meant that every lookup of such a macro would find two
(identical) definitions. It also made it difficult to determine the correct
owner for a macro when reaching the end of a module: the most recent
MacroDirective in the chain could be from an #included submodule rather than
the current module.
Simplify this: whenever we convert a MacroDirective to a ModuleMacro when
leaving a module, clear out the MacroDirective chain for that identifier, and
just rely on the ModuleMacro to provide the macro definition information.
(We don't want to do this for local submodule visibility mode, because in that
mode we maintain a distinct MacroDirective chain for each submodule, and we
need to keep around the prior MacroDirective in case we re-enter the submodule
-- for instance, if its header is #included more than once in a module build,
we need the include guard directive to stick around. But the problem doesn't
arise in this case for the same reason: each submodule has its own
MacroDirective chain, so the macros don't leak out of submodules in the first
place.)
llvm-svn: 302932
In r298391 we fixed the umbrella framework model to work when submodules
named "Private" are used. This complements the work by allowing the
umbrella framework model to work in general.
rdar://problem/31790067
llvm-svn: 302491
To support this, an optional marker "#pragma clang module contents" is
recognized in module map files, and the rest of the module map file from that
point onwards is treated as the source of the module. Preprocessing a module
map produces the input module followed by the marker and then the preprocessed
contents of the module.
Ignoring line markers, a preprocessed module might look like this:
module A {
header "a.h"
}
#pragma clang module contents
#pragma clang module begin A
// ... a.h ...
#pragma clang module end
The preprocessed output generates line markers, which are not accepted by the
module map parser, so -x c++-module-map-cpp-output should be used to compile
such outputs.
A couple of major parts do not work yet:
1) The files that are listed in the module map must exist on disk, in order to
build the on-disk header -> module lookup table in the PCM file. To fix
this, we need the preprocessed output to track the file size and other stat
information we might use to build the lookup table.
2) Declaration ownership semantics don't work properly yet, since mapping from
a source location to a module relies on mapping from FileIDs to modules,
which we can't do if module transitions can occur in the middle of a file.
llvm-svn: 302309
This patch adds a fix-it for the -Wunguarded-availability warning. This fix-it
is similar to the Swift one: it suggests that you wrap the statement in an
`if (@available)` check. The produced fixits are indented (just like the Swift
ones) to make them look nice in Xcode's fix-it preview.
rdar://31680358
Differential Revision: https://reviews.llvm.org/D32424
llvm-svn: 302253
The whitespace should come from the argument name in the macro
expansion, rather than from the token passed to the macro (same as it
does when not pasting).
Added a new test case for the change in behavior to stringize_space.c.
FileCheck'ized macro_paste_commaext.c, tweaked the test case, and
added a comment; no behavioral change to this test.
Differential Revision: https://reviews.llvm.org/D30427
llvm-svn: 302195
These pragmas are intended to simulate the effect of entering or leaving a file
with an associated module. This is not completely implemented yet: declarations
between the pragmas will not be attributed to the correct module, but macro
visibility is already functional.
Modules named by #pragma clang module begin must already be known to clang (in
some module map that's either loaded or on the search path).
llvm-svn: 302098
Many of our supported configurations support modules but do not have any
first-class syntax to perform a module import. This leaves us with a problem:
there is no way to represent the expansion of a #include that imports a module
in the -E output for such languages. (We don't want to just leave it as a
#include because that requires the consumer of the preprocessed source to have
the same file system layout and include paths as the creator.)
This patch adds a new pragma:
#pragma clang module import MODULE.NAME.HERE
that imports a module, and changes -E and -frewrite-includes to use it when
rewriting a #include that maps to a module import. We don't make any attempt
to use a native language syntax import if one exists, to get more consistent
output. (If in the future, @import and #include have different semantics in
some way, the pragma will track the #include semantics.)
llvm-svn: 301725
One of the -Wincomplete-umbrella warnings diagnoses when a header is present in
the directory but it's not present in the umbrella header. Currently, this
warning only happens on top level modules; any submodule using an umbrella
header does not get this warning. Fix that by also considering the submodules.
Differential Revision: https://reviews.llvm.org/D32576
rdar://problem/22623686
llvm-svn: 301597
If a file search involves a header map, suppress
-Wnonportable-include-path. It's firing lots of false positives for
framework authors internally, and it's not trivial to fix.
Consider a framework called "Foo" with a main (installed) framework header
"Foo/Foo.h". It's atypical for "Foo.h" to actually live inside a
directory called "Foo" in the source repository. Instead, the
build system generates a header map while building the framework.
If Foo.h lives at the top-level of the source repository (common), and
the git repo is called ssh://some.url/foo.git, then the header map will
have something like:
Foo/Foo.h -> /Users/myname/code/foo/Foo.h
where "/Users/myname/code/foo" is the clone of ssh://some.url/foo.git.
After #import <Foo/Foo.h>, the current implementation of
-Wnonportable-include-path will falsely assume that Foo.h was found in a
nonportable way, because of the name of the git clone (.../foo/Foo.h).
However, that directory name was not involved in the header search at
all.
This commit adds an extra parameter to Preprocessor::LookupFile and
HeaderSearch::LookupFile to track if the search used a header map,
making it easy to suppress the warning. Longer term, once we find a way
to avoid the false positive, we should turn the warning back on.
rdar://problem/28863903
llvm-svn: 301592
This reverts commit r301449. It breaks the build with:
MacroPPCallbacks.h:114:50: error: non-virtual member function marked 'override' hides virtual member function
llvm-svn: 301469
Summary:
The PPCallbacks::MacroUndefined callback is currently insufficient for clients that need to track the MacroDirectives.
This patch adds an additional argument to PPCallbacks::MacroUndefined that is the undef MacroDirective.
Reviewers: bruno, manmanren
Reviewed By: bruno
Subscribers: nemanjai, cfe-commits
Differential Revision: https://reviews.llvm.org/D29923
llvm-svn: 301449
Drive-by fix (noticed while working on https://reviews.llvm.org/D32205):
DebugOverflowStack() is supposed to provoke a stack overflow, however
LLVM was smart enough to use the red-zone and fold the load into a tail
jump on x86_64 optimizing this to an endless loop instead of a stack
overflow.
llvm-svn: 301218
This commit teaches Clang to recognize editor placeholders that are produced
when an IDE like Xcode inserts a code-completion result that includes a
placeholder. Now when the lexer sees a placeholder token, it emits an
'editor placeholder in source file' error and creates an identifier token
that represents the placeholder. The parser/sema can now recognize the
placeholders and can suppress the diagnostics related to the placeholders. This
ensures that live issues in an IDE like Xcode won't get spurious diagnostics
related to placeholders.
This commit also adds a new compiler option named '-fallow-editor-placeholders'
that silences the 'editor placeholder in source file' error. This is useful
for an IDE like Xcode as we don't want to display those errors in live issues.
rdar://31581400
Differential Revision: https://reviews.llvm.org/D32081
llvm-svn: 300667
Previously, if an escaped newline was followed by a newline or a nul, we'd lex
the escaped newline as a bogus space character. This led to a bunch of
different broken corner cases:
For the pattern "\\\n\0#", we would then have a (horizontal) space whose
spelling ends in a newline, and would decide that the '#' is at the start of a
line, and incorrectly start preprocessing a directive in the middle of a
logical source line. If we were already in the middle of a directive, this
would result in our attempting to process multiple directives at the same time!
This resulted in crashes, asserts, and hangs on invalid input, as discovered by
fuzz-testing.
For the pattern "\\\n" at EOF (with an implicit following nul byte), we would
produce a bogus trailing space character with spelling "\\\n". This was mostly
harmless, but would lead to clang-format getting confused and misformatting in
rare cases. We now produce a trailing EOF token with spelling "\\\n",
consistent with our handling for other similar cases -- an escaped newline is
always part of the token containing the next character, if any.
For the pattern "\\\n\n", this was somewhat more benign, but would produce an
extraneous whitespace token to clients who care about preserving whitespace.
However, it turns out that our lexing for line comments was relying on this bug
due to an off-by-one error in its computation of the end of the comment, on the
slow path where the comment might contain escaped newlines.
llvm-svn: 300515
This allows using and testing these two features separately. (noteably,
debug info is, so far as I know, always a win (basically). But function
modular codegen is currently a loss for highly optimized code - where
most of the linkonce_odr definitions are optimized away, so providing
weak_odr definitions is only overhead)
llvm-svn: 300104
Summary: When using the C preprocessor with assembly files, either with a
capital `S` file extension, or with `-xassembler-with-cpp`, the Unicode escape
sequence `\u` is ignored. The `\u` pattern can be used for expanding a macro
argument that starts with `u`.
Author: Salman Arif <salman.arif@arm.com>
Reviewers: rengolin, olista01
Reviewed By: olista01
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D31765
llvm-svn: 299754
Fix the current parsing of subframeworks in modulemaps to lookup for
headers based on whether they are frameworks.
rdar://problem/30563982
llvm-svn: 298391
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298278
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
Richard Smith