Commit Graph

511 Commits

Author SHA1 Message Date
Richard Smith f3f846162a Track the set of module maps read while building a .pcm file and reload those when preprocessing from that .pcm file.
llvm-svn: 306628
2017-06-29 02:19:42 +00:00
Richard Smith 9565c75b29 Support non-identifier module names when preprocessing modules.
llvm-svn: 305758
2017-06-19 23:09:36 +00:00
Richard Smith 86a3ef5b03 Add -frewrite-imports flag.
If specified, when preprocessing, the contents of imported .pcm files will be
included in preprocessed output. The resulting preprocessed file can then be
compiled standalone without the module sources or .pcm files.

llvm-svn: 305116
2017-06-09 21:24:02 +00:00
Richard Smith 5d2ed48987 Add #pragma clang module build/endbuild pragmas for performing a module build
as part of a compilation.

This is intended for two purposes:

1) Writing self-contained test cases for modules: we can now write a single
source file test that builds some number of module files on the side and
imports them.

2) Debugging / test case reduction. A single-source testcase is much more
amenable to reduction, compared to a VFS tarball or .pcm files.

llvm-svn: 305101
2017-06-09 19:22:32 +00:00
Richard Smith 27e5aa0892 Factor out and unify emission of "module is unavailable" diagnostics.
Inspired by post-commit review of r304190.

llvm-svn: 304728
2017-06-05 18:57:56 +00:00
Galina Kistanova e37ad5a79e Added LLVM_FALLTHROUGH to address warning: this statement may fall through. NFC.
llvm-svn: 304645
2017-06-03 06:27:16 +00:00
Richard Smith f74d946624 Move functionality for handling module maps as inputs from the -emit-module
action to the general FrontendAction infrastructure.

This permits applying -E, -ast-dump, -fsyntax-only, and so on to a module map
compilation. (The -E form is not currently especially useful yet as there's no
good way to take the output and use it to actually build a module.)

In order to support this, -cc1 now accepts -x <lang>-module-map in all cases
where it accepts -x <lang> for a language we can parse (not ir/ast). And for
uniformity, we also accept -x <lang>-header for all such languages (we used
to reject for cuda and renderscript), and -x <lang>-cpp-output for all such
languages (we used to reject for c, cl, and renderscript).

(None of these new alternatives are accepted by the driver yet, so no
user-visible changes.)

llvm-svn: 301610
2017-04-28 01:49:42 +00:00
Duncan P. N. Exon Smith cfc1f6a6ee Preprocessor: Suppress -Wnonportable-include-path for header maps
If a file search involves a header map, suppress
-Wnonportable-include-path.  It's firing lots of false positives for
framework authors internally, and it's not trivial to fix.

Consider a framework called "Foo" with a main (installed) framework header
"Foo/Foo.h".  It's atypical for "Foo.h" to actually live inside a
directory called "Foo" in the source repository.  Instead, the
build system generates a header map while building the framework.
If Foo.h lives at the top-level of the source repository (common), and
the git repo is called ssh://some.url/foo.git, then the header map will
have something like:

    Foo/Foo.h -> /Users/myname/code/foo/Foo.h

where "/Users/myname/code/foo" is the clone of ssh://some.url/foo.git.

After #import <Foo/Foo.h>, the current implementation of
-Wnonportable-include-path will falsely assume that Foo.h was found in a
nonportable way, because of the name of the git clone (.../foo/Foo.h).
However, that directory name was not involved in the header search at
all.

This commit adds an extra parameter to Preprocessor::LookupFile and
HeaderSearch::LookupFile to track if the search used a header map,
making it easy to suppress the warning.  Longer term, once we find a way
to avoid the false positive, we should turn the warning back on.

rdar://problem/28863903

llvm-svn: 301592
2017-04-27 21:41:51 +00:00
Richard Smith 40c0efa515 Refactor frontend InputKind to prepare for treating module maps as a distinct kind of input.
No functionality change intended.

llvm-svn: 301442
2017-04-26 18:57:40 +00:00
Graydon Hoare ce539b54e3 [PCH] Attach instance's dependency collectors to PCH external AST sources.
Summary:
When a PCH is included via -include-pch, clang should treat the
current TU as dependent on the sourcefile that the PCH was generated from.

This is currently _partly_ accomplished by InitializePreprocessor calling
AddImplicitIncludePCH to synthesize an implicit #include of the sourcefile,
into the preprocessor's Predefines buffer.

For FrontendActions such as PreprocessOnlyAction (which is, curiously, what the
driver winds up running one of in response to a plain clang -M) this is
sufficient: the preprocessor cranks over its Predefines and emits a dependency
reference to the initial sourcefile.

For other FrontendActions (for example -emit-obj or -fsyntax-only) the
Predefines buffer is reset to the suggested predefines buffer from the PCH, so
the dependency edge is lost. The result is that clang emits a .d file in those
cases that lacks a reference to the .h file responsible for the input (and in
Swift's case, our .swiftdeps file winds up not including a reference to the
source file for a PCH bridging header.)

This patch fixes the problem by taking a different tack: ignoring the
Predefines buffer (which seems a bit like a hack anyways) and directly
attaching the CompilerInstance's DependencyCollectors (and legacy
DependencyFileGenerator) to the ASTReader for the external AST.

This approach is similar to the one chosen in earlier consultation with Bruno
and Ben, and I think it's the least-bad solution, given several options.

Reviewers: bruno, benlangmuir, doug.gregor

Reviewed By: bruno, doug.gregor

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31378

llvm-svn: 299009
2017-03-29 17:33:09 +00:00
Eric Christopher 3646e6289d Move setting of LangOpts based on target flags out of CompilerInstance
and into TargetInfo::adjust so that it gets called in more places
throughout the compiler (AST serialization in particular).

Should fix PPC modules after removing of faltivec.

llvm-svn: 298487
2017-03-22 06:36:09 +00:00
Eric Christopher 758aad76d8 Remove the -faltivec alias option and replace it with -maltivec everywhere.
The alias was only ever used on darwin and had some issues there,
and isn't used in practice much. Also fixes a problem with -mno-altivec
not turning off -maltivec.

Also add a diagnostic for faltivec/fno-altivec that directs users to use
maltivec options and include the altivec.h file explicitly.

llvm-svn: 298449
2017-03-21 22:06:18 +00:00
Duncan P. N. Exon Smith 030d7d6daa Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338).  The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298278
2017-03-20 17:58:26 +00:00
Renato Golin f1966cf646 Revert "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298165, as it broke the ARM builds.

llvm-svn: 298185
2017-03-18 12:31:32 +00:00
Bruno Cardoso Lopes 5a0af1fcd1 [Modules] In case of lock timeout, fallback and build module
Duncan's r298165 introduced the PCMCache mechanism, which guarantees
that locks aren't necessary anymore for correctness but only for
performance, by avoiding building it twice when possible.

Change the logic to avoid an error but actually build the module in case
the timeout happens. Instead of an error, still emit a remark for
debugging purposes.

rdar://problem/30297862

llvm-svn: 298175
2017-03-18 00:26:18 +00:00
Duncan P. N. Exon Smith 079c40e886 Modules: Cache PCMs in memory and avoid a use-after-free
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

  - The ModuleManager stores PCMs there that it loads from disk, never
    touching the disk if the cache is hot.

  - When modules fail to validate, they're removed from the cache.

  - When a CompilerInstance is spawned to build a new module, each
    already-loaded PCM is assumed to be valid, and is frozen to avoid
    the use-after-free.

  - Any newly-built module is written directly to the cache to avoid the
    round-trip to the filesystem, making lock files unnecessary for
    correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298165
2017-03-17 22:55:13 +00:00
Duncan P. N. Exon Smith 60fa28882e Modules: Use hash of PCM content for SIGNATURE
Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.

  - Move definition ASTFileSignature to Basic/Module.h so Module and
    ASTSourceDescriptor can use it.

  - Change the signature from uint64_t to std::array<uint32_t,5>.

  - Stop using (saving/reading) the size and modification time of PCM
    files when there is a valid SIGNATURE.

  - Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
    and other records that shouldn't affect the hash.  Because implicit
    modules reuses the same file for multiple levels of -Werror, this
    includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.

This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system.  This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).

Design and original patch by Manman Ren!

llvm-svn: 297655
2017-03-13 18:45:08 +00:00
Duncan P. N. Exon Smith 1116d6915c Modules: Simplify CompilerInstance constructor, NFC
Initialize fields directly in header.  Note that the ModuleManager field is an
IntrusiveRefCntPtr, so there's no need for explicit initialization.

llvm-svn: 293863
2017-02-02 05:09:51 +00:00
David Blaikie ea4395ebcd Reapply "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"
Aleksey Shlypanikov pointed out my mistake in migrating an explicit
unique_ptr to auto - I was expecting the function returned a unique_ptr,
but instead it returned a raw pointer - introducing a leak.

Thanks Aleksey!

This reapplies r291184, reverted in r291249.

llvm-svn: 291270
2017-01-06 19:49:01 +00:00
David Blaikie 81d0829438 Revert "IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer"
Caused a memory leak reported by asan. Reverting while I investigate.

This reverts commit r291184.

llvm-svn: 291249
2017-01-06 17:47:10 +00:00
David Blaikie 9c28cb3f65 shared_ptrify (from InclusiveRefCntPtr) HeaderSearchOptions
llvm-svn: 291202
2017-01-06 01:04:46 +00:00
David Blaikie 9280a857bc IntrusiveRefCntPtr -> std::shared_ptr for CompilerInvocationBase and CodeCompleteConsumer
llvm-svn: 291184
2017-01-05 22:19:11 +00:00
David Blaikie 41565463bd Move Preprocessor over to std::shared_ptr rather than IntrusiveRefCntPtr
llvm-svn: 291166
2017-01-05 19:48:07 +00:00
David Blaikie e304168853 Move PreprocessorOptions to std::shared_ptr from IntrusiveRefCntPtr
llvm-svn: 291160
2017-01-05 19:11:36 +00:00
David Blaikie f95113dacf Move FailedModulesSet over to shared_ptr from IntrusiveRefCntPtr
llvm-svn: 291159
2017-01-05 19:11:31 +00:00
David Blaikie 61137e1a50 Use shared_ptr instead of IntrusiveRefCntPtr for ModuleFileExtension
The intrusiveness wasn't needed here, so this simplifies/clarifies the
ownership model.

llvm-svn: 291150
2017-01-05 18:23:18 +00:00
Bruno Cardoso Lopes 82ec4fde42 [CrashReproducer] Add support for merging -ivfsoverlay
Merge all VFS mapped files inside -ivfsoverlay inputs into the vfs
overlay provided by the crash reproducer. This is the last missing piece
to allow crash reproducers to fully work with user frameworks; when
combined with headermaps, it allows clang to find additional frameworks.

rdar://problem/27913709

llvm-svn: 290326
2016-12-22 07:06:03 +00:00
Bruno Cardoso Lopes 7aff2bb3d2 [CrashReproducer] Collect PCH included via -include-pch
Collect the necessary input PCH files.

Do not try to validate the AST before copying it out because if the
crash is in this path, we won't be able to collect it. Instead only
check if it's a file containg an AST.

rdar://problem/27913709

llvm-svn: 289460
2016-12-12 19:28:25 +00:00
Bruno Cardoso Lopes 181225b8a3 [CrashReproducer] Collect headermap files
Include headermaps (.hmap files) in the .cache directory and
add VFS entries. All headermaps are known after HeaderSearch
setup, collect them right after.

rdar://problem/27913709

llvm-svn: 289360
2016-12-11 04:27:28 +00:00
Richard Smith a114c46e87 Revert r288626, which reverts r288449. Original commit message:
Recover better from an incompatible .pcm file being provided by -fmodule-file=. We try to include the headers of the module textually in this case, still enforcing the modules semantic rules. In order to make that work, we need to still track that we're entering and leaving the module. Also, if the module was also marked as unavailable (perhaps because it was missing a file), we shouldn't mark the module unavailable -- we don't need the module to be complete if we're going to enter it textually.

llvm-svn: 288741
2016-12-06 00:40:17 +00:00
Daniel Jasper ad3ba6be31 Revert "Recover better from an incompatible .pcm file being provided by -fmodule-file=. We try to include the headers of the module textually in this case, still enforcing the modules semantic rules. In order to make that work, we need to still track that we're entering and leaving the module. Also, if the module was also marked as unavailable (perhaps because it was missing a file), we shouldn't mark the module unavailable -- we don't need the module to be complete if we're going to enter it textually."
This reverts commit r288449.

I believe that this is currently faulty wrt. modules being imported
inside namespaces. Adding these lines to the new test:

  namespace n {
  #include "foo.h"
  }

Makes it break with

  fatal error: import of module 'M' appears within namespace 'n'

However, I believe it should fail with

  error: redundant #include of module 'M' appears within namespace 'n'

I have tracked this down to us now inserting a tok::annot_module_begin
instead of a tok::annot_module_include in
Preprocessor::HandleIncludeDirective() and then later in
Parser::parseMisplacedModuleImport(), we hit the code path for
tok::annot_module_begin, which doesn't set FromInclude of
checkModuleImportContext to true (thus leading to the "wrong"
diagnostic).

llvm-svn: 288626
2016-12-04 22:34:37 +00:00
Richard Smith 444e6f3d82 Recover better from an incompatible .pcm file being provided by -fmodule-file=.
We try to include the headers of the module textually in this case, still
enforcing the modules semantic rules. In order to make that work, we need to
still track that we're entering and leaving the module. Also, if the module was
also marked as unavailable (perhaps because it was missing a file), we
shouldn't mark the module unavailable -- we don't need the module to be
complete if we're going to enter it textually.

llvm-svn: 288449
2016-12-02 01:52:28 +00:00
Matthias Braun ae032b6cf5 Adapt to llvm NamedRegionTimer changes
We have to specify a name and description for the timers and groups now.

llvm-svn: 287371
2016-11-18 19:43:25 +00:00
Justin Lebar f91086b0a8 [CUDA] Initialize our header search using the host triple.
Summary:
This used to work because system headers are found in a (somewhat)
predictable set of locations on Linux.  But this is not the case on
MacOS; without this change, we don't look in the right places for our
headers when doing device-side compilation on Mac.

Reviewers: tra

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D26776

llvm-svn: 287286
2016-11-18 00:41:27 +00:00
Benjamin Kramer 7de9969bb0 [Frontend] Allow attaching an external sema source to compiler instance and extra diags to TypoCorrections
This can be used to append alternative typo corrections to an existing diag.
include-fixer can use it to suggest includes to be added.

Differential Revision: https://reviews.llvm.org/D26745

llvm-svn: 287128
2016-11-16 18:15:26 +00:00
Justin Lebar 5e83dfedb8 Switch SmallSetVector to use DenseSet when it overflows its inline space.
Summary:
SetVector already used DenseSet, but SmallSetVector used std::set.  This
leads to surprising performance differences.  Moreover, it means that
the set of key types accepted by SetVector and SmallSetVector are
quite different!

In order to make this change, we had to convert some callsites that used
SmallSetVector<std::string, N> to use SmallSetVector<CachedHashString, N>
instead.

Reviewers: timshen

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D25648

llvm-svn: 284887
2016-10-21 21:45:01 +00:00
Matthias Braun ec1c5a2048 Adapt to LLVM EnableStatistics() change.
llvm-svn: 282533
2016-09-27 19:38:59 +00:00
Matthias Braun abb6eea19c CC1: Add -save-stats option
This option behaves in a similar spirit as -save-temps and writes
internal llvm statistics in json format to a file.

Differential Revision: https://reviews.llvm.org/D24820

llvm-svn: 282426
2016-09-26 18:53:34 +00:00
Richard Smith bbcc9f0462 C++ Modules TS: add frontend support for building pcm files from module
interface files. At the moment, all declarations (and no macros) are exported,
and 'export' declarations are not supported yet.

llvm-svn: 279794
2016-08-26 00:14:38 +00:00
Manman Ren 11f2a47772 Module: add -fprebuilt-module-path to support loading prebuilt modules.
In this mode, there is no need to load any module map and the programmer can
simply use "@import" syntax to load the module directly from a prebuilt
module path. When loading from prebuilt module path, we don't support
rebuilding of the module files and we ignore compatible configuration
mismatches.

rdar://27290316
Differential Revision: http://reviews.llvm.org/D23125

llvm-svn: 279096
2016-08-18 17:42:15 +00:00
Yaxun Liu 2c17e82bc7 [OpenCL][AMDGPU] Add support for -cl-denorms-are-zero
Adjust target features for amdgcn target when -cl-denorms-are-zero is set.

Denormal support is controlled by feature strings fp32-denormals fp64-denormals in amdgcn target. If -cl-denorms-are-zero is not set and the command line does not set fp32/64-denormals feature string, +fp32-denormals +fp64-denormals will be on for GPU's supporting them.

A new virtual function virtual void TargetInfo::adjustTargetOptions(const CodeGenOptions &CGOpts, TargetOptions &TargetOpts) const is introduced to allow adjusting target option by codegen option.

Differential Revision: https://reviews.llvm.org/D22815

llvm-svn: 278151
2016-08-09 19:43:38 +00:00
Mehdi Amini 9670f847b8 [NFC] Header cleanup
Summary: Removed unused headers, replaced some headers with forward class declarations

Patch by: Eugene <claprix@yandex.ru>

Differential Revision: https://reviews.llvm.org/D20100

llvm-svn: 275882
2016-07-18 19:02:11 +00:00
Peter Collingbourne 03f8907f65 Frontend: Simplify ownership model for clang's output streams.
This changes the CompilerInstance::createOutputFile function to return
a std::unique_ptr<llvm::raw_ostream>, rather than an llvm::raw_ostream
implicitly owned by the CompilerInstance. This in most cases required that
I move ownership of the output stream to the relevant ASTConsumer.

The motivation for this change is to allow BackendConsumer to be a client
of interfaces such as D20268 which take ownership of the output stream.

Differential Revision: http://reviews.llvm.org/D21537

llvm-svn: 275507
2016-07-15 00:55:40 +00:00
Benjamin Kramer d6da1a097b Add some std::move where the value is only read otherwise.
This mostly affects smart pointers. No functionality change intended.

llvm-svn: 272520
2016-06-12 20:05:23 +00:00
Bruno Cardoso Lopes 4a52222c9d [Modules] Improve diagnostics for LockFileManager errors
Uses error message now provided by LockFileManager in LLVM r271755.

rdar://problem/26529101

llvm-svn: 271758
2016-06-04 01:13:22 +00:00
Benjamin Kramer cfeacf56f0 Apply clang-tidy's misc-move-constructor-init throughout Clang.
No functionality change intended, maybe a tiny performance improvement.

llvm-svn: 270996
2016-05-27 14:27:13 +00:00
Justin Lebar 76945b2f44 [CUDA] Copy host builtin types to NVPTXTargetInfo.
Summary:
Host and device types must match, otherwise when we pass values back and
forth between the host and device, we will get the wrong result.

This patch makes NVPTXTargetInfo inherit most of its type information
from the host's target info.

Reviewers: rsmith

Subscribers: cfe-commits, jhen, tra

Differential Revision: http://reviews.llvm.org/D19346

llvm-svn: 268131
2016-04-29 23:05:19 +00:00
Saleem Abdulrasool 10a4972a8d revert SVN r265702, r265640
Revert the two changes to thread CodeGenOptions into the TargetInfo allocation
and to fix the layering violation by moving CodeGenOptions into Basic.
Code Generation is arguably not particularly "basic".  This addresses Richard's
post-commit review comments.  This change purely does the mechanical revert and
will be followed up with an alternate approach to thread the desired information
into TargetInfo.

llvm-svn: 265806
2016-04-08 16:52:00 +00:00
Saleem Abdulrasool 724275ba5f Basic: thread CodeGenOptions into TargetInfo
This threads CodeGenOptions into the TargetInfo hierarchy.  This is motivated by
ARM which can change some target information based on the EABI selected
(-meabi).  Similar options exist for other platforms (e.g. MIPS) and thus is
generally useful.  NFC.

llvm-svn: 265640
2016-04-07 05:41:11 +00:00
Bruno Cardoso Lopes b1631d9161 [CrashReproducer] Cleanup and move functionality around in ModuleDependencyCollector. NFC
- Make ModuleDependencyCollector use the DependencyCollector interface
- Move some methods from ModuleDependencyListener to ModuleDependencyCollector
in order to share common functionality with other future possible
callbacks.

llvm-svn: 264808
2016-03-29 23:47:40 +00:00
Nico Weber f54146c178 clang-cl: Fix remaining bugs in interaction of /Yc and /FI /showIncludes.
Instead of putting the /Yc header into ExtraDeps, give DependencyOutputOptions
a dedicated field for /Yc mode, and let HeaderIncludesCallback hang on to the
full DependencyOutputOptions object, not just ExtraDeps.

Reverts parts of r263352 that are now no longer needed.

llvm-svn: 264182
2016-03-23 18:46:57 +00:00
Nico Weber 149d9522fb clang-cl: Include /FI headers in /showIncludes output.
-H in gcc mode doesn't print -include headers, but they are included in
depfiles written by MMD and friends. Since /showIncludes is what's used instead
of depfiles, printing /FI there seems important (and matches cl.exe).

Instead of giving HeaderIncludeGen more options, just switch on ShowAllHeaders
in clang-cl mode and let clang::InitializePreprocessor() not put -include flags
in the <command line> block. This changes the behavior of -E slightly, and it
removes the <command line> flag from the output triggered by setting the
obscure CC_PRINT_HEADERS=1 env var to true while running clang. Both of these
seem ok to change.

http://reviews.llvm.org/D18401

llvm-svn: 264174
2016-03-23 18:00:22 +00:00
Nico Weber 4b5aedef16 clang-cl: Add /Yc argument to /showIncludes output.
To make this work, delay printing of ExtraDeps in HeaderIncludesCallback a bit,
so that it happens after CompilerInstance::InitializeSourceManager() has run.

General /FI arguments are still missing from /showIncludes output, this still
needs to be fixed.

llvm-svn: 263352
2016-03-13 02:44:13 +00:00
Nico Weber 2ca4be97de clang-cl: Implement initial limited support for precompiled headers.
In the gcc precompiled header model, one explicitly runs clang with `-x
c++-header` on a .h file to produce a gch file, and then includes the header
with `-include foo.h` and if a .gch file exists for that header it gets used.
This is documented at
http://clang.llvm.org/docs/UsersManual.html#precompiled-headers

cl.exe's model is fairly different, and controlled by the two flags /Yc and
/Yu. A pch file is generated as a side effect of a regular compilation when
/Ycheader.h is passed. While the compilation is running, the compiler keeps
track of #include lines in the main translation unit and writes everything up
to an `#include "header.h"` line into a pch file. Conversely, /Yuheader.h tells
the compiler to skip all code in the main TU up to and including `#include
"header.h"` and instead load header.pch. (It's also possible to use /Yc and /Yu
without an argument, in that case a `#pragma hrdstop` takes the role of
controlling the point where pch ends and real code begins.)

This patch implements limited support for this in that it requires the pch
header to be passed as a /FI force include flag – with this restriction,
it can be implemented almost completely in the driver with fairly small amounts
of code. For /Yu, this is trivial, and for /Yc a separate pch action is added
that runs before the actual compilation. After r261774, the first failing
command makes a compilation stop – this means if the pch fails to build the
main compilation won't run, which is what we want. However, in /fallback builds
we need to run the main compilation even if the pch build fails so that the
main compilation's fallback can run. To achieve this, add a ForceSuccessCommand
that pretends that the pch build always succeeded in /fallback builds (the main
compilation will then fail to open the pch and run the fallback cl.exe
invocation).

If /Yc /Yu are used in a setup that clang-cl doesn't implement yet, clang-cl
will now emit a "not implemented yet; flag ignored" warning that can be
disabled using -Wno-clang-cl-pch.

Since clang-cl doesn't yet serialize some important things (most notably
`pragma comment(lib, ...)`, this feature is disabled by default and only
enabled by an internal driver flag. Once it's more stable, this internal flag
will disappear.

(The default stdafx.h setup passes stdafx.h as explicit argument to /Yc but not
as /FI – instead every single TU has to `#include <stdafx.h>` as first thing it
does. Implementing support for this should be possible with the approach in
this patch with minimal frontend changes by passing a --stop-at / --start-at
flag from the driver to the frontend. This is left for a follow-up. I don't
think we ever want to support `#pragma hdrstop`, and supporting it with this
approach isn't easy: This approach relies on the driver knowing the pch
filename in advance, and `#pragma hdrstop(out.pch)` can set the output
filename, so the driver can't know about it in advance.)

clang-cl now also honors /Fp and puts pch files in the same spot that cl.exe
would put them, but the pch file format is of course incompatible. This has
ramifications on /fallback, so /Yc /Yu aren't passed through to cl.exe in
/fallback builds.

http://reviews.llvm.org/D17695

llvm-svn: 262420
2016-03-01 23:16:44 +00:00
Richard Smith 7e82e019c6 [modules] Flatten -fmodule-name= and -fmodule-implementation-of= into a single
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.

This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.

llvm-svn: 261372
2016-02-19 22:25:36 +00:00
Benjamin Kramer 0772c42385 Reduce the number of implicit StringRef->std::string conversions by threading StringRef through more APIs.
No functionality change intended.

llvm-svn: 260815
2016-02-13 13:42:54 +00:00
Rafael Espindola fa1eedee3a Update for llvm API change.
llvm-svn: 255838
2015-12-16 22:59:09 +00:00
Vedant Kumar 5b60ad68ce [Frontend] Rangify for loop. NFC.
llvm-svn: 253178
2015-11-16 00:59:34 +00:00
Richard Smith 8a308ec24d [modules] If we're given a module file, via -fmodule-file=, for a module, but
we can't load that file due to a configuration mismatch, and implicit module
building is disabled, and the user turns off the error-by-default warning for
that situation, then fall back to textual inclusion for the module rather than
giving an error if any of its headers are included.

llvm-svn: 252114
2015-11-05 00:54:55 +00:00
Douglas Gregor 6623e1f10f Introduce module file extensions to piggy-back data onto module files.
Introduce the notion of a module file extension, which introduces
additional information into a module file at the time it is built that
can then be queried when the module file is read. Module file
extensions are identified by a block name (which must be unique to the
extension) and can write any bitstream records into their own
extension block within the module file. When a module file is loaded,
any extension blocks are matched up with module file extension
readers, that are per-module-file and are given access to the input
bitstream.

Note that module file extensions can only be introduced by
programmatic clients that have access to the CompilerInvocation. There
is only one such extension at the moment, which is used for testing
the module file extension harness. As a future direction, one could
imagine allowing the plugin mechanism to introduce new module file
extensions.

llvm-svn: 251955
2015-11-03 18:33:07 +00:00
Richard Smith 95dc57a611 [modules] Allow the error when explicitly loading an incompatible module file
via -fmodule-file= to be turned off; in that case, just include the relevant
files textually. This allows module files to be unconditionally passed to all
compile actions via CXXFLAGS, and to be ignored for rules that specify custom
incompatible flags.

llvm-svn: 250577
2015-10-16 23:20:19 +00:00
Rafael Espindola ee4e08ba94 Use llvm::errc instead of std::errc.
llvm-svn: 249302
2015-10-05 11:49:35 +00:00
Yaron Keren 8b563665c3 Replace double negation of !FileID.isInvalid() with FileID.isValid().
+couple more of double-negated !SourceLocation.isInvalid() unfixed in r249228.

llvm-svn: 249235
2015-10-03 10:46:20 +00:00
Artem Belevich b5bc923af4 [CUDA] Allow parsing of host and device code simultaneously.
* adds -aux-triple option to specify target triple
 * propagates aux target info to AST context and Preprocessor
 * pulls in target specific preprocessor macros.
 * pulls in target-specific builtins from aux target.
 * sets appropriate host or device attribute on builtins.

Differential Revision: http://reviews.llvm.org/D12917

llvm-svn: 248299
2015-09-22 17:23:22 +00:00
Douglas Katzman 8bfac2c553 createOutputFile should set Error to something if it returns null.
This is not portably unit-testable because the only visible
effect is a change from one random message string to another.

llvm-svn: 247900
2015-09-17 16:45:12 +00:00
Richard Smith 293534b1a5 Initialize the AST consumer as soon as we have both an ASTConsumer and an
ASTContext. Fixes some cases where we could previously initialize the AST
consumer more than once.

llvm-svn: 245346
2015-08-18 20:39:29 +00:00
Richard Smith e75ee0f0c8 [modules] When explicitly building a module file, don't include timestamps in
the produced pcm file for stable file creation across distributed build
systems.

llvm-svn: 245199
2015-08-17 07:13:32 +00:00
Richard Smith 3938f0c728 [modules] Stop dropping 'module.timestamp' files into the current directory
when building with implicit modules disabled.

llvm-svn: 245136
2015-08-15 00:34:15 +00:00
Ivan Krasin 1193f2cbc0 Add sanitizer blacklists to the rules generated with -M/-MM/-MD/-MMD.
Summary:
Clang sanitizers, such as AddressSanitizer, ThreadSanitizer, MemorySanitizer,
Control Flow Integrity and others, use blacklists to specify which types / functions
should not be instrumented to avoid false positives or suppress known failures.

This change adds the blacklist filenames to the list of dependencies of the rules,
generated with -M/-MM/-MD/-MMD. This lets CMake/Ninja recognize that certain
C/C++/ObjC files need to be recompiled (if a blacklist is updated).

Reviewers: pcc

Subscribers: rsmith, honggyu.kim, pcc, cfe-commits

Differential Revision: http://reviews.llvm.org/D11968

llvm-svn: 244867
2015-08-13 04:04:37 +00:00
Richard Smith 629d8e6f18 Fix some tabs.
llvm-svn: 244537
2015-08-11 00:03:28 +00:00
Richard Smith 86cc82906f [modules] Remove now-dead code for lazy loading of files specified by -fmodule-file=.
llvm-svn: 244417
2015-08-09 08:58:36 +00:00
Richard Smith 0f99d6a441 [modules] PR22534: Load files specified by -fmodule-file= eagerly. In particular, this avoids the need to re-parse module map files when using such a module.
llvm-svn: 244416
2015-08-09 08:48:41 +00:00
Richard Smith 38c1e6d355 Unrevert r244412 (reverted in r244414), and delete the bogus line left behind
in the unit test that was checking a file the test no longer creates.

llvm-svn: 244415
2015-08-09 06:03:55 +00:00
Justin Bogner b102d1a432 Revert "[modules] Don't leak -M flags for dependency file generation into the module"
This was failing tests on a bunch of bots:

http://lab.llvm.org:8011/builders/clang-hexagon-elf/builds/29919/steps/check-all
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/29627/steps/check-all
http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_check/9959/
http://lab.llvm.org:8080/green/job/clang-stage2-configure-Rlto_check/5591/

This reverts r244412

llvm-svn: 244414
2015-08-09 05:40:38 +00:00
Richard Smith 2d7fbbc7be [modules] Don't leak -M flags for dependency file generation into the module
build process when we implicitly build a module. Previously, we'd create the
specified .d file once for each implicitly-built module and then finally
overwrite it with the correct contents after the requested build completes.
(This fails if you use stdout as a dependency file, which is what the provided
testcase does, and is how I discovered this brokenness.)

llvm-svn: 244412
2015-08-09 02:44:20 +00:00
Richard Smith 03f7e611c6 [modules] Attach dependency listeners to the module manager once when it's
created, rather than creating and attaching a new listener each time we load a
module file (yes, the old ones were kept around too!). No functionality change
intended, but a bit more sanity.

llvm-svn: 244411
2015-08-09 02:28:42 +00:00
Richard Smith d520a250b6 [modules] Produce an error if -cc1 wants to implicitly build a module and no
module cache has been provided, rather than creating one in the current
directory.

llvm-svn: 242819
2015-07-21 18:07:47 +00:00
Adrian Prantl fb2398d0c4 Make the clang module container format selectable from the command line.
- introduces a new cc1 option -fmodule-format=[raw,obj]
  with 'raw' being the default
- supports arbitrary module container formats that libclang is agnostic to
- adds the format to the module hash to avoid collisions
- splits the old PCHContainerOperations into PCHContainerWriter and
  a PCHContainerReader.

Thanks to Richard Smith for reviewing this patch!

llvm-svn: 242499
2015-07-17 01:19:54 +00:00
Richard Smith ce18a187f7 Extend -ftime-report to give more information about time spent reading module files.
llvm-svn: 242094
2015-07-14 00:26:00 +00:00
Yaron Keren 5b816061ba Replace some const std::string & with llvm::StringRef or std::string
and std::move to avoid implicit std::string construction.

Patch by Eugene Kosov.

llvm-svn: 241433
2015-07-06 08:47:15 +00:00
Yaron Keren 40178c352e Revert r241330. It compiled with Visual C++ 2013 and gcc 4.9.1 (mingw) but now fails the bots.
llvm-svn: 241335
2015-07-03 09:30:33 +00:00
Yaron Keren 452670165c Replace some const std::string & with llvm::StringRef or std::string
and std::move to avoid implicit std::string construction.

Patch by Eugene Kosov.

llvm-svn: 241330
2015-07-03 09:16:20 +00:00
Yaron Keren d30f5ffddc Revert r241319, investigating.
llvm-svn: 241321
2015-07-03 05:31:54 +00:00
Yaron Keren 25bb2e4709 Replace some const std::string & with llvm::StringRef or std::string
and std::move to avoid implicit std::string construction.

Part 1/2.

Patch by Eugene Kosov.

llvm-svn: 241319
2015-07-03 05:09:59 +00:00
Adrian Prantl bb165fb04d Introduce a PCHContainerOperations interface (NFC).
A PCHContainerOperations abstract interface provides operations for
creating and unwrapping containers for serialized ASTs (precompiled
headers and clang modules). The default implementation is
RawPCHContainerOperations, which uses a flat file for the output.

The main application for this interface will be an
ObjectFilePCHContainerOperations implementation that uses LLVM to
wrap the module in an ELF/Mach-O/COFF container to store debug info
alongside the AST.

rdar://problem/20091852

llvm-svn: 240225
2015-06-20 18:53:08 +00:00
Benjamin Kramer 3204b152b5 Replace push_back(Constructor(foo)) with emplace_back(foo) for non-trivial types
If the type isn't trivially moveable emplace can skip a potentially
expensive move. It also saves a couple of characters.


Call sites were found with the ASTMatcher + some semi-automated cleanup.

memberCallExpr(
    argumentCountIs(1), callee(methodDecl(hasName("push_back"))),
    on(hasType(recordDecl(has(namedDecl(hasName("emplace_back")))))),
    hasArgument(0, bindTemporaryExpr(
                       hasType(recordDecl(hasNonTrivialDestructor())),
                       has(constructExpr()))),
    unless(isInTemplateInstantiation()))

No functional change intended.

llvm-svn: 238601
2015-05-29 19:42:19 +00:00
Richard Smith bca31b731b [modules] Move implicit creation of ImportDecls for #includes transformed into module imports from the frontend into Sema where it belongs.
llvm-svn: 237555
2015-05-18 05:35:52 +00:00
Richard Smith 2b63d15f49 [modules] Retain the name as written for umbrella headers and directories, rather than converting to an absolute path. No observable change expected, but this allows us to correctly compute the module for an umbrella header, which later changes will require.
llvm-svn: 237508
2015-05-16 02:28:53 +00:00
Richard Smith 4241314164 [modules] Add local submodule visibility support for declarations.
With this change, enabling -fmodules-local-submodule-visibility results in name
visibility rules being applied to submodules of the current module in addition
to imported modules (that is, names no longer "leak" between submodules of the
same top-level module). This also makes it much safer to textually include a
non-modular library into a module: each submodule that textually includes that
library will get its own "copy" of that library, and so the library becomes
visible no matter which including submodule you import.

llvm-svn: 237473
2015-05-15 20:05:43 +00:00
Richard Smith a7e2cc684f [modules] Start moving the module visibility information off the Module itself.
It has no place there; it's not a property of the Module, and it makes
restoring the visibility set when we leave a submodule more difficult.

llvm-svn: 236300
2015-05-01 01:53:09 +00:00
Richard Smith 20e883e59b [modules] Stop trying to fake up a linear MacroDirective history.
Modules builds fundamentally have a non-linear macro history. In the interest
of better source fidelity, represent the macro definition information
faithfully: we have a linear macro directive history within each module, and at
any point we have a unique "latest" local macro directive and a collection of
visible imported directives. This also removes the attendent complexity of
attempting to create a correct MacroDirective history (which we got wrong
in the general case).

No functionality change intended.

llvm-svn: 236176
2015-04-29 23:20:19 +00:00
Rafael Espindola 2f16bc1095 Use raw_pwrite_stream in clang.
This is a small improvement to -emit-pth and allows llvm to start requiring it.

llvm-svn: 234897
2015-04-14 15:15:49 +00:00
Reid Kleckner 0aa128e219 [Frontend] Close open file handles before renaming output files
The placement of the 'delete' call that was removed in the unique_ptr
migration in r234597 was not an accident. The raw_ostream has to be
destroyed before you do the rename on Windows, otherwise you get
ERROR_ACCESS_DENIED. We can still use unique_ptr, we just need to do a
manual reset().

Also, range-for-loop-ify this code.

llvm-svn: 234612
2015-04-10 17:27:58 +00:00
Rafael Espindola c80a40661c Return std::unique_ptr to avoid a release and recreate.
llvm-svn: 234598
2015-04-10 14:30:43 +00:00
Rafael Espindola 269ec0f470 Use a std::unique_ptr to make it easier to see who owns the stream.
llvm-svn: 234597
2015-04-10 14:11:52 +00:00
Chandler Carruth ff8d943cb8 [Modules] Don't compute a modules cache path if we're not using modules!
Notably, this prevents us from doing *tons* of work to compute the
modules hash, including trying to read a darwin specific plist file off
of the system. There is a lot that needs cleaning up below this layer
too.

llvm-svn: 233462
2015-03-28 01:10:44 +00:00
Chandler Carruth 580dd296fa [Modules] Stop creating timestamps for the modules cache and trying to
prune it when we have disabled implicit module generation and thus are
not using any cached modules.

Also update a test of explicitly generated modules to pass this CC1 flag
correctly.

This fixes an issue where Clang was dropping files into the source tree
while running its tests.

llvm-svn: 233117
2015-03-24 21:44:25 +00:00
Yaron Keren 92e1b62d45 Remove many superfluous SmallString::str() calls.
Now that SmallString is a first-class citizen, most SmallString::str()
calls are not required. This patch removes a whole bunch of them, yet
there are lots more.

There are two use cases where str() is really needed:
1) To use one of StringRef member functions which is not available in
SmallString.
2) To convert to std::string, as StringRef implicitly converts while 
SmallString do not. We may wish to change this, but it may introduce
ambiguity.

llvm-svn: 232622
2015-03-18 10:17:07 +00:00
Richard Smith 7f330cdb31 Make module files passed to a module build via -fmodule-file= available to
consumers of that module.

Previously, such a file would only be available if the module happened to
actually import something from that module.

llvm-svn: 232583
2015-03-18 01:42:29 +00:00
Richard Smith 37bd29a5e6 Give better diagnostics when -fmodule-file= finds a bad file: if the file is
found indirectly, explain how we got there, and distinguish between 'file not
found' and 'file found but invalid'.

llvm-svn: 230839
2015-02-28 03:09:52 +00:00