Commit Graph

3905 Commits

Author SHA1 Message Date
Krasimir Georgiev 1f64bf6c69 [Frontend] Remove unused TemporaryFiles
Summary:
OnDiskData.TemporaryFiles is filled only by ASTUnit::addTemporaryFile, which is
dead. Also these files are used nowhere in the frontend nor in libclang.

Reviewers: bkramer, ilya-biryukov

Reviewed By: bkramer, ilya-biryukov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D33270

llvm-svn: 303265
2017-05-17 14:51:44 +00:00
Evgeniy Stepanov d991cdd50b [asan] A clang flag to enable ELF globals-gc.
This feature is subtly broken when the linker is gold 2.26 or
earlier. See the following bug for details:
  https://sourceware.org/bugzilla/show_bug.cgi?id=19002

Since the decision needs to be made at compilation time, we can not
test the linker version. The flag is off by default on ELF targets,
and on otherwise.

llvm-svn: 302591
2017-05-09 21:57:43 +00:00
Kostya Serebryany 50fb618b93 [sanitizer-coverage] implement -fsanitize-coverage=no-prune,... instead of a hidden -mllvm flag. clang part.
llvm-svn: 302320
2017-05-05 23:28:18 +00:00
Richard Smith 8128f3327f Add support for building modules from preprocessed source.
To support this, an optional marker "#pragma clang module contents" is
recognized in module map files, and the rest of the module map file from that
point onwards is treated as the source of the module. Preprocessing a module
map produces the input module followed by the marker and then the preprocessed
contents of the module.

Ignoring line markers, a preprocessed module might look like this:

  module A {
    header "a.h"
  }
  #pragma clang module contents
  #pragma clang module begin A
  // ... a.h ...
  #pragma clang module end

The preprocessed output generates line markers, which are not accepted by the
module map parser, so -x c++-module-map-cpp-output should be used to compile
such outputs.

A couple of major parts do not work yet:

1) The files that are listed in the module map must exist on disk, in order to
   build the on-disk header -> module lookup table in the PCM file. To fix
   this, we need the preprocessed output to track the file size and other stat
   information we might use to build the lookup table.

2) Declaration ownership semantics don't work properly yet, since mapping from
   a source location to a module relies on mapping from FileIDs to modules,
   which we can't do if module transitions can occur in the middle of a file.

llvm-svn: 302309
2017-05-05 22:18:51 +00:00
Richard Smith d13863008b Add #pragma clang module begin/end pragmas and generate them when preprocessing a module.
These pragmas are intended to simulate the effect of entering or leaving a file
with an associated module. This is not completely implemented yet: declarations
between the pragmas will not be attributed to the correct module, but macro
visibility is already functional.

Modules named by #pragma clang module begin must already be known to clang (in
some module map that's either loaded or on the search path).

llvm-svn: 302098
2017-05-04 00:29:54 +00:00
Yuka Takahashi 5511fc2eb3 Revert rL301998: "Fix a bug that -isysroot is completely ignored on Unix"
This reverts commit because it broke
sanitizer-x86_64-linux-autoconf bot and
clang-ppc64be-linux-multistage  bot.

llvm-svn: 302000
2017-05-03 06:02:45 +00:00
Yuka Takahashi 79d310713a Fix a bug that -isysroot is completely ignored on Unix
-isysroot is the flag which set the system root directory.
This bug report https://bugs.llvm.org//show_bug.cgi?id=11503
shows that -isysroot is not handled at all on Unix, so fixed this bug.

After this diff, I could get this result https://pastebin.com/TeCmn9mj .

Differential Revision: https://reviews.llvm.org/D31495

llvm-svn: 301998
2017-05-03 04:58:39 +00:00
Richard Smith c7cacdc332 PR26771: don't forget the " 2" (returning from #included file) linemarker after including an empty file with -frewrite-includes.
llvm-svn: 301727
2017-04-29 00:54:03 +00:00
Richard Smith c51c38b4ec Add pragma to perform module import and use it in -E output.
Many of our supported configurations support modules but do not have any
first-class syntax to perform a module import. This leaves us with a problem:
there is no way to represent the expansion of a #include that imports a module
in the -E output for such languages. (We don't want to just leave it as a
#include because that requires the consumer of the preprocessed source to have
the same file system layout and include paths as the creator.)

This patch adds a new pragma:

  #pragma clang module import MODULE.NAME.HERE

that imports a module, and changes -E and -frewrite-includes to use it when
rewriting a #include that maps to a module import. We don't make any attempt
to use a native language syntax import if one exists, to get more consistent
output. (If in the future, @import and #include have different semantics in
some way, the pragma will track the #include semantics.)

llvm-svn: 301725
2017-04-29 00:34:47 +00:00
Richard Smith f74d946624 Move functionality for handling module maps as inputs from the -emit-module
action to the general FrontendAction infrastructure.

This permits applying -E, -ast-dump, -fsyntax-only, and so on to a module map
compilation. (The -E form is not currently especially useful yet as there's no
good way to take the output and use it to actually build a module.)

In order to support this, -cc1 now accepts -x <lang>-module-map in all cases
where it accepts -x <lang> for a language we can parse (not ir/ast). And for
uniformity, we also accept -x <lang>-header for all such languages (we used
to reject for cuda and renderscript), and -x <lang>-cpp-output for all such
languages (we used to reject for c, cl, and renderscript).

(None of these new alternatives are accepted by the driver yet, so no
user-visible changes.)

llvm-svn: 301610
2017-04-28 01:49:42 +00:00
Duncan P. N. Exon Smith cfc1f6a6ee Preprocessor: Suppress -Wnonportable-include-path for header maps
If a file search involves a header map, suppress
-Wnonportable-include-path.  It's firing lots of false positives for
framework authors internally, and it's not trivial to fix.

Consider a framework called "Foo" with a main (installed) framework header
"Foo/Foo.h".  It's atypical for "Foo.h" to actually live inside a
directory called "Foo" in the source repository.  Instead, the
build system generates a header map while building the framework.
If Foo.h lives at the top-level of the source repository (common), and
the git repo is called ssh://some.url/foo.git, then the header map will
have something like:

    Foo/Foo.h -> /Users/myname/code/foo/Foo.h

where "/Users/myname/code/foo" is the clone of ssh://some.url/foo.git.

After #import <Foo/Foo.h>, the current implementation of
-Wnonportable-include-path will falsely assume that Foo.h was found in a
nonportable way, because of the name of the git clone (.../foo/Foo.h).
However, that directory name was not involved in the header search at
all.

This commit adds an extra parameter to Preprocessor::LookupFile and
HeaderSearch::LookupFile to track if the search used a header map,
making it easy to suppress the warning.  Longer term, once we find a way
to avoid the false positive, we should turn the warning back on.

rdar://problem/28863903

llvm-svn: 301592
2017-04-27 21:41:51 +00:00
Vassil Vassilev 2b676cfa55 Remove leaking UnknownPragmaHandlers right after we are done with them.
The UnknownPragmaHandlers added by DoPrintPreprocessedInput conflict with the
real PragmaHandlers from clang::Parser because they try to handle the same
#pragma directives. This makes it impossible to use a Preprocessor (that was
previously passed to DoPrintPreprocessedInput), as an Preprocessor for a
clang::Parser instance which is what we currently do in cling.

This patch removes the added UnknownPragmaHandler to avoid conflicts these
conflicts and leave the PragmaHandlers of the Preprocessors in a the same state
as before calling DoPrintPreprocessedInput.

Patch by Raphael Isemann (D32486)!

llvm-svn: 301563
2017-04-27 16:58:33 +00:00
Richard Smith 8c8fb80490 Improve diagnostics for bad -std= flag.
Don't list deprecated -std= values (c++0x etc). Only produce one line of output
per standard, even if we know it by multiple names.

In passing, add missing -std=gnu++03 alias (supported by GCC), and add new
spelling '-std=cl1.0' for OpenCL 1.0 for consistency with the other values,
with the same meaning as the preexisting '-std=cl'.

llvm-svn: 301507
2017-04-27 01:17:05 +00:00
Richard Smith 6379cfc553 Remove unnecessary and somewhat inaccurate "C89" flag from language standards.
llvm-svn: 301500
2017-04-26 23:49:57 +00:00
Richard Smith b0fe70451e Don't accept -std= values that would switch us to a different source language.
We already prohibited this in most cases (in r130710), but had some bugs in our
enforcement of this rule. Specifically, this prevents the following
combinations:

 * -x c -std=clN.M, which would previously effectively act as if -x cl were
   used, despite the input being a C source file. (-x cl -std=cNN continues
   to be disallowed.)

 * -x c++ -std=cuda, which would previously select C++98 + CUDA, despite that
   not being a C++ standard. (-x cuda -std=c++NN is still permitted, and
   selects CUDA with the given C++ standard as its base language.
   -x cuda -std=cuda is still supported with the meaning of CUDA + C++98.)

 * -x renderscript -std=c++NN, which would previously form a hybrid "C++ with
   RenderScript extensions" language. We could support such a thing, but
   shouldn't do so by accident.

llvm-svn: 301497
2017-04-26 23:44:33 +00:00
Vedant Kumar 349a624ab9 Revert "Revert "PPCallbacks::MacroUndefined, change signature and add test.""
This reverts commit r301469. It isn't needed with r301470, which fixes
the API break introduced in the original commit.

llvm-svn: 301472
2017-04-26 21:05:44 +00:00
Vedant Kumar 5e511ed49c Revert "PPCallbacks::MacroUndefined, change signature and add test."
This reverts commit r301449. It breaks the build with:

  MacroPPCallbacks.h:114:50: error: non-virtual member function marked 'override' hides virtual member function

llvm-svn: 301469
2017-04-26 20:58:19 +00:00
Frederich Munch b7755111a2 PPCallbacks::MacroUndefined, change signature and add test.
Summary:
The PPCallbacks::MacroUndefined callback is currently insufficient for clients that need to track the MacroDirectives.
This patch adds an additional argument to PPCallbacks::MacroUndefined that is the undef MacroDirective.

Reviewers: bruno, manmanren

Reviewed By: bruno

Subscribers: nemanjai, cfe-commits

Differential Revision: https://reviews.llvm.org/D29923

llvm-svn: 301449
2017-04-26 19:47:31 +00:00
Richard Smith 40c0efa515 Refactor frontend InputKind to prepare for treating module maps as a distinct kind of input.
No functionality change intended.

llvm-svn: 301442
2017-04-26 18:57:40 +00:00
David Blaikie 8150355498 Move Split DWARF handling to an MC option/command line argument rather than using metadata
Since Split DWARF needs to name the actual .dwo file that is generated,
it can't be known at the time the llvm::Module is produced as it may be
merged with other Modules before the object is generated and that object
may be generated with any name.

By passing the Split DWARF file name when LLVM is producing object code
the .dwo file name in the object file can match correctly.

The support for Split DWARF for implicit modules remains the same -
using metadata to store the dwo name and dwo id so that potentially
multiple skeleton CUs referring to different dwo files can be generated
from one llvm::Module.

llvm-svn: 301063
2017-04-21 23:35:36 +00:00
Eric Fiselier 4b2c8f75a1 Add __CLANG_ATOMIC_<TYPE>_LOCK_FREE macros for use in MSVC compatibility mode.
Summary:
Libc++ currently implements the `ATOMIC_<TYPE>_LOCK_FREE` macros using the `__GCC_ATOMIC_<TYPE>_LOCK_FREE` macros. However these are not available when MSVC compatibility is enabled even though C11 `_Atomic` is.  This prevents libc++ from correctly implementing `ATOMIC_<TYPE>_LOCK_FREE`.

This patch adds an alternative spelling `__CLANG_ATOMIC_<TYPE>_LOCK_FREE` that is enabled with `-fms-compatibility`.

Reviewers: rsmith, aaron.ballman, majnemer, zturner, compnerd, jfb, rnk

Reviewed By: rsmith

Subscribers: BillyONeal, smeenai, jfb, cfe-commits, dschuff

Differential Revision: https://reviews.llvm.org/D32265

llvm-svn: 300914
2017-04-20 22:53:57 +00:00
Alex Lorenz 1be800c511 Add support for editor placeholders to Clang
This commit teaches Clang to recognize editor placeholders that are produced
when an IDE like Xcode inserts a code-completion result that includes a
placeholder. Now when the lexer sees a placeholder token, it emits an
'editor placeholder in source file' error and creates an identifier token
that represents the placeholder. The parser/sema can now recognize the
placeholders and can suppress the diagnostics related to the placeholders. This
ensures that live issues in an IDE like Xcode won't get spurious diagnostics
related to placeholders.

This commit also adds a new compiler option named '-fallow-editor-placeholders'
that silences the 'editor placeholder in source file' error. This is useful
for an IDE like Xcode as we don't want to display those errors in live issues.

rdar://31581400

Differential Revision: https://reviews.llvm.org/D32081

llvm-svn: 300667
2017-04-19 08:58:56 +00:00
Richard Smith 34e485f922 [modules-ts] Fold together -x c++ and -x c++-module at -cc1 level.
The driver needs to know whether it's building a module interface or
implementation unit because it affects which outputs it produces and how it
builds the command pipeline. But the frontend doesn't need to know and should
not care: all it needs to know is what action it is being asked to perform on
the input.

(This is in preparation for permitting -emit-obj to be used on a module
interface unit to produce object code without going via a "full" PCM file.)

llvm-svn: 300611
2017-04-18 21:55:37 +00:00
Bruno Cardoso Lopes c69f9f3530 [Modules] Enable local submodule visibility for ObjC/C
Remove the restriction where this is only valid with C++

rdar://problem/29055656

Differential Revision: https://reviews.llvm.org/D31781

llvm-svn: 300108
2017-04-12 21:46:16 +00:00
David Blaikie f63556d8b4 Modular Codegen: Separate flags for function and debug info support
This allows using and testing these two features separately. (noteably,
debug info is, so far as I know, always a win (basically). But function
modular codegen is currently a loss for highly optimized code - where
most of the linkonce_odr definitions are optimized away, so providing
weak_odr definitions is only overhead)

llvm-svn: 300104
2017-04-12 20:58:33 +00:00
Bruno Cardoso Lopes 69cdc901f0 [Modules] Remove darwin specific code to check for SystemVersion.plist
This isn't need anymore and modules options -fbuild-session-file and
-fmodules-validate-once-per-build-session already provide a sane
mechanism to validate the system headers.

rdar://problem/19767523

llvm-svn: 300027
2017-04-12 04:49:00 +00:00
Jin-Gu Kang e7cdcdea73 Preserve vec3 type.
Summary: Preserve vec3 type with CodeGen option.

Reviewers: Anastasia, bruno

Reviewed By: Anastasia

Subscribers: bruno, ahatanak, cfe-commits

Differential Revision: https://reviews.llvm.org/D30810

llvm-svn: 299445
2017-04-04 16:40:25 +00:00
Davide Italiano 1a3665bb82 [Driver] Don't crash on invalid values of -mrelocation-model=.
This is handled in a similar way we handle invalid -mcode-model.

PR: 31840
llvm-svn: 299315
2017-04-01 21:07:07 +00:00
Dean Michael Berris 504fc2262a [XRay][clang] Fix the -fxray-instruction-threshold flag processing
Summary:
The refactoring introduced a regression in the flag processing for
-fxray-instruction-threshold which causes it to not get passed properly.
This change should restore the previous behaviour.

Reviewers: rnk, pelikan

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31491

llvm-svn: 299126
2017-03-30 22:46:45 +00:00
Simon Pilgrim 2c51880a82 Spelling mistakes in comments. NFCI. (PR27635)
llvm-svn: 299083
2017-03-30 14:13:19 +00:00
Dean Michael Berris 835832d37a [XRay] Add -fxray-{always,never}-instrument= flags to clang
Summary:
The -fxray-always-instrument= and -fxray-never-instrument= flags take
filenames that are used to imbue the XRay instrumentation attributes
using a whitelist mechanism (similar to the sanitizer special cases
list). We use the same syntax and semantics as the sanitizer blacklists
files in the implementation.

As implemented, we respect the attributes that are already defined in
the source file (i.e. those that have the
[[clang::xray_{always,never}_instrument]] attributes) before applying
the always/never instrument lists.

Reviewers: rsmith, chandlerc

Subscribers: jfb, mgorny, cfe-commits

Differential Revision: https://reviews.llvm.org/D30388

llvm-svn: 299041
2017-03-30 00:29:36 +00:00
Adam Nemet 049a31d53d Use FPContractModeKind universally
FPContractModeKind is the codegen option flag which is already ternary (off,
on, fast).  This makes it universally the type for the contractable info
across the front-end:

* In FPOptions (i.e. in the Sema + in the expression nodes).
* In LangOpts::DefaultFPContractMode which is the option that initializes
FPOptions in the Sema.

Another way to look at this change is that before fp-contractable on/off were
the only states handled to the front-end:
 * For "on", FMA folding was performed by  the front-end
 * For "fast", we simply forwarded the flag to TargetOptions to handle it in
 LLVM

Now off/on/fast are all exposed because for fast we will generate
fast-math-flags during CodeGen.

This is toward moving fp-contraction=fast from an LLVM TargetOption to a
FastMathFlag in order to fix PR25721.

---
This is a recommit of r299027 with an adjustment to the test
CodeGenCUDA/fp-contract.cu.  The test assumed that even
though -ffp-contract=on is passed FE-based folding of FMA won't happen.

This is obviously wrong since the user is asking for this explicitly with the
option.  CUDA is different that -ffp-contract=fast is on by default.

The test used to "work" because contract=fast and contract=on were maintained
separately and we didn't fold in the FE because contract=fast was on due to
the target-default.  This patch consolidates the contract=on/fast/off state
into a ternary state hence the change in behavior.
---

Differential Revision: https://reviews.llvm.org/D31167

llvm-svn: 299033
2017-03-29 21:54:24 +00:00
Adam Nemet e940f358c8 Revert "Use FPContractModeKind universally"
This reverts commit r299027.

It's causing a test failure in clang's CodeGenCUDE/fp-contract.cu

llvm-svn: 299029
2017-03-29 21:24:19 +00:00
Adam Nemet 32093a1c28 Use FPContractModeKind universally
FPContractModeKind is the codegen option flag which is already ternary (off,
on, fast).  This makes it universally the type for the contractable info
across the front-end:

* In FPOptions (i.e. in the Sema + in the expression nodes).
* In LangOpts::DefaultFPContractMode which is the option that initializes
FPOptions in the Sema.

Another way to look at this change is that before fp-contractable on/off were
the only states handled to the front-end:
 * For "on", FMA folding was performed by  the front-end
 * For "fast", we simply forwarded the flag to TargetOptions to handle it in
 LLVM

Now off/on/fast are all exposed because for fast we will generate
fast-math-flags during CodeGen.

This is toward moving fp-contraction=fast from an LLVM TargetOption to a
FastMathFlag in order to fix PR25721.

Differential Revision: https://reviews.llvm.org/D31167

llvm-svn: 299027
2017-03-29 20:39:49 +00:00
Graydon Hoare ce539b54e3 [PCH] Attach instance's dependency collectors to PCH external AST sources.
Summary:
When a PCH is included via -include-pch, clang should treat the
current TU as dependent on the sourcefile that the PCH was generated from.

This is currently _partly_ accomplished by InitializePreprocessor calling
AddImplicitIncludePCH to synthesize an implicit #include of the sourcefile,
into the preprocessor's Predefines buffer.

For FrontendActions such as PreprocessOnlyAction (which is, curiously, what the
driver winds up running one of in response to a plain clang -M) this is
sufficient: the preprocessor cranks over its Predefines and emits a dependency
reference to the initial sourcefile.

For other FrontendActions (for example -emit-obj or -fsyntax-only) the
Predefines buffer is reset to the suggested predefines buffer from the PCH, so
the dependency edge is lost. The result is that clang emits a .d file in those
cases that lacks a reference to the .h file responsible for the input (and in
Swift's case, our .swiftdeps file winds up not including a reference to the
source file for a PCH bridging header.)

This patch fixes the problem by taking a different tack: ignoring the
Predefines buffer (which seems a bit like a hack anyways) and directly
attaching the CompilerInstance's DependencyCollectors (and legacy
DependencyFileGenerator) to the ASTReader for the external AST.

This approach is similar to the one chosen in earlier consultation with Bruno
and Ben, and I think it's the least-bad solution, given several options.

Reviewers: bruno, benlangmuir, doug.gregor

Reviewed By: bruno, doug.gregor

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31378

llvm-svn: 299009
2017-03-29 17:33:09 +00:00
Adam Nemet 484aa45153 Encapsulate FPOptions and use it consistently
Sema holds the current FPOptions which is adjusted by 'pragma STDC
FP_CONTRACT'.  This then gets propagated into expression nodes as they are
built.

This encapsulates FPOptions so that this propagation happens opaquely rather
than directly with the fp_contractable on/off bit.  This allows controlled
transitioning of fp_contractable to a ternary value (off, on, fast).  It will
also allow adding more fast-math flags later.

This is toward moving fp-contraction=fast from an LLVM TargetOption to a
FastMathFlag in order to fix PR25721.

Differential Revision: https://reviews.llvm.org/D31166

llvm-svn: 298877
2017-03-27 19:17:25 +00:00
Egor Churaev 45c26ee0bf [OpenCL] Extended mapping of parcing CodeGen arguments
Summary: Enable cl_mad_enamle and cl_no_signed_zeros options when user turns on cl_unsafe_math_optimizations or cl_fast_relaxed_math options.

Reviewers: Anastasia, cfe-commits

Reviewed By: Anastasia

Subscribers: bader, yaxunl

Differential Revision: https://reviews.llvm.org/D31324

llvm-svn: 298838
2017-03-27 10:38:01 +00:00
Anastasia Stulova 671db7027c [OpenCL][Bug 10573] Don't set CXXOperatorNames flag
The flag CXXOperatorNames was overwritten unconditionally
after being set for OpenCL. 

There seems to be no necessity to set it, so removing the line.

llvm-svn: 298709
2017-03-24 16:43:51 +00:00
Teresa Johnson 488d1dc0ed [ThinLTO] Clang support for emitting minimized bitcode for thin link
Summary:
Clang companion patch to LLVM patch D31027, which adds support
for emitting minimized bitcode file for use in the thin link step.
Add a cc1 option -fthin-link-bitcode=<file> to trigger this behavior.

Depends on D31027.

Reviewers: mehdi_amini, pcc

Subscribers: cfe-commits, Prazek

Differential Revision: https://reviews.llvm.org/D31050

llvm-svn: 298639
2017-03-23 19:47:49 +00:00
Eric Christopher 3646e6289d Move setting of LangOpts based on target flags out of CompilerInstance
and into TargetInfo::adjust so that it gets called in more places
throughout the compiler (AST serialization in particular).

Should fix PPC modules after removing of faltivec.

llvm-svn: 298487
2017-03-22 06:36:09 +00:00
Eric Christopher 758aad76d8 Remove the -faltivec alias option and replace it with -maltivec everywhere.
The alias was only ever used on darwin and had some issues there,
and isn't used in practice much. Also fixes a problem with -mno-altivec
not turning off -maltivec.

Also add a diagnostic for faltivec/fno-altivec that directs users to use
maltivec options and include the altivec.h file explicitly.

llvm-svn: 298449
2017-03-21 22:06:18 +00:00
Zachary Turner 82a0c97b32 Add a function to MD5 a file's contents.
In doing so, clean up the MD5 interface a little.  Most
existing users only care about the lower 8 bytes of an MD5,
but for some users that care about the upper and lower,
there wasn't a good interface.  Furthermore, consumers
of the MD5 checksum were required to handle endianness
details on their own, so it seems reasonable to abstract
this into a nicer interface that just gives you the right
value.

Differential Revision: https://reviews.llvm.org/D31105

llvm-svn: 298322
2017-03-20 23:33:18 +00:00
Richard Smith 4b0cad0bb8 Bump __cplusplus for C++17 to 201703L per the C++17 DIS.
llvm-svn: 298299
2017-03-20 20:12:48 +00:00
Duncan P. N. Exon Smith 030d7d6daa Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338).  The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298278
2017-03-20 17:58:26 +00:00
Renato Golin f1966cf646 Revert "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298165, as it broke the ARM builds.

llvm-svn: 298185
2017-03-18 12:31:32 +00:00
Bruno Cardoso Lopes 5a0af1fcd1 [Modules] In case of lock timeout, fallback and build module
Duncan's r298165 introduced the PCMCache mechanism, which guarantees
that locks aren't necessary anymore for correctness but only for
performance, by avoiding building it twice when possible.

Change the logic to avoid an error but actually build the module in case
the timeout happens. Instead of an error, still emit a remark for
debugging purposes.

rdar://problem/30297862

llvm-svn: 298175
2017-03-18 00:26:18 +00:00
Duncan P. N. Exon Smith 079c40e886 Modules: Cache PCMs in memory and avoid a use-after-free
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

  - The ModuleManager stores PCMs there that it loads from disk, never
    touching the disk if the cache is hot.

  - When modules fail to validate, they're removed from the cache.

  - When a CompilerInstance is spawned to build a new module, each
    already-loaded PCM is assumed to be valid, and is frozen to avoid
    the use-after-free.

  - Any newly-built module is written directly to the cache to avoid the
    round-trip to the filesystem, making lock files unnecessary for
    correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298165
2017-03-17 22:55:13 +00:00
Adrian Prantl 0ebdeac13a Canonicalize the path provided by -fmodules-cache-path.
This fixes lookup mismatches that could happen when the module cache
path contained a '/./' component.

<rdar://problem/30413458>

Differential Revision: https://reviews.llvm.org/D30915

llvm-svn: 297790
2017-03-14 23:07:49 +00:00
Duncan P. N. Exon Smith 60fa28882e Modules: Use hash of PCM content for SIGNATURE
Change ASTFileSignature from a random 32-bit number to the hash of the
PCM content.

  - Move definition ASTFileSignature to Basic/Module.h so Module and
    ASTSourceDescriptor can use it.

  - Change the signature from uint64_t to std::array<uint32_t,5>.

  - Stop using (saving/reading) the size and modification time of PCM
    files when there is a valid SIGNATURE.

  - Add UNHASHED_CONTROL_BLOCK, and use it to store the SIGNATURE record
    and other records that shouldn't affect the hash.  Because implicit
    modules reuses the same file for multiple levels of -Werror, this
    includes DIAGNOSTIC_OPTIONS and DIAG_PRAGMA_MAPPINGS.

This helps to solve a PCH + implicit Modules dependency issue: PCH files
are handled by the external build system, whereas implicit modules are
handled by internal compiler build system.  This prevents invalidating a
PCH when the compiler overwrites a PCM file with the same content
(modulo the diagnostic differences).

Design and original patch by Manman Ren!

llvm-svn: 297655
2017-03-13 18:45:08 +00:00
Jonas Devlieghere 5eb9c81d82 [Linker] Provide callback for internalization
Differential Revision: https://reviews.llvm.org/D30738

llvm-svn: 297649
2017-03-13 18:08:11 +00:00