Commit Graph

337 Commits

Author SHA1 Message Date
Richard Smith d13863008b Add #pragma clang module begin/end pragmas and generate them when preprocessing a module.
These pragmas are intended to simulate the effect of entering or leaving a file
with an associated module. This is not completely implemented yet: declarations
between the pragmas will not be attributed to the correct module, but macro
visibility is already functional.

Modules named by #pragma clang module begin must already be known to clang (in
some module map that's either loaded or on the search path).

llvm-svn: 302098
2017-05-04 00:29:54 +00:00
Duncan P. N. Exon Smith 030d7d6daa Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338).  The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298278
2017-03-20 17:58:26 +00:00
Renato Golin f1966cf646 Revert "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298165, as it broke the ARM builds.

llvm-svn: 298185
2017-03-18 12:31:32 +00:00
Duncan P. N. Exon Smith 079c40e886 Modules: Cache PCMs in memory and avoid a use-after-free
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

  - The ModuleManager stores PCMs there that it loads from disk, never
    touching the disk if the cache is hot.

  - When modules fail to validate, they're removed from the cache.

  - When a CompilerInstance is spawned to build a new module, each
    already-loaded PCM is assumed to be valid, and is frozen to avoid
    the use-after-free.

  - Any newly-built module is written directly to the cache to avoid the
    round-trip to the filesystem, making lock files unnecessary for
    correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298165
2017-03-17 22:55:13 +00:00
David Blaikie e304168853 Move PreprocessorOptions to std::shared_ptr from IntrusiveRefCntPtr
llvm-svn: 291160
2017-01-05 19:11:36 +00:00
Eugene Zelenko e95e7d5d64 Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes.
Differential revision: https://reviews.llvm.org/D24115

llvm-svn: 280870
2016-09-07 21:53:17 +00:00
Richard Smith bbcc9f0462 C++ Modules TS: add frontend support for building pcm files from module
interface files. At the moment, all declarations (and no macros) are exported,
and 'export' declarations are not supported yet.

llvm-svn: 279794
2016-08-26 00:14:38 +00:00
Richard Smith 49cc1ccb00 C++ Modules TS: Add parsing support for module import declaration.
llvm-svn: 279163
2016-08-18 21:59:42 +00:00
Richard Smith 3dba7ebde4 PR28438: Update the information on an identifier with local definitions before
trying to write out its macro graph, in case we imported a module that added
another module macro between the most recent local definition and the end of
the module.

llvm-svn: 279024
2016-08-18 01:16:55 +00:00
John Brawn 4d79ec7fe8 Reapply r276973 "Adjust Registry interface to not require plugins to export a registry"
This differs from the previous version by being more careful about template
instantiation/specialization in order to prevent errors when building with
clang -Werror. Specifically:
 * begin is not defined in the template and is instead instantiated when Head
   is. I think the warning when we don't do that is wrong (PR28815) but for now
   at least do it this way to avoid the warning.
 * Instead of performing template specializations in LLVM_INSTANTIATE_REGISTRY
   instead provide a template definition then do explicit instantiation. No
   compiler I've tried has problems with doing it the other way, but strictly
   speaking it's not permitted by the C++ standard so better safe than sorry.

Original commit message:

Currently the Registry class contains the vestiges of a previous attempt to
allow plugins to be used on Windows without using BUILD_SHARED_LIBS, where a
plugin would have its own copy of a registry and export it to be imported by
the tool that's loading the plugin. This only works if the plugin is entirely
self-contained with the only interface between the plugin and tool being the
registry, and in particular this conflicts with how IR pass plugins work.

This patch changes things so that instead the add_node function of the registry
is exported by the tool and then imported by the plugin, which solves this
problem and also means that instead of every plugin having to export every
registry they use instead LLVM only has to export the add_node functions. This
allows plugins that use a registry to work on Windows if
LLVM_EXPORT_SYMBOLS_FOR_PLUGINS is used.

llvm-svn: 277806
2016-08-05 11:01:08 +00:00
John Brawn 2853269224 Revert r276973 "Adjust Registry interface to not require plugins to export a registry"
Buildbot failures when building with clang -Werror. Reverting while I try to
figure this out.

llvm-svn: 277008
2016-07-28 17:17:22 +00:00
John Brawn 778c3c6c61 Reapply r276856 "Adjust Registry interface to not require plugins to export a registry"
This version has two fixes compared to the original:
 * In Registry.h the template static members are instantiated before they are
   used, as clang gives an error if you do it the other way around.
 * The use of the Registry template in clang-tidy is updated in the same way as
   has been done everywhere else.

Original commit message:

Currently the Registry class contains the vestiges of a previous attempt to
allow plugins to be used on Windows without using BUILD_SHARED_LIBS, where a
plugin would have its own copy of a registry and export it to be imported by
the tool that's loading the plugin. This only works if the plugin is entirely
self-contained with the only interface between the plugin and tool being the
registry, and in particular this conflicts with how IR pass plugins work.

This patch changes things so that instead the add_node function of the registry
is exported by the tool and then imported by the plugin, which solves this
problem and also means that instead of every plugin having to export every
registry they use instead LLVM only has to export the add_node functions. This
allows plugins that use a registry to work on Windows if
LLVM_EXPORT_SYMBOLS_FOR_PLUGINS is used.

llvm-svn: 276973
2016-07-28 12:48:17 +00:00
Vassil Vassilev 644ea61d2d Implement filtering for code completion of identifiers.
Patch by Cristina Cristescu and Axel Naumann!

Agreed on post commit review (D17820).

llvm-svn: 276878
2016-07-27 14:56:59 +00:00
John Brawn 3839263204 Revert r276856 "Adjust Registry interface to not require plugins to export a registry"
This is causing a huge pile of buildbot failures.

llvm-svn: 276857
2016-07-27 11:41:18 +00:00
John Brawn 63aff61019 Adjust Registry interface to not require plugins to export a registry
Currently the Registry class contains the vestiges of a previous attempt to
allow plugins to be used on Windows without using BUILD_SHARED_LIBS, where a
plugin would have its own copy of a registry and export it to be imported by
the tool that's loading the plugin. This only works if the plugin is entirely
self-contained with the only interface between the plugin and tool being the
registry, and in particular this conflicts with how IR pass plugins work.

This patch changes things so that instead the add_node function of the registry
is exported by the tool and then imported by the plugin, which solves this
problem and also means that instead of every plugin having to export every
registry they use instead LLVM only has to export the add_node functions. This
allows plugins that use a registry to work on Windows if
LLVM_EXPORT_SYMBOLS_FOR_PLUGINS is used.

Differential Revision: http://reviews.llvm.org/D21385

llvm-svn: 276856
2016-07-27 11:18:38 +00:00
Mehdi Amini 9670f847b8 [NFC] Header cleanup
Summary: Removed unused headers, replaced some headers with forward class declarations

Patch by: Eugene <claprix@yandex.ru>

Differential Revision: https://reviews.llvm.org/D20100

llvm-svn: 275882
2016-07-18 19:02:11 +00:00
Benjamin Kramer cfeacf56f0 Apply clang-tidy's misc-move-constructor-init throughout Clang.
No functionality change intended, maybe a tiny performance improvement.

llvm-svn: 270996
2016-05-27 14:27:13 +00:00
NAKAMURA Takumi cacd94e242 AnnotateFunctions: Tweak for mingw.
- Externalize the registry.
  - Update libdeps.

llvm-svn: 265301
2016-04-04 15:30:44 +00:00
Richard Smith 7e82e019c6 [modules] Flatten -fmodule-name= and -fmodule-implementation-of= into a single
option. Previously these options could both be used to specify that you were
compiling the implementation file of a module, with a different set of minor
bugs in each case.

This change removes -fmodule-implementation-of, and instead tracks a flag to
determine whether we're currently building a module. -fmodule-name now behaves
the same way that -fmodule-implementation-of previously did.

llvm-svn: 261372
2016-02-19 22:25:36 +00:00
Reid Kleckner 738d48d5fe Sink some PTHManager includes out of Preprocessor.h
This reduces the number of .cpp files needed to be rebuilt after
touching OnDiskHashTable from 120 to 21 for me.

llvm-svn: 251810
2015-11-02 17:53:55 +00:00
Angel Garcia Gomez 637d1e6694 Roll-back r250822.
Summary: It breaks the build for the ASTMatchers

Subscribers: klimek, cfe-commits

Differential Revision: http://reviews.llvm.org/D13893

llvm-svn: 250827
2015-10-20 13:23:58 +00:00
Angel Garcia Gomez b5250d3448 Apply modernize-use-default to clang.
Summary: Replace empty bodies of default constructors and destructors with '= default'.

Reviewers: bkramer, klimek

Subscribers: klimek, alexfh, cfe-commits

Differential Revision: http://reviews.llvm.org/D13890

llvm-svn: 250822
2015-10-20 12:52:55 +00:00
Yaron Keren 8b563665c3 Replace double negation of !FileID.isInvalid() with FileID.isValid().
+couple more of double-negated !SourceLocation.isInvalid() unfixed in r249228.

llvm-svn: 249235
2015-10-03 10:46:20 +00:00
Yaron Keren 716f3a68c9 Fix typo.
llvm-svn: 248807
2015-09-29 16:51:08 +00:00
Artem Belevich b5bc923af4 [CUDA] Allow parsing of host and device code simultaneously.
* adds -aux-triple option to specify target triple
 * propagates aux target info to AST context and Preprocessor
 * pulls in target specific preprocessor macros.
 * pulls in target-specific builtins from aux target.
 * sets appropriate host or device attribute on builtins.

Differential Revision: http://reviews.llvm.org/D12917

llvm-svn: 248299
2015-09-22 17:23:22 +00:00
Eric Christopher 02d5d86b4e Rename the non-coding style conformant functions in namespace Builtins
to match the rest of their brethren and reformat the bits that need it.

llvm-svn: 244186
2015-08-06 01:01:12 +00:00
Jordan Rose a46bfa6b83 [Preprocessor] Iterating over all macros should include those from modules.
So, iterate over the list of macros mentioned in modules, and make sure those
are in the master table.

This isn't particularly efficient, but hopefully it's something that isn't
done too often.

PR23929 and rdar://problem/21480635

llvm-svn: 240571
2015-06-24 19:27:02 +00:00
Richard Smith 04765ae01e [modules] If we re-enter a submodule from within itself (when submodule
visibility is enabled) or leave and re-enter it, restore the macro and module
visibility state from last time we were in that submodule.

This allows mutually-#including header files to stand a chance at being
modularized with local visibility enabled.

llvm-svn: 237871
2015-05-21 01:20:10 +00:00
Richard Smith 31d5184dc5 Generalize future keyword compat diagnostics.
This, in preparation for the introduction of more new keywords in the
implementation of the C++ language, generalizes the support for future keyword
compat diagnostics (e.g., diag::warn_cxx11_keyword) by extending the
applicability of the relevant property in IdentifierTable with appropriate
renaming.

Patch by Hubert Tong!

llvm-svn: 237332
2015-05-14 04:00:59 +00:00
Richard Smith dbbc523639 [modules] Rearrange preprocessor module visibility handling, no observable change intended.
llvm-svn: 237331
2015-05-14 02:25:44 +00:00
Richard Smith 10434f307c [modules] Remove dead code from Module for tracking macro import locations.
llvm-svn: 236376
2015-05-02 02:08:26 +00:00
Richard Smith 38477db8c4 [modules] If a module #includes a modular header that #undef's its macro, it
should not export the macro.

... at least, not unless we have local submodule visibility enabled.

llvm-svn: 236369
2015-05-02 00:45:56 +00:00
Richard Smith a7e2cc684f [modules] Start moving the module visibility information off the Module itself.
It has no place there; it's not a property of the Module, and it makes
restoring the visibility set when we leave a submodule more difficult.

llvm-svn: 236300
2015-05-01 01:53:09 +00:00
Richard Smith f5ec2acca9 Add an assert to get information on buildbot failure.
llvm-svn: 236181
2015-04-29 23:40:48 +00:00
Richard Smith 20e883e59b [modules] Stop trying to fake up a linear MacroDirective history.
Modules builds fundamentally have a non-linear macro history. In the interest
of better source fidelity, represent the macro definition information
faithfully: we have a linear macro directive history within each module, and at
any point we have a unique "latest" local macro directive and a collection of
visible imported directives. This also removes the attendent complexity of
attempting to create a correct MacroDirective history (which we got wrong
in the general case).

No functionality change intended.

llvm-svn: 236176
2015-04-29 23:20:19 +00:00
Richard Smith 1e1728536a Refactor to make MacroState ownership and lifetime clearer.
llvm-svn: 236032
2015-04-28 21:05:07 +00:00
Richard Smith 66f5ab26e3 Fix memory leak found by asan buildbot.
llvm-svn: 235957
2015-04-28 01:11:23 +00:00
Richard Smith 753e007091 [modules] Incrementally compute the list of overridden module macros based on
the active module macros at the point of definition, rather than reconstructing
it from the macro history. No functionality change intended.

llvm-svn: 235941
2015-04-27 23:21:38 +00:00
Richard Smith b8b2ed6529 [modules] Determine the set of macros exported by a submodule at the end of that submodule.
Previously we'd defer this determination until writing the AST, which doesn't
allow us to use this information when building other submodules of the same
module. This change also allows us to use a uniform mechanism for writing
module macro records, independent of whether they are local or imported.

llvm-svn: 235614
2015-04-23 18:18:26 +00:00
Argyrios Kyrtzidis 5c585253e5 [Modules] Fix crash in Preprocessor::getLastMacroWithSpelling().
Macro names that got undefined inside a module may not have their MacroInfo set.

llvm-svn: 231251
2015-03-04 16:03:07 +00:00
Sean Callanan 8759649013 Modified the Objective-C lexer and parser (only
in debugger mode) to accept @import declarations
and pass them to the debugger.  

In the preprocessor, accept import declarations
if the debugger is enabled, but don't actually
load the module, just pass the import path on to 
the preprocessor callbacks.

In the Objective-C parser, if it sees an import
declaration in statement context (usual for LLDB),
ignore it and return a NullStmt.

llvm-svn: 223855
2014-12-09 23:47:56 +00:00
Richard Smith 3c1a41ad99 [modules] Track how 'header' directives were written in module map files,
rather than trying to extract this information from the FileEntry after the
fact.

This has a number of beneficial effects. For instance, diagnostic messages for
failed module builds give a path relative to the "module root" rather than an
absolute file path, and the contents of the module includes file is no longer
dependent on what files the including TU happened to inspect prior to
triggering the module build.

llvm-svn: 223095
2014-12-02 00:08:08 +00:00
Argyrios Kyrtzidis ee301f941c [libclang] If the code-completion point is inside the preamble, adjust the position at the beginning of the file after the preamble.
Otherwise we will not hit the code-completion point.

llvm-svn: 220136
2014-10-18 06:23:50 +00:00
Argyrios Kyrtzidis e62d682a27 [libclang] Allow code-completion when pointing at the end-of-file.
llvm-svn: 220135
2014-10-18 06:19:36 +00:00
David Blaikie 9f0af9d862 Unique_ptrify Preprocessor::PragmaHandlersBackup
Follow up to r217656

llvm-svn: 217829
2014-09-15 21:31:42 +00:00
Craig Topper be25030137 Use unique_ptr for ScratchBuf and PragmaHandlers in the preprocessor.
llvm-svn: 217656
2014-09-12 05:19:24 +00:00
Craig Topper b8a7053055 Unique_ptrify PPCallbacks ownership.
Unique_ptr creation stil needs to be moved earlier at some of the call sites.

llvm-svn: 217474
2014-09-10 04:53:53 +00:00
David Blaikie 6d5038c19c unique_ptrify Preprocessor's TokenLexerCache
llvm-svn: 216756
2014-08-29 19:36:52 +00:00
David Blaikie 50a5f97e82 unique_ptrify SourceManager::createFileID
llvm-svn: 216715
2014-08-29 07:59:55 +00:00
David Blaikie 49cc3181a2 Overload SourceManager::overrideFileContents so that unconditionally passing ownership is explicitly done using unique_ptr.
Only those callers who are dynamically passing ownership should need the
3 argument form. Those accepting the default ("do pass ownership")
should do so explicitly with a unique_ptr now.

llvm-svn: 216614
2014-08-27 20:54:45 +00:00