llvm-project/clang/lib/Lex
Duncan P. N. Exon Smith 030d7d6daa Reapply "Modules: Cache PCMs in memory and avoid a use-after-free"
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338).  The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.

Original commit message follows:

----

Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands).  Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly.  Furthermore, the LockFileManager is not
robust in every environment.  Other -cc1 commands can stall until
timeout (after about eight minutes).

This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack.  The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module.  Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout.  Perhaps we should
reconsider blocking at all.

This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.

The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename.  Its ownership is shared by the
CompilerInstance and ModuleManager.

- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.

- When modules fail to validate, they're removed from the cache.

- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.

- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.

Original patch by Manman Ren; most testcases by Adrian Prantl!

llvm-svn: 298278
2017-03-20 17:58:26 +00:00
..
CMakeLists.txt
HeaderMap.cpp Use StringRef for MemoryBuffer identifier API (NFC) 2016-10-01 16:38:28 +00:00
HeaderSearch.cpp Fix handling of -fmodule-map-file=X where X has no directory component. 2017-03-09 00:58:22 +00:00
Lexer.cpp Allow lexer to handle string_view literals. Patch from Anton Bikineev. 2016-12-30 04:51:10 +00:00
LiteralSupport.cpp Allow lexer to handle string_view literals. Patch from Anton Bikineev. 2016-12-30 04:51:10 +00:00
MacroArgs.cpp [modules] Stop trying to fake up a linear MacroDirective history. 2015-04-29 23:20:19 +00:00
MacroInfo.cpp Retire llvm::alignOf in favor of C++11 alignof. 2016-10-20 14:27:22 +00:00
ModuleMap.cpp Fix modules codegen to be compatible with modules-ts 2017-01-31 21:28:19 +00:00
PPCaching.cpp [Preprocessor] Fix incorrect token caching that occurs when lexing _Pragma 2017-02-24 17:45:16 +00:00
PPCallbacks.cpp
PPConditionalDirectiveRecord.cpp Switch PPCallbacks to take the new MacroDefinition instead of MacroDirective*, in order to preserve full information on module macro expansion. 2015-05-04 03:15:40 +00:00
PPDirectives.cpp Fix for pr31836 - pp_nonportable_path on absolute paths: broken delimiters 2017-02-21 22:30:55 +00:00
PPExpressions.cpp Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes. 2016-09-07 21:53:17 +00:00
PPLexerChange.cpp Fix MSVC warning about missing 'this' from lambda default capture mode 2016-10-27 10:51:29 +00:00
PPMacroExpansion.cpp P0426: Make the library implementation of constexpr char_traits a little easier 2017-01-20 00:45:35 +00:00
PTHLexer.cpp Turn FileManager DirectoryEntry::Name from raw pointer to StringRef (NFC) 2016-10-11 07:31:29 +00:00
Pragma.cpp [Preprocessor] Fix incorrect token caching that occurs when lexing _Pragma 2017-02-24 17:45:16 +00:00
PreprocessingRecord.cpp Retire llvm::alignOf in favor of C++11 alignof. 2016-10-20 14:27:22 +00:00
Preprocessor.cpp Reapply "Modules: Cache PCMs in memory and avoid a use-after-free" 2017-03-20 17:58:26 +00:00
PreprocessorLexer.cpp
ScratchBuffer.cpp MSan told me that we actually dump the entire scratch buffer into PCH files, initialize it. 2015-04-06 20:01:49 +00:00
TokenConcatenation.cpp [PP] Replace some index based for loops with range based ones 2016-10-26 13:06:13 +00:00
TokenLexer.cpp [PP] Replace some index based for loops with range based ones 2016-10-26 13:06:13 +00:00
UnicodeCharSets.h Header guard canonicalization, clang part. 2014-08-13 16:25:19 +00:00