Leverage the InMemoryModuleCache to invalidate a module the first time
it fails to import (and to lock a module as soon as it's built or
imported successfully). For implicit module builds, this optimizes
importing deep graphs where the leaf module is out-of-date; see example
near the end of the commit message.
Previously the cache finalized ("locked in") all modules imported so far
when starting a new module build. This was sufficient to prevent
loading two versions of the same module, but was somewhat arbitrary and
hard to reason about.
Now the cache explicitly tracks module state, where each module must be
one of:
- Unknown: module not in the cache (yet).
- Tentative: module in the cache, but not yet fully imported.
- ToBuild: module found on disk could not be imported; need to build.
- Final: module in the cache has been successfully built or imported.
Preventing repeated failed imports avoids variation in builds based on
shifting filesystem state. Now it's guaranteed that a module is loaded
from disk exactly once. It now seems safe to remove
FileManager::invalidateCache, but I'm leaving that for a later commit.
The new, precise logic uncovered a pre-existing problem in the cache:
the map key is the module filename, and different contexts use different
filenames for the same PCM file. (In particular, the test
Modules/relative-import-path.c does not build without this commit.
r223577 started using a relative path to describe a module's base
directory when importing it within another module. As a result, the
module cache sees an absolute path when (a) building the module or
importing it at the top-level, and a relative path when (b) importing
the module underneath another one.)
The "obvious" fix is to resolve paths using FileManager::getVirtualFile
and change the map key for the cache to a FileEntry, but some contexts
(particularly related to ASTUnit) have a shorter lifetime for their
FileManager than the InMemoryModuleCache. This is worth pursuing
further in a later commit; perhaps by tying together the FileManager and
InMemoryModuleCache lifetime, or moving the in-memory PCM storage into a
VFS layer.
For now, use the PCM's base directory as-written for constructing the
filename to check the ModuleCache.
Example
=======
To understand the build optimization, first consider the build of a
module graph TU -> A -> B -> C -> D with an empty cache:
TU builds A'
A' builds B'
B' builds C'
C' builds D'
imports D'
B' imports C'
imports D'
A' imports B'
imports C'
imports D'
TU imports A'
imports B'
imports C'
imports D'
If we build TU again, where A, B, C, and D are in the cache and D is
out-of-date, we would previously get this build:
TU imports A
imports B
imports C
imports D (out-of-date)
TU builds A'
A' imports B
imports C
imports D (out-of-date)
builds B'
B' imports C
imports D (out-of-date)
builds C'
C' imports D (out-of-date)
builds D'
imports D'
B' imports C'
imports D'
A' imports B'
imports C'
imports D'
TU imports A'
imports B'
imports C'
imports D'
After this commit, we'll immediateley invalidate A, B, C, and D when we
first observe that D is out-of-date, giving this build:
TU imports A
imports B
imports C
imports D (out-of-date)
TU builds A' // The same graph as an empty cache.
A' builds B'
B' builds C'
C' builds D'
imports D'
B' imports C'
imports D'
A' imports B'
imports C'
imports D'
TU imports A'
imports B'
imports C'
imports D'
The new build matches what we'd naively expect, pretty closely matching
the original build with the empty cache.
rdar://problem/48545366
llvm-svn: 355778
Change MemoryBufferCache to InMemoryModuleCache, moving it from Basic to
Serialization. Another patch will start using it to manage module build
more explicitly, but this is split out because it's mostly mechanical.
Because of the move to Serialization we can no longer abuse the
Preprocessor to forward it to the ASTReader. Besides the rename and
file move, that means Preprocessor::Preprocessor has one fewer parameter
and ASTReader::ASTReader has one more.
llvm-svn: 355777
Add a remark for importing modules. Depending on whether this is a
direct import (into the TU being built by this compiler instance) or
transitive import (into an already-imported module), the diagnostic has
two forms:
importing module 'Foo' from 'path/to/Foo.pcm'
importing module 'Foo' into 'Bar' from 'path/to/Foo.pcm'
Also drop a redundant FileCheck invocation in Rmodule-build.m that was
using -Reverything, since the notes from -Rmodule-import were confusing
it.
https://reviews.llvm.org/D58891
llvm-svn: 355477
This patch implements the parsing and sema support for the OpenMP
'from'-clause with potential user-defined mappers attached.
User-defined mappers are a new feature in OpenMP 5.0. A 'from'-clause
can have an explicit or implicit associated mapper, which instructs the
compiler to generate and use customized mapping functions. An example is
shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(id: struct S s) map(s, s.d[0:s.len])
struct S ss;
#pragma omp target update from(mapper(id): ss) // use the mapper with name 'id' to map ss from device
Contributed-by: Lingda Li <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D58638
llvm-svn: 354817
This patch implements the parsing and sema support for OpenMP to clause
with potential user-defined mappers attached. User defined mapper is a
new feature in OpenMP 5.0. A to/from clause can have an explicit or
implicit associated mapper, which instructs the compiler to generate and
use customized mapping functions. An example is shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(id: struct S s) map(s, s.d[0:s.len])
struct S ss;
#pragma omp target update to(mapper(id): ss) // use the mapper with name 'id' to map ss to device
Contributed-by: <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D58523
llvm-svn: 354698
This patch implements the parsing and sema support for OpenMP map
clauses with potential user-defined mapper attached. User defined mapper
is a new feature in OpenMP 5.0. A map clause can have an explicit or
implicit associated mapper, which instructs the compiler to generate
extra data mapping. An example is shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(id: struct S s) map(s, s.d[0:s.len])
struct S ss;
#pragma omp target map(mapper(id) tofrom: ss) // use the mapper with name 'id' to map ss
Contributed-by: Lingda Li <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D58074
llvm-svn: 354347
This patch implements parsing and sema for "omp declare mapper"
directive. User defined mapper, i.e., declare mapper directive, is a new
feature in OpenMP 5.0. It is introduced to extend existing map clauses
for the purpose of simplifying the copy of complex data structures
between host and device (i.e., deep copy). An example is shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(struct S s) map(s, s.d[0:s.len]) // Memory region that d points to is also mapped using this mapper.
Contributed-by: Lingda Li <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D56326
llvm-svn: 352906
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
A map clause with the close map-type-modifier is a hint to
prefer that the variables are mapped using a copy into faster
memory.
Patch by Ahsan Saghir (saghir)
Differential Revision: https://reviews.llvm.org/D55719
llvm-svn: 349551
Address spaces are cast into generic before invoking the constructor.
Added support for a trailing Qualifiers object in FunctionProtoType.
Note: This recommits the previously reverted patch,
but now it is commited together with a fix for lldb.
Differential Revision: https://reviews.llvm.org/D54862
llvm-svn: 349019
Address spaces are cast into generic before invoking the constructor.
Added support for a trailing Qualifiers object in FunctionProtoType.
Differential Revision: https://reviews.llvm.org/D54862
llvm-svn: 348927
When debugging a boost build with a modified
version of Clang, I discovered that the PTH implementation
stores TokenKind in 8 bits. However, we currently have 368
TokenKinds.
The result is that the value gets truncated and the wrong token
gets picked up when including PTH files. It seems that this will
go wrong every time someone uses a token that uses the 9th bit.
Upon asking on IRC, it was brought up that this was a highly
experimental features that was considered a failure. I discovered
via googling that BoostBuild (mostly Boost.Math) is the only user of
this
feature, using the CC1 flag directly. I believe that this can be
transferred over to normal PCH with minimal effort:
https://github.com/boostorg/build/issues/367
Based on advice on IRC and research showing that this is a nearly
completely unused feature, this patch removes it entirely.
Note: I considered leaving the build-flags in place and making them
emit an error/warning, however since I've basically identified and
warned the only user, it seemed better to just remove them.
Differential Revision: https://reviews.llvm.org/D54547
Change-Id: If32744275ef1f585357bd6c1c813d96973c4d8d9
llvm-svn: 348266
This patch breaks Index/opencl-types.cl LIT test:
Script:
--
: 'RUN: at line 1'; stage1/bin/c-index-test -test-print-type llvm/tools/clang/test/Index/opencl-types.cl -cl-std=CL2.0 | stage1/bin/FileCheck llvm/tools/clang/test/Index/opencl-types.cl
--
Command Output (stderr):
--
llvm/tools/clang/test/Index/opencl-types.cl:3:26: warning: unsupported OpenCL extension 'cl_khr_fp16' - ignoring [-Wignored-pragmas]
llvm/tools/clang/test/Index/opencl-types.cl:4:26: warning: unsupported OpenCL extension 'cl_khr_fp64' - ignoring [-Wignored-pragmas]
llvm/tools/clang/test/Index/opencl-types.cl:8:9: error: use of type 'double' requires cl_khr_fp64 extension to be enabled
llvm/tools/clang/test/Index/opencl-types.cl:11:8: error: declaring variable of type 'half' is not allowed
llvm/tools/clang/test/Index/opencl-types.cl:15:3: error: use of type 'double' requires cl_khr_fp64 extension to be enabled
llvm/tools/clang/test/Index/opencl-types.cl:16:3: error: use of type 'double4' (vector of 4 'double' values) requires cl_khr_fp64 extension to be enabled
llvm/tools/clang/test/Index/opencl-types.cl:26:26: warning: unsupported OpenCL extension 'cl_khr_gl_msaa_sharing' - ignoring [-Wignored-pragmas]
llvm/tools/clang/test/Index/opencl-types.cl:35:44: error: use of type '__read_only image2d_msaa_t' requires cl_khr_gl_msaa_sharing extension to be enabled
llvm/tools/clang/test/Index/opencl-types.cl:36:49: error: use of type '__read_only image2d_array_msaa_t' requires cl_khr_gl_msaa_sharing extension to be enabled
llvm/tools/clang/test/Index/opencl-types.cl:37:49: error: use of type '__read_only image2d_msaa_depth_t' requires cl_khr_gl_msaa_sharing extension to be enabled
llvm/tools/clang/test/Index/opencl-types.cl:38:54: error: use of type '__read_only image2d_array_msaa_depth_t' requires cl_khr_gl_msaa_sharing extension to be enabled
llvm-svn: 346338
Add support for OMP5.0 requires directive and unified_address clause.
Patches to follow will include support for additional clauses.
Differential Revision: https://reviews.llvm.org/D52359
llvm-svn: 343063
Introduce the following optimizations in DeclarationName(Table):
1. Store common kinds inline in DeclarationName instead of
DeclarationNameExtra. Currently the kind of C++ constructor, destructor,
conversion function and overloaded operator names is stored in
DeclarationNameExtra. Instead store it inline in DeclarationName.
To do this align IdentifierInfo, CXXSpecialName, DeclarationNameExtra
and CXXOperatorIdName to 8 bytes so that we can use the lower 3 bits of
DeclarationName::Ptr. This is already the case on 64 bits archs anyway.
This also allow us to remove DeclarationNameExtra from CXXSpecialName
and CXXOperatorIdName, which shave off a pointer from CXXSpecialName.
2. Synchronize the enumerations DeclarationName::NameKind,
DeclarationName::StoredNameKind and Selector::IdentifierInfoFlag.
This makes DeclarationName::getNameKind much more efficient since we can
replace the switch table by a single comparison and an addition.
3. Put the overloaded operator names inline in DeclarationNameTable to remove
an indirection. This increase the size of DeclarationNameTable a little
bit but this is not important since it is only used in ASTContext, and
never copied nor moved from. This also get rid of the last dynamic
allocation in DeclarationNameTable.
Altogether these optimizations cut the run time of parsing all of Boost by
about 0.8%. While we are at it, do the following NFC modifications:
1. Put the internal classes CXXSpecialName, CXXDeductionGuideNameExtra,
CXXOperatorIdName, CXXLiteralOperatorIdName and DeclarationNameExtra
in a namespace detail since these classes are only meant to be used by
DeclarationName and DeclarationNameTable. Make this more explicit by making
the members of these classes private and friending DeclarationName(Table).
2. Make DeclarationName::getFETokenInfo a non-template since every users are
using it to get a void *. It was supposed to be used with a type to avoid
a subsequent static_cast.
3. Change the internal functions DeclarationName::getAs* to castAs* since when
we use them we already know the correct kind. This has no external impact
since all of these are private.
Reviewed By: erichkeane, rjmccall
Differential Revision: https://reviews.llvm.org/D52267
llvm-svn: 342729
Move declarations for OMPClauseReader, OMPClauseWriter to ASTReader.h
and ASTWriter.h and move implementation to ASTReader.cpp and
ASTWriter.cpp. This change helps generalize the serialization of
OpenMP clauses and will be used in the future implementation of new
OpenMP directives (e.g. requires).
Patch by Patrick Lyster
Differential Revision: https://reviews.llvm.org/D52097
llvm-svn: 342322
submodule visibility is disabled.
Attempting to pick a specific declaration to make visible when the
module containing the merged declaration becomes visible is error-prone,
as we don't yet know which declaration we'll choose to be the definition
when we are informed of the merging.
This reinstates r342019, reverted in r342020. The regression previously
observed after this commit was fixed in r342096.
llvm-svn: 342097
submodule visibility is disabled.
Attempting to pick a specific declaration to make visible when the
module containing the merged declaration becomes visible is error-prone,
as we don't yet know which declaration we'll choose to be the definition
when we are informed of the merging.
llvm-svn: 342019
Fix a bug in the deserialization of IMPORTS section and allow for
imported modules to also be printed with -module-file-info.
rdar://problem/43867753
llvm-svn: 341902
Specifically, AttributedType now tracks a regular attr::Kind rather than
having its own parallel Kind enumeration, and AttributedTypeLoc now
holds an Attr* instead of holding an ad-hoc collection of Attr fields.
Differential Revision: https://reviews.llvm.org/D50526
This reinstates r339623, reverted in r339638, with a fix to not fail
template instantiation if we instantiate a QualType with no associated
type source information and we encounter an AttributedType.
llvm-svn: 340215
This breaks compiling atlwin.h in Chromium. I'm sure the code is invalid
in some way, but we put a lot of work into accepting it, and I'm sure
rejecting it was not an intended consequence of this refactoring. :)
llvm-svn: 339638
Specifically, AttributedType now tracks a regular attr::Kind rather than
having its own parallel Kind enumeration, and AttributedTypeLoc now
holds an Attr* instead of holding an ad-hoc collection of Attr fields.
Differential Revision: https://reviews.llvm.org/D50526
llvm-svn: 339623
We can't read a deduced return type until we are sure that the types referred
to by it are not in the middle of being loaded. So defer all reading of such
deduced return types until the end of the recursive deserialization step.
Also, when we load a function type that has a deduced return type, update all
other redeclarations of the function to have that deduced return type.
llvm-svn: 338798
Summary:
Clang emits error when implicit modules was relocated from the
first build directory. However this was biting our usecase where we copy
the contents of build directory to another directory in order to
distribute.
Differential Revision: https://reviews.llvm.org/D49852
llvm-svn: 338503
Summary:
Reproducer and errors:
https://bugs.llvm.org/show_bug.cgi?id=37878
lookupModule was falling back to loadSubdirectoryModuleMaps when it couldn't
find ModuleName in (proper) search paths. This was causing iteration over all
files in the search path subdirectories for example "/usr/include/foobar" in
bugzilla case.
Users don't expect Clang to load modulemaps in subdirectories implicitly, and
also the disk access is not cheap.
if (AllowExtraModuleMapSearch) true with ObjC with @import ModuleName.
Reviewers: rsmith, aprantl, bruno
Subscribers: cfe-commits, teemperor, v.g.vassilev
Differential Revision: https://reviews.llvm.org/D48367
llvm-svn: 337430
As listed in the above PRs, vector_size doesn't allow
dependent types/values. This patch introduces a new
DependentVectorType to handle a VectorType that has a dependent
size or type.
In the future, ALL the vector-types should be able to create one
of these to handle dependent types/sizes as well. For example,
DependentSizedExtVectorType could likely be switched to just use
this instead, though that is left as an exercise for the future.
Differential Revision: https://reviews.llvm.org/D49045
llvm-svn: 337036
The member init list for the sole constructor for CodeGenFunction
has gotten out of hand, so this patch moves the non-parameter-dependent
initializations into the member value inits.
llvm-svn: 336726
Summary:
Reproducer and errors:
https://bugs.llvm.org/show_bug.cgi?id=37878
lookupModule was falling back to loadSubdirectoryModuleMaps when it couldn't
find ModuleName in (proper) search paths. This was causing iteration over all
files in the search path subdirectories for example "/usr/include/foobar" in
bugzilla case.
Users don't expect Clang to load modulemaps in subdirectories implicitly, and
also the disk access is not cheap.
if (AllowExtraModuleMapSearch) true with ObjC with @import ModuleName.
Reviewers: rsmith, aprantl, bruno
Subscribers: cfe-commits, teemperor, v.g.vassilev
Differential Revision: https://reviews.llvm.org/D48367
llvm-svn: 336660
Functions that are a sub-Decl of a record were hashed differently than other
functions. This change keeps the AddFunctionDecl function and the hash of
records now calls this function. In addition, AddFunctionDecl has an option
to perform a hash as if the body was absent, which is required for some
checks after loading modules. Additional logic prevents multiple error
message from being printed.
llvm-svn: 336632
Implement support for MS-style PCH through headers.
This enables support for /Yc and /Yu where the through header is either
on the command line or included in the source. It replaces the current
support the requires the header also be specified with /FI.
This change adds a -cc1 option -pch-through-header that is used to either
start or stop compilation during PCH create or use.
When creating a PCH, the compilation ends after compilation of the through
header.
When using a PCH, tokens are skipped until after the through header is seen.
Patch By: mikerice
Differential Revision: https://reviews.llvm.org/D46652
llvm-svn: 336379
Fangrui Song