This reverts commit f0cf544d6f.
Just a small change to fix:
```
/home/buildbot/as-builder-4/llvm-clang-x86_64-expensive-checks-ubuntu/llvm-project/llvm/lib/Support/VirtualFileSystem.cpp: In static member function ‘static llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> > llvm::vfs::File::getWithPath(llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >, const llvm::Twine&)’:
/home/buildbot/as-builder-4/llvm-clang-x86_64-expensive-checks-ubuntu/llvm-project/llvm/lib/Support/VirtualFileSystem.cpp:2084:10: error: could not convert ‘F’ from ‘std::unique_ptr<llvm::vfs::File>’ to ‘llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >’
return F;
^
```
Differential Revision: https://reviews.llvm.org/D113832
```
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp: In static member function 'static llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> > llvm::vfs::File::getWithPath(llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >, const llvm::Twine&)':
/work/omp-vega20-0/openmp-offload-amdgpu-runtime/llvm.src/llvm/lib/Support/VirtualFileSystem.cpp:2084:10: error: could not convert 'F' from 'std::unique_ptr<llvm::vfs::File>' to 'llvm::ErrorOr<std::unique_ptr<llvm::vfs::File> >'
return F;
^
```
This reverts commit c972175649.
This is a follow up to 0be9ca7c0f to make
paths in the case of falling back to the external file system use the
original format, preserving relative paths, and allow the external
filesystem to canonicalize them if needed.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D109128
Reading modules first reads each control block in the chain and then all
AST blocks.
The first phase is intended to find recoverable errors, eg. an out of
date or missing module. If any error occurs during this phase, it is
safe to remove all modules in the chain as no references to them will
exist.
While reading the AST blocks, however, various fields in ASTReader are
updated with references to the module. Removing modules at this point
can cause dangling pointers which can be accessed later. These would be
otherwise harmless, eg. a binary search over `GlobalSLocEntryMap` may
access a failed module that could error, but shouldn't crash. Do not
remove modules in this phase, regardless of failures.
Since this is the case, it also doesn't make sense to return OutOfDate
during this phase, so remove the two cases where this happens.
When they were originally added these checks would return a failure when
the serialized and current path didn't match up. That was updated to an
OutOfDate as it was found to be hit when using VFS and overriding the
umbrella. Later on the path was changed to instead be the name as
written in the module file, resolved using the serialized base
directory. At this point the check is really only comparing the name of
the umbrella and only works for frameworks since those don't include
`Headers/` in the name (which means the resolved path will never exist)
Given all that, it seems safe to ignore this case entirely for now.
This makes the handling of an umbrella header/directory the same as
regular headers, which also don't check for differences in the path
caused by VFS.
Resolves rdar://79329355
Differential Revision: https://reviews.llvm.org/D107690
Previously file entries in the -ivfsoverlay yaml could map to a file in the
external file system, but directories had to list their contents in the form of
other file entries or directories. Allowing directory entries to map to a
directory in the external file system makes it possible to present an external
directory's contents in a different location and (in combination with the
'fallthrough' option) overlay one directory's contents on top of another.
rdar://problem/72485443
Differential Revision: https://reviews.llvm.org/D94844
Removed some #ifdefs specific to Windows handling of VFS paths. This
eliminates most of the differences between the Windows and non-Windows
code paths.
Making this work required some changes to account for the fact that VFS
file paths can be Posix style or Windows style, so you cannot just assume
that they use the host's native path style. In one case, this means
implementing our own version of make_absolute, since the filesystem code
in Support doesn't have styles in the sense that the path code does.
Differential Review: https://reviews.llvm.org/D71092
This fixes the following failure:
C:\[...]\llvm\tools\clang\test\VFS\external-names.c:34:26: error: CHECK-DEBUG-EXTERNAL: expected string not found in input
// CHECK-DEBUG-EXTERNAL: ![[Num]] = !DIFile(filename: "{{[^"]*}}Inputs{{.}}external-names.h"
^
[...]
<stdin>:42:54: note: possible intended match here
!10 = !DIFile(filename: "C:/[...]\\llvm\\tools\\clang\\test\\VFS\\Inputs\\external-names.h", directory: "")
Differential Revision: https://reviews.llvm.org/D71991
This test was XFAILed because of symlinks, but some versions of ln -s
seem to work on Windows, so the test was unexpectedly passing on our
bot:
http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/13233
Unexpected Passing Tests (1):
Clang :: VFS/subframework-symlink.m
I don't know how or why, but it seems dependent on system configuration,
and is not something worth debugging. Avoid the problem by marking the
test UNSUPPORTED: system-windows instead of XFAIL: system-windows.
Since VFS paths can be in either Posix or Windows style, we have to use
a more flexible definition of "absolute" path.
The key here is that FileSystem::makeAbsolute is now virtual, and the
RedirectingFileSystem override checks for either concept of absolute
before trying to make the path absolute by combining it with the current
directory.
Differential Revision: https://reviews.llvm.org/D70701
Jenkins sometimes starts a new working directory by appending @2 (or
incrementing the number if the @n suffix is already there). This causes
several clang tests to fail as:
s@INPUT_DIR@%/S/Inputs@g
gets expanded to the invalid:
s@INPUT_DIR@/path/to/workdir@2/Inputs@g
~~~~~~~~~~
where the part marked with ~'s is interpreted as the flags. These are
invalid and the test fails.
Previous fixes simply exchanged the @ character for another like | but
that's just moving the problem. Address it by adding an expansion that
escapes the @ character we're using as a delimiter as well as other magic
characters in the replacement of sed's s@@@.
There's still room for expansions to cause trouble though. One I ran into
while testing this was that having a directory called foo@bar causes lots
of `CHECK-NOT: foo` directives to match. There's also things like
directories containing `\1`
Keys in a virtual file system can be in Posix or Windows form or even
a combination of the two. Many VFS tests (and a few Clang tests) were
XFAILed on Windows because of false negatives when comparing paths.
First, we default CaseSenstive to false on Windows. This allows
drive letters like "D:" to match "d:". Windows filesystems are, by
default, case insensitive, so this makes sense even beyond the drive
letter.
Second, we allow slashes to match backslashes when they're used as the
root component of a path.
Both of these changes are limited to RedirectingFileSystems, so there's
little chance of affecting other path handling.
These changes allow eleven of the VFS tests to pass on Windows as well
as three other Clang tests, so they have re-enabled.
This solves the majority of PR43272. Additional VFS test failures will
be fixed in separate patches.
Differential Revision: https://reviews.llvm.org/D69958
Part of PR43272, the changes are:
1. Use @ as the sed pattern delimiter instead of : so that the drive
letter in lit substitutions isn't an issue.
2. Use the %/t and %/S substitutions to get paths with forward slashes
to work around string quoting issues in the yaml file.
3. Replace REQUIRES:shell with XFAIL:windows. These tests should pass on
Windows, but do not for reasons that are not yet understood. We would
like to know if they pass unexpectedly.
I was able to remove the XFAILs from two tests, since they already pass
with my sed fix:
clang/test/VFS/module_missing_vfs.m
clang/test/VFS/test_nonmodular.c
Reviewers: amccarth
Differential Revision: https://reviews.llvm.org/D67454
llvm-svn: 371663
when the FileManager is reused across invocations
This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.
The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.
Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.
Differential Revision: https://reviews.llvm.org/D65907
llvm-svn: 369680
It fixes the case when Objective-C framework is added as a subframework
through a symlink. When parent framework infers a module map and fails
to detect a symlink, it would add a subframework as a submodule. And
when we parse module map for the subframework, we would encounter an
error like
> error: umbrella for module 'WithSubframework.Foo' already covers this directory
By implementing `getRealPath` "an egregious but useful hack" in
`ModuleMap::inferFrameworkModule` works as expected.
LLVM commit is r347009.
rdar://problem/45821279
Reviewers: bruno, benlangmuir, erik.pilkington
Reviewed By: bruno
Subscribers: hiraditya, dexonsmith, JDevlieghere, cfe-commits, llvm-commits
Differential Revision: https://reviews.llvm.org/D54245
llvm-svn: 347012
Default property value 'true' preserves current behavior. Value 'false' can be
used to create VFS "root", file system that gives better control over which
files compiler can use during compilation as there are no unpredictable
accesses to real file system.
Non-fallthrough use case changes how we treat multiple VFS overlay
files. Instead of all of them being at the same level just above a real
file system, now they are nested and subsequent overlays can refer to
files in previous overlays.
Change is done both in LLVM and Clang, corresponding LLVM commit is r345431.
rdar://problem/39465552
Reviewers: bruno, benlangmuir
Reviewed By: bruno
Subscribers: dexonsmith, cfe-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D50539
llvm-svn: 345432
'ignore-non-existent-contents' stopped working after r342232 in a way
that the actual attribute value isn't used and it works as if it is
always `true`.
Common use case for VFS iteration is iterating through files in umbrella
directories for modules. Ability to detect if some VFS entries point to
non-existing files is nice but non-critical. Instead of adding back
support for `'ignore-non-existent-contents': false` I am removing the
attribute, because such scenario isn't used widely enough and stricter
checks don't provide enough value to justify the maintenance.
rdar://problem/45176119
Reviewers: bruno
Reviewed By: bruno
Subscribers: hiraditya, dexonsmith, sammccall, cfe-commits
Differential Revision: https://reviews.llvm.org/D53228
llvm-svn: 345212
Changes the default Windows target triple returned by
GetHostTriple.cmake from the old environment names (which we wanted to
move away from) to newer, normalized ones. This also requires updating
all tests to use the new systems names in constraints.
Differential Revision: https://reviews.llvm.org/D47381
llvm-svn: 339307
Using @import in framework headers inhibit the use of such headers
when not using modules, this is specially bad for headers that end
up in the SDK (or any other system framework). Add a warning to give
users some indication that this is discouraged.
rdar://problem/39192894
llvm-svn: 335780
This make -ivfsoverlay behave more like other fatal errors (e.g. missing
-include file) by skipping the missing file instead of bailing out of
the whole compilation. This makes it possible for libclang to still
provide some functionallity as well as to correctly produce the fatal
error diagnostic (previously we lost the diagnostic in libclang since
there was no TU to tie it to).
rdar://33385423
llvm-svn: 328337
Summary:
cp -a is neither part of POSIX nor the LSB. The nearest equivalent under
POSIX is cp -RPp; however, cp -R is sufficient for the intended purpose.
test/Modules/crash-vfs-headermaps.m is not updated since it requires
system-darwin anyway.
Reviewers: bruno
Reviewed By: bruno
Subscribers: bruno, rcraik, cfe-commits
Differential Revision: https://reviews.llvm.org/D41545
llvm-svn: 321778
This is triggered when we are compiling an implementation of a module,
it has relative includes to a VFS-mapped module with umbrella headers.
Currently we will find the real path to headers under the umbrella directory,
but the umbrella directories are using virtual path.
rdar://27951255
Thanks Ben and Richard for reviewing the patch!
Differential Revision: http://reviews.llvm.org/D23858
llvm-svn: 279838
Reapply r278457 with test fixed to not abouse fs case sensitivity.
When the VFS uses a YAML file, the real file path for a
virtual file is described in the "external-contents" field. Example:
...
{
'type': 'file',
'name': 'a.h',
'external-contents': '/a/b/c/a.h'
}
Currently, when parsing umbrella directories, we use
vfs::recursive_directory_iterator to gather the header files to generate the
equivalent modules for. If the external contents for a header does not exist,
we currently are unable to build a module, since the VFS
vfs::recursive_directory_iterator will fail when it finds an entry without a
reliable real path.
Since the YAML file could be prepared ahead of time and shared among
different compiler invocations, an entry might not yet have a reliable
path in 'external-contents', breaking the iteration.
Give the VFS the capability to skip such entries whenever
'ignore-non-existent-contents' property is set in the YAML file.
rdar://problem/27531549
llvm-svn: 278543
When the VFS uses a YAML file, the real file path for a
virtual file is described in the "external-contents" field. Example:
...
{
'type': 'file',
'name': 'a.h',
'external-contents': '/a/b/c/a.h'
}
Currently, when parsing umbrella directories, we use
vfs::recursive_directory_iterator to gather the header files to generate the
equivalent modules for. If the external contents for a header does not exist,
we currently are unable to build a module, since the VFS
vfs::recursive_directory_iterator will fail when it finds an entry without a
reliable real path.
Since the YAML file could be prepared ahead of time and shared among
different compiler invocations, an entry might not yet have a reliable
path in 'external-contents', breaking the iteration.
Give the VFS the capability to skip such entries whenever
'ignore-non-existent-contents' property is set in the YAML file.
rdar://problem/27531549
llvm-svn: 278457
Add 'ignore-non-existent-contents' to tell the VFS whether an invalid path
obtained via 'external-contents' should cause iteration on the VFS to stop.
If 'true', the VFS should ignore the entry and continue with the next. Allows
YAML files to be shared across multiple compiler invocations regardless of
prior existent paths in 'external-contents'. This global value is overridable
on a per-file basis.
This adds the parsing and write test part, but use by VFS comes next.
Differential Revision: https://reviews.llvm.org/D23422
rdar://problem/27531549
llvm-svn: 278456
Make RedirectedFileSystem::openFilForRead(path)->status() the same as
RedirectedFileSystem::status(path). Previously we would just get the
status of the underlying real file, which would not have the IsVFSMapped
bit set.
This fixes rebuilding a module that has an include that is relative to
the includer where we will lookup the real path of that file before we
lookup the VFS location.
rdar://problem/23640339
llvm-svn: 255312
With this change, most 'g' options are rejected by CompilerInvocation.
They remain only as Driver options. The new way to request debug info
from cc1 is with "-debug-info-kind={line-tables-only|limited|standalone}"
and "-dwarf-version={2|3|4}". In the absence of a command-line option
to specify Dwarf version, the Toolchain decides it, rather than placing
Toolchain-specific logic in CompilerInvocation.
Also fix a bug in the Windows compatibility argument parsing
in which the "rightmost argument wins" principle failed.
Differential Revision: http://reviews.llvm.org/D13221
llvm-svn: 249655
LLVM r236120 renamed debug info IR constructs to use a `DI` prefix, now
that the `DIDescriptor` hierarchy has been gone for about a week. This
commit was generated using the rename-md-di-nodes.sh upgrade script
attached to PR23080, followed by running clang-format-diff.py on the
`lib/` portion of the patch.
llvm-svn: 236121
There are two issues here:
1) It's too late to rebuild at this point, because we won't go through
removeModules and when we try to reload the new .pcm we'll get the old
one instead. We might be able to call removeModules after an OutOfDate
here, but I'm not yet confident that it is always safe to do so.
2) In practice, this check fails spuriously when the umbrella header
appears to change because of a VFS change that means it maps to a
different copy of the same file. Because of this, we just skip the
check for now.
llvm-svn: 230064
This reverts commit r218917, effectively reapplying r218913. Original
commit message follows.
--
Update debug info testcases for an LLVM metadata schema change to fold
metadata constant operands into a single `MDString`.
Part of PR17891.
llvm-svn: 219011
Update debug info testcases for an LLVM metadata schema change to fold
metadata constant operands into a single `MDString`.
Part of PR17891.
llvm-svn: 218913
If we lookup a path using its 'real' path first, we need to ensure that
when we run header search we still use the VFS-mapped path or we will
not be able to find the corresponding module for the header.
The real problem is that we tie the name of a file to its underlying
FileEntry, which is uniqued by inode, so we only ever get the first name
it is looked up by. This doesn't work with modules, which rely on a
specific file system structure. I'm hoping to have time to write up a
proposal for fixing this more permanently soon, but as a stopgap this
patch updates the name of the file's directory if it comes from a VFS
mapping.
llvm-svn: 209534
Use llvm::sys::fs::make_absolute to get an absolute path before
matching. Also, allow "." directories to enable testing. ".." is still
not supported, and will require crossing file system boundaries to
implement correctly.
llvm-svn: 202903
Was r202442
There were two issues with the original patch that have now been fixed.
1. We were memset'ing over a FileEntry in a test case. After adding a
std::string to FileEntry, this still happened to not break for me.
2. I didn't pass the FileManager into the new compiler instance in
compileModule. This was hidden in some cases by the fact I didn't
clear the module cache in the test.
Also, I changed the copy constructor for FileEntry, which was memcpy'ing
in a (now) unsafe way.
llvm-svn: 202539
Pass through the externally-visible names that we got from the VFS down
to FileManager, and test that this is the name showing up in __FILE__,
diagnostics, and debug information.
llvm-svn: 202442
Keith Smiley