We currently use target_link_libraries without an explicit scope
specifier (INTERFACE, PRIVATE or PUBLIC) when linking executables.
Dependencies added in this way apply to both the target and its
dependencies, i.e. they become part of the executable's link interface
and are transitive.
Transitive dependencies generally don't make sense for executables,
since you wouldn't normally be linking against an executable. This also
causes issues for generating install export files when using
LLVM_DISTRIBUTION_COMPONENTS. For example, clang has a lot of LLVM
library dependencies, which are currently added as interface
dependencies. If clang is in the distribution components but the LLVM
libraries it depends on aren't (which is a perfectly legitimate use case
if the LLVM libraries are being built static and there are therefore no
run-time dependencies on them), CMake will complain about the LLVM
libraries not being in export set when attempting to generate the
install export file for clang. This is reasonable behavior on CMake's
part, and the right thing is for LLVM's build system to explicitly use
PRIVATE dependencies for executables.
Unfortunately, CMake doesn't allow you to mix and match the keyword and
non-keyword target_link_libraries signatures for a single target; i.e.,
if a single call to target_link_libraries for a particular target uses
one of the INTERFACE, PRIVATE, or PUBLIC keywords, all other calls must
also be updated to use those keywords. This means we must do this change
in a single shot. I also fully expect to have missed some instances; I
tested by enabling all the projects in the monorepo (except dragonegg),
and configuring both with and without shared libraries, on both Darwin
and Linux, but I'm planning to rely on the buildbots for other
configurations (since it should be pretty easy to fix those).
Even after this change, we still have a lot of target_link_libraries
calls that don't specify a scope keyword, mostly for shared libraries.
I'm thinking about addressing those in a follow-up, but that's a
separate change IMO.
Differential Revision: https://reviews.llvm.org/D40823
llvm-svn: 319840
Summary:
In https://reviews.llvm.org/D39572 , I added support for specifying
`Type` when invoking `InMemoryFileSystem::addFile()`.
However, I didn't account for the fact that when `Type` is
`directory_file`, we need to construct an `InMemoryDirectory`, not an
`InMemoryFile`, or else clients cannot create files inside that
directory.
This diff fixes the bug and adds a test.
Test Plan: New test added. Ran test with:
% make -j12 check-clang-tools
Reviewers: bkramer, hokein
Reviewed By: bkramer
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D40140
llvm-svn: 318445
Clang's DiagnosticError is an llvm::Error payload that stores a partial
diagnostic and its location. I'll be using it in the refactoring engine.
Differential Revision: https://reviews.llvm.org/D36969
llvm-svn: 311778
as part of a compilation.
This is intended for two purposes:
1) Writing self-contained test cases for modules: we can now write a single
source file test that builds some number of module files on the side and
imports them.
2) Debugging / test case reduction. A single-source testcase is much more
amenable to reduction, compared to a VFS tarball or .pcm files.
llvm-svn: 305101
The intent for an explicit module build is that the diagnostics produced within
the module are those that were configured when the module was built, not those
that are enabled within a user of the module. This includes diagnostics that
don't actually show up until the module is used (for instance, diagnostics
produced during template instantiation and weird cases like -Wpadded).
We serialized and restored the diagnostic state for individual warning groups,
but previously did not track the state for flags like -Werror and -Weverything,
which are implemented as separate bits rather than as part of the diagnostics
mapping information.
llvm-svn: 301992
This reverts commit r301449. It breaks the build with:
MacroPPCallbacks.h:114:50: error: non-virtual member function marked 'override' hides virtual member function
llvm-svn: 301469
Summary:
The PPCallbacks::MacroUndefined callback is currently insufficient for clients that need to track the MacroDirectives.
This patch adds an additional argument to PPCallbacks::MacroUndefined that is the undef MacroDirective.
Reviewers: bruno, manmanren
Reviewed By: bruno
Subscribers: nemanjai, cfe-commits
Differential Revision: https://reviews.llvm.org/D29923
llvm-svn: 301449
The getVirtualFile method would create entries for e.g. libclang's
CXUnsavedFile but not mark them as valid. The effect is that a lookup
through getFile where the file name is not exactly matching the virtual
file (e.g. through mixing slashes and backslashes on Windows) would
result in a normal file "lookup", and re-using the file entry found
by using the UniqueID, and overwrite the file entry fields. Because the
lookup involves opening the file, and moving it into the file entry, the
file is now open. The SourceManager keys its buffers on the UniqueID
(which is still the same), so it will find an already loaded buffer.
Because only the loading a buffer from disk will close the file, the
FileEntry will hold on to an open file for as long as the FileManager
is around. As the FileManager will only get destroyed at a reparse,
you can't safe to the "leaked" and locked file on Windows.
llvm-svn: 298905
This is a fixup for the unit tests from r298278 (originally r298165).
Since the buffer that RawB2 pointed at was later deleted, a new call to
getBuffer may very well return a buffer at the same/old address. Which is
fine. Just delete the spurious check.
A Windows bot was occasionally hitting this in practice:
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast/builds/7086
llvm-svn: 298414
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298278
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
llvm-svn: 298165
Modified the tests to accept any iteration order, to run only on Unix, and added
additional error reporting to investigate SystemZ bot issue.
The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts don't stat entries unless they have to
descend into the next directory, which allows to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds similar behavior to the VFS iterators. There should be no
change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297693
Modified the tests to accept any iteration order.
The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297528
The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297510
Unlike sys::fs::recursive_directory_iterator,
vfs::recursive_directory_iterator does not implement the level() method,
which tells how deep in the directory tree the current iterator is. This
is needed in the vfs::recursive_directory_iterator so that future
improvements to the crash reproducer will be able to properly access
header for umbrellas when looking into the VFS.
rdar://problem/25880368
llvm-svn: 269520
Reapply r269100 and r269270, reverted due to
https://llvm.org/bugs/show_bug.cgi?id=27725. Isolate the testcase that
corresponds to the new feature side of this commit and skip it on
windows hosts until we find why it does not work on these platforms.
Original commit message:
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
Build a proper virtual directory tree out of the YAML representation,
allowing faster search and proper iteration. Besides the crash
reproducer, this potentially benefits other VFS clients.
llvm-svn: 269327
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
This patch builds a proper virtual directory tree out of the YAML
representation, allowing faster search and proper iteration. Besides the
crash reproducer, this potentially benefits other VFS clients.
llvm-svn: 269270
Bruno made a couple valiant attempts but the bot is still red.
This reverts r269100 (primary commit), r269108 (fix attempt), r269133
(fix attempt).
llvm-svn: 269160
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
This patch builds a proper virtual directory tree out of the YAML
representation, allowing faster search and proper iteration. Besides the
crash reproducer, this potentially benefits other VFS clients.
llvm-svn: 269100
This is a mechanical move of CodeGenOptions from libFrontend to libBasic. This
fixes the layering violation introduced earlier by threading CodeGenOptions into
TargetInfo. It should also fix the modules based self-hosting builds. NFC.
llvm-svn: 265702
This threads CodeGenOptions into the TargetInfo hierarchy. This is motivated by
ARM which can change some target information based on the EABI selected
(-meabi). Similar options exist for other platforms (e.g. MIPS) and thus is
generally useful. NFC.
llvm-svn: 265640
This reapplies r261552 and r263748. Fixed testcase to reapply.
The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.
When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.
To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.
Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:
"overlay-relative": "true",
"roots": [
...
"type": "directory",
"name": "/usr/include",
"contents": [
{
"type": "file",
"name": "stdio.h",
"external-contents": "/usr/include/stdio.h"
},
...
Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.
This is a useful feature for debugging module crashes in machines other than
the one where the error happened.
Differential Revision: http://reviews.llvm.org/D17457
rdar://problem/24499339
llvm-svn: 263893
This reapplies r261552.
The VFS overlay mapping between virtual paths and real paths is done through
the 'external-contents' entries in YAML files, which contains hardcoded paths
to the real files.
When a module compilation crashes, headers are dumped into <name>.cache/vfs
directory and are mapped via the <name>.cache/vfs/vfs.yaml. The script
generated for reproduction uses -ivfsoverlay pointing to file to gather the
mapping between virtual paths and files inside <name>.cache/vfs. Currently, we
are only capable of reproducing such crashes in the same machine as they
happen, because of the hardcoded paths in 'external-contents'.
To be able to reproduce a crash in another machine, this patch introduces a new
option in the VFS yaml file called 'overlay-relative'. When it's equal to
'true' it means that the provided path to the YAML file through the
-ivfsoverlay option should also be used to prefix the final path for every
'external-contents'.
Example, given the invocation snippet "... -ivfsoverlay
<name>.cache/vfs/vfs.yaml" and the following entry in the yaml file:
"overlay-relative": "true",
"roots": [
...
"type": "directory",
"name": "/usr/include",
"contents": [
{
"type": "file",
"name": "stdio.h",
"external-contents": "/usr/include/stdio.h"
},
...
Here, a file manager request for virtual "/usr/include/stdio.h", that will map
into real path "/<absolute_path_to>/<name>.cache/vfs/usr/include/stdio.h.
This is a useful feature for debugging module crashes in machines other than
the one where the error happened.
Differential Revision: http://reviews.llvm.org/D17457
rdar://problem/24499339
llvm-svn: 263748
Use it to calculate UserLabelPrefix, instead of specifying it (often
incorrectly).
Note that the *actual* user label prefix has always come from the
DataLayout, and is handled within LLVM. The main thing clang's
TargetInfo::UserLabelPrefix did was to set the #define value. Having
these be different from each-other is just silly.
Differential Revision: http://reviews.llvm.org/D17183
llvm-svn: 262737
This behavior is enabled when the new CXTranslationUnit_KeepGoing
option is passed to clang_parseTranslationUnit{,2}. It is geared
towards use by IDEs and similar consumers of the clang-c API where
fatal errors may arise when parsing incomplete code mid-edit, or
when include paths are not properly configured yet. In such situations
one still wants to get as much information as possible about a TU.
Previously, the semantic analysis would not instantiate templates
or report additional fatal errors after the first fatal error was
encountered.
Fixes PR24268.
Patch by Milian Wolff.
llvm-svn: 262318
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html
"This is the way [autoconf] ends
Not with a bang but a whimper."
-T.S. Eliot
Reviewers: chandlerc, grosbach, bob.wilson, echristo
Subscribers: klimek, cfe-commits
Differential Revision: http://reviews.llvm.org/D16472
llvm-svn: 258862
FixedCompilationDatabase sets the working dir to "." by default. For
chdir(".") this is a noop but this lead to InMemoryFileSystem to create
bogus paths. Fixes PR25327.
llvm-svn: 257260
Make RedirectedFileSystem::openFilForRead(path)->status() the same as
RedirectedFileSystem::status(path). Previously we would just get the
status of the underlying real file, which would not have the IsVFSMapped
bit set.
This fixes rebuilding a module that has an include that is relative to
the includer where we will lookup the real path of that file before we
lookup the VFS location.
rdar://problem/23640339
llvm-svn: 255312
This is a more principled version of what I did earlier. Path
normalization is generally a good thing, but may break users in strange
environments, e. g. using lots of symlinks. Let the user choose and
default it to on.
This also changes adding a duplicated file into returning an error if
the file contents are different instead of an assertion failure.
Differential Revision: http://reviews.llvm.org/D13658
llvm-svn: 250060
Actually the only special path we have to handle is ./foo, the rest is
tricky to get right so do the same thing as the existing YAML vfs here.
llvm-svn: 250036
This can fail badly if we're overlaying a real file system and there are
symlinks there. Just keep the path as-is for now.
This essentially reverts r249830.
llvm-svn: 250021
Apart from being cleaner this also means that clang-format no longer has
access to the host file system. This isn't necessary because clang-format
never reads includes :)
Includes minor tweaks and bugfixes found in the VFS implementation while
running clang-format tests.
llvm-svn: 249385
For RealFileSystem this is getcwd()/chdir(), the synthetic file systems can
make up one for themselves. OverlayFileSystem now synchronizes the working
directories when a new FS is added to the overlay or the overlay working
directory is set. This allows purely artificial file systems that have zero
ties to the underlying disks.
Differential Revision: http://reviews.llvm.org/D13430
llvm-svn: 249316
This is a simple file system tree of memory buffers that can be filled by a
client. In conjunction with an OverlayFS it can be used to make virtual
files accessible right next to physical files. This can be used as a
replacement for the virtual file handling in FileManager and which I intend
to remove eventually.
llvm-svn: 249315
The patch is generated using this command:
$ tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
work/llvm/tools/clang
To reduce churn, not touching namespaces spanning less than 10 lines.
llvm-svn: 240270
It has no place there; it's not a property of the Module, and it makes
restoring the visibility set when we leave a submodule more difficult.
llvm-svn: 236300
Only those callers who are dynamically passing ownership should need the
3 argument form. Those accepting the default ("do pass ownership")
should do so explicitly with a unique_ptr now.
llvm-svn: 216614
And in the process, discover that FileManager::removeStatCache had a
double-delete when removing an element from the middle of the list (at
the beginning or the end of the list, there was no problem) and add a
unit test to exercise the code path (which successfully crashed when run
(with modifications to match the old API) without this patch applied)
llvm-svn: 215388
For now, this is only used by its unit tests. It is similar to the API
in llvm::sys::fs::recursive_directory_iterator, but without some of the
more complex features like requesting that the iterator not recurse into
the next directory, for example.
llvm-svn: 211732
1) missing iterator bits needed by libstdc++4.7
Using find_if was convenient, but since operator++ wasn't a good
interface anyway, I just replaced with a range-based for loop and
removed operator++ from the directory_iterator class.
2) stop relying on order of iterating real files
llvm-svn: 211633
The API is based on sys::fs::directory_iterator, but it allows iterating
over overlays and the yaml-based VFS. For now, it isn't used by
anything (except its tests).
llvm-svn: 211623
Shoaib Meenai