This patch makes the minimizer more conservative to avoid missing dependency files that are brought in by __has_include
PP expressions that occur in a condition of an #if/#elif that was previously skipped. The __has_include PP expressions
can be used in an #if/#elif either directly, or through macro expansion, so we can't detect them at the time of minimization.
Differential Revision: https://reviews.llvm.org/D70936
It is tricky to use replace_path_prefix correctly on Windows which uses
backslashes as native path separators. Switch back to the old approach
(startswith is not ideal) to appease build bots for now.
GCC 8 implements -fmacro-prefix-map. Like -fdebug-prefix-map, it replaces a string prefix for the __FILE__ macro.
-ffile-prefix-map is the union of -fdebug-prefix-map and -fmacro-prefix-map
Reviewed By: rnk, Lekensteyn, maskray
Differential Revision: https://reviews.llvm.org/D49466
__has_cpp_attribute is not available in C mode, and __has_c_attribute
should not be available in C++ mode. This also adds a test to
demonstrate that we properly handle scoped attribute tokens even in C
mode.
Refactor the logic on CompilerInstance::loadModule and a couple of
surrounding methods in order to clarify what's going on.
- Rename ModuleLoader::loadModuleFromSource to compileModuleFromSource
and fix its documentation, since it never loads a module. It just
creates/compiles one.
- Rename one of the overloads of compileModuleImpl to compileModule,
making it more obvious which one calls the other.
- Rename compileAndLoadModule to compileModuleAndReadAST. This
clarifies the relationship between this helper and its caller,
CompilerInstance::loadModule (the old name implied the opposite
relationship). It also (correctly) indicates that more needs to be
done to load the module than this function is responsible for.
- Split findOrCompileModuleAndReadAST out of loadModule. Besides
reducing nesting for this code thanks to early returns and the like,
this refactor clarifies the logic in loadModule, particularly around
calls to ModuleMap::cacheModuleLoad and
ModuleMap::getCachedModuleLoad. findOrCompileModuleAndReadAST also
breaks early if the initial ReadAST call returns Missing or OutOfDate,
allowing the last ditch call to compileModuleAndReadAST to come at the
end of the function body.
- Additionally split out selectModuleSource, clarifying the logic
due to early returns.
- Add ModuleLoadResult::isNormal and OtherUncachedFailure, so that
loadModule knows whether to cache the result.
OtherUncachedFailure was added to keep this patch NFC, but there's
a chance that these cases were uncached by accident, through
copy/paste/modify failures. These should be audited as a
follow-up (maybe we can eliminate this case).
- Do *not* lift the setting of `ModuleLoadFailed = true` to
loadModule because there isn't a clear pattern for when it's set.
This should be reconsidered in a follow-up, in case it would be
correct to set `ModuleLoadFailed` whenever no module is returned
and the result is either Normal or OtherUncachedFailure.
- Add some header documentation where it was missing, and fix it where
it was wrong.
This should have no functionality change.
https://reviews.llvm.org/D70556
Builtins are rarely if ever accessed via the Preprocessor. They are
typically found on the ASTContext, so there should be no performance
penalty to using a pointer indirection to store the builtin context.
In Microsoft-compatibility mode, single commas from nested macro expansions
should not be considered as argument separators; we already emulated this by
marking them to be ignored. However, in MSVC's preprocessor, subsequent
expansions DO treat these commas as argument separators... so we now ignore
each comma at most once.
Includes a small unit test that validates we match MSVC's behavior as shown
in https://gcc.godbolt.org/z/y0twaq
Fixes PR43282
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D69626
The existing string/character literal skipping code in the
dependency directives source minimizer has two issues:
- It doesn't stop the scanning when a newline is reached before the terminating character,
unlike the lexer which considers the token to be done (even if it's invalid) at the end of the line.
- It doesn't support whitespace between '\' and the newline when looking if the '\' is used as a line continuation character.
This commit fixes both issues.
Differential Revision: https://reviews.llvm.org/D68436
llvm-svn: 374127
Summary:
Avoid parsing __pragma into an annotation token when macro arguments are pre-expanded.
This is what clang currently does when parsing _Pragmas.
Fixes https://bugs.llvm.org/show_bug.cgi?id=41128, where clang crashed
when trying to get the length of an annotation token.
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D68114
llvm-svn: 373950
in the dependency source minimizer
Clang allows continuations that have whitespace between the backslash and the newline.
This patch ensures that the dependency source minimizer can handle the whitespace between
the backslash and the newline when looking for a line continuation.
Differential Revision: https://reviews.llvm.org/D68052
llvm-svn: 373007
This include search logic has an extra parameter to deal with Windows
includes with backslashes, which get normalized to forward slashes on
non-Windows under -fms-compatibility.
Hoist the conditional operator out of LookupHeaderIncludeOrImport and
pass the result in instead of repeating the ?: expression everywhere.
llvm-svn: 372926
In case of certain #define'd macros, there's a space just before line continuation
that the minimized-source lexer was missing to include, resulting in invalid stringize.
Patch by: kousikk (Kousik Kumar)
Differential Revision: https://reviews.llvm.org/D67635
llvm-svn: 372360
-frewrite-includes calls PP.SetMacroExpansionOnlyInDirectives() to avoid
macro expansions that are useless in that mode, but this can lead
to -Wunused-macros false positives. As -frewrite-includes does not emit
normal warnings, block -Wunused-macros too.
Differential Revision: https://reviews.llvm.org/D65371
llvm-svn: 372026
In order to enable future improvements to our attribute diagnostics,
this moves info from ParsedAttr into CommonAttributeInfo, then makes
this type the base of the *Attr and ParsedAttr types. Quite a bit of
refactoring took place, including removing a bunch of redundant Spelling
Index propogation.
Differential Revision: https://reviews.llvm.org/D67368
llvm-svn: 371875
This commit adds an optimization to clang-scan-deps and clang's preprocessor that skips excluded preprocessor
blocks by bumping the lexer pointer, and not lexing the tokens until reaching appropriate #else/#endif directive.
The skip positions and lexer offsets are computed when the file is minimized, directly from the minimized tokens.
On an 18-core iMacPro with macOS Catalina Beta I got 10-15% speed-up from this optimization when running clang-scan-deps on
the compilation database for a recent LLVM and Clang (3511 files).
Differential Revision: https://reviews.llvm.org/D67127
llvm-svn: 371656
In `DirectoryLookup::LookupFile` parameter `HasBeenMapped` doesn't cover
the case when clang finds a file through a header map but doesn't remap
the lookup filename because the target path is an absolute path. As a
result, -Wnonportable-include-path suppression for header maps
introduced in r301592 wasn't triggered.
Change parameter `HasBeenMapped` to `IsInHeaderMap` and use parameter
`MappedName` to track the filename remapping. This way we can handle
both relative and absolute paths in header maps, and account for their
specific properties, like filename remapping being a property preserved
across lookups in multiple directories.
rdar://problem/39516483
Reviewers: dexonsmith, bruno
Reviewed By: dexonsmith
Subscribers: jkorous, cfe-commits, ributzka
Differential Revision: https://reviews.llvm.org/D58094
llvm-svn: 371655
accessed name to the directory entry
This commit introduces a parallel API that returns a DirectoryEntryRef
to the FileManager, similar to the parallel FileEntryRef API. All
uses will have to be update in follow-up patches. The immediate use of the new API in this
patch fixes the issue where a file manager was reused in clang-scan-deps,
but reported an different file path whenever a framework lookup was done through a symlink.
Differential Revision: https://reviews.llvm.org/D67026
llvm-svn: 370562
Previously, an #error directive with quoted, multi-line content, along with CR+LF line endings wasn't handled correctly.
Differential Revision: https://reviews.llvm.org/D66556
llvm-svn: 370129
This fixes the issue where a filename dependendency was missing if the file that
was referenced with __has_include() was accessed through a symlink in an earlier run,
if the file manager was reused between runs.
llvm-svn: 370081
This reverts commit r369986.
This change added a dependency on the 'dos2unix' tool, which is not one
of our accepted test dependencies and may not exist on all machines that
build Clang.
llvm-svn: 370000
This fixes the issue where a filename dependendency was missing if the file that
was skipped was included through a symlink in an earlier run, if the file
manager was reused between runs.
llvm-svn: 369998
Previously, an #error directive with quoted, multi-line content, along with CR+LF line endings wasn't handled correctly.
Differential Revision: https://reviews.llvm.org/D66556
llvm-svn: 369986
`FileManager::getFileRef` is a modern API which we expect to convert to
over time. We should modernize the error handling as well, using
`llvm::Expected` instead of `llvm::ErrorOr`, to help clients that care
about errors to ensure nothing is missed.
However, not all clients care. I've also added another path for those
that don't:
- `FileEntryRef` is now copy- and move-assignable (using a pointer
instead of a reference).
- `FileManager::getOptionalFileRef` returns an `llvm::Optional` instead
of `llvm::Expected`.
- Added an `llvm::expectedToOptional` utility in case this is useful
elsewhere.
https://reviews.llvm.org/D66705
llvm-svn: 369943
when the FileManager is reused across invocations
This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.
The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.
Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.
Differential Revision: https://reviews.llvm.org/D65907
llvm-svn: 369680
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368942
Summary:
Previously __has_builtin(__builtin_*) would return false for
__builtin_*s that we modeled as keywords rather than as functions
(because they take type arguments). With this patch, all builtins
that are called with function-call-like syntax return true from
__has_builtin (covering __builtin_* and also the __is_* and __has_* type
traits and the handful of similar builtins without such a prefix).
Update the documentation on __has_builtin and on type traits to match.
While doing this I noticed the type trait documentation was out of date
and incomplete; that's fixed here too.
Reviewers: aaron.ballman
Subscribers: jfb, kristina, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66100
llvm-svn: 368785
Summary:
As noted on Errc.h:
// * std::errc is just marked with is_error_condition_enum. This means that
// common patters like AnErrorCode == errc::no_such_file_or_directory take
// 4 virtual calls instead of two comparisons.
And on some libstdc++ those virtual functions conclude that
------------------------
int main() {
std::error_code foo = std::make_error_code(std::errc::no_such_file_or_directory);
return foo == std::errc::no_such_file_or_directory;
}
-------------------------
should exit with 0.
Reviewers: thakis, rnk, jfb
Reviewed By: thakis
Subscribers: lebedev.ri, dexonsmith, xbolva00, cfe-commits, caomhin
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66143
llvm-svn: 368739
This regressed in r368322, and was reported as PR42948 and on the
mailing list. The fix is to ignore the specific error code for this
case. The problem doesn't seem to reproduce on Windows, where a
different error code is used instead.
llvm-svn: 368475
Apparently Windows returns the "invalid argument" error code when the
path contains invalid characters such as '<'. The
test/Preprocessor/include-likely-typo.c test does this, so it was
failing after r368322.
Also, the diagnostic requires two arguments, so add the filename.
llvm-svn: 368348
clang would only print "file not found" when it's unable to find a
header file. If the reason for that is a file handle leak, that's not a
very useful error message. For errors that aren't in a small whitelist
("file not found", "file is directory"), print an error with the
strerror() output.
This changes behavior in corner cases: If clang was out of file handles
while looking in one -I dir but then suddenly wasn't when looking in the
next -I dir, and both directories contained a file with the desired
name, previously we'd silently return the file from the second
directory. For this reason, it's important to ignore "is a directory"
for this new diag: if a file foo/foo exists and -I -Ifoo are passed, an
include of "foo" should successfully open file "foo" in directory "foo/"
instead of complaining that "./foo" is a directory.
No test since we mostly hit this when there's a handle leak somewhere,
and currently there isn't one. I manually tested this with the repro
steps in comment 2 on the bug below.
Fixes PR42524.
Differential Revision: https://reviews.llvm.org/D65956
llvm-svn: 368322
Update the callers of FileManager::getFile and FileManager::getDirectory to handle the new llvm::ErrorOr-returning methods.
Signed-off-by: Harlan Haskins <harlan@apple.com>
llvm-svn: 367616
Summary:
Preivously we would only discard it if we failed to parse parameter lists.
If we do not consume the body, parser sees tokens inside directive. In
turn, this leads to spurious diagnostics and a crash in TokenBuffer, see
the added tests.
Reviewers: sammccall
Reviewed By: sammccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65517
llvm-svn: 367530
Summary:
The cache recorded the wrong expansion location for all but the first
stringization. It seems uncommon to stringize the same macro argument
multiple times, so this cache doesn't seem that important.
Fixes PR39942
Reviewers: vsk, rsmith
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D65428
llvm-svn: 367337
We should re-emit `#pragma once` to ensure the preprocessor will
still honor it when running on minimized sources.
Differential Revision: https://reviews.llvm.org/D64945
llvm-svn: 366509
single quotes are not digit separators after a valid character literal prefix
The single quote character can act as a c++ digit separator.
However, the minimizer shouldn't treat it as such when it's actually following
a valid character literal prefix, like L, U, u, or u8.
Differential Revision: https://reviews.llvm.org/D64525
llvm-svn: 365700
Summary:
Currently HeaderSearch only looks at SearchDir's passed into it, but in
addition to those paths headers can be relative to including file's directory.
This patch makes sure that is taken into account.
Reviewers: gribozavr
Subscribers: jkorous, arphaman, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D63295
llvm-svn: 365005
Alex Lorenz