llvm-project/clang-tools-extra/clangd/unittests/TestTU.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

268 lines
9.6 KiB
C++
Raw Normal View History

//===--- TestTU.cpp - Scratch source files for testing --------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "TestTU.h"
#include "CompileCommands.h"
[clangd] Surface errors from command-line parsing Summary: Those errors are exposed at the first character of a file, for a lack of a better place. Previously, all errors were stored inside the AST and report accordingly. However, errors in command-line argument parsing could result in failure to produce the AST, so we need an alternative ways to report those errors. We take the following approach in this patch: - buildCompilerInvocation() now requires an explicit DiagnosticConsumer. - TUScheduler and TestTU now collect the diagnostics produced when parsing command line arguments. If pasing of the AST failed, diagnostics are reported via a new ParsingCallbacks::onFailedAST method. If parsing of the AST succeeded, any errors produced during command-line parsing are stored alongside the AST inside the ParsedAST instance and reported as previously by calling the ParsingCallbacks::onMainAST method; - The client code that uses ClangdServer's DiagnosticConsumer does not need to change, it will receive new diagnostics in the onDiagnosticsReady() callback Errors produced when parsing command-line arguments are collected using the same StoreDiags class that is used to collect all other errors. They are recognized by their location being invalid. IIUC, the location is invalid as there is no source manager at this point, it is created at a later stage. Although technically we might also get diagnostics that mention the command-line arguments FileID with after the source manager was created (and they have valid source locations), we choose to not handle those and they are dropped as not coming from the main file. AFAICT, those diagnostics should always be notes, therefore it's safe to drop them without loosing too much information. Reviewers: kadircet Reviewed By: kadircet Subscribers: nridge, javed.absar, MaskRay, jkorous, arphaman, cfe-commits, gribozavr Tags: #clang Differential Revision: https://reviews.llvm.org/D66759 llvm-svn: 370177
2019-08-28 17:24:55 +08:00
#include "Compiler.h"
#include "Diagnostics.h"
#include "TestFS.h"
#include "index/FileIndex.h"
#include "clang/AST/RecursiveASTVisitor.h"
#include "clang/Basic/Diagnostic.h"
#include "clang/Frontend/CompilerInvocation.h"
#include "llvm/ADT/ScopeExit.h"
#include "llvm/Support/ScopedPrinter.h"
#include "llvm/Support/raw_ostream.h"
#include <cstdlib>
namespace clang {
namespace clangd {
ParseInputs TestTU::inputs(MockFS &FS) const {
std::string FullFilename = testPath(Filename),
[clangd] Include insertion: require header guards, drop other heuristics, treat .def like .inc. Summary: We do have some reports of include insertion behaving badly in some codebases. Requiring header guards both makes sense in principle, and is likely to disable this "nice-to-have" feature in codebases where headers don't follow the expected pattern. With this we can drop some other heuristics, such as looking at file extensions to detect known non-headers - implementation files have no guards. One wrinkle here is #import - objc headers may not have guards because they're intended to be used via #import. If the header is the main file or is #included, we won't collect locations - merge should take care of this if we see the file #imported somewhere. Seems likely to be OK. Headers which have a canonicalization (stdlib, IWYU) are exempt from this check. *.inc files continue to be handled by looking up to the including file. This patch also adds *.def here - tablegen wants this pattern too. In terms of code structure, the division between SymbolCollector and CanonicalIncludes has shifted: SymbolCollector is responsible for more. This is because SymbolCollector has all the SourceManager/HeaderSearch access needed for checking for guards, and we interleave these checks with the *.def checks in a loop (potentially). We could hand all the info into CanonicalIncludes and put the logic there if that's preferable. Reviewers: ioeric Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60316 llvm-svn: 358571
2019-04-17 18:36:02 +08:00
FullHeaderName = testPath(HeaderFilename),
ImportThunk = testPath("import_thunk.h");
// We want to implicitly include HeaderFilename without messing up offsets.
// -include achieves this, but sometimes we want #import (to simulate a header
// guard without messing up offsets). In this case, use an intermediate file.
std::string ThunkContents = "#import \"" + FullHeaderName + "\"\n";
FS.Files = AdditionalFiles;
FS.Files[FullFilename] = Code;
FS.Files[FullHeaderName] = HeaderCode;
FS.Files[ImportThunk] = ThunkContents;
ParseInputs Inputs;
Inputs.FeatureModules = FeatureModules;
auto &Argv = Inputs.CompileCommand.CommandLine;
// In tests, omit predefined macros (__GNUC__ etc) for a 25% speedup.
// There are hundreds, and we'd generate, parse, serialize, and re-parse them!
Argv = {"clang", "-Xclang", "-undef"};
// FIXME: this shouldn't need to be conditional, but it breaks a
// GoToDefinition test for some reason (getMacroArgExpandedLocation fails).
if (!HeaderCode.empty()) {
Argv.push_back("-include");
Argv.push_back(ImplicitHeaderGuard ? ImportThunk : FullHeaderName);
// ms-compatibility changes the meaning of #import.
// The default is OS-dependent (on on windows), ensure it's off.
if (ImplicitHeaderGuard)
Inputs.CompileCommand.CommandLine.push_back("-fno-ms-compatibility");
}
Argv.insert(Argv.end(), ExtraArgs.begin(), ExtraArgs.end());
// Put the file name at the end -- this allows the extra arg (-xc++) to
// override the language setting.
Argv.push_back(FullFilename);
auto Mangler = CommandMangler::forTests();
Mangler.adjust(Inputs.CompileCommand.CommandLine, FullFilename);
Inputs.CompileCommand.Filename = FullFilename;
Inputs.CompileCommand.Directory = testRoot();
Inputs.Contents = Code;
if (OverlayRealFileSystemForModules)
FS.OverlayRealFileSystemForModules = true;
Inputs.TFS = &FS;
Inputs.Opts = ParseOptions();
if (ClangTidyProvider)
Inputs.ClangTidyProvider = ClangTidyProvider;
Inputs.Index = ExternalIndex;
return Inputs;
}
void initializeModuleCache(CompilerInvocation &CI) {
llvm::SmallString<128> ModuleCachePath;
if (llvm::sys::fs::createUniqueDirectory("module-cache", ModuleCachePath)) {
llvm::errs() << "Failed to create temp directory for module-cache";
std::abort();
}
CI.getHeaderSearchOpts().ModuleCachePath = ModuleCachePath.c_str();
}
void deleteModuleCache(const std::string ModuleCachePath) {
if (!ModuleCachePath.empty()) {
if (llvm::sys::fs::remove_directories(ModuleCachePath)) {
llvm::errs() << "Failed to delete temp directory for module-cache";
std::abort();
}
}
}
std::shared_ptr<const PreambleData>
TestTU::preamble(PreambleParsedCallback PreambleCallback) const {
MockFS FS;
auto Inputs = inputs(FS);
IgnoreDiagnostics Diags;
auto CI = buildCompilerInvocation(Inputs, Diags);
assert(CI && "Failed to build compilation invocation.");
if (OverlayRealFileSystemForModules)
initializeModuleCache(*CI);
auto ModuleCacheDeleter = llvm::make_scope_exit(
std::bind(deleteModuleCache, CI->getHeaderSearchOpts().ModuleCachePath));
return clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
/*StoreInMemory=*/true, PreambleCallback);
}
ParsedAST TestTU::build() const {
MockFS FS;
auto Inputs = inputs(FS);
Inputs.Opts = ParseOpts;
[clangd] Surface errors from command-line parsing Summary: Those errors are exposed at the first character of a file, for a lack of a better place. Previously, all errors were stored inside the AST and report accordingly. However, errors in command-line argument parsing could result in failure to produce the AST, so we need an alternative ways to report those errors. We take the following approach in this patch: - buildCompilerInvocation() now requires an explicit DiagnosticConsumer. - TUScheduler and TestTU now collect the diagnostics produced when parsing command line arguments. If pasing of the AST failed, diagnostics are reported via a new ParsingCallbacks::onFailedAST method. If parsing of the AST succeeded, any errors produced during command-line parsing are stored alongside the AST inside the ParsedAST instance and reported as previously by calling the ParsingCallbacks::onMainAST method; - The client code that uses ClangdServer's DiagnosticConsumer does not need to change, it will receive new diagnostics in the onDiagnosticsReady() callback Errors produced when parsing command-line arguments are collected using the same StoreDiags class that is used to collect all other errors. They are recognized by their location being invalid. IIUC, the location is invalid as there is no source manager at this point, it is created at a later stage. Although technically we might also get diagnostics that mention the command-line arguments FileID with after the source manager was created (and they have valid source locations), we choose to not handle those and they are dropped as not coming from the main file. AFAICT, those diagnostics should always be notes, therefore it's safe to drop them without loosing too much information. Reviewers: kadircet Reviewed By: kadircet Subscribers: nridge, javed.absar, MaskRay, jkorous, arphaman, cfe-commits, gribozavr Tags: #clang Differential Revision: https://reviews.llvm.org/D66759 llvm-svn: 370177
2019-08-28 17:24:55 +08:00
StoreDiags Diags;
auto CI = buildCompilerInvocation(Inputs, Diags);
assert(CI && "Failed to build compilation invocation.");
if (OverlayRealFileSystemForModules)
initializeModuleCache(*CI);
auto ModuleCacheDeleter = llvm::make_scope_exit(
std::bind(deleteModuleCache, CI->getHeaderSearchOpts().ModuleCachePath));
auto Preamble = clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
/*StoreInMemory=*/true,
/*PreambleCallback=*/nullptr);
auto AST = ParsedAST::build(testPath(Filename), Inputs, std::move(CI),
Diags.take(), Preamble);
if (!AST) {
llvm::errs() << "Failed to build code:\n" << Code;
std::abort();
}
assert(AST->getDiagnostics() &&
"TestTU should always build an AST with a fresh Preamble");
// Check for error diagnostics and report gtest failures (unless expected).
// This guards against accidental syntax errors silently subverting tests.
// error-ok is awfully primitive - using clang -verify would be nicer.
// Ownership and layering makes it pretty hard.
bool ErrorOk = [&, this] {
llvm::StringLiteral Marker = "error-ok";
if (llvm::StringRef(Code).contains(Marker) ||
llvm::StringRef(HeaderCode).contains(Marker))
return true;
for (const auto &KV : this->AdditionalFiles)
if (llvm::StringRef(KV.second).contains(Marker))
return true;
return false;
}();
if (!ErrorOk) {
// We always build AST with a fresh preamble in TestTU.
for (const auto &D : *AST->getDiagnostics())
if (D.Severity >= DiagnosticsEngine::Error) {
llvm::errs()
<< "TestTU failed to build (suppress with /*error-ok*/): \n"
<< D << "\n\nFor code:\n"
<< Code;
std::abort(); // Stop after first error for simplicity.
}
}
return std::move(*AST);
}
SymbolSlab TestTU::headerSymbols() const {
auto AST = build();
return std::get<0>(indexHeaderSymbols(/*Version=*/"null", AST.getASTContext(),
AST.getPreprocessor(),
AST.getCanonicalIncludes()));
}
RefSlab TestTU::headerRefs() const {
auto AST = build();
return std::get<1>(indexMainDecls(AST));
}
std::unique_ptr<SymbolIndex> TestTU::index() const {
auto AST = build();
auto Idx = std::make_unique<FileIndex>();
Idx->updatePreamble(testPath(Filename), /*Version=*/"null",
AST.getASTContext(), AST.getPreprocessor(),
AST.getCanonicalIncludes());
Idx->updateMain(testPath(Filename), AST);
return std::move(Idx);
}
const Symbol &findSymbol(const SymbolSlab &Slab, llvm::StringRef QName) {
const Symbol *Result = nullptr;
for (const Symbol &S : Slab) {
if (QName != (S.Scope + S.Name).str())
continue;
if (Result) {
llvm::errs() << "Multiple symbols named " << QName << ":\n"
<< *Result << "\n---\n"
<< S;
assert(false && "QName is not unique");
}
Result = &S;
}
if (!Result) {
llvm::errs() << "No symbol named " << QName << " in "
<< llvm::to_string(Slab);
assert(false && "No symbol with QName");
}
return *Result;
}
// RAII scoped class to disable TraversalScope for a ParsedAST.
class TraverseHeadersToo {
ASTContext &Ctx;
std::vector<Decl *> ScopeToRestore;
public:
TraverseHeadersToo(ParsedAST &AST)
: Ctx(AST.getASTContext()), ScopeToRestore(Ctx.getTraversalScope()) {
Ctx.setTraversalScope({Ctx.getTranslationUnitDecl()});
}
~TraverseHeadersToo() { Ctx.setTraversalScope(std::move(ScopeToRestore)); }
};
const NamedDecl &findDecl(ParsedAST &AST, llvm::StringRef QName) {
auto &Ctx = AST.getASTContext();
auto LookupDecl = [&Ctx](const DeclContext &Scope,
llvm::StringRef Name) -> const NamedDecl & {
auto LookupRes = Scope.lookup(DeclarationName(&Ctx.Idents.get(Name)));
assert(!LookupRes.empty() && "Lookup failed");
assert(LookupRes.isSingleResult() && "Lookup returned multiple results");
return *LookupRes.front();
};
const DeclContext *Scope = Ctx.getTranslationUnitDecl();
StringRef Cur, Rest;
for (std::tie(Cur, Rest) = QName.split("::"); !Rest.empty();
std::tie(Cur, Rest) = Rest.split("::")) {
Scope = &cast<DeclContext>(LookupDecl(*Scope, Cur));
}
return LookupDecl(*Scope, Cur);
}
const NamedDecl &findDecl(ParsedAST &AST,
std::function<bool(const NamedDecl &)> Filter) {
TraverseHeadersToo Too(AST);
struct Visitor : RecursiveASTVisitor<Visitor> {
decltype(Filter) F;
llvm::SmallVector<const NamedDecl *, 1> Decls;
bool VisitNamedDecl(const NamedDecl *ND) {
if (F(*ND))
Decls.push_back(ND);
return true;
}
} Visitor;
Visitor.F = Filter;
Visitor.TraverseDecl(AST.getASTContext().getTranslationUnitDecl());
if (Visitor.Decls.size() != 1) {
llvm::errs() << Visitor.Decls.size() << " symbols matched.\n";
assert(Visitor.Decls.size() == 1);
}
return *Visitor.Decls.front();
}
const NamedDecl &findUnqualifiedDecl(ParsedAST &AST, llvm::StringRef Name) {
return findDecl(AST, [Name](const NamedDecl &ND) {
if (auto *ID = ND.getIdentifier())
if (ID->getName() == Name)
return true;
return false;
});
}
} // namespace clangd
} // namespace clang