2018-05-28 20:23:17 +08:00
|
|
|
//===--- TestTU.cpp - Scratch source files for testing --------------------===//
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
//
|
2019-01-19 16:50:56 +08:00
|
|
|
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
|
|
|
|
// See https://llvm.org/LICENSE.txt for license information.
|
|
|
|
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
//
|
2018-08-15 00:03:32 +08:00
|
|
|
//===----------------------------------------------------------------------===//
|
|
|
|
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
#include "TestTU.h"
|
[clangd] Surface errors from command-line parsing
Summary:
Those errors are exposed at the first character of a file,
for a lack of a better place.
Previously, all errors were stored inside the AST and report
accordingly. However, errors in command-line argument parsing could
result in failure to produce the AST, so we need an alternative ways to
report those errors.
We take the following approach in this patch:
- buildCompilerInvocation() now requires an explicit DiagnosticConsumer.
- TUScheduler and TestTU now collect the diagnostics produced when
parsing command line arguments.
If pasing of the AST failed, diagnostics are reported via a new
ParsingCallbacks::onFailedAST method.
If parsing of the AST succeeded, any errors produced during
command-line parsing are stored alongside the AST inside the
ParsedAST instance and reported as previously by calling the
ParsingCallbacks::onMainAST method;
- The client code that uses ClangdServer's DiagnosticConsumer
does not need to change, it will receive new diagnostics in the
onDiagnosticsReady() callback
Errors produced when parsing command-line arguments are collected using
the same StoreDiags class that is used to collect all other errors. They
are recognized by their location being invalid. IIUC, the location is
invalid as there is no source manager at this point, it is created at a
later stage.
Although technically we might also get diagnostics that mention the
command-line arguments FileID with after the source manager was created
(and they have valid source locations), we choose to not handle those
and they are dropped as not coming from the main file. AFAICT, those
diagnostics should always be notes, therefore it's safe to drop them
without loosing too much information.
Reviewers: kadircet
Reviewed By: kadircet
Subscribers: nridge, javed.absar, MaskRay, jkorous, arphaman, cfe-commits, gribozavr
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66759
llvm-svn: 370177
2019-08-28 17:24:55 +08:00
|
|
|
#include "Compiler.h"
|
|
|
|
#include "Diagnostics.h"
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
#include "TestFS.h"
|
|
|
|
#include "index/FileIndex.h"
|
|
|
|
#include "index/MemIndex.h"
|
2018-06-06 00:30:25 +08:00
|
|
|
#include "clang/AST/RecursiveASTVisitor.h"
|
[clangd] Errors in TestTU cause test failures unless suppressed with error-ok.
Summary:
The historic behavior of TestTU is to gather diagnostics and otherwise ignore
them. So if a test has a syntax error, and doesn't assert diagnostics, it
silently misbehaves.
This can be annoying when developing tests, as evidenced by various tests
gaining "assert no diagnostics" where that's not really the point of the test.
This patch aims to make that default behavior. For the first error
(not warning), TestTU will call ADD_FAILURE().
This can be suppressed with a comment containing "error-ok". For now that will
suppress any errors in the TU. We can make this stricter later -verify style.
(-verify itself is hard to reuse because of DiagnosticConsumer interfaces...)
A magic-comment was chosen over a TestTU option because of table-driven tests.
In addition to the behavior change, this patch:
- adds //error-ok where we're knowingly testing invalid code
(e.g. for diagnostics, crash-resilience, or token-level tests)
- fixes a bunch of errors in the checked-in tests, mostly trivial (missing ;)
- removes a bunch of now-redundant instances of "assert no diagnostics"
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73199
2020-01-22 23:38:41 +08:00
|
|
|
#include "clang/Basic/Diagnostic.h"
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
#include "clang/Frontend/CompilerInvocation.h"
|
|
|
|
#include "clang/Frontend/Utils.h"
|
2020-08-13 23:24:22 +08:00
|
|
|
#include "llvm/ADT/ScopeExit.h"
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
|
|
|
|
namespace clang {
|
|
|
|
namespace clangd {
|
|
|
|
|
2020-06-18 00:09:54 +08:00
|
|
|
ParseInputs TestTU::inputs(MockFS &FS) const {
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
std::string FullFilename = testPath(Filename),
|
[clangd] Include insertion: require header guards, drop other heuristics, treat .def like .inc.
Summary:
We do have some reports of include insertion behaving badly in some
codebases. Requiring header guards both makes sense in principle, and is
likely to disable this "nice-to-have" feature in codebases where headers don't
follow the expected pattern.
With this we can drop some other heuristics, such as looking at file
extensions to detect known non-headers - implementation files have no guards.
One wrinkle here is #import - objc headers may not have guards because
they're intended to be used via #import. If the header is the main file
or is #included, we won't collect locations - merge should take care of
this if we see the file #imported somewhere. Seems likely to be OK.
Headers which have a canonicalization (stdlib, IWYU) are exempt from this check.
*.inc files continue to be handled by looking up to the including file.
This patch also adds *.def here - tablegen wants this pattern too.
In terms of code structure, the division between SymbolCollector and
CanonicalIncludes has shifted: SymbolCollector is responsible for more.
This is because SymbolCollector has all the SourceManager/HeaderSearch access
needed for checking for guards, and we interleave these checks with the *.def
checks in a loop (potentially).
We could hand all the info into CanonicalIncludes and put the logic there
if that's preferable.
Reviewers: ioeric
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, kadircet, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60316
llvm-svn: 358571
2019-04-17 18:36:02 +08:00
|
|
|
FullHeaderName = testPath(HeaderFilename),
|
|
|
|
ImportThunk = testPath("import_thunk.h");
|
|
|
|
// We want to implicitly include HeaderFilename without messing up offsets.
|
|
|
|
// -include achieves this, but sometimes we want #import (to simulate a header
|
|
|
|
// guard without messing up offsets). In this case, use an intermediate file.
|
|
|
|
std::string ThunkContents = "#import \"" + FullHeaderName + "\"\n";
|
2019-04-29 18:25:44 +08:00
|
|
|
|
2020-06-18 00:09:54 +08:00
|
|
|
FS.Files = AdditionalFiles;
|
|
|
|
FS.Files[FullFilename] = Code;
|
|
|
|
FS.Files[FullHeaderName] = HeaderCode;
|
|
|
|
FS.Files[ImportThunk] = ThunkContents;
|
2019-04-29 18:25:44 +08:00
|
|
|
|
2020-04-01 05:09:28 +08:00
|
|
|
ParseInputs Inputs;
|
2020-03-16 04:43:00 +08:00
|
|
|
auto &Argv = Inputs.CompileCommand.CommandLine;
|
2020-04-01 05:09:28 +08:00
|
|
|
Argv = {"clang"};
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
// FIXME: this shouldn't need to be conditional, but it breaks a
|
|
|
|
// GoToDefinition test for some reason (getMacroArgExpandedLocation fails).
|
|
|
|
if (!HeaderCode.empty()) {
|
2020-04-01 05:09:28 +08:00
|
|
|
Argv.push_back("-include");
|
|
|
|
Argv.push_back(ImplicitHeaderGuard ? ImportThunk : FullHeaderName);
|
2019-07-17 21:21:25 +08:00
|
|
|
// ms-compatibility changes the meaning of #import.
|
|
|
|
// The default is OS-dependent (on on windows), ensure it's off.
|
|
|
|
if (ImplicitHeaderGuard)
|
2020-04-01 05:09:28 +08:00
|
|
|
Inputs.CompileCommand.CommandLine.push_back("-fno-ms-compatibility");
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
}
|
2020-04-01 05:09:28 +08:00
|
|
|
Argv.insert(Argv.end(), ExtraArgs.begin(), ExtraArgs.end());
|
2019-06-26 16:10:26 +08:00
|
|
|
// Put the file name at the end -- this allows the extra arg (-xc++) to
|
|
|
|
// override the language setting.
|
2020-04-01 05:09:28 +08:00
|
|
|
Argv.push_back(FullFilename);
|
2018-11-09 20:56:49 +08:00
|
|
|
Inputs.CompileCommand.Filename = FullFilename;
|
|
|
|
Inputs.CompileCommand.Directory = testRoot();
|
|
|
|
Inputs.Contents = Code;
|
2020-08-13 23:24:22 +08:00
|
|
|
if (OverlayRealFileSystemForModules)
|
|
|
|
FS.OverlayRealFileSystemForModules = true;
|
2020-06-18 00:09:54 +08:00
|
|
|
Inputs.TFS = &FS;
|
2019-01-28 22:01:55 +08:00
|
|
|
Inputs.Opts = ParseOptions();
|
2020-11-26 02:35:34 +08:00
|
|
|
if (ClangTidyProvider)
|
|
|
|
Inputs.ClangTidyProvider = ClangTidyProvider;
|
2019-01-28 22:01:55 +08:00
|
|
|
Inputs.Index = ExternalIndex;
|
2020-04-01 05:09:28 +08:00
|
|
|
return Inputs;
|
|
|
|
}
|
|
|
|
|
2020-08-13 23:24:22 +08:00
|
|
|
void initializeModuleCache(CompilerInvocation &CI) {
|
|
|
|
llvm::SmallString<128> ModuleCachePath;
|
|
|
|
ASSERT_FALSE(
|
|
|
|
llvm::sys::fs::createUniqueDirectory("module-cache", ModuleCachePath));
|
|
|
|
CI.getHeaderSearchOpts().ModuleCachePath = ModuleCachePath.c_str();
|
|
|
|
}
|
|
|
|
|
|
|
|
void deleteModuleCache(const std::string ModuleCachePath) {
|
|
|
|
if (!ModuleCachePath.empty()) {
|
|
|
|
ASSERT_FALSE(llvm::sys::fs::remove_directories(ModuleCachePath));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-10-13 14:09:45 +08:00
|
|
|
std::shared_ptr<const PreambleData>
|
|
|
|
TestTU::preamble(PreambleParsedCallback PreambleCallback) const {
|
2020-06-18 00:09:54 +08:00
|
|
|
MockFS FS;
|
|
|
|
auto Inputs = inputs(FS);
|
2020-05-14 18:23:21 +08:00
|
|
|
IgnoreDiagnostics Diags;
|
|
|
|
auto CI = buildCompilerInvocation(Inputs, Diags);
|
|
|
|
assert(CI && "Failed to build compilation invocation.");
|
2020-08-13 23:24:22 +08:00
|
|
|
if (OverlayRealFileSystemForModules)
|
|
|
|
initializeModuleCache(*CI);
|
|
|
|
auto ModuleCacheDeleter = llvm::make_scope_exit(
|
|
|
|
std::bind(deleteModuleCache, CI->getHeaderSearchOpts().ModuleCachePath));
|
2020-05-14 18:23:21 +08:00
|
|
|
return clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
|
2020-10-13 14:09:45 +08:00
|
|
|
/*StoreInMemory=*/true, PreambleCallback);
|
2020-05-14 18:23:21 +08:00
|
|
|
}
|
|
|
|
|
2020-04-01 05:09:28 +08:00
|
|
|
ParsedAST TestTU::build() const {
|
2020-06-18 00:09:54 +08:00
|
|
|
MockFS FS;
|
|
|
|
auto Inputs = inputs(FS);
|
[clangd] Surface errors from command-line parsing
Summary:
Those errors are exposed at the first character of a file,
for a lack of a better place.
Previously, all errors were stored inside the AST and report
accordingly. However, errors in command-line argument parsing could
result in failure to produce the AST, so we need an alternative ways to
report those errors.
We take the following approach in this patch:
- buildCompilerInvocation() now requires an explicit DiagnosticConsumer.
- TUScheduler and TestTU now collect the diagnostics produced when
parsing command line arguments.
If pasing of the AST failed, diagnostics are reported via a new
ParsingCallbacks::onFailedAST method.
If parsing of the AST succeeded, any errors produced during
command-line parsing are stored alongside the AST inside the
ParsedAST instance and reported as previously by calling the
ParsingCallbacks::onMainAST method;
- The client code that uses ClangdServer's DiagnosticConsumer
does not need to change, it will receive new diagnostics in the
onDiagnosticsReady() callback
Errors produced when parsing command-line arguments are collected using
the same StoreDiags class that is used to collect all other errors. They
are recognized by their location being invalid. IIUC, the location is
invalid as there is no source manager at this point, it is created at a
later stage.
Although technically we might also get diagnostics that mention the
command-line arguments FileID with after the source manager was created
(and they have valid source locations), we choose to not handle those
and they are dropped as not coming from the main file. AFAICT, those
diagnostics should always be notes, therefore it's safe to drop them
without loosing too much information.
Reviewers: kadircet
Reviewed By: kadircet
Subscribers: nridge, javed.absar, MaskRay, jkorous, arphaman, cfe-commits, gribozavr
Tags: #clang
Differential Revision: https://reviews.llvm.org/D66759
llvm-svn: 370177
2019-08-28 17:24:55 +08:00
|
|
|
StoreDiags Diags;
|
|
|
|
auto CI = buildCompilerInvocation(Inputs, Diags);
|
2018-12-14 21:19:38 +08:00
|
|
|
assert(CI && "Failed to build compilation invocation.");
|
2020-08-13 23:24:22 +08:00
|
|
|
if (OverlayRealFileSystemForModules)
|
|
|
|
initializeModuleCache(*CI);
|
|
|
|
auto ModuleCacheDeleter = llvm::make_scope_exit(
|
|
|
|
std::bind(deleteModuleCache, CI->getHeaderSearchOpts().ModuleCachePath));
|
|
|
|
|
2020-05-14 18:23:21 +08:00
|
|
|
auto Preamble = clang::clangd::buildPreamble(testPath(Filename), *CI, Inputs,
|
|
|
|
/*StoreInMemory=*/true,
|
|
|
|
/*PreambleCallback=*/nullptr);
|
2020-04-27 06:13:56 +08:00
|
|
|
auto AST = ParsedAST::build(testPath(Filename), Inputs, std::move(CI),
|
|
|
|
Diags.take(), Preamble);
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
if (!AST.hasValue()) {
|
|
|
|
ADD_FAILURE() << "Failed to build code:\n" << Code;
|
|
|
|
llvm_unreachable("Failed to build TestTU!");
|
|
|
|
}
|
[clangd] Errors in TestTU cause test failures unless suppressed with error-ok.
Summary:
The historic behavior of TestTU is to gather diagnostics and otherwise ignore
them. So if a test has a syntax error, and doesn't assert diagnostics, it
silently misbehaves.
This can be annoying when developing tests, as evidenced by various tests
gaining "assert no diagnostics" where that's not really the point of the test.
This patch aims to make that default behavior. For the first error
(not warning), TestTU will call ADD_FAILURE().
This can be suppressed with a comment containing "error-ok". For now that will
suppress any errors in the TU. We can make this stricter later -verify style.
(-verify itself is hard to reuse because of DiagnosticConsumer interfaces...)
A magic-comment was chosen over a TestTU option because of table-driven tests.
In addition to the behavior change, this patch:
- adds //error-ok where we're knowingly testing invalid code
(e.g. for diagnostics, crash-resilience, or token-level tests)
- fixes a bunch of errors in the checked-in tests, mostly trivial (missing ;)
- removes a bunch of now-redundant instances of "assert no diagnostics"
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73199
2020-01-22 23:38:41 +08:00
|
|
|
// Check for error diagnostics and report gtest failures (unless expected).
|
|
|
|
// This guards against accidental syntax errors silently subverting tests.
|
|
|
|
// error-ok is awfully primitive - using clang -verify would be nicer.
|
|
|
|
// Ownership and layering makes it pretty hard.
|
2020-04-01 05:09:28 +08:00
|
|
|
bool ErrorOk = [&, this] {
|
|
|
|
llvm::StringLiteral Marker = "error-ok";
|
|
|
|
if (llvm::StringRef(Code).contains(Marker) ||
|
|
|
|
llvm::StringRef(HeaderCode).contains(Marker))
|
|
|
|
return true;
|
2020-03-16 04:43:00 +08:00
|
|
|
for (const auto &KV : this->AdditionalFiles)
|
2020-04-01 05:09:28 +08:00
|
|
|
if (llvm::StringRef(KV.second).contains(Marker))
|
|
|
|
return true;
|
|
|
|
return false;
|
|
|
|
}();
|
|
|
|
if (!ErrorOk) {
|
[clangd] Errors in TestTU cause test failures unless suppressed with error-ok.
Summary:
The historic behavior of TestTU is to gather diagnostics and otherwise ignore
them. So if a test has a syntax error, and doesn't assert diagnostics, it
silently misbehaves.
This can be annoying when developing tests, as evidenced by various tests
gaining "assert no diagnostics" where that's not really the point of the test.
This patch aims to make that default behavior. For the first error
(not warning), TestTU will call ADD_FAILURE().
This can be suppressed with a comment containing "error-ok". For now that will
suppress any errors in the TU. We can make this stricter later -verify style.
(-verify itself is hard to reuse because of DiagnosticConsumer interfaces...)
A magic-comment was chosen over a TestTU option because of table-driven tests.
In addition to the behavior change, this patch:
- adds //error-ok where we're knowingly testing invalid code
(e.g. for diagnostics, crash-resilience, or token-level tests)
- fixes a bunch of errors in the checked-in tests, mostly trivial (missing ;)
- removes a bunch of now-redundant instances of "assert no diagnostics"
Reviewers: kadircet
Subscribers: ilya-biryukov, MaskRay, jkorous, arphaman, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D73199
2020-01-22 23:38:41 +08:00
|
|
|
for (const auto &D : AST->getDiagnostics())
|
|
|
|
if (D.Severity >= DiagnosticsEngine::Error) {
|
|
|
|
ADD_FAILURE()
|
|
|
|
<< "TestTU failed to build (suppress with /*error-ok*/): \n"
|
|
|
|
<< D << "\n\nFor code:\n"
|
|
|
|
<< Code;
|
|
|
|
break; // Just report first error for simplicity.
|
|
|
|
}
|
|
|
|
}
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
return std::move(*AST);
|
|
|
|
}
|
|
|
|
|
|
|
|
SymbolSlab TestTU::headerSymbols() const {
|
|
|
|
auto AST = build();
|
[clangd] Track document versions, include them with diags, enhance logs
Summary:
This ties to an LSP feature (diagnostic versioning) but really a lot
of the value is in being able to log what's happening with file versions
and queues more descriptively and clearly.
As such it's fairly invasive, for a logging patch :-\
Key decisions:
- at the LSP layer, we don't reqire the client to provide versions (LSP
makes it mandatory but we never enforced it). If not provided,
versions start at 0 and increment. DraftStore handles this.
- don't propagate magically using contexts, but rather manually:
addDocument -> ParseInputs -> (ParsedAST, Preamble, various callbacks)
Context-propagation would hide the versions from ClangdServer, which
would make producing good log messages hard
- within ClangdServer, treat versions as opaque and unordered.
std::string is a convenient type for this, and allows richer versions
for embedders. They're "mandatory" but "null" is a reasonable default.
Subscribers: ilya-biryukov, javed.absar, MaskRay, jkorous, arphaman, kadircet, usaxena95, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D75582
2020-03-04 07:33:29 +08:00
|
|
|
return std::get<0>(indexHeaderSymbols(/*Version=*/"null", AST.getASTContext(),
|
2019-06-15 10:26:47 +08:00
|
|
|
AST.getPreprocessorPtr(),
|
|
|
|
AST.getCanonicalIncludes()));
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
}
|
|
|
|
|
2020-05-19 22:59:04 +08:00
|
|
|
RefSlab TestTU::headerRefs() const {
|
|
|
|
auto AST = build();
|
|
|
|
return std::get<1>(indexMainDecls(AST));
|
|
|
|
}
|
|
|
|
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
std::unique_ptr<SymbolIndex> TestTU::index() const {
|
2018-09-05 18:33:36 +08:00
|
|
|
auto AST = build();
|
2020-11-16 11:59:10 +08:00
|
|
|
auto Idx = std::make_unique<FileIndex>(/*UseDex=*/true,
|
|
|
|
/*CollectMainFileRefs=*/true);
|
2020-04-08 22:14:53 +08:00
|
|
|
Idx->updatePreamble(testPath(Filename), /*Version=*/"null",
|
|
|
|
AST.getASTContext(), AST.getPreprocessorPtr(),
|
|
|
|
AST.getCanonicalIncludes());
|
|
|
|
Idx->updateMain(testPath(Filename), AST);
|
2018-10-05 01:15:41 +08:00
|
|
|
return std::move(Idx);
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
}
|
|
|
|
|
2019-01-07 23:45:19 +08:00
|
|
|
const Symbol &findSymbol(const SymbolSlab &Slab, llvm::StringRef QName) {
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
const Symbol *Result = nullptr;
|
|
|
|
for (const Symbol &S : Slab) {
|
|
|
|
if (QName != (S.Scope + S.Name).str())
|
|
|
|
continue;
|
|
|
|
if (Result) {
|
|
|
|
ADD_FAILURE() << "Multiple symbols named " << QName << ":\n"
|
|
|
|
<< *Result << "\n---\n"
|
|
|
|
<< S;
|
|
|
|
assert(false && "QName is not unique");
|
|
|
|
}
|
|
|
|
Result = &S;
|
|
|
|
}
|
|
|
|
if (!Result) {
|
|
|
|
ADD_FAILURE() << "No symbol named " << QName << " in "
|
|
|
|
<< ::testing::PrintToString(Slab);
|
|
|
|
assert(false && "No symbol with QName");
|
|
|
|
}
|
|
|
|
return *Result;
|
|
|
|
}
|
|
|
|
|
2019-01-07 23:45:19 +08:00
|
|
|
const NamedDecl &findDecl(ParsedAST &AST, llvm::StringRef QName) {
|
2018-05-28 20:23:17 +08:00
|
|
|
auto &Ctx = AST.getASTContext();
|
|
|
|
auto LookupDecl = [&Ctx](const DeclContext &Scope,
|
2019-01-07 23:45:19 +08:00
|
|
|
llvm::StringRef Name) -> const NamedDecl & {
|
2018-05-28 20:23:17 +08:00
|
|
|
auto LookupRes = Scope.lookup(DeclarationName(&Ctx.Idents.get(Name)));
|
|
|
|
assert(!LookupRes.empty() && "Lookup failed");
|
|
|
|
assert(LookupRes.size() == 1 && "Lookup returned multiple results");
|
|
|
|
return *LookupRes.front();
|
|
|
|
};
|
|
|
|
|
|
|
|
const DeclContext *Scope = Ctx.getTranslationUnitDecl();
|
2020-12-10 22:59:16 +08:00
|
|
|
|
|
|
|
StringRef Cur, Rest;
|
|
|
|
for (std::tie(Cur, Rest) = QName.split("::"); !Rest.empty();
|
|
|
|
std::tie(Cur, Rest) = Rest.split("::")) {
|
|
|
|
Scope = &cast<DeclContext>(LookupDecl(*Scope, Cur));
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
}
|
2020-12-10 22:59:16 +08:00
|
|
|
return LookupDecl(*Scope, Cur);
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
}
|
|
|
|
|
2018-11-09 20:56:49 +08:00
|
|
|
const NamedDecl &findDecl(ParsedAST &AST,
|
2018-11-10 01:33:48 +08:00
|
|
|
std::function<bool(const NamedDecl &)> Filter) {
|
2018-06-06 00:30:25 +08:00
|
|
|
struct Visitor : RecursiveASTVisitor<Visitor> {
|
2018-11-10 01:33:48 +08:00
|
|
|
decltype(Filter) F;
|
2019-01-07 23:45:19 +08:00
|
|
|
llvm::SmallVector<const NamedDecl *, 1> Decls;
|
2018-06-06 00:30:25 +08:00
|
|
|
bool VisitNamedDecl(const NamedDecl *ND) {
|
2018-11-10 01:33:48 +08:00
|
|
|
if (F(*ND))
|
2018-07-05 16:14:04 +08:00
|
|
|
Decls.push_back(ND);
|
2018-06-06 00:30:25 +08:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
} Visitor;
|
2018-11-10 01:33:48 +08:00
|
|
|
Visitor.F = Filter;
|
2018-11-09 20:56:49 +08:00
|
|
|
Visitor.TraverseDecl(AST.getASTContext().getTranslationUnitDecl());
|
2018-06-06 00:30:25 +08:00
|
|
|
if (Visitor.Decls.size() != 1) {
|
2018-07-05 16:14:04 +08:00
|
|
|
ADD_FAILURE() << Visitor.Decls.size() << " symbols matched.";
|
2018-06-06 00:30:25 +08:00
|
|
|
assert(Visitor.Decls.size() == 1);
|
|
|
|
}
|
|
|
|
return *Visitor.Decls.front();
|
|
|
|
}
|
|
|
|
|
2019-01-07 23:45:19 +08:00
|
|
|
const NamedDecl &findUnqualifiedDecl(ParsedAST &AST, llvm::StringRef Name) {
|
2018-11-09 20:56:49 +08:00
|
|
|
return findDecl(AST, [Name](const NamedDecl &ND) {
|
2018-07-05 16:14:04 +08:00
|
|
|
if (auto *ID = ND.getIdentifier())
|
|
|
|
if (ID->getName() == Name)
|
|
|
|
return true;
|
|
|
|
return false;
|
|
|
|
});
|
|
|
|
}
|
|
|
|
|
[clangd] Extract scoring/ranking logic, and shave yaks.
Summary:
Code completion scoring was embedded in CodeComplete.cpp, which is bad:
- awkward to test. The mechanisms (extracting info from index/sema) can be
unit-tested well, the policy (scoring) should be quantitatively measured.
Neither was easily possible, and debugging was hard.
The intermediate signal struct makes this easier.
- hard to reuse. This is a bug in workspaceSymbols: it just presents the
results in the index order, which is not sorted in practice, it needs to rank
them!
Also, index implementations care about scoring (both query-dependent and
independent) in order to truncate result lists appropriately.
The main yak shaved here is the build() function that had 3 variants across
unit tests is unified in TestTU.h (rather than adding a 4th variant).
Reviewers: ilya-biryukov
Subscribers: klimek, mgorny, ioeric, MaskRay, jkorous, mgrang, cfe-commits
Differential Revision: https://reviews.llvm.org/D46524
llvm-svn: 332378
2018-05-16 01:43:27 +08:00
|
|
|
} // namespace clangd
|
|
|
|
} // namespace clang
|