llvm-project/clang/tools/clang-import-test/clang-import-test.cpp

384 lines
14 KiB
C++
Raw Normal View History

Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
//===-- import-test.cpp - ASTImporter/ExternalASTSource testbed -----------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#include "clang/AST/ASTContext.h"
#include "clang/AST/ASTImporter.h"
#include "clang/AST/DeclObjC.h"
#include "clang/AST/ExternalASTMerger.h"
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
#include "clang/Basic/Builtins.h"
#include "clang/Basic/IdentifierTable.h"
#include "clang/Basic/SourceLocation.h"
#include "clang/Basic/TargetInfo.h"
#include "clang/Basic/TargetOptions.h"
#include "clang/CodeGen/ModuleBuilder.h"
#include "clang/Driver/Types.h"
#include "clang/Frontend/ASTConsumers.h"
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
#include "clang/Frontend/CompilerInstance.h"
#include "clang/Frontend/MultiplexConsumer.h"
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
#include "clang/Frontend/TextDiagnosticBuffer.h"
#include "clang/Lex/Lexer.h"
#include "clang/Lex/Preprocessor.h"
#include "clang/Parse/ParseAST.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Module.h"
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
#include "llvm/Support/CommandLine.h"
#include "llvm/Support/Error.h"
#include "llvm/Support/Host.h"
#include "llvm/Support/Signals.h"
#include <memory>
#include <string>
using namespace clang;
static llvm::cl::opt<std::string> Expression(
"expression", llvm::cl::Required,
llvm::cl::desc("Path to a file containing the expression to parse"));
static llvm::cl::list<std::string>
Imports("import", llvm::cl::ZeroOrMore,
llvm::cl::desc("Path to a file containing declarations to import"));
static llvm::cl::opt<bool>
Direct("direct", llvm::cl::Optional,
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
llvm::cl::desc("Use the parsed declarations without indirection"));
static llvm::cl::opt<bool> UseOrigins(
"use-origins", llvm::cl::Optional,
llvm::cl::desc(
"Use DeclContext origin information for more accurate lookups"));
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
static llvm::cl::list<std::string>
ClangArgs("Xcc", llvm::cl::ZeroOrMore,
llvm::cl::desc("Argument to pass to the CompilerInvocation"),
llvm::cl::CommaSeparated);
static llvm::cl::opt<std::string>
Input("x", llvm::cl::Optional,
llvm::cl::desc("The language to parse (default: c++)"),
llvm::cl::init("c++"));
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
static llvm::cl::opt<bool> DumpAST("dump-ast", llvm::cl::init(false),
llvm::cl::desc("Dump combined AST"));
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
static llvm::cl::opt<bool> DumpIR("dump-ir", llvm::cl::init(false),
llvm::cl::desc("Dump IR from final parse"));
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
namespace init_convenience {
class TestDiagnosticConsumer : public DiagnosticConsumer {
private:
std::unique_ptr<TextDiagnosticBuffer> Passthrough;
const LangOptions *LangOpts = nullptr;
public:
TestDiagnosticConsumer()
: Passthrough(llvm::make_unique<TextDiagnosticBuffer>()) {}
virtual void BeginSourceFile(const LangOptions &LangOpts,
const Preprocessor *PP = nullptr) override {
this->LangOpts = &LangOpts;
return Passthrough->BeginSourceFile(LangOpts, PP);
}
virtual void EndSourceFile() override {
this->LangOpts = nullptr;
Passthrough->EndSourceFile();
}
virtual bool IncludeInDiagnosticCounts() const override {
return Passthrough->IncludeInDiagnosticCounts();
}
private:
static void PrintSourceForLocation(const SourceLocation &Loc,
SourceManager &SM) {
const char *LocData = SM.getCharacterData(Loc, /*Invalid=*/nullptr);
unsigned LocColumn =
SM.getSpellingColumnNumber(Loc, /*Invalid=*/nullptr) - 1;
FileID FID = SM.getFileID(Loc);
llvm::MemoryBuffer *Buffer = SM.getBuffer(FID, Loc, /*Invalid=*/nullptr);
assert(LocData >= Buffer->getBufferStart() &&
LocData < Buffer->getBufferEnd());
const char *LineBegin = LocData - LocColumn;
assert(LineBegin >= Buffer->getBufferStart());
const char *LineEnd = nullptr;
for (LineEnd = LineBegin; *LineEnd != '\n' && *LineEnd != '\r' &&
LineEnd < Buffer->getBufferEnd();
++LineEnd)
;
llvm::StringRef LineString(LineBegin, LineEnd - LineBegin);
llvm::errs() << LineString << '\n';
llvm::errs().indent(LocColumn);
llvm::errs() << '^';
llvm::errs() << '\n';
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
}
virtual void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
const Diagnostic &Info) override {
if (Info.hasSourceManager() && LangOpts) {
SourceManager &SM = Info.getSourceManager();
if (Info.getLocation().isValid()) {
Info.getLocation().print(llvm::errs(), SM);
llvm::errs() << ": ";
}
SmallString<16> DiagText;
Info.FormatDiagnostic(DiagText);
llvm::errs() << DiagText << '\n';
if (Info.getLocation().isValid()) {
PrintSourceForLocation(Info.getLocation(), SM);
}
for (const CharSourceRange &Range : Info.getRanges()) {
bool Invalid = true;
StringRef Ref = Lexer::getSourceText(Range, SM, *LangOpts, &Invalid);
if (!Invalid) {
llvm::errs() << Ref << '\n';
}
}
}
DiagnosticConsumer::HandleDiagnostic(DiagLevel, Info);
}
};
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
std::unique_ptr<CompilerInstance> BuildCompilerInstance() {
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
auto Ins = llvm::make_unique<CompilerInstance>();
auto DC = llvm::make_unique<TestDiagnosticConsumer>();
const bool ShouldOwnClient = true;
Ins->createDiagnostics(DC.release(), ShouldOwnClient);
auto Inv = llvm::make_unique<CompilerInvocation>();
std::vector<const char *> ClangArgv(ClangArgs.size());
std::transform(ClangArgs.begin(), ClangArgs.end(), ClangArgv.begin(),
[](const std::string &s) -> const char * { return s.data(); });
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
CompilerInvocation::CreateFromArgs(*Inv, ClangArgv.data(),
&ClangArgv.data()[ClangArgv.size()],
Ins->getDiagnostics());
{
using namespace driver::types;
ID Id = lookupTypeForTypeSpecifier(Input.c_str());
assert(Id != TY_INVALID);
if (isCXX(Id)) {
Inv->getLangOpts()->CPlusPlus = true;
Inv->getLangOpts()->CPlusPlus11 = true;
Inv->getHeaderSearchOpts().UseLibcxx = true;
}
if (isObjC(Id)) {
Inv->getLangOpts()->ObjC = 1;
}
}
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
Inv->getLangOpts()->Bool = true;
Inv->getLangOpts()->WChar = true;
Inv->getLangOpts()->Blocks = true;
Inv->getLangOpts()->DebuggerSupport = true;
Inv->getLangOpts()->SpellChecking = false;
Inv->getLangOpts()->ThreadsafeStatics = false;
Inv->getLangOpts()->AccessControl = false;
Inv->getLangOpts()->DollarIdents = true;
Inv->getLangOpts()->Exceptions = true;
Inv->getLangOpts()->CXXExceptions = true;
// Needed for testing dynamic_cast.
Inv->getLangOpts()->RTTI = true;
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
Inv->getCodeGenOpts().setDebugInfo(codegenoptions::FullDebugInfo);
Inv->getTargetOpts().Triple = llvm::sys::getDefaultTargetTriple();
Ins->setInvocation(std::move(Inv));
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
TargetInfo *TI = TargetInfo::CreateTargetInfo(
Ins->getDiagnostics(), Ins->getInvocation().TargetOpts);
Ins->setTarget(TI);
Ins->getTarget().adjust(Ins->getLangOpts());
Ins->createFileManager();
Ins->createSourceManager(Ins->getFileManager());
Ins->createPreprocessor(TU_Complete);
return Ins;
}
std::unique_ptr<ASTContext>
BuildASTContext(CompilerInstance &CI, SelectorTable &ST, Builtin::Context &BC) {
auto AST = llvm::make_unique<ASTContext>(
CI.getLangOpts(), CI.getSourceManager(),
CI.getPreprocessor().getIdentifierTable(), ST, BC);
AST->InitBuiltinTypes(CI.getTarget());
return AST;
}
std::unique_ptr<CodeGenerator> BuildCodeGen(CompilerInstance &CI,
llvm::LLVMContext &LLVMCtx) {
StringRef ModuleName("$__module");
return std::unique_ptr<CodeGenerator>(CreateLLVMCodeGen(
CI.getDiagnostics(), ModuleName, CI.getHeaderSearchOpts(),
CI.getPreprocessorOpts(), CI.getCodeGenOpts(), LLVMCtx));
}
} // namespace init_convenience
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
namespace {
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
/// A container for a CompilerInstance (possibly with an ExternalASTMerger
/// attached to its ASTContext).
///
/// Provides an accessor for the DeclContext origins associated with the
/// ExternalASTMerger (or an empty list of origins if no ExternalASTMerger is
/// attached).
///
/// This is the main unit of parsed source code maintained by clang-import-test.
struct CIAndOrigins {
using OriginMap = clang::ExternalASTMerger::OriginMap;
std::unique_ptr<CompilerInstance> CI;
ASTContext &getASTContext() { return CI->getASTContext(); }
FileManager &getFileManager() { return CI->getFileManager(); }
const OriginMap &getOriginMap() {
static const OriginMap EmptyOriginMap{};
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
if (ExternalASTSource *Source = CI->getASTContext().getExternalSource())
return static_cast<ExternalASTMerger *>(Source)->GetOrigins();
return EmptyOriginMap;
}
DiagnosticConsumer &getDiagnosticClient() {
return CI->getDiagnosticClient();
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
CompilerInstance &getCompilerInstance() { return *CI; }
};
void AddExternalSource(CIAndOrigins &CI,
llvm::MutableArrayRef<CIAndOrigins> Imports) {
ExternalASTMerger::ImporterTarget Target(
{CI.getASTContext(), CI.getFileManager()});
llvm::SmallVector<ExternalASTMerger::ImporterSource, 3> Sources;
for (CIAndOrigins &Import : Imports)
Sources.push_back({Import.getASTContext(), Import.getFileManager(),
Import.getOriginMap()});
auto ES = llvm::make_unique<ExternalASTMerger>(Target, Sources);
CI.getASTContext().setExternalSource(ES.release());
CI.getASTContext().getTranslationUnitDecl()->setHasExternalVisibleStorage();
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
CIAndOrigins BuildIndirect(CIAndOrigins &CI) {
CIAndOrigins IndirectCI{init_convenience::BuildCompilerInstance()};
auto ST = llvm::make_unique<SelectorTable>();
auto BC = llvm::make_unique<Builtin::Context>();
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
std::unique_ptr<ASTContext> AST = init_convenience::BuildASTContext(
IndirectCI.getCompilerInstance(), *ST, *BC);
IndirectCI.getCompilerInstance().setASTContext(AST.release());
AddExternalSource(IndirectCI, CI);
return IndirectCI;
}
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
llvm::Error ParseSource(const std::string &Path, CompilerInstance &CI,
ASTConsumer &Consumer) {
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
SourceManager &SM = CI.getSourceManager();
const FileEntry *FE = CI.getFileManager().getFile(Path);
if (!FE) {
return llvm::make_error<llvm::StringError>(
llvm::Twine("Couldn't open ", Path), std::error_code());
}
SM.setMainFileID(SM.createFileID(FE, SourceLocation(), SrcMgr::C_User));
ParseAST(CI.getPreprocessor(), &Consumer, CI.getASTContext());
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
return llvm::Error::success();
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
llvm::Expected<CIAndOrigins> Parse(const std::string &Path,
llvm::MutableArrayRef<CIAndOrigins> Imports,
bool ShouldDumpAST, bool ShouldDumpIR) {
CIAndOrigins CI{init_convenience::BuildCompilerInstance()};
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
auto ST = llvm::make_unique<SelectorTable>();
auto BC = llvm::make_unique<Builtin::Context>();
std::unique_ptr<ASTContext> AST =
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
init_convenience::BuildASTContext(CI.getCompilerInstance(), *ST, *BC);
CI.getCompilerInstance().setASTContext(AST.release());
if (Imports.size())
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
AddExternalSource(CI, Imports);
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
std::vector<std::unique_ptr<ASTConsumer>> ASTConsumers;
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
auto LLVMCtx = llvm::make_unique<llvm::LLVMContext>();
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
ASTConsumers.push_back(
init_convenience::BuildCodeGen(CI.getCompilerInstance(), *LLVMCtx));
auto &CG = *static_cast<CodeGenerator *>(ASTConsumers.back().get());
if (ShouldDumpAST)
ASTConsumers.push_back(CreateASTDumper(nullptr /*Dump to stdout.*/,
"", true, false, false));
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
CI.getDiagnosticClient().BeginSourceFile(
CI.getCompilerInstance().getLangOpts(),
&CI.getCompilerInstance().getPreprocessor());
MultiplexConsumer Consumers(std::move(ASTConsumers));
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
Consumers.Initialize(CI.getASTContext());
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
if (llvm::Error PE = ParseSource(Path, CI.getCompilerInstance(), Consumers))
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
return std::move(PE);
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
CI.getDiagnosticClient().EndSourceFile();
if (ShouldDumpIR)
CG.GetModule()->print(llvm::outs(), nullptr);
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
if (CI.getDiagnosticClient().getNumErrors())
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
return llvm::make_error<llvm::StringError>(
"Errors occurred while parsing the expression.", std::error_code());
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
return std::move(CI);
}
void Forget(CIAndOrigins &CI, llvm::MutableArrayRef<CIAndOrigins> Imports) {
llvm::SmallVector<ExternalASTMerger::ImporterSource, 3> Sources;
for (CIAndOrigins &Import : Imports)
Sources.push_back({Import.getASTContext(), Import.getFileManager(),
Import.getOriginMap()});
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
ExternalASTSource *Source = CI.CI->getASTContext().getExternalSource();
auto *Merger = static_cast<ExternalASTMerger *>(Source);
Merger->RemoveSources(Sources);
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
}
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
} // end namespace
int main(int argc, const char **argv) {
const bool DisableCrashReporting = true;
llvm::sys::PrintStackTraceOnErrorSignal(argv[0], DisableCrashReporting);
llvm::cl::ParseCommandLineOptions(argc, argv);
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
std::vector<CIAndOrigins> ImportCIs;
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
for (auto I : Imports) {
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
llvm::Expected<CIAndOrigins> ImportCI = Parse(I, {}, false, false);
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
if (auto E = ImportCI.takeError()) {
llvm::errs() << llvm::toString(std::move(E));
exit(-1);
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
ImportCIs.push_back(std::move(*ImportCI));
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
std::vector<CIAndOrigins> IndirectCIs;
if (!Direct || UseOrigins) {
for (auto &ImportCI : ImportCIs) {
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
CIAndOrigins IndirectCI = BuildIndirect(ImportCI);
IndirectCIs.push_back(std::move(IndirectCI));
}
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
if (UseOrigins)
for (auto &ImportCI : ImportCIs)
IndirectCIs.push_back(std::move(ImportCI));
llvm::Expected<CIAndOrigins> ExpressionCI =
Parse(Expression, (Direct && !UseOrigins) ? ImportCIs : IndirectCIs,
DumpAST, DumpIR);
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
if (auto E = ExpressionCI.takeError()) {
llvm::errs() << llvm::toString(std::move(E));
exit(-1);
}
Add support for remembering origins to ExternalASTMerger ExternalASTMerger has hitherto relied on being able to look up any Decl through its named DeclContext chain. This works for many cases, but causes problems for function-local structs, which cannot be looked up in their containing FunctionDecl. An example case is void f() { { struct S { int a; }; } { struct S { bool b; }; } } It is not possible to lookup either of the two Ses individually (or even to provide enough information to disambiguate) after parsing is over; and there is typically no need to, since they are invisible to the outside world. However, ExternalASTMerger needs to be able to complete either S on demand. This led to an XFAIL on test/Import/local-struct, which this patch removes. The way the patch works is: It defines a new data structure, ExternalASTMerger::OriginMap, which clients are expected to maintain (default-constructing if the origin does not have an ExternalASTMerger servicing it) As DeclContexts are imported, if they cannot be looked up by name they are placed in the OriginMap. This allows ExternalASTMerger to complete them later if necessary. As DeclContexts are imported from an origin that already has its own OriginMap, the origins are forwarded – but only for those DeclContexts that are actually used. This keeps the amount of stored data minimal. The patch also applies several improvements from review: - Thoroughly documents the interface to ExternalASTMerger; - Adds optional logging to help track what's going on; and - Cleans up a bunch of braces and dangling elses. Differential Revision: https://reviews.llvm.org/D38208 llvm-svn: 314336
2017-09-28 03:57:58 +08:00
Forget(*ExpressionCI, (Direct && !UseOrigins) ? ImportCIs : IndirectCIs);
return 0;
Testbed and skeleton of a new expression parser Recommitted after formal approval. LLVM's JIT is now the foundation of dynamic-compilation features for many languages. Clang also has low-level support for dynamic compilation (ASTImporter and ExternalASTSource, notably). How the compiler is set up for dynamic parsing is generally left up to individual clients, for example LLDB's C/C++/Objective-C expression parser and the ROOT project. Although this arrangement offers external clients the flexibility to implement dynamic features as they see fit, the lack of an in-tree client means that subtle bugs can be introduced that cause regressions in the external clients but aren't caught by tests (or users) until much later. LLDB for example regularly encounters complicated ODR violation scenarios where it is not immediately clear who is at fault. Other external clients (notably, Cling) rely on similar functionality, and another goal is to break this functionality up into composable parts so that any client can be built easily on top of Clang without requiring extensive additional code. I propose that the parts required to build a simple expression parser be added to Clang. Initially, I aim to have the following features: A piece that looks up external declarations from a variety of sources (e.g., from previous dynamic compilations, from modules, or from DWARF) and uses clear conflict resolution rules to reconcile differences, with easily understood errors. This functionality will be supported by in-tree tests. A piece that works hand in hand with the LLVM JIT to resolve the locations of external declarations so that e.g. variables can be redeclared and (for high-performance applications like DTrace) external variables can be accessed directly from the registers where they reside. This commit adds a tester that parses a sequence of source files and then uses them as source data for an expression. External references are resolved using an ExternalASTSource that responds to name queries using an ASTImporter. This is the setup that LLDB uses, and the motivating reason for MinimalImport in ASTImporter. When complete, this tester will implement the first of the above goals. Differential Revision: https://reviews.llvm.org/D27180 llvm-svn: 290367
2016-12-23 04:03:14 +08:00
}