forked from OSchip/llvm-project
282 lines
13 KiB
ReStructuredText
282 lines
13 KiB
ReStructuredText
|
=======================================================
|
||
|
|
Kaleidoscope: Building an ORC-based JIT in LLVM
|
||
|
|
=======================================================
|
||
|
|
|
||
|
|
.. contents::
|
||
|
|
:local:
|
||
|
|
|
||
|
|
**This tutorial is under active development. It is incomplete and details may
|
||
|
|
change frequently.** Nonetheless we invite you to try it out as it stands, and
|
||
|
|
we welcome any feedback.
|
||
|
|
|
||
|
|
Chapter 1 Introduction
|
||
|
|
======================
|
||
|
|
|
||
|
|
Welcome to Chapter 1 of the "Building an ORC-based JIT in LLVM" tutorial. This
|
||
|
|
tutorial runs through the implementation of a JIT compiler using LLVM's
|
||
|
|
On-Request-Compilation (ORC) APIs. It begins with a simplified version of the
|
||
|
|
KaleidoscopeJIT class used in the
|
||
|
|
`Implementing a language with LLVM <LangImpl1.html>`_ tutorials and then
|
||
|
|
introduces new features like optimization, lazy compilation and remote
|
||
|
|
execution.
|
||
|
|
|
||
|
|
The goal of this tutorial is to introduce you to LLVM's ORC JIT APIs, show how
|
||
|
|
these APIs interact with other parts of LLVM, and to teach you how to recombine
|
||
|
|
them to build a custom JIT that is suited to your use-case.
|
||
|
|
|
||
|
|
The structure of the tutorial is:
|
||
|
|
|
||
|
|
- Chapter #1: Investigate the simple KaleidoscopeJIT class. This will
|
||
|
|
introduce some of the basic concepts of the ORC JIT APIs, including the
|
||
|
|
idea of an ORC *Layer*.
|
||
|
|
|
||
|
|
- `Chapter #2 <BuildingAJIT2.html>`_: Extend the basic KaleidoscopeJIT by adding
|
||
|
|
a new layer that will optimize IR and generated code.
|
||
|
|
|
||
|
|
- `Chapter #3 <BuildingAJIT3.html>`_: Further extend the JIT by adding a
|
||
|
|
Compile-On-Demand layer to lazily compile IR.
|
||
|
|
|
||
|
|
- `Chapter #4 <BuildingAJIT4.html>`_: Improve the laziness of our JIT by
|
||
|
|
replacing the Compile-On-Demand layer with a custom layer that uses the ORC
|
||
|
|
Compile Callbacks API directly to defer IR-generation until functions are
|
||
|
|
called.
|
||
|
|
|
||
|
|
- `Chapter #5 <BuildingAJIT5.html>`_: Add process isolation by JITing code into
|
||
|
|
a remote process with reduced privileges using the JIT Remote APIs.
|
||
|
|
|
||
|
|
To provide input for our JIT we will use the Kaleidoscope REPL from
|
||
|
|
`Chapter 7 <LangImpl7.html>`_ of the "Implementing a language in LLVM tutorial",
|
||
|
|
with one minor modification: We will remove the FunctionPassManager from the
|
||
|
|
code for that chapter and replace it with optimization support in our JIT class
|
||
|
|
in Chapter #2.
|
||
|
|
|
||
|
|
Finally, a word on API generations: ORC is the 3rd generation of LLVM JIT API.
|
||
|
|
It was preceeded by MCJIT, and before that by the (now deleted) legacy JIT.
|
||
|
|
These tutorials don't assume any experience with these earlier APIs, but
|
||
|
|
readers acquainted with them will see many familiar elements. Where appropriate
|
||
|
|
we will make this connection with the earlier APIs explicit to help people who
|
||
|
|
are transitioning from them to ORC.
|
||
|
|
|
||
|
|
JIT API Basics
|
||
|
|
==============
|
||
|
|
|
||
|
|
The purpose of a JIT compiler is to compile code "on-the-fly" as it is needed,
|
||
|
|
rather than compiling whole programs to disk ahead of time as a traditional
|
||
|
|
compiler does. To support that aim our initial, bare-bones JIT API will be:
|
||
|
|
|
||
|
|
1. Handle addModule(Module &M) -- Make the given IR module available for
|
||
|
|
execution.
|
||
|
|
2. JITSymbol findSymbol(const std::string &Name) -- Search for pointers to
|
||
|
|
symbols (functions or variables) that have been added to the JIT.
|
||
|
|
3. void removeModule(Handle H) -- Remove a module from the JIT, releasing any
|
||
|
|
memory that had been used for the compiled code.
|
||
|
|
|
||
|
|
A basic use-case for this API, executing the 'main' function from a module,
|
||
|
|
will look like:
|
||
|
|
|
||
|
|
.. code-block:: c++
|
||
|
|
|
||
|
|
std::unique_ptr<Module> M = buildModule();
|
||
|
|
JIT J;
|
||
|
|
Handle H = J.addModule(*M);
|
||
|
|
int (*Main)(int, char*[]) =
|
||
|
|
(int(*)(int, char*[])J.findSymbol("main").getAddress();
|
||
|
|
int Result = Main();
|
||
|
|
J.removeModule(H);
|
||
|
|
|
||
|
|
The APIs that we build in these tutorials will all be variations on this simple
|
||
|
|
theme. Behind the API we will refine the implementation of the JIT to add
|
||
|
|
support for optimization and lazy compilation. Eventually we will extend the
|
||
|
|
API itself to allow higher-level program representations (e.g. ASTs) to be
|
||
|
|
added to the JIT.
|
||
|
|
|
||
|
|
KaleidoscopeJIT
|
||
|
|
===============
|
||
|
|
|
||
|
|
In the previous section we described our API, now we examine a simple
|
||
|
|
implementation of it: The KaleidoscopeJIT class [1]_ that was used in the
|
||
|
|
`Implementing a language with LLVM <LangImpl1.html>`_ tutorials. We will use
|
||
|
|
the REPL code from `Chapter 7 <LangImpl7.html>`_ of that tutorial to supply the
|
||
|
|
input for our JIT: Each time the user enters an expression the REPL will add a
|
||
|
|
new IR module containing the code for that expression to the JIT. If the
|
||
|
|
expression is a top-level expression like '1+1' or 'sin(x)', the REPL will also
|
||
|
|
use the findSymbol method of our JIT class find and execute the code for the
|
||
|
|
expression, and then use the removeModule method to remove the code again
|
||
|
|
(since there's no way to re-invoke an anonymous expression). In later chapters
|
||
|
|
of this tutorial we'll modify the REPL to enable new interactions with our JIT
|
||
|
|
class, but for now we will take this setup for granted and focus our attention on
|
||
|
|
the implementation of our JIT itself.
|
||
|
|
|
||
|
|
Our KaleidoscopeJIT class is defined in the KaleidoscopeJIT.h header. After the
|
||
|
|
usual include guards and #includes [2]_, we get to the definition of our class:
|
||
|
|
|
||
|
|
.. code-block:: c++
|
||
|
|
|
||
|
|
#ifndef LLVM_EXECUTIONENGINE_ORC_KALEIDOSCOPEJIT_H
|
||
|
|
#define LLVM_EXECUTIONENGINE_ORC_KALEIDOSCOPEJIT_H
|
||
|
|
|
||
|
|
#include "llvm/ExecutionEngine/ExecutionEngine.h"
|
||
|
|
#include "llvm/ExecutionEngine/RTDyldMemoryManager.h"
|
||
|
|
#include "llvm/ExecutionEngine/Orc/CompileUtils.h"
|
||
|
|
#include "llvm/ExecutionEngine/Orc/IRCompileLayer.h"
|
||
|
|
#include "llvm/ExecutionEngine/Orc/LambdaResolver.h"
|
||
|
|
#include "llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h"
|
||
|
|
#include "llvm/IR/Mangler.h"
|
||
|
|
#include "llvm/Support/DynamicLibrary.h"
|
||
|
|
|
||
|
|
namespace llvm {
|
||
|
|
namespace orc {
|
||
|
|
|
||
|
|
class KaleidoscopeJIT {
|
||
|
|
private:
|
||
|
|
|
||
|
|
std::unique_ptr<TargetMachine> TM;
|
||
|
|
const DataLayout DL;
|
||
|
|
ObjectLinkingLayer<> ObjectLayer;
|
||
|
|
IRCompileLayer<decltype(ObjectLayer)> CompileLayer;
|
||
|
|
|
||
|
|
public:
|
||
|
|
|
||
|
|
typedef decltype(CompileLayer)::ModuleSetHandleT ModuleHandleT;
|
||
|
|
|
||
|
|
Our class begins with four members: A TargetMachine, TM, which will be used
|
||
|
|
to build our LLVM compiler instance; A DataLayout, DL, which will be used for
|
||
|
|
symbol mangling (more on that later), and two ORC *layers*: An
|
||
|
|
ObjectLinkingLayer, and an IRCompileLayer. The ObjectLinkingLayer is the
|
||
|
|
foundation of our JIT: it takes in-memory object files produced by a
|
||
|
|
compiler and links them on the fly to make them executable. This
|
||
|
|
JIT-on-top-of-a-linker design was introduced in MCJIT, where the linker was
|
||
|
|
hidden inside the MCJIT class itself. In ORC we expose the linker as a visible,
|
||
|
|
reusable component so that clients can access and configure it directly
|
||
|
|
if they need to. In this tutorial our ObjectLinkingLayer will just be used to
|
||
|
|
support the next layer in our stack: the IRCompileLayer, which will be
|
||
|
|
responsible for taking LLVM IR, compiling it, and passing the resulting
|
||
|
|
in-memory object files down to the object linking layer below.
|
||
|
|
|
||
|
|
After our member variables comes typedef: ModuleHandle. This is the handle
|
||
|
|
type that will be returned from our JIT's addModule method, and which can be
|
||
|
|
used to remove a module again using the removeModule method. The IRCompileLayer
|
||
|
|
class already provides a convenient handle type
|
||
|
|
(IRCompileLayer::ModuleSetHandleT), so we will just provide a type-alias for
|
||
|
|
this.
|
||
|
|
|
||
|
|
.. code-block:: c++
|
||
|
|
|
||
|
|
KaleidoscopeJIT()
|
||
|
|
: TM(EngineBuilder().selectTarget()), DL(TM->createDataLayout()),
|
||
|
|
CompileLayer(ObjectLayer, SimpleCompiler(*TM)) {
|
||
|
|
llvm::sys::DynamicLibrary::LoadLibraryPermanently(nullptr);
|
||
|
|
}
|
||
|
|
|
||
|
|
TargetMachine &getTargetMachine() { return *TM; }
|
||
|
|
|
||
|
|
Next up we have our class constructor. We begin by initializing TM using the
|
||
|
|
EngineBuilder::selectTarget helper method, which constructs a TargetMachine for
|
||
|
|
the current process. Next we use our newly created TargetMachine to initialize
|
||
|
|
DL, our DataLayout. Then we initialize our IRCompileLayer. Our IRCompile layer
|
||
|
|
needs two things: (1) A reference to our object linking layer, and (2) a
|
||
|
|
compiler instance to use to perform the actual compilation from IR to object
|
||
|
|
files. We use the off-the-shelf SimpleCompiler instance for now, but in later
|
||
|
|
chapters we will substitute our own configurable compiler classes. Finally, in
|
||
|
|
the body of the constructor, we call the DynamicLibrary::LoadLibraryPermanently
|
||
|
|
method with a nullptr argument. Normally the LoadLibraryPermanently method is
|
||
|
|
called with the path of a dynamic library to load, but when passed a null
|
||
|
|
pointer it will 'load' the host process itself, making its exported symbols
|
||
|
|
available for execution.
|
||
|
|
|
||
|
|
.. code-block:: c++
|
||
|
|
|
||
|
|
ModuleHandleT addModule(std::unique_ptr<Module> M) {
|
||
|
|
// We need a memory manager to allocate memory and resolve symbols for this
|
||
|
|
// new module. Create one that resolves symbols by looking back into the
|
||
|
|
// JIT.
|
||
|
|
auto Resolver = createLambdaResolver(
|
||
|
|
[&](const std::string &Name) {
|
||
|
|
if (auto Sym = CompileLayer.findSymbol(Name, false))
|
||
|
|
return RuntimeDyld::SymbolInfo(Sym.getAddress(), Sym.getFlags());
|
||
|
|
return RuntimeDyld::SymbolInfo(nullptr);
|
||
|
|
},
|
||
|
|
[](const std::string &S) { return nullptr; });
|
||
|
|
std::vector<std::unique_ptr<Module>> Ms;
|
||
|
|
Ms.push_back(std::move(M));
|
||
|
|
return CompileLayer.addModuleSet(singletonSet(std::move(M)),
|
||
|
|
make_unique<SectionMemoryManager>(),
|
||
|
|
std::move(Resolver));
|
||
|
|
}
|
||
|
|
|
||
|
|
*To be done: describe addModule -- createLambdaResolver, resolvers, memory
|
||
|
|
managers, why 'module set' rather than a single module...*
|
||
|
|
|
||
|
|
.. code-block:: c++
|
||
|
|
|
||
|
|
JITSymbol findSymbol(const std::string Name) {
|
||
|
|
std::string MangledName;
|
||
|
|
raw_string_ostream MangledNameStream(MangledName);
|
||
|
|
Mangler::getNameWithPrefix(MangledNameStream, Name, DL);
|
||
|
|
return CompileLayer.findSymbol(MangledNameStream.str(), true);
|
||
|
|
}
|
||
|
|
|
||
|
|
void removeModule(ModuleHandle H) {
|
||
|
|
CompileLayer.removeModuleSet(H);
|
||
|
|
}
|
||
|
|
|
||
|
|
*To be done: describe findSymbol and removeModule -- why do we mangle? what's
|
||
|
|
the relationship between findSymbol and resolvers, why remove modules...*
|
||
|
|
|
||
|
|
*To be done: Conclusion, exercises (maybe a utility for a standalone IR JIT,
|
||
|
|
like a mini-LLI), feed to next chapter.*
|
||
|
|
|
||
|
|
Full Code Listing
|
||
|
|
=================
|
||
|
|
|
||
|
|
Here is the complete code listing for our running example, enhanced with
|
||
|
|
mutable variables and var/in support. To build this example, use:
|
||
|
|
|
||
|
|
.. code-block:: bash
|
||
|
|
|
||
|
|
# Compile
|
||
|
|
clang++ -g toy.cpp `llvm-config --cxxflags --ldflags --system-libs --libs core orc native` -O3 -o toy
|
||
|
|
# Run
|
||
|
|
./toy
|
||
|
|
|
||
|
|
Here is the code:
|
||
|
|
|
||
|
|
.. literalinclude:: ../../examples/Kaleidoscope/BuildingAJIT/Chapter1/KaleidoscopeJIT.h
|
||
|
|
:language: c++
|
||
|
|
|
||
|
|
`Next: Extending the KaleidoscopeJIT <BuildingAJIT2.html>`_
|
||
|
|
|
||
|
|
.. [1] Actually we use a cut-down version of KaleidoscopeJIT that makes a
|
||
|
|
simplifying assumption: symbols cannot be re-defined. This will make it
|
||
|
|
impossible to re-define symbols in the REPL, but will make our symbol
|
||
|
|
lookup logic simpler. Re-introducing support for symbol redefinition is
|
||
|
|
left as an exercise for the reader. (The KaleidoscopeJIT.h used in the
|
||
|
|
original tutorials will be a helpful reference).
|
||
|
|
|
||
|
|
.. [2] +-----------------------+-----------------------------------------------+
|
||
|
|
| File | Reason for inclusion |
|
||
|
|
+=======================+===============================================+
|
||
|
|
| ExecutionEngine.h | Access to the EngineBuilder::selectTarget |
|
||
|
|
| | method. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| | Access to the |
|
||
|
|
| RTDyldMemoryManager.h | RTDyldMemoryManager::getSymbolAddressInProcess|
|
||
|
|
| | method. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| CompileUtils.h | Provides the SimpleCompiler class. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| IRCompileLayer.h | Provides the IRCompileLayer class. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| | Access the createLambdaResolver function, |
|
||
|
|
| LambdaResolver.h | which provides easy construction of symbol |
|
||
|
|
| | resolvers. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| ObjectLinkingLayer.h | Provides the ObjectLinkingLayer class. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| Mangler.h | Provides the Mangler class for platform |
|
||
|
|
| | specific name-mangling. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|
||
|
|
| DynamicLibrary.h | Provides the DynamicLibrary class, which |
|
||
|
|
| | makes symbols in the host process searchable. |
|
||
|
|
+-----------------------+-----------------------------------------------+
|