Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html
"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi
Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark
Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits
Differential Revision: http://reviews.llvm.org/D16471
llvm-svn: 258861
error: field 'CCMgr' will be initialized after field 'IndirectStubsMgr' [-Werror,-Wreorder]
: DL(TM.createDataLayout()), CCMgr(std::move(CCMgr)),
llvm-svn: 258354
they're needed.
Prior to this patch objects were loaded (via RuntimeDyld::loadObject) when they
were added to the ObjectLinkingLayer, but were not relocated and finalized until
a symbol address was requested. In the interim, another object could be loaded
and finalized with the same memory manager, causing relocation/finalization of
the first object to fail (as the first finalization call may have marked the
allocated memory for the first object read-only).
By deferring the loadObject call (and subsequent memory allocations) until an
object file is needed we can avoid prematurely finalizing memory.
llvm-svn: 258185
The cases of this switch are all perfectly regular (except for the first case).
A macro is more readable here.
Thanks to Dave Blaikie for the suggestion.
llvm-svn: 257951
classes.
OrcRemoteTargetClient::RCMemoryManager will now register EH frames with the
server automatically. This allows remote-execution of code that uses exceptions.
llvm-svn: 257816
The new ORC remote-JITing support provides a superset of the old code's
functionality, so we can replace the old stuff. As a bonus, a couple of
previously XFAILed tests have started passing.
llvm-svn: 257343
This patch adds utilities to ORC for managing a remote JIT target. It consists
of:
1. A very primitive RPC system for making calls over a byte-stream. See
RPCChannel.h, RPCUtils.h.
2. An RPC API defined in the above system for managing memory, looking up
symbols, creating stubs, etc. on a remote target. See OrcRemoteTargetRPCAPI.h.
3. An interface for creating high-level JIT components (memory managers,
callback managers, stub managers, etc.) that operate over the RPC API. See
OrcRemoteTargetClient.h.
4. A helper class for building servers that can handle the RPC calls. See
OrcRemoteTargetServer.h.
The system is designed to work neatly with the existing ORC components and
functionality. In particular, the ORC callback API (and consequently the
CompileOnDemandLayer) is supported, enabling lazy compilation of remote code.
Assuming this doesn't trigger any builder failures, a follow-up patch will be
committed which tests these utilities by using them to replace LLI's existing
remote-JITing demo code.
llvm-svn: 257305
RuntimeDyld::MemoryManager.
The RuntimeDyld::MemoryManager::reserveAllocationSpace method is called when
object files are loaded, and gives clients a chance to pre-allocate memory for
all segments. Previously only the size of each segment (code, ro-data, rw-data)
was supplied but not the alignment. This hasn't caused any problems so far, as
most clients allocate via the MemoryBlock interface which returns page-aligned
blocks. Adding alignment arguments enables finer grained allocation while still
satisfying alignment restrictions.
llvm-svn: 257294
This class is turning into a useful interface, rather than an implementation
detail, so I'm dropping the 'Base' suffix.
No functional change.
llvm-svn: 254693
The needed lld matching changes to be submitted immediately next,
but this revision will cause lld failures with this alone which is expected.
This removes the eating of the error in Archive::Child::getSize() when the characters
in the size field in the archive header for the member is not a number. To do this we
have all of the needed methods return ErrorOr to push them up until we get out of lib.
Then the tools and can handle the error in whatever way is appropriate for that tool.
So the solution is to plumb all the ErrorOr stuff through everything that touches archives.
This include its iterators as one can create an Archive object but the first or any other
Child object may fail to be created due to a bad size field in its header.
Thanks to Lang Hames on the changes making child_iterator contain an
ErrorOr<Child> instead of a Child and the needed changes to ErrorOr.h to add
operator overloading for * and -> .
We don’t want to use llvm_unreachable() as it calls abort() and is produces a “crash”
and using report_fatal_error() to move the error checking will cause the program to
stop, neither of which are really correct in library code. There are still some uses of
these that should be cleaned up in this library code for other than the size field.
The test cases use archives with text files so one can see the non-digit character,
in this case a ‘%’, in the size field.
These changes will require corresponding changes to the lld project. That will be
committed immediately after this change. But this revision will cause lld failures
with this alone which is expected.
llvm-svn: 252192
Bypassing LLVM for this has a number of benefits:
1) Laziness support becomes asm-syntax agnostic (previously lazy jitting didn't
work on Windows as the resolver block was in Darwin asm).
2) For cross-process JITs, it allows resolver blocks and trampolines to be
emitted directly in the target process, reducing cross process traffic.
3) It should be marginally faster.
llvm-svn: 251933
in the size field in the archive header for the member is not a number. To do this we
have all of the needed methods return ErrorOr to push them up until we get out of lib.
Then the tools and can handle the error in whatever way is appropriate for that tool.
So the solution is to plumb all the ErrorOr stuff through everything that touches archives.
This include its iterators as one can create an Archive object but the first or any other
Child object may fail to be created due to a bad size field in its header.
Thanks to Lang Hames on the changes making child_iterator contain an
ErrorOr<Child> instead of a Child and the needed changes to ErrorOr.h to add
operator overloading for * and -> .
We don’t want to use llvm_unreachable() as it calls abort() and is produces a “crash”
and using report_fatal_error() to move the error checking will cause the program to
stop, neither of which are really correct in library code. There are still some uses of
these that should be cleaned up in this library code for other than the size field.
Also corrected the code where the size gets us to the “at the end of the archive”
which is OK but past the end of the archive will return object_error::parse_failed now.
The test cases use archives with text files so one can see the non-digit character,
in this case a ‘%’, in the size field.
llvm-svn: 250906
memory, rather than representing the stubs in IR. Update the CompileOnDemand
layer to use this functionality.
Directly emitting stubs is much cheaper than building them in IR and codegen'ing
them (see below). It also plays well with remote JITing - stubs can be emitted
directly in the target process, rather than having to send them over the wire.
The downsides are:
(1) Care must be taken when resolving symbols, as stub symbols are held in a
separate symbol table. This is only a problem for layer writers and other
people using this API directly. The CompileOnDemand layer hides this detail.
(2) Aliases of function stubs can't be symbolic any more (since there's no
symbol definition in IR), but must be converted into a constant pointer
expression. This means that modules containing aliases of stubs cannot be
cached. In practice this is unlikely to be a problem: There's no benefit to
caching such a module anyway.
On balance I think the extra performance is more than worth the trade-offs: In a
simple stress test with 10000 dummy functions requiring stubs and a single
executed "hello world" main function, directly emitting stubs reduced user time
for JITing / executing by over 90% (1.5s for IR stubs vs 0.1s for direct
emission).
llvm-svn: 250712
This allows modules containing aliases to be lazily jit'd. Previously these
failed with missing symbol errors because the aliases weren't cloned from the
original module.
llvm-svn: 249481
Summary:
Replace getDataLayout() with a createDataLayout() method to make
explicit that it is intended to create a DataLayout only and not
accessing it for other purpose.
This change is the last of a series of commits dedicated to have a
single DataLayout during compilation by using always the one owned
by the module.
Reviewers: echristo
Subscribers: jholewinski, llvm-commits, rafael, yaron.keren
Differential Revision: http://reviews.llvm.org/D11103
(cherry picked from commit 5609fc56bca971e5a7efeaa6ca4676638eaec5ea)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 243114
This reverts commit 0f720d984f419c747709462f7476dff962c0bc41.
It breaks clang too badly, I need to prepare a proper patch for clang
first.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 243089
Summary:
Replace getDataLayout() with a createDataLayout() method to make
explicit that it is intended to create a DataLayout only and not
accessing it for other purpose.
This change is the last of a series of commits dedicated to have a
single DataLayout during compilation by using always the one owned
by the module.
Reviewers: echristo
Subscribers: jholewinski, llvm-commits, rafael, yaron.keren
Differential Revision: http://reviews.llvm.org/D11103
(cherry picked from commit 5609fc56bca971e5a7efeaa6ca4676638eaec5ea)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 243083
Summary:
This change is part of a series of commits dedicated to have a single
DataLayout during compilation by using always the one owned by the
module.
The ExecutionEngine will act as an exception and will be unsafe to
be reused across context. We don't enforce this rule but undefined
behavior can occurs if the user tries to do it.
Reviewers: lhames
Subscribers: echristo, llvm-commits, rafael, yaron.keren
Differential Revision: http://reviews.llvm.org/D11110
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 242414
Summary:
This change is part of a series of commits dedicated to have a single
DataLayout during compilation by using always the one owned by the
module.
The ExecutionEngine will act as an exception and will be unsafe to
be reused across context. We don't enforce this rule but undefined
behavior can occurs if the user tries to do it.
Reviewers: lhames
Subscribers: echristo, llvm-commits, rafael, yaron.keren
Differential Revision: http://reviews.llvm.org/D11110
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 242387
Summary:
This is an implementation of RuntimeDyld::SymbolResolver that simply
rejects all resolution requests; useful for clients that do not have any
cross-object symbol references.
Reviewers: lhames
Reviewed By: lhames
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D10455
llvm-svn: 240288
`LLVM_ENABLE_MODULES` builds sometimes fail because `Intrinsics.td`
needs to regenerate `Instrinsics.h` before anyone can include anything
from the LLVM_IR module. Represent the dependency explicitly to prevent
that.
llvm-svn: 239796
into partitions. Also, add an option to clone stub definitions (not just decls)
into partitions: these definitions could be inlined in some places to avoid the
overhead of calling via the stub.
Found by inspection - no test case yet, although I plan to add a unit test for
this once the CompileOnDemand layer refactoring settles down.
llvm-svn: 239640
and avoid cloning unused decls into every partition.
Module partitioning showed up as a source of significant overhead when I
profiled some trivial test cases. Avoiding the overhead of partitionging
for uncalled functions helps to mitigate this.
This change also means that it is no longer necessary to have a
LazyEmittingLayer underneath the CompileOnDemand layer, since the
CompileOnDemandLayer will not extract or emit function bodies until they are
called.
llvm-svn: 236465
Many of the callers already have the pointer type anyway, and for the
couple of callers that don't it's pretty easy to call PointerType::get
on the pointee type and address space.
This avoids LLParser from using PointerType::getElementType when parsing
GlobalAliases from IR.
llvm-svn: 236160
the function body.
This is necessary for correctness when lazily compiling.
Also, flesh out the Orc unit test infrastructure slightly, and add a unit test
for this.
llvm-svn: 235347
The patch is generated using clang-tidy misc-use-override check.
This command was used:
tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py \
-checks='-*,misc-use-override' -header-filter='llvm|clang' \
-j=32 -fix -format
http://reviews.llvm.org/D8925
llvm-svn: 234679
use these to add support for C++ static ctors/dtors to the Orc-lazy JIT in LLI.
Replace the trivial_retval_1 regression test - the new 'hello' test is covering
strictly more code.
llvm-svn: 233885
MCJIT.
This patch decouples the two responsibilities of the RTDyldMemoryManager class,
memory management and symbol resolution, into two new classes:
RuntimeDyld::MemoryManager and RuntimeDyld::SymbolResolver.
The symbol resolution interface is modified slightly, from:
uint64_t getSymbolAddress(const std::string &Name);
to:
RuntimeDyld::SymbolInfo findSymbol(const std::string &Name);
The latter passes symbol flags along with symbol addresses, allowing RuntimeDyld
and others to reason about non-strong/non-exported symbols.
The memory management interface removes the following method:
void notifyObjectLoaded(ExecutionEngine *EE,
const object::ObjectFile &) {}
as it is not related to memory management. (Note: Backwards compatibility *is*
maintained for this method in MCJIT and OrcMCJITReplacement, see below).
The RTDyldMemoryManager class remains in-tree for backwards compatibility.
It inherits directly from RuntimeDyld::SymbolResolver, and indirectly from
RuntimeDyld::MemoryManager via the new MCJITMemoryManager class, which
just subclasses RuntimeDyld::MemoryManager and reintroduces the
notifyObjectLoaded method for backwards compatibility).
The EngineBuilder class retains the existing method:
EngineBuilder&
setMCJITMemoryManager(std::unique_ptr<RTDyldMemoryManager> mcjmm);
and includes two new methods:
EngineBuilder&
setMemoryManager(std::unique_ptr<MCJITMemoryManager> MM);
EngineBuilder&
setSymbolResolver(std::unique_ptr<RuntimeDyld::SymbolResolver> SR);
Clients should use EITHER:
A single call to setMCJITMemoryManager with an RTDyldMemoryManager.
OR (exclusive)
One call each to each of setMemoryManager and setSymbolResolver.
This patch should be fully compatible with existing uses of RTDyldMemoryManager.
If it is not it should be considered a bug, and the patch either fixed or
reverted.
If clients find the new API to be an improvement the goal will be to deprecate
and eventually remove the RTDyldMemoryManager class in favor of the new classes.
llvm-svn: 233509
Summary:
DataLayout keeps the string used for its creation.
As a side effect it is no longer needed in the Module.
This is "almost" NFC, the string is no longer
canonicalized, you can't rely on two "equals" DataLayout
having the same string returned by getStringRepresentation().
Get rid of DataLayoutPass: the DataLayout is in the Module
The DataLayout is "per-module", let's enforce this by not
duplicating it more than necessary.
One more step toward non-optionality of the DataLayout in the
module.
Make DataLayout Non-Optional in the Module
Module->getDataLayout() will never returns nullptr anymore.
Reviewers: echristo
Subscribers: resistor, llvm-commits, jholewinski
Differential Revision: http://reviews.llvm.org/D7992
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 231270
This patch replaces most of the Orc indirection utils API with a new class:
JITCompileCallbackManager, which creates and manages JIT callbacks.
Exposing this functionality directly allows the user to create callbacks that
are associated with user supplied compilation actions. For example, you can
create a callback to lazyily IR-gen something from an AST. (A kaleidoscope
example demonstrating this will be committed shortly).
This patch also refactors the CompileOnDemand layer to use the
JITCompileCallbackManager API.
llvm-svn: 229461
This allows IDEs to recognize the entire set of header files for
each of the core LLVM projects.
Differential Revision: http://reviews.llvm.org/D7526
Reviewed By: Chris Bieneman
llvm-svn: 228798
This patch refactors a key piece of the Orc APIs: It removes the
*::getSymbolAddress and *::lookupSymbolAddressIn methods, which returned target
addresses (uint64_ts), and replaces them with *::findSymbol and *::findSymbolIn
respectively, which return instances of the new JITSymbol type. Unlike the old
methods, calling findSymbol or findSymbolIn does not cause the symbol to be
immediately materialized when found. Instead, the symbol will be materialized
if/when the getAddress method is called on the returned JITSymbol. This allows
us to query for the existence of symbols without actually materializing them. In
the future I expect more information to be attached to the JITSymbol class, for
example whether the returned symbol is a weak or strong definition. This will
allow us to properly handle weak symbols and multiple definitions.
llvm-svn: 228557
ObjectLinkingLayer.
There are a two of overloads for addObject, one of which transfers ownership of
the underlying buffer to OrcMCJITReplacement. This commit makes the ownership
transfering version pass ownership down to the ObjectLinkingLayer in order to
prevent the issue described in r227778.
I think this commit will fix the sanitizer bot failures that necessitated the
removal of the load-object-a.ll regression test in r227785, so I'm reinstating
that test.
llvm-svn: 227845
ExecutionEngine's Modules list instead.
This makes the owned modules visibile to ExecutionEngine. In particular,
it is required for ExecutionEngine::runStaticConstructorsAndDestructors to
work.
Regression tests for Orc (which test this issue) will be committed shortly.
llvm-svn: 227779
derived classes.
Since global data alignment, layout, and mangling is often based on the
DataLayout, move it to the TargetMachine. This ensures that global
data is going to be layed out and mangled consistently if the subtarget
changes on a per function basis. Prior to this all targets(*) have
had subtarget dependent code moved out and onto the TargetMachine.
*One target hasn't been migrated as part of this change: R600. The
R600 port has, as a subtarget feature, the size of pointers and
this affects global data layout. I've currently hacked in a FIXME
to enable progress, but the port needs to be updated to either pass
the 64-bitness to the TargetMachine, or fix the DataLayout to
avoid subtarget dependent features.
llvm-svn: 227113
This patch adds a new set of JIT APIs to LLVM. The aim of these new APIs is to
cleanly support a wider range of JIT use cases in LLVM, and encourage the
development and contribution of re-usable infrastructure for LLVM JIT use-cases.
These APIs are intended to live alongside the MCJIT APIs, and should not affect
existing clients.
Included in this patch:
1) New headers in include/llvm/ExecutionEngine/Orc that provide a set of
components for building JIT infrastructure.
Implementation code for these headers lives in lib/ExecutionEngine/Orc.
2) A prototype re-implementation of MCJIT (OrcMCJITReplacement) built out of the
new components.
3) Minor changes to RTDyldMemoryManager needed to support the new components.
These changes should not impact existing clients.
4) A new flag for lli, -use-orcmcjit, which will cause lli to use the
OrcMCJITReplacement class as its underlying execution engine, rather than
MCJIT itself.
Tests to follow shortly.
Special thanks to Michael Ilseman, Pete Cooper, David Blaikie, Eric Christopher,
Justin Bogner, and Jim Grosbach for extensive feedback and discussion.
llvm-svn: 226940
Lang Hames