Commit Graph

536 Commits

Author SHA1 Message Date
Lang Hames 7c4814306d [ORC] Simplify LLJIT::Create by removing the ExecutionSession parameter.
The Create method can just construct the ExecutionSession, rather than having the
client pass it in.

llvm-svn: 341872
2018-09-10 22:08:57 +00:00
Lang Hames 37a66413c1 [ORC] Add an addObjectFile method to LLJIT.
The addObjectFile method adds the given object file to the JIT session, making
its code available for execution.

Support for the -extra-object flag is added to lli when operating in
-jit-kind=orc-lazy mode to support testing of this feature.

llvm-svn: 340870
2018-08-28 20:20:31 +00:00
Lang Hames d5f56c5979 [ORC] Rename VSO to JITDylib.
VSO was a little close to VDSO (an acronym on Linux for Virtual Dynamic Shared
Object) for comfort. It also risks giving the impression that instances of this
class could be shared between ExecutionSessions, which they can not.

JITDylib seems moderately less confusing, while still hinting at how this
class is intended to be used, i.e. as a JIT-compiled stand-in for a dynamic
library (code that would have been a dynamic library if you had wanted to
compile it ahead of time).

llvm-svn: 340084
2018-08-17 21:18:18 +00:00
Andres Freund 376a3d3659 Add PerfJITEventListener for perf profiling support.
This new JIT event listener supports generating profiling data for
the linux 'perf' profiling tool, allowing it to generate function and
instruction level profiles.

Currently this functionality is not enabled by default, but must be
enabled with LLVM_USE_PERF=yes.  Given that the listener has no
dependencies, it might be sensible to enable by default once the
initial issues have been shaken out.

I followed existing precedent in registering the listener by default
in lli. Should there be a decision to enable this by default on linux,
that should probably be changed.

Please note that until https://reviews.llvm.org/D47343 is resolved,
using this functionality with mcjit rather than orcjit will not
reliably work.

Disregarding the previous comment, here's an example:

$ cat /tmp/expensive_loop.c

bool stupid_isprime(uint64_t num)
{
        if (num == 2)
                return true;
        if (num < 1 || num % 2 == 0)
                return false;
        for(uint64_t i = 3; i < num / 2; i+= 2) {
                if (num % i == 0)
                        return false;
        }
        return true;
}

int main(int argc, char **argv)
{
        int numprimes = 0;

        for (uint64_t num = argc; num < 100000; num++)
        {
                if (stupid_isprime(num))
                        numprimes++;
        }

        return numprimes;
}

$ clang -ggdb -S -c -emit-llvm /tmp/expensive_loop.c -o
/tmp/expensive_loop.ll

$ perf record -o perf.data -g -k 1 ./bin/lli -jit-kind=mcjit /tmp/expensive_loop.ll 1

$ perf inject --jit -i perf.data -o perf.jit.data

$ perf report -i perf.jit.data
-   92.59%  lli      jitted-5881-2.so                   [.] stupid_isprime
     stupid_isprime
     main
     llvm::MCJIT::runFunction
     llvm::ExecutionEngine::runFunctionAsMain
     main
     __libc_start_main
     0x4bf6258d4c544155
+    0.85%  lli      ld-2.27.so                         [.] do_lookup_x

And line-level annotations also work:
       │              for(uint64_t i = 3; i < num / 2; i+= 2) {
       │1 30:   movq   $0x3,-0x18(%rbp)
  0.03 │1 38:   mov    -0x18(%rbp),%rax
  0.03 │        mov    -0x10(%rbp),%rcx
       │        shr    $0x1,%rcx
  3.63 │     ┌──cmp    %rcx,%rax
       │     ├──jae    6f
       │     │                if (num % i == 0)
  0.03 │     │  mov    -0x10(%rbp),%rax
       │     │  xor    %edx,%edx
 89.00 │     │  divq   -0x18(%rbp)
       │     │  cmp    $0x0,%rdx
  0.22 │     │↓ jne    5f
       │     │                        return false;
       │     │  movb   $0x0,-0x1(%rbp)
       │     │↓ jmp    73
       │     │        }
  3.22 │1 5f:│↓ jmp    61
       │     │        for(uint64_t i = 3; i < num / 2; i+= 2) {

Subscribers: mgorny, llvm-commits

Differential Revision: https://reviews.llvm.org/D44892

llvm-svn: 337789
2018-07-24 00:54:06 +00:00
Lang Hames adae9bfa24 [ORC] Verify modules when running LLLazyJIT in LLI, and deal with fallout.
The verifier identified several modules that were broken due to incorrect
linkage on declarations. To fix this, CompileOnDemandLayer2::extractFunction
has been updated to change decls to external linkage.

llvm-svn: 336150
2018-07-02 22:30:18 +00:00
Lang Hames 6a94134b11 [ORC] Add LLJIT and LLLazyJIT, and replace OrcLazyJIT in LLI with LLLazyJIT.
LLJIT is a prefabricated ORC based JIT class that is meant to be the go-to
replacement for MCJIT. Unlike OrcMCJITReplacement (which will continue to be
supported) it is not API or bug-for-bug compatible, but targets the same
use cases: Simple, non-lazy compilation and execution of LLVM IR.

LLLazyJIT extends LLJIT with support for function-at-a-time lazy compilation,
similar to what was provided by LLVM's original (now long deprecated) JIT APIs.

This commit also contains some simple utility classes (CtorDtorRunner2,
LocalCXXRuntimeOverrides2, JITTargetMachineBuilder) to support LLJIT and
LLLazyJIT.

Both of these classes are works in progress. Feedback from JIT clients is very
welcome!

llvm-svn: 335670
2018-06-26 21:35:48 +00:00
Lang Hames bd0cb787d0 [ORC] Update JITCompileCallbackManager to support multi-threaded code.
Previously JITCompileCallbackManager only supported single threaded code. This
patch embeds a VSO (see include/llvm/ExecutionEngine/Orc/Core.h) in the callback
manager. The VSO ensures that the compile callback is only executed once and that
the resulting address cached for use by subsequent re-entries.

llvm-svn: 333490
2018-05-30 01:57:45 +00:00
Lang Hames d261e1258c [ORC] Rewrite the VSO symbol table yet again. Update related utilities.
VSOs now track dependencies for materializing symbols. Each symbol must have its
dependencies registered with the VSO prior to finalization. Usually this will
involve registering the dependencies returned in
AsynchronousSymbolQuery::ResolutionResults for queries made while linking the
symbols being materialized.

Queries against symbols are notified that a symbol is ready once it and all of
its transitive dependencies are finalized, allowing compilation work to be
broken up and moved between threads without queries returning until their
symbols fully safe to access / execute.

Related utilities (VSO, MaterializationUnit, MaterializationResponsibility) are
updated to support dependence tracking and more explicitly track responsibility
for symbols from the point of definition until they are finalized.

llvm-svn: 332541
2018-05-16 22:24:30 +00:00
Nicola Zaghen d34e60ca85 Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.

In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.

Differential Revision: https://reviews.llvm.org/D43624

llvm-svn: 332240
2018-05-14 12:53:11 +00:00
Nico Weber 432a38838d IWYU for llvm-config.h in llvm, additions.
See r331124 for how I made a list of files missing the include.
I then ran this Python script:

    for f in open('filelist.txt'):
        f = f.strip()
        fl = open(f).readlines()

        found = False
        for i in xrange(len(fl)):
            p = '#include "llvm/'
            if not fl[i].startswith(p):
                continue
            if fl[i][len(p):] > 'Config':
                fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
                found = True
                break
        if not found:
            print 'not found', f
        else:
            open(f, 'w').write(''.join(fl))

and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.

No intended behavior change.

llvm-svn: 331184
2018-04-30 14:59:11 +00:00
Jonas Devlieghere 7b5fa24bcd [lli] Fix syntax error: missing ';'
Fixes build issue on the windows bots:
  error C2143: syntax error: missing ';'

llvm-svn: 330538
2018-04-22 08:35:00 +00:00
Jonas Devlieghere 4a2863ccbc [lli] Make error handling more consistent.
Makes error handling more consistent by using the helpers in support.

llvm-svn: 330537
2018-04-22 08:02:11 +00:00
Rui Ueyama 197194b6c9 Define InitLLVM to do common initialization all at once.
We have a few functions that virtually all command wants to run on
process startup/shutdown. This patch adds InitLLVM class to do that
all at once, so that we don't need to copy-n-paste boilerplate code
to each llvm command's main() function.

Differential Revision: https://reviews.llvm.org/D45602

llvm-svn: 330046
2018-04-13 18:26:06 +00:00
David Blaikie 4333f9700d Rename *CommandFlags.def to *CommandFlags.inc
These aren't the .def style files used in LLVM that require a macro
defined before their inclusion - they're just basic non-modular includes
to stamp out command line flag variables.

llvm-svn: 329840
2018-04-11 18:49:37 +00:00
Lang Hames 3fdfc04e53 [ORC] Create a new SymbolStringPool by default in ExecutionSession constructor.
This makes the common case of constructing an ExecutionSession tidier.

llvm-svn: 329013
2018-04-02 20:57:56 +00:00
Lang Hames 5721ee48a2 [ORC] Re-apply r327566 with a fix for test-global-ctors.ll.
Also clang-formats the patch, which I should have done the first time around.

llvm-svn: 327594
2018-03-15 00:30:14 +00:00
Reid Kleckner c7fd1540b3 Revert "[ORC] Switch from shared_ptr to unique_ptr for addModule methods."
This reverts commit r327566, it breaks
test/ExecutionEngine/OrcMCJIT/test-global-ctors.ll.

The test doesn't crash with a stack trace, unfortunately. It merely
returns 1 as the exit code.

ASan didn't produce a report, and I reproduced this on my Linux machine
and Windows box.

llvm-svn: 327576
2018-03-14 21:32:34 +00:00
Lang Hames 7bea03c2bb [ORC] Switch from shared_ptr to unique_ptr for addModule methods.
Layer implementations typically mutate module state, and this is better
reflected by having layers own the Module they are operating on.

llvm-svn: 327566
2018-03-14 20:29:45 +00:00
Lang Hames 1cd3dd0bd8 [ORC] Consolidate RTDyldObjectLinkingLayer GetMemMgr and GetResolver into a
unified GetResources callback.

Having a single 'GetResources' callback will simplify adding new resources in
the future.

llvm-svn: 325180
2018-02-14 22:13:02 +00:00
Lang Hames e833fe8ec3 [ORC] Switch to shared_ptr ownership for AsynchronousSymbolQueries.
Queries need to stay alive until each owner has set the values they are
responsible for.

llvm-svn: 325179
2018-02-14 22:12:56 +00:00
Lang Hames 0976cee8e9 [ORC] Remove Layer handles from the layer concept.
Handles were returned by addModule and used as keys for removeModule,
findSymbolIn, and emitAndFinalize. Their job is now subsumed by VModuleKeys,
which simplify resource management by providing a consistent handle across all
layers.

llvm-svn: 324700
2018-02-09 02:30:40 +00:00
Lang Hames e33798f53d [ORC] Remove some unused lambda captures.
llvm-svn: 324410
2018-02-06 21:52:46 +00:00
Lang Hames 4b546c9145 [ORC] Start migrating ORC layers to use the new ORC Core.h APIs.
In particular this patch switches RTDyldObjectLinkingLayer to use
orc::SymbolResolver and threads the requried changse (ExecutionSession
references and VModuleKeys) through the existing layer APIs.

The purpose of the new resolver interface is to improve query performance and
better support parallelism, both in JIT'd code and within the compiler itself.

The most visibile change is switch of the <Layer>::addModule signatures from:

Expected<Handle> addModule(std::shared_ptr<ModuleType> Mod,
                           std::shared_ptr<JITSymbolResolver> Resolver)

to:

Expected<Handle> addModule(VModuleKey K, std::shared_ptr<ModuleType> Mod);

Typical usage of addModule will now look like:

auto K = ES.allocateVModuleKey();
Resolvers[K] = createSymbolResolver(...);
Layer.addModule(K, std::move(Mod));

See the BuildingAJIT tutorial code for example usage.

llvm-svn: 324405
2018-02-06 21:25:11 +00:00
Lang Hames b72f48452c [ORC] Re-apply r322913 with a fix for a read-after-free error.
ExternalSymbolMap now stores the string key (rather than using a StringRef),
as the object file backing the key may be removed at any time.

llvm-svn: 323001
2018-01-19 22:24:13 +00:00
Lang Hames 44efd042a2 [ORC] Revert r322913 while I investigate an ASan failure.
llvm-svn: 322914
2018-01-19 01:40:26 +00:00
Lang Hames 817df9fa0c [ORC] Redesign the JITSymbolResolver interface to support bulk queries.
Bulk queries reduce IPC/RPC overhead for cross-process JITing and expose
opportunities for parallel compilation.

The two new query methods are lookupFlags, which finds the flags for each of a
set of symbols; and lookup, which finds the address and flags for each of a
set of symbols. (See doxygen comments for more details.)

The existing JITSymbolResolver class is renamed LegacyJITSymbolResolver, and
modified to extend the new JITSymbolResolver class using the following scheme:

- lookupFlags is implemented by calling findSymbolInLogicalDylib for each of the
symbols, then returning the result of calling getFlags() on each of these
symbols. (Importantly: lookupFlags does NOT call getAddress on the returned
symbols, so lookupFlags will never trigger materialization, and lookupFlags will
never call findSymbol, so only symbols that are part of the logical dylib will
return results.)

- lookup is implemented by calling findSymbolInLogicalDylib for each symbol and
falling back to findSymbol if findSymbolInLogicalDylib returns a null result.
Assuming a symbol is found its getAddress method is called to materialize it and
the result (if getAddress succeeds) is stored in the result map, or the error
(if getAddress fails) is returned immediately from lookup. If any symbol is not
found then lookup returns immediately with an error.

This change will break any out-of-tree derivatives of JITSymbolResolver. This
can be fixed by updating those classes to derive from LegacyJITSymbolResolver
instead.

llvm-svn: 322913
2018-01-19 01:12:40 +00:00
Craig Topper 243f20f117 [lli] Make lli support -mcpu=native for CPU autodetection
llc, opt, and clang can all autodetect the CPU and supported features. lli cannot as far as I could tell.

This patch uses the getCPUStr() and introduces a new getCPUFeatureList() and uses those in lli in place of MCPU and MAttrs.

Ideally, we would merge getCPUFeatureList and getCPUFeatureStr, but opt and llc need a string and lli wanted a list. Maybe we should just return the SubtargetFeature object and let the caller decide what it needs?

Differential Revision: https://reviews.llvm.org/D41833

llvm-svn: 322100
2018-01-09 18:14:18 +00:00
David Blaikie c14bfec487 Rename CommandFlags.h -> CommandFlags.def
Since this isn't a real header - it includes static functions and had
external linkage variables (though this change makes them static, since
that's what they should be) so can't be included more than once in a
program.

llvm-svn: 319082
2017-11-27 19:43:58 +00:00
Lang Hames 9e68b734d6 [ORC] Refactor OrcRemoteTarget code to expose its RPC API, reduce
code duplication in the client, and improve error propagation.

This patch moves the OrcRemoteTarget rpc::Function declarations from
OrcRemoteTargetRPCAPI into their own namespaces under llvm::orc::remote so that
they can be used in new contexts (in particular, a remote-object-file adapter
layer that I will commit shortly).

Code duplication in OrcRemoteTargetClient (especially in loops processing the
code, rw-data and ro-data allocations) is removed by moving the loop bodies
into their own functions.

Error propagation is (slightly) improved by adding an ErrorReporter functor to
the OrcRemoteTargetClient -- Errors that can't be returned (because they occur
in destructors, or behind stable APIs that don't provide error returns) can be
sent to the ErrorReporter instead. Some methods in the Client API are also
changed to make better use of the Expected class: returning Expected<T>s rather
than returning Errors and taking T&s to store the results.

llvm-svn: 312500
2017-09-04 20:54:46 +00:00
NAKAMURA Takumi a1e97a77f5 Untabify.
llvm-svn: 311875
2017-08-28 06:47:47 +00:00
David Blaikie 32512e161f Orc: PR33769: Don't rely on comparisons with default constructed iterators
llvm-svn: 310729
2017-08-11 16:38:28 +00:00
Rafael Espindola 79e238afee Delete Default and JITDefault code models
IMHO it is an antipattern to have a enum value that is Default.

At any given piece of code it is not clear if we have to handle
Default or if has already been mapped to a concrete value. In this
case in particular, only the target can do the mapping and it is nice
to make sure it is always done.

This deletes the two default enum values of CodeModel and uses an
explicit Optional<CodeModel> when it is possible that it is
unspecified.

llvm-svn: 309911
2017-08-03 02:16:21 +00:00
Hiroshi Inoue 0ca79dcf4b fix typos in comments; NFC
llvm-svn: 307626
2017-07-11 06:04:59 +00:00
Lang Hames 4ce98662e7 [ORC] Errorize the ORC APIs.
This patch updates the ORC layers and utilities to return and propagate
llvm::Errors where appropriate. This is necessary to allow ORC to safely handle
error cases in cross-process and remote JITing.

llvm-svn: 307350
2017-07-07 02:59:13 +00:00
Lang Hames 5b51816020 [Orc] Remove the memory manager argument to addModule, and de-templatize the
symbol resolver argument.

De-templatizing the symbol resolver is part of the ongoing simplification of
ORC layer API.

Removing the memory management argument (and delegating construction of memory
managers for RTDyldObjectLinkingLayer to a functor passed in to the constructor)
allows us to build JITs whose base object layers need not be compatible with
RTDyldObjectLinkingLayer's memory mangement scheme. For example, a 'remote
object layer' that sends fully relocatable objects directly to the remote does
not need a memory management scheme at all (that will be handled by the remote).

llvm-svn: 307058
2017-07-04 04:42:30 +00:00
Lang Hames cd9d49b605 [ORC] Re-apply r306166 and r306168 with fix for regression test.
llvm-svn: 306182
2017-06-23 23:25:28 +00:00
Rafael Espindola f6242c3e90 This reverts commit r306166 and r306168.
Revert "[ORC] Remove redundant semicolons from DEFINE_SIMPLE_CONVERSION_FUNCTIONS uses."
Revert "[ORC] Move ORC IR layer interface from addModuleSet to addModule and fix the module type as std::shared_ptr<Module>."

They broke ExecutionEngine/OrcMCJIT/test-global-ctors.ll on linux.

llvm-svn: 306176
2017-06-23 22:50:24 +00:00
Lang Hames 2c19c1be56 [ORC] Move ORC IR layer interface from addModuleSet to addModule and fix the
module type as std::shared_ptr<Module>.

llvm-svn: 306166
2017-06-23 21:45:29 +00:00
Lang Hames 266202236f [ORC] Switch the object layer API from addObjectSet to addObject (singular), and
move the ObjectCache from the IRCompileLayer to SimpleCompiler.

This is the first in a series of patches aimed at cleaning up and improving the
robustness and performance of the ORC APIs.

llvm-svn: 306058
2017-06-22 21:06:54 +00:00
Eugene Zelenko f292a2feca [ExecutionEngine] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 305760
2017-06-19 23:37:52 +00:00
Lang Hames c936ac7f37 [ExecutionEngine] Make RuntimeDyld::MemoryManager responsible for tracking EH
frames.

RuntimeDyld was previously responsible for tracking allocated EH frames, but it
makes more sense to have the RuntimeDyld::MemoryManager track them (since the
frames are allocated through the memory manager, and written to memory owned by
the memory manager). This patch moves the frame tracking into
RTDyldMemoryManager, and changes the deregisterFrames method on
RuntimeDyld::MemoryManager from:

void deregisterEHFrames(uint8_t *Addr, uint64_t LoadAddr, size_t Size);

to:

void deregisterEHFrames();

Separating this responsibility will allow ORC to continue to throw the
RuntimeDyld instances away post-link (saving a few dozen bytes per lazy
function) while properly deregistering frames when modules are unloaded.

This patch also updates ORC to call deregisterEHFrames when modules are
unloaded. This fixes a bug where an exception that tears down the JIT can then
unwind through dangling EH frames that have been deallocated but not
deregistered, resulting in UB.

For people using SectionMemoryManager this should be pretty much a no-op. For
people with custom allocators that override registerEHFrames/deregisterEHFrames,
you will now be responsible for tracking allocated EH frames.

Reviewed in https://reviews.llvm.org/D32829

llvm-svn: 302589
2017-05-09 21:32:18 +00:00
Serge Guelton 59a2d7b909 Module::getOrInsertFunction is using C-style vararg instead of variadic templates.
From a user prospective, it forces the use of an annoying nullptr to mark the end of the vararg, and there's not type checking on the arguments.
The variadic template is an obvious solution to both issues.

Differential Revision: https://reviews.llvm.org/D31070

llvm-svn: 299949
2017-04-11 15:01:18 +00:00
Diana Picus b050c7fbe0 Revert "Turn some C-style vararg into variadic templates"
This reverts commit r299925 because it broke the buildbots. See e.g.
http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15/builds/6008

llvm-svn: 299928
2017-04-11 10:07:12 +00:00
Serge Guelton 5fd75fb72e Turn some C-style vararg into variadic templates
Module::getOrInsertFunction is using C-style vararg instead of
variadic templates.

From a user prospective, it forces the use of an annoying nullptr
to mark the end of the vararg, and there's not type checking on the
arguments. The variadic template is an obvious solution to both
issues.

llvm-svn: 299925
2017-04-11 08:36:52 +00:00
Mehdi Amini db11fdfda5 Revert "Turn some C-style vararg into variadic templates"
This reverts commit r299699, the examples needs to be updated.

llvm-svn: 299702
2017-04-06 20:23:57 +00:00
Mehdi Amini 579540a8f7 Turn some C-style vararg into variadic templates
Module::getOrInsertFunction is using C-style vararg instead of
variadic templates.

From a user prospective, it forces the use of an annoying nullptr
to mark the end of the vararg, and there's not type checking on the
arguments. The variadic template is an obvious solution to both
issues.

Patch by: Serge Guelton <serge.guelton@telecom-bretagne.eu>

Differential Revision: https://reviews.llvm.org/D31070

llvm-svn: 299699
2017-04-06 20:09:31 +00:00
Lang Hames 67de5d24a9 [Orc] Rename ObjectLinkingLayer -> RTDyldObjectLinkingLayer.
The current ObjectLinkingLayer (now RTDyldObjectLinkingLayer) links objects
in-process using MCJIT's RuntimeDyld class. In the near future I hope to add new
object linking layers (e.g. a remote linking layer that links objects in the JIT
target process, rather than the client), so I'm renaming this class to be more
descriptive.

llvm-svn: 295636
2017-02-20 05:45:14 +00:00
NAKAMURA Takumi 689493bb12 Prune unused libdeps.
llvm-svn: 289060
2016-12-08 15:28:02 +00:00
Simon Pilgrim dae11f7aab Fix spelling mistakes in Tools/Tests comments. NFC.
Identified by Pedro Giffuni in PR27636.

llvm-svn: 287489
2016-11-20 13:31:13 +00:00
Chris Bieneman d7f71b5187 [CMake] lli-child-target depends on intrinsics gen
Messed up in r287420, it isn't just lli, but also but lli-child-target that need to depend on intrinsics_gen.

llvm-svn: 287421
2016-11-19 02:09:51 +00:00
Chris Bieneman 3bd0191c5b [CMake] lli depends on intrinsics_gen
ChildTarget.cpp has the following include chain:

llvm/ExecutionEngine/Orc/OrcABISupport.h
llvm/ExecutionEngine/Orc/IndirectionUtils.h
llvm/IR/IRBuilder.h
llvm/IR/ConstantFolder.h
llvm/IR/InstrTypes.h
llvm/IR/Attributes.h
llvm/IR/Attributes.gen

This means lli needs to depend on intrinsics_gen.

llvm-svn: 287420
2016-11-19 02:05:19 +00:00
Davide Italiano 8651144353 [lli] Prefer `exit(1)` to `return 1` for consistency.
llvm-svn: 287277
2016-11-17 22:59:13 +00:00
Davide Italiano da8e6b2ec7 [lli] Factor out error handling. NFCI.
llvm-svn: 287276
2016-11-17 22:58:13 +00:00
Lang Hames 1f2bf2d3e1 [ORC] Re-apply 286620 with fixes for the ErrorSuccess class.
llvm-svn: 286639
2016-11-11 21:42:09 +00:00
Lang Hames 4f734f254e [ORC] Revert r286620 while I investigate a bot failure.
llvm-svn: 286621
2016-11-11 19:46:46 +00:00
Lang Hames ae1fdddbc4 [ORC] Refactor the ORC RPC utilities to add some new features.
(1) Add support for function key negotiation.

The previous version of the RPC required both sides to maintain the same
enumeration for functions in the API. This means that any version skew between
the client and server would result in communication failure.

With this version of the patch functions (and serializable types) are defined
with string names, and the derived function signature strings are used to
negotiate the actual function keys (which are used for efficient call
serialization). This allows clients to connect to any server that supports a
superset of the API (based on the function signatures it supports).

(2) Add a callAsync primitive.

The callAsync primitive can be used to install a return value handler that will
run as soon as the RPC function's return value is sent back from the remote.

(3) Launch policies for RPC function handlers.

The new addHandler method, which installs handlers for RPC functions, takes two
arguments: (1) the handler itself, and (2) an optional "launch policy". When the
RPC function is called, the launch policy (if present) is invoked to actually
launch the handler. This allows the handler to be spawned on a background
thread, or added to a work list. If no launch policy is used, the handler is run
on the server thread itself. This should only be used for short-running
handlers, or entirely synchronous RPC APIs.

(4) Zero cost cross type serialization.

You can now define serialization from any type to a different "wire" type. For
example, this allows you to call an RPC function that's defined to take a
std::string while passing a StringRef argument. If a serializer from StringRef
to std::string has been defined for the channel type this will be used to
serialize the argument without having to construct a std::string instance.

This allows buffer reference types to be used as arguments to RPC calls without
requiring a copy of the buffer to be made.

llvm-svn: 286620
2016-11-11 19:42:44 +00:00
Teresa Johnson ad17679abd Split Bitcode/ReaderWriter.h into separate reader and writer headers
Summary:
Split ReaderWriter.h which contains the APIs into both the BitReader and
BitWriter libraries into BitcodeReader.h and BitcodeWriter.h.

This is to address Chandler's concern about sharing the same API header
between multiple libraries (BitReader and BitWriter). That concern is
why we create a single bitcode library in our downstream build of clang,
which led to r286297 being reverted as it added a dependency that
created a cycle only when there is a single bitcode library (not two as
in upstream).

Reviewers: mehdi_amini

Subscribers: dlj, mehdi_amini, llvm-commits

Differential Revision: https://reviews.llvm.org/D26502

llvm-svn: 286566
2016-11-11 05:34:58 +00:00
Davide Italiano b0e067b08d [tools] Unbreak the GCC build (workaround a GCC bug).
../tools/llvm-extract/llvm-extract.cpp: In function ‘int main(int, char**)’:
warning: ISO C++ forbids zero-size array ‘argv’ [-Wpedantic]

GCC reference bug https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61259

llvm-svn: 286396
2016-11-09 21:30:33 +00:00
Peter Collingbourne 7f00d0a125 Bitcode: Change the materializer interface to return llvm::Error.
Differential Revision: https://reviews.llvm.org/D26439

llvm-svn: 286382
2016-11-09 17:49:19 +00:00
Malcolm Parsons 06ac79c210 Fix Clang-tidy readability-redundant-string-cstr warnings
Reviewers: beanz, lattner, jlebar

Subscribers: jholewinski, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D26235

llvm-svn: 285832
2016-11-02 16:43:50 +00:00
Davide Italiano 543283bad2 [lli] Don't strip away const qualifier. Unbreak the gcc6 build.
llvm-svn: 285592
2016-10-31 16:21:48 +00:00
Lang Hames 1a2e656c67 [lli] Pass command line arguments in to the orc-lazy JIT.
This brings the LLI orc-lazy JIT's behavior more closely in-line with LLI's
mcjit bahavior.

llvm-svn: 285413
2016-10-28 16:52:34 +00:00
Mehdi Amini 732afdd09a Turn cl::values() (for enum) from a vararg function to using C++ variadic template
The core of the change is supposed to be NFC, however it also fixes
what I believe was an undefined behavior when calling:

 va_start(ValueArgs, Desc);

with Desc being a StringRef.

Differential Revision: https://reviews.llvm.org/D25342

llvm-svn: 283671
2016-10-08 19:41:06 +00:00
Lang Hames e98bc7af8b [ORC] Rename RPCChannel to RPCByteChannel. NFC.
llvm-svn: 281171
2016-09-11 18:41:05 +00:00
Lang Hames 38c7927b6f [ORC] Clone module flags metadata into the globals module in the
CompileOnDemandLayer.

Also contains a tweak to the orc-lazy jit in LLI to enable the test case.

llvm-svn: 280632
2016-09-04 17:53:30 +00:00
Lang Hames 6b21751ba9 [Orc] Simplify LogicalDylib and move it back inside CompileOnDemandLayer. Also
switch to using one indirect stub manager per logical dylib rather than one per
input module.

LogicalDylib is a helper class used by the CompileOnDemandLayer to manage
symbol resolution between modules during lazy compilation. In particular, it
ensures that internal symbols resolve correctly even in the case where multiple
input modules contain the same internal symbol name (which must to be promoted
to external hidden linkage so that functions in any given module can be split
out by lazy compilation). LogicalDylib's resolution scheme (before this commit)
required one stub-manager per input module. This made recompilation of functions
(by adding a module containing a new definition) difficult, as the stub manager
for any given symbol was bound to the module that supplied the original
definition. By using one stubs manager for the whole logical dylib symbols can
be more easily replaced, although support for doing this is not included in this
patch (it will be implemented in a follow up).

llvm-svn: 279952
2016-08-29 00:54:29 +00:00
Lang Hames 73976f622d [ORC] Re-apply r277896, removing bogus triples and datalayouts that broke tests
on linux last time.

llvm-svn: 277942
2016-08-06 22:36:26 +00:00
Nico Weber c893e603ab Revert r277896.
It breaks ExecutionEngine/OrcLazy/weak-function.ll on most bots.

Script:
--
...
--
Exit Code: 1

Command Output (stderr):
--
Could not find main function.

llvm-svn: 277907
2016-08-06 02:00:45 +00:00
Lang Hames 62a459603c [ORC] Add (partial) weak symbol support to the CompileOnDemand layer.
This adds partial support for weak functions to the CompileOnDemandLayer by
modifying the addLogicalModule method to check for existing stub definitions
before building a new stub for a weak function. This scheme is sufficient to
support ODR definitions, but fails for general weak definitions if strong
definition is encountered after the first weak definition. (A more extensive
refactor will be required to fully support weak symbols).

This patch does *not* add weak symbol support to RuntimeDyld: I hope to add
that in the near future.

llvm-svn: 277896
2016-08-06 00:54:43 +00:00
Lang Hames 368c4223b8 [lli] Add the ability for OrcLazyJIT to accept multiple input modules.
LLI already supported passing multiple input modules to MCJIT via the
-extra-module option. This patch adds the plumbing to pass these modules to
the OrcLazy JIT too.

This functionality will be used in an upcoming test case for weak symbol
handling.

llvm-svn: 277521
2016-08-02 21:00:40 +00:00
Lang Hames ad4a911fea [ExecutionEngine][MCJIT][Orc] Replace RuntimeDyld::SymbolInfo with JITSymbol.
This patch replaces RuntimeDyld::SymbolInfo with JITSymbol: A symbol class
that is capable of lazy materialization (i.e. the symbol definition needn't be
emitted until the address is requested). This can be used to support common
and weak symbols in the JIT (though this is not implemented in this patch).

For consistency, RuntimeDyld::SymbolResolver is renamed to JITSymbolResolver.

For space efficiency a new class, JITEvaluatedSymbol, is introduced that
behaves like the old RuntimeDyld::SymbolInfo - i.e. it is just a pair of an
address and symbol flags. Instances of JITEvaluatedSymbol can be used in
symbol-tables to avoid paying the space cost of the materializer.

llvm-svn: 277386
2016-08-01 20:49:11 +00:00
Michael Gottesman daaf9f1379 [cmake] Change lli-child-target to use add_llvm_utility instead of add_llvm_executable.
We are currently using add_llvm_utility for executable targets that:

1. Are built by default.
2. Used for testing.
3. Are not installed by default.

Originally, lli-child-target used add_llvm_tool instead of add_llvm_executable
directly. This was changed so that lli-child-target would not be installed. This
was good since this is only used for testing and should never be installed for
users. This also had the unfortunate side effect that one can never turn off the
building of lli-child-target by default, a regression for projects that by
default do not want to compile any LLVM tools beyond tablegen/llvm-config.

This patch changes lli-child-target to use add_llvm_utility. This makes sense
since:

1. lli-child-target matches the semantics of executables created with
add_llvm_utility.
2. We fix the regression since now one can use the flag LLVM_BUILD_UTILS to
eliminate default compilation.

llvm-svn: 275008
2016-07-10 02:43:50 +00:00
Kevin Enderby c60a321c6b Change Archive::create() from ErrorOr<...> to Expected<...> and update
its clients.

This commit will break the next lld builds.  I’ll be committing the matching
change for lld next.

llvm-svn: 274160
2016-06-29 20:35:44 +00:00
Richard Smith 2ad6d48b0c Search for llvm-symbolizer binary in the same directory as argv[0], before
looking for it along $PATH. This allows installs of LLVM tools outside of
$PATH to find the symbolizer and produce pretty backtraces if they crash.

llvm-svn: 272232
2016-06-09 00:53:21 +00:00
Benjamin Kramer 46e38f3678 Avoid copies of std::strings and APInt/APFloats where we only read from it
As suggested by clang-tidy's performance-unnecessary-copy-initialization.
This can easily hit lifetime issues, so I audited every change and ran the
tests under asan, which came back clean.

llvm-svn: 272126
2016-06-08 10:01:20 +00:00
Lang Hames 656532075b [Orc] Add conversion to/from RuntimeDyld::SymbolInfo for JITSymbol.
This tidies up some code that was manually constructing RuntimeDyld::SymbolInfo
instances from JITSymbols. It will save more mess in the future when
JITSymbol::getAddress is extended to return an Expected<TargetAddress> rather
than just a TargetAddress, since we'll be able to embed the error checking in
the conversion.

llvm-svn: 271350
2016-05-31 23:14:26 +00:00
Lang Hames 105518fe3c [Orc] Merge some common code for creating CompileCallbackManagers and
IndirectStubsManagers.

llvm-svn: 270874
2016-05-26 17:20:35 +00:00
Lang Hames bf9d1aa931 [RuntimeDyld] Call the SymbolResolver::findSymbolInLogicalDylib method when
searching for external symbols, and fall back to the SymbolResolver::findSymbol
method if the former returns null.

This makes RuntimeDyld behave more like a static linker: Symbol definitions
from within the current module's "logical dylib" will be preferred to
external definitions. We can build on this behavior in the future to properly
support weak symbol handling.

Custom symbol resolvers that override the findSymbolInLogicalDylib method may
notice changes due to this patch. Clients who have not overridden this method
should generally be unaffected, however users of the OrcMCJITReplacement class
may notice changes.

llvm-svn: 270716
2016-05-25 16:23:59 +00:00
Rafael Espindola 8c34dd8257 Delete Reloc::Default.
Having an enum member named Default is quite confusing: Is it distinct
from the others?

This patch removes that member and instead uses Optional<Reloc> in
places where we have a user input that still hasn't been maped to the
default value, which is now clear has no be one of the remaining 3
options.

llvm-svn: 269988
2016-05-18 22:04:49 +00:00
Lang Hames 2a1ee0fc23 [Orc] Fix missing rename from r268845.
llvm-svn: 268846
2016-05-07 03:48:56 +00:00
Lang Hames 4821c24d08 [Orc] Rename OrcArchitectureSupport to OrcABISupport and add Win32 ABI support.
This enables lazy JITing on Windows x86-64.

Patch by David. Thanks David!

llvm-svn: 268845
2016-05-07 03:36:38 +00:00
Lang Hames e246c4390e [lli] Fix a sign-compare warning.
llvm-svn: 267512
2016-04-26 01:45:25 +00:00
Lang Hames ef5a0ee2c3 [ORC] Thread Error/Expected through the RPC library.
This replaces use of std::error_code and ErrorOr in the ORC RPC support library
with Error and Expected. This required updating the OrcRemoteTarget API, Client,
and server code, as well as updating the Orc C API.

This patch also fixes several instances where Errors were dropped.

llvm-svn: 267457
2016-04-25 19:56:45 +00:00
Lang Hames 3fde652e18 [Orc] Re-commit r266581 with fixes for MSVC, and format cleanups.
Fixes:

(1) Removes constexpr (unsupported in MSVC)
(2) Move constructors (remove explicitly defaulted ones)
(3) <future> - Add warning suppression for MSVC.

llvm-svn: 266663
2016-04-18 19:55:43 +00:00
Mehdi Amini a7de820a20 lli: avoid global variables, use a local unique_ptr instead
There was issue with order of destruction in some cases.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266652
2016-04-18 18:52:39 +00:00
Nico Weber ca94d0ec0c Revert 266581 (and follow-up 266588), it doesn't build on Windows.
Three problems:
1. <future> can't be easily used.  If you must use it, see
   include/Support/ThreadPool.h for how.
2. constexpr problems, even after 266588.
3. Move assignment operators can't be defaulted in MSVC2013.

llvm-svn: 266615
2016-04-18 13:57:08 +00:00
Lang Hames 236cea74df [ORC] Generalize the ORC RPC utils to support RPC function return values and
asynchronous call/handle. Also updates the ORC remote JIT API to use the new
scheme.

The previous version of the RPC tools only supported void functions, and
required the user to manually call a paired function to return results. This
patch replaces the Procedure typedef (which only supported void functions) with
the Function typedef which supports return values, e.g.:

  Function<FooId, int32_t(std::string)> Foo;

The RPC primitives and channel operations are also expanded. RPC channels must
support four new operations: startSendMessage, endSendMessage,
startRecieveMessage and endRecieveMessage, to handle channel locking. In
addition, serialization support for tuples to RPCChannels is added to enable
multiple return values.

The RPC primitives are expanded from callAppend, call, expect and handle, to:

appendCallAsync - Make an asynchronous call to the given function.

callAsync - The same as appendCallAsync, but calls send on the channel when
            done.

callSTHandling - Blocking call for single-threaded code. Wraps a call to
                 callAsync then waits on the result, using a user-supplied
                 handler to handle any callbacks from the remote.

callST - The same as callSTHandling, except that it doesn't handle
         callbacks - it expects the result to be the first return.

expect and handle - as before.

handleResponse - Handle a response from the remote.

waitForResult - Wait for the response with the given sequence number to arrive.

llvm-svn: 266581
2016-04-18 01:06:49 +00:00
NAKAMURA Takumi c5727ecd88 lli.cpp: Appease some builders. There might be ownership issues between LLVMContext and ExecutionEngine. Investigating.
llvm-svn: 266441
2016-04-15 15:31:28 +00:00
Mehdi Amini 03b42e41bf Remove every uses of getGlobalContext() in LLVM (but the C API)
At the same time, fixes InstructionsTest::CastInst unittest: yes
you can leave the IR in an invalid state and exit when you don't
destroy the context (like the global one), no longer now.

This is the first part of http://reviews.llvm.org/D19094

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266379
2016-04-14 21:59:01 +00:00
Kevin Enderby 3fcdf6ae2a Thread Expected<...> up from createMachOObjectFile() to allow llvm-objdump to produce a real error message
Produce the first specific error message for a malformed Mach-O file describing
the problem instead of the generic message for object_error::parse_failed of
"Invalid data was encountered while parsing the file”.  Many more good error
messages will follow after this first one.

This is built on Lang Hames’ great work of adding the ’Error' class for
structured error handling and threading Error through MachOObjectFile
construction.  And making createMachOObjectFile return Expected<...> .

So to to get the error to the llvm-obdump tool, I changed the stack of
these methods to also return Expected<...> :

  object::ObjectFile::createObjectFile()
  object::SymbolicFile::createSymbolicFile()
  object::createBinary()

Then finally in ParseInputMachO() in MachODump.cpp the error can
be reported and the specific error message can be printed in llvm-objdump
and can be seen in the existing test case for the existing malformed binary
but with the updated error message.

Converting these interfaces to Expected<> from ErrorOr<> does involve
touching a number of places. To contain the changes for now use of
errorToErrorCode() and errorOrToExpected() are used where the callers
are yet to be converted.

Also there some were bugs in the existing code that did not deal with the
old ErrorOr<> return values.  So now with Expected<> since they must be
checked and the error handled, I added a TODO and a comment:
“// TODO: Actually report errors helpfully” and a call something like
consumeError(ObjOrErr.takeError()) so the buggy code will not crash
since needed to deal with the Error.

Note there is one fix also needed to lld/COFF/InputFiles.cpp that goes along
with this that I will commit right after this.  So expect lld not to built
after this commit and before the next one.

llvm-svn: 265606
2016-04-06 22:14:09 +00:00
Lang Hames 4f8194e81d [Orc] Add lazy-JITting support for i386.
This patch adds a new class, OrcI386, which contains the hooks needed to
support lazy-JITing on i386 (currently only for Pentium 2 or above, as the JIT
re-entry code uses the FXSAVE/FXRSTOR instructions).

Support for i386 is enabled in the LLI lazy JIT and the Orc C API, and
regression and unit tests are enabled for this architecture.

llvm-svn: 260338
2016-02-10 01:02:33 +00:00
Chris Bieneman e49730d4ba Remove autoconf support
Summary:
This patch is provided in preparation for removing autoconf on 1/26. The proposal to remove autoconf on 1/26 was discussed on the llvm-dev thread here: http://lists.llvm.org/pipermail/llvm-dev/2016-January/093875.html

"I felt a great disturbance in the [build system], as if millions of [makefiles] suddenly cried out in terror and were suddenly silenced. I fear something [amazing] has happened."
- Obi Wan Kenobi

Reviewers: chandlerc, grosbach, bob.wilson, tstellarAMD, echristo, whitequark

Subscribers: chfast, simoncook, emaste, jholewinski, tberghammer, jfb, danalbert, srhines, arsenm, dschuff, jyknight, dsanders, joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D16471

llvm-svn: 258861
2016-01-26 21:29:08 +00:00
Lang Hames 4ce7cc0f4c Remove some stale comments and fix a typo as suggested by David Blaikie in his
review of r257343.

Thanks Dave!

llvm-svn: 258002
2016-01-17 01:49:46 +00:00
Lang Hames 22c5af782a [Orc] Make FDRPCChannel final.
This class is in the LLI tool, and isn't subclassed.

llvm-svn: 257937
2016-01-15 21:35:24 +00:00
NAKAMURA Takumi cef0a821fa lli: use llvm::utostr() instead of std::to_string().
llvm-svn: 257857
2016-01-15 02:14:46 +00:00
Lang Hames 52c4724165 [Orc] Add support for EH-frame registration to the Orc Remote Target utility
classes.

OrcRemoteTargetClient::RCMemoryManager will now register EH frames with the
server automatically. This allows remote-execution of code that uses exceptions.

llvm-svn: 257816
2016-01-14 22:02:03 +00:00
NAKAMURA Takumi 65c4eafaa5 lli: Fix warnings. [-Wsign-compare]
llvm-svn: 257430
2016-01-12 01:23:30 +00:00
NAKAMURA Takumi d9d2a896fa lli-child-target: Introduce a new dependency on RuntimeDyld.
llvm-svn: 257410
2016-01-11 23:12:30 +00:00
Lang Hames 4b6e021fad XFAIL the LLI remote JIT tests on Win32.
llvm-svn: 257391
2016-01-11 21:41:34 +00:00
Lang Hames 081103aeb6 lli/ChildTarget now depends on OrcJIT. Add that component to the Makefile.
llvm-svn: 257360
2016-01-11 17:44:07 +00:00