Commit Graph

2384 Commits

Author SHA1 Message Date
Lang Hames 6fe6616c47 [ORC] Add a getRequestedSymbols method to MaterializationResponsibility.
This method returns the set of symbols in the target VSO that have queries
waiting on them. This can be used to make decisions about which symbols to
delegate to another MaterializationUnit (typically this will involve
delegating all symbols that have *not* been requested to another
MaterializationUnit so that materialization of those symbols can be
deferred until they are requested).

llvm-svn: 333684
2018-05-31 19:29:03 +00:00
Lang Hames d3a76f5bbc [ORC] Rename IRMaterializationUnit's Discardable member to SymbolToDefinition,
and make it protected rather than private.

The new name reflects the actual information in the map, and this information
can be useful to derived classes (for example, to quickly look up the IR
definition of a requested symbol).

llvm-svn: 333683
2018-05-31 19:29:01 +00:00
Hiroshi Inoue 3872c6c633 [PowerPC] fix broken JIT-compiled code with tail call optimization
The relocation for branch instructions in the dynamic loader of ExecutionEngine assumes branch instructions with R_PPC64_REL24 relocation type are only bl. However, with the tail call optimization, b instructions can be also used to jump into another function.
This patch makes the relocation to keep bits in the branch instruction other than the jump offset to avoid relocation rewrites a b instruction into bl.

Differential Revision: https://reviews.llvm.org/D47456

llvm-svn: 333502
2018-05-30 04:48:29 +00:00
Lang Hames e9bdfc16d7 [ORC] Fix an ambiguous make_unique call.
llvm-svn: 333492
2018-05-30 02:40:40 +00:00
Lang Hames bd0cb787d0 [ORC] Update JITCompileCallbackManager to support multi-threaded code.
Previously JITCompileCallbackManager only supported single threaded code. This
patch embeds a VSO (see include/llvm/ExecutionEngine/Orc/Core.h) in the callback
manager. The VSO ensures that the compile callback is only executed once and that
the resulting address cached for use by subsequent re-entries.

llvm-svn: 333490
2018-05-30 01:57:45 +00:00
Andres Freund 4a3e2dc8f9 [C-API] Add functions to create GDB, Intel, Oprofile event listeners.
The additions of Intel, Oprofile listeners were done blindly.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D44890

llvm-svn: 333230
2018-05-24 21:32:54 +00:00
Andres Freund 7aa54e956e [ORC][C-API] Expose LLVMOrc{Unr,R}egisterJITEventListener().
Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D44890

llvm-svn: 333229
2018-05-24 21:32:52 +00:00
Andres Freund 6801feee07 [ORC] Add ability [un]register JITEventListener on Orc C stack.
Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D44890

llvm-svn: 333228
2018-05-24 21:32:50 +00:00
Andres Freund c735f9e228 [ORC] Extend object layer callbacks so JITEventListener can be supported.
Currently RTDyldObjectLinkingLayer makes it hard to support
JITEventListeners. Which in turn means debugging and profiling JIT
generated code hard.

Supporting JITEventListeners at minimum requries a freed
callback (added).

As listeners expect the ObjectFile to be passed as well, an adaptor
between RTDyldObjectLinkingLayer and JITEventListeners would currently
need to also maintain ObjectFiles for all loaded modules. To make that
less awkward, extend the callbacks to pass the ObjectFile to both
Finalized and Freed callbacks.  That requires extending the lifetime
of the object file when callbacks are present.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D44890

llvm-svn: 333227
2018-05-24 21:32:48 +00:00
Lang Hames 7d60b9052a Add handling for GlobalAliases in ExecutionEngine::getConstantValue.
Patch by Brad Moody. Thanks Brad!

https://reviews.llvm.org/D42160

llvm-svn: 333217
2018-05-24 19:07:34 +00:00
Andres Freund 54ddd7426e [ORC] Add findSymbolIn() wrapper to C bindings, take #2.
Re-appply r333147, reverted in r333152 due to a pre-existing bug. As
D47308 has been merged in r333206, the OSX issue should now be
resolved.

In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D44889

llvm-svn: 333215
2018-05-24 18:44:34 +00:00
Andres Freund ac15cabd89 [ORC] Perform name mangling in findSymbolIn(), as done in findSymbol().
The lack of name mangling caused a unittest failure after r333147 (in
TestEagerIRCompilation), as OSX prefixes symbol names with '_'. The
lack of name mangling therefore leads to a NULL pointer being returned
and then called, hence the failure.

While it may look like it, this isn't an actual behavioral change, as
findSymbolIn() previously was not exposed externally, and essentially
dead code. Which explains why nobody noticed the issue previously.

Reviewers: lhames

Reviewed By: lhames

Subscribers: chandlerc, llvm-commits

Differential Revision: https://reviews.llvm.org/D47308

llvm-svn: 333206
2018-05-24 17:03:06 +00:00
Andres Freund 361941283f Revert r333147 "[ORC] Add findSymbolIn() wrapper to C bindings."
This reverts r333147 until https://reviews.llvm.org/D47308 is ready to
be reviewed. r333147 exposed a behavioural difference between
OrcCBindingsStack::findSymbolIn() and OrcCBindingsStack::findSymbol(),
where only the latter does name mangling. After r333147 that causes a
test failure on OSX, because the new test looks for main using
findSymbolIn() but the mangled name is _main.

llvm-svn: 333152
2018-05-24 05:10:19 +00:00
Andres Freund b0b67b07f5 [ORC] Add findSymbolIn() wrapper to C bindings.
In many cases JIT users will know in which module a symbol
resides. Avoiding to search other modules can be more efficient. It
also allows to handle duplicate symbol names between modules.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D44889

llvm-svn: 333147
2018-05-24 01:01:42 +00:00
Lang Hames 4c4a2ba353 [RuntimeDyld][MachO] Add support for MachO::ARM64_RELOC_POINTER_TO_GOT reloc.
llvm-svn: 333130
2018-05-23 21:27:07 +00:00
Lang Hames 5216ac9685 [LKH] Add a new IRTransformLayer.
llvm-svn: 333129
2018-05-23 21:27:07 +00:00
Lang Hames 85642262b2 [LKH] Add ObjectTransformLayer2.
llvm-svn: 333128
2018-05-23 21:27:06 +00:00
Lang Hames 4caa2f70ac [LKH] Add a new IRCompileLayer.
llvm-svn: 333127
2018-05-23 21:27:01 +00:00
Lang Hames 5261aa9f91 [ORC] Move symbol-scanning and discard from BasicIRLayerMaterializationUnit in
to a base class (IRMaterializationUnit).

The new class, IRMaterializationUnit, provides a convenient base for any client
that wants to write a materializer for LLVM IR.

llvm-svn: 332993
2018-05-22 16:15:38 +00:00
Lang Hames 373f4628a5 [LKH] Add a replacement RTDyldLayer.
llvm-svn: 332918
2018-05-21 23:45:40 +00:00
Lang Hames 502f81e37e [ORC] Preserve Materializing symbol flag during resolution.
llvm-svn: 332899
2018-05-21 21:11:22 +00:00
Lang Hames 0b0b41fcce [ORC] Lookup now returns an error if any symbols are not found.
Also tightens the behavior of ExecutionSession::failQuery. Queries can usually
only be failed by marking a symbol as failed-to-materialize, but
ExecutionSession::failQuery provides a second route, and both routes may be
executed from different threads. In the case that a query has already been
failed due to a materialization error, ExecutionSession::failQuery will
direct the error to ExecutionSession::reportError instead.

llvm-svn: 332898
2018-05-21 21:11:21 +00:00
Lang Hames add9b6805c [ORC] Remove the optional MaterializationResponsibility argument from lookup.
The lookup function provides blocking symbol resolution for JIT clients (not
layers themselves) so it does not need to track symbol dependencies via a
MaterializationResponsibility.

llvm-svn: 332897
2018-05-21 21:11:21 +00:00
Lang Hames 1cf9987f6e [ORC] Add IRLayer and ObjectLayer interfaces and related MaterializationUnits.
llvm-svn: 332896
2018-05-21 21:11:13 +00:00
Lang Hames ecb3e50041 [ORC] Consolidate materialization errors, and generate them in VSO's
notifyFailed method rather than passing in an error generator.

VSO::notifyFailed is responsible for notifying queries that they will not
succeed due to error. In practice the queries don't care about the details
of the failure, just the fact that a failure occurred for some symbols.
Having VSO::notifyFailed take care of this simplifies the interface.

llvm-svn: 332666
2018-05-17 20:48:58 +00:00
Lang Hames d261e1258c [ORC] Rewrite the VSO symbol table yet again. Update related utilities.
VSOs now track dependencies for materializing symbols. Each symbol must have its
dependencies registered with the VSO prior to finalization. Usually this will
involve registering the dependencies returned in
AsynchronousSymbolQuery::ResolutionResults for queries made while linking the
symbols being materialized.

Queries against symbols are notified that a symbol is ready once it and all of
its transitive dependencies are finalized, allowing compilation work to be
broken up and moved between threads without queries returning until their
symbols fully safe to access / execute.

Related utilities (VSO, MaterializationUnit, MaterializationResponsibility) are
updated to support dependence tracking and more explicitly track responsibility
for symbols from the point of definition until they are finalized.

llvm-svn: 332541
2018-05-16 22:24:30 +00:00
Nicola Zaghen d34e60ca85 Rename DEBUG macro to LLVM_DEBUG.
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.

In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.

Differential Revision: https://reviews.llvm.org/D43624

llvm-svn: 332240
2018-05-14 12:53:11 +00:00
Lang Hames 3ae85708c9 [RuntimeDyld][MachO] Properly handle thumb to thumb calls within a section.
Previously thumb bits were only checked for external relocations (thumb to arm
code and vice-versa). This patch adds detection for thumb callees in the same
section asthe (also thumb) caller.

The MachO/Thumb test case is updated to cover this, and redundant checks
(handled by the MachO/ARM test) are removed.

llvm-svn: 331838
2018-05-09 01:38:13 +00:00
Adrian Prantl 4dfcc4a788 Remove @brief commands from doxygen comments, too.
This is a follow-up to r331272.

We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by
  for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done

https://reviews.llvm.org/D46290

llvm-svn: 331275
2018-05-01 16:10:38 +00:00
Adrian Prantl 5f8f34e459 Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

  for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done

Differential Revision: https://reviews.llvm.org/D46290

llvm-svn: 331272
2018-05-01 15:54:18 +00:00
Nico Weber 432a38838d IWYU for llvm-config.h in llvm, additions.
See r331124 for how I made a list of files missing the include.
I then ran this Python script:

    for f in open('filelist.txt'):
        f = f.strip()
        fl = open(f).readlines()

        found = False
        for i in xrange(len(fl)):
            p = '#include "llvm/'
            if not fl[i].startswith(p):
                continue
            if fl[i][len(p):] > 'Config':
                fl.insert(i, '#include "llvm/Config/llvm-config.h"\n')
                found = True
                break
        if not found:
            print 'not found', f
        else:
            open(f, 'w').write(''.join(fl))

and then looked through everything with `svn diff | diffstat -l | xargs -n 1000 gvim -p`
and tried to fix include ordering and whatnot.

No intended behavior change.

llvm-svn: 331184
2018-04-30 14:59:11 +00:00
Lang Hames ee68ec06a1 [ORC] Fix an assertion condition from r329934.
Thanks to Alexander Ivchenko for finding the issue!

llvm-svn: 330359
2018-04-19 19:30:35 +00:00
Lang Hames 9bbd653084 [ORC] Make VSO symbol resolution/finalization operations private.
This forces these operations to be carried out via a
MaterializationResponsibility instance, ensuring responsibility is explicitly
tracked.

llvm-svn: 330356
2018-04-19 18:42:49 +00:00
Lang Hames 67feadb2c7 [ORC] Add a MaterializationResponsibility class to track responsibility for
materializing function definitions.

MaterializationUnit instances are responsible for resolving and finalizing
symbol definitions when their materialize method is called. By contract, the
MaterializationUnit must materialize all definitions it is responsible for and
no others. If it can not materialize all definitions (because of some error)
then it must notify the associated VSO about each definition that could not be
materialized. The MaterializationResponsibility class tracks this
responsibility, asserting that all required symbols are resolved and finalized,
and that no extraneous symbols are resolved or finalized. In the event of an
error it provides a convenience method for notifying the VSO about each
definition that could not be materialized.

llvm-svn: 330142
2018-04-16 18:05:24 +00:00
Lang Hames ffeacb7b1a [ORC] Merge VSO notifyResolutionFailed and notifyFinalizationFailed in to
notifyMaterializationFailed.

The notifyMaterializationFailed method can determine which error to raise by
looking at which queue the pending queries are in (resolution or finalization).

llvm-svn: 330141
2018-04-16 18:05:22 +00:00
Weiming Zhao 79f2d093dd Rename ObjectMemoryBuffer to SmallVectorMemoryBuffer; NFCI
Summary: As discussed in https://reviews.llvm.org/D45606, it makes more sense to name the class as SmallVectorMemoryBuffer

Reviewers: bkramer, dblaikie

Reviewed By: dblaikie

Subscribers: mehdi_amini, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D45661

llvm-svn: 330107
2018-04-16 03:44:03 +00:00
Weiming Zhao 7c16977a8f NFC: Move ObjectMemoryBuffer to support
Summary:
Since the class is used by both MCJIT and LTO, it makes more sense to move it to Support lib.
This is a follow up patch to r329929 and https://reviews.llvm.org/D45244

Reviewers: bkramer, dblaikie

Reviewed By: bkramer

Subscribers: mehdi_amini, eraman, llvm-commits

Differential Revision: https://reviews.llvm.org/D45606

llvm-svn: 330093
2018-04-15 05:17:14 +00:00
Lang Hames e7c4502a87 [ORC] Use insert rather than emplace.
Hopefully this will fix the build failure at
http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/9028

llvm-svn: 329944
2018-04-12 19:54:41 +00:00
Lang Hames 9d8877baf8 [ORC] Plumb error notifications through the VSO interface.
This allows materializers to notify the VSO that they were unable to
resolve or finalize symbols.

llvm-svn: 329934
2018-04-12 18:35:08 +00:00
Benjamin Kramer 4c5fa6be27 [MCJIT] Remove the anchor from mcjit.
This is a layering violation. LTO shouldn't depend on MCJIT. The right
fix for this is moving the class somewhere else.

llvm-svn: 329929
2018-04-12 17:28:30 +00:00
Weiming Zhao 1bd40005ba Add missing vtable anchors
Summary: This patch adds anchor() for MemoryBuffer, raw_fd_ostream, RTDyldMemoryManager, SectionMemoryManager, etc.

Reviewers: jlebar, eli.friedman, dblaikie

Reviewed By: dblaikie

Subscribers: mehdi_amini, mgorny, dblaikie, weimingz, llvm-commits

Differential Revision: https://reviews.llvm.org/D45244

llvm-svn: 329861
2018-04-11 23:09:20 +00:00
Lang Hames 7a598477aa [RuntimeDyld][PowerPC] Use global entry points for calls between sections.
Functions in different objects may use different TOCs, so calls between such
functions should use the global entry point of the callee which updates the
TOC pointer.

This should fix a bug that the Numba developers encountered (see
https://github.com/numba/numba/issues/2451).

Patch by Olexa Bilaniuk. Thanks Olexa!

No RuntimeDyld checker test case yet as I am not familiar enough with how
RuntimeDyldELF fixes up call-sites, but I do not want to hold up landing
this. I will continue to work on it and see if I can rope some powerpc
experts in.

llvm-svn: 329335
2018-04-05 19:37:05 +00:00
Lang Hames b1e5043cff Reapply r329133 with fix.
llvm-svn: 329136
2018-04-04 00:34:54 +00:00
Lang Hames 4e319acd84 Revert r329133 "[RuntimeDyld][AArch64] Add some error pluming / generation..."
This broke a number of buildbots. Looking in to it now...

llvm-svn: 329135
2018-04-04 00:12:12 +00:00
Lang Hames b92b10f3ec [RuntimeDyld][AArch64] Add some error pluming / generation to catch unhandled
relocation types on AArch64.

llvm-svn: 329133
2018-04-03 23:19:20 +00:00
Lang Hames 3fdfc04e53 [ORC] Create a new SymbolStringPool by default in ExecutionSession constructor.
This makes the common case of constructing an ExecutionSession tidier.

llvm-svn: 329013
2018-04-02 20:57:56 +00:00
Lang Hames a95b0df5ed [ORC] Fix ORC on platforms without indirection support.
Previously this crashed because a nullptr (returned by
createLocalIndirectStubsManagerBuilder() on platforms without
indirection support) functor was unconditionally invoked.

Patch by Andres Freund. Thanks Andres!

llvm-svn: 328687
2018-03-28 03:41:45 +00:00
David Blaikie 7c4b5d92f1 Remove unused file, ExecutionEngine/MCJIT/ObjectBuffer.h
This header also wasn't self contained/modular - but with no users, it
didn't seem worth fixing because it'd break so easily again.

llvm-svn: 328565
2018-03-26 18:10:31 +00:00
David Blaikie bf121cf44a Fix layering by moving Support/CodeGenCWrappers.h to Target
This includes llvm-c/TargetMachine.h which is logically part of
libTarget (since libTarget implements llvm-c/TargetMachine.h's
functions).

llvm-svn: 328394
2018-03-23 23:58:21 +00:00
Lang Hames 2c83285716 [ORC] Don't fully qualify explicit destructor call -- it confuses some compilers.
This should fix the builder failure at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/19224

llvm-svn: 327955
2018-03-20 05:56:58 +00:00
Lang Hames 4cca7d229e [ORC] Rename SymbolSource to MaterializationUnit, and make the materialization
operation all-or-nothing, rather than allowing materialization on a per-symbol
basis.

This addresses a shortcoming of per-symbol materialization: If a
MaterializationUnit (/SymbolSource) wants to materialize more symbols than
requested (which is likely: most materializers will want to materialize whole
modules) then it needs a way to notify the symbol table about the extra symbols
being materialized. This process (checking what has been requested against what
is being provided and notifying the symbol table about the difference) has to
be repeated at every level of the JIT stack. Making materialization
all-or-nothing eliminates this issue, simplifying both materializer
implementations and the symbol table (VSO class) API. The cost is that
per-symbol materialization (e.g. for individual symbols in a module) now
requires multiple MaterializationUnits.

llvm-svn: 327946
2018-03-20 03:49:29 +00:00
Lang Hames 5721ee48a2 [ORC] Re-apply r327566 with a fix for test-global-ctors.ll.
Also clang-formats the patch, which I should have done the first time around.

llvm-svn: 327594
2018-03-15 00:30:14 +00:00
Reid Kleckner c7fd1540b3 Revert "[ORC] Switch from shared_ptr to unique_ptr for addModule methods."
This reverts commit r327566, it breaks
test/ExecutionEngine/OrcMCJIT/test-global-ctors.ll.

The test doesn't crash with a stack trace, unfortunately. It merely
returns 1 as the exit code.

ASan didn't produce a report, and I reproduced this on my Linux machine
and Windows box.

llvm-svn: 327576
2018-03-14 21:32:34 +00:00
Lang Hames 7bea03c2bb [ORC] Switch from shared_ptr to unique_ptr for addModule methods.
Layer implementations typically mutate module state, and this is better
reflected by having layers own the Module they are operating on.

llvm-svn: 327566
2018-03-14 20:29:45 +00:00
Lang Hames 2d603a1860 [RuntimeDyld] Silence a compiler error.
This should fix the error at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/19008

llvm-svn: 327478
2018-03-14 06:39:49 +00:00
Lang Hames b2facd6479 [ORC] Fix a data race in the lookup function.
The Error locals need to be protected by a mutex. (This could be fixed by
having the promises / futures contain Expected and Error values, but
MSVC's future implementation does not support this yet).

Hopefully this will fix some of the errors seen on the builders due to
r327474.

llvm-svn: 327477
2018-03-14 06:25:08 +00:00
Lang Hames 313f590aee [ExecutionEngine] Add a getSymbolTable method to RuntimeDyld.
This can be used to extract the symbol table from a RuntimeDyld instance prior
to disposing of it.

This patch also updates RTDyldObjectLinkingLayer to use the new method, rather
than requesting symbols one at a time via getSymbol.

llvm-svn: 327476
2018-03-14 06:25:07 +00:00
Lang Hames d4a768e78f [ORC] Silence a compiler error.
This should fix the builder error at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/19006

llvm-svn: 327475
2018-03-14 05:23:56 +00:00
Lang Hames 817f1f64d9 [ORC] Add a 'lookup' convenience function for finding symbols in a list of VSOs.
The lookup function takes a list of VSOs, a set of symbol names (or just one
symbol name) and a materialization function object. It returns an
Expected<SymbolMap> (if given a set of names) or an Expected<JITEvaluatedSymbol>
(if given just one name). The lookup method constructs an
AsynchronousSymbolQuery for the given names, applies that query to each VSO in
the list in turn, and then blocks waiting for the query to complete. If
threading is enabled then the materialization function object can be used to
execute the materialization on different threads. If threading is disabled the
MaterializeOnCurrentThread utility must be used.

llvm-svn: 327474
2018-03-14 04:18:04 +00:00
Lang Hames 811343cfd8 [RuntimeDyld][MachO] Fix assertion in encodeAddend, add missing directive to
test case.

r326290 fixed the assertion for decodeAddend, but not encodeAddend. The
regression test failed to catch this because it was missing the
subsections_via_symbols flag, so the desired relocation was not applied.

This patch also fixes the formatting of the assertion from r326290.

llvm-svn: 326406
2018-03-01 01:44:33 +00:00
Chih-Hung Hsieh 9f9e4681ac [TLS] use emulated TLS if the target supports only this mode
Emulated TLS is enabled by llc flag -emulated-tls,
which is passed by clang driver.
When llc is called explicitly or from other drivers like LTO,
missing -emulated-tls flag would generate wrong TLS code for targets
that supports only this mode.
Now use useEmulatedTLS() instead of Options.EmulatedTLS to decide whether
emulated TLS code should be generated.
Unit tests are modified to run with and without the -emulated-tls flag.

Differential Revision: https://reviews.llvm.org/D42999

llvm-svn: 326341
2018-02-28 17:48:55 +00:00
Lang Hames 6588f14a6c [RuntimeDyld][MachO] Support ARM64_RELOC_BRANCH26 for BL instructions by
relaxing an assertion.

llvm-svn: 326290
2018-02-28 00:58:21 +00:00
Lang Hames a944589cc5 [ORC] Switch to shared_ptr ownership for SymbolSources in VSOs.
This makes it easy to free a SymbolSource (and any related
resources) when the last reference in a VSO is dropped.

llvm-svn: 325727
2018-02-21 21:55:57 +00:00
Lang Hames 589eece132 [ORC] Switch RTDyldObjectLinkingLayer to take a unique_ptr<MemoryBuffer> rather
than a shared ObjectFile/MemoryBuffer pair.

There's no need to pre-parse the buffer into an ObjectFile before passing it
down to the linking layer, and moving the parsing into the linking layer allows
us remove the parsing code at each call site.

llvm-svn: 325725
2018-02-21 21:55:49 +00:00
Frederich Munch 33ef594c58 Handle IMAGE_REL_AMD64_ADDR32NB in RuntimeDyldCOFF
Summary:
IMAGE_REL_AMD64_ADDR32NB relocations are currently set to zero in all cases.
This patch sets the relocation to the correct value when possible and shows an error when not.

Reviewers: enderby, lhames, compnerd

Reviewed By: compnerd

Subscribers: LepelTsmok, compnerd, martell, llvm-commits

Differential Revision: https://reviews.llvm.org/D30709

llvm-svn: 325700
2018-02-21 17:18:20 +00:00
Serge Pavlov 76d8ccee2e Report fatal error in the case of out of memory
This is the second part of recommit of r325224. The previous part was
committed in r325426, which deals with C++ memory allocation. Solution
for C memory allocation involved functions `llvm::malloc` and similar.
This was a fragile solution because it caused ambiguity errors in some
cases. In this commit the new functions have names like `llvm::safe_malloc`.

The relevant part of original comment is below, updated for new function
names.

Analysis of fails in the case of out of memory errors can be tricky on
Windows. Such error emerges at the point where memory allocation function
fails, but manifests itself when null pointer is used. These two points
may be distant from each other. Besides, next runs may not exhibit
allocation error.

In some cases memory is allocated by a call to some of C allocation
functions, malloc, calloc and realloc. They are used for interoperability
with C code, when allocated object has variable size and when it is
necessary to avoid call of constructors. In many calls the result is not
checked for null pointer. To simplify checks, new functions are defined
in the namespace 'llvm': `safe_malloc`, `safe_calloc` and `safe_realloc`.
They behave as corresponding standard functions but produce fatal error if
allocation fails. This change replaces the standard functions like 'malloc'
in the cases when the result of the allocation function is not checked
for null pointer.

Finally, there are plain C code, that uses malloc and similar functions. If
the result is not checked, assert statement is added.

Differential Revision: https://reviews.llvm.org/D43010

llvm-svn: 325551
2018-02-20 05:41:26 +00:00
Serge Pavlov 4500001905 Revert r325224 "Report fatal error in the case of out of memory"
It caused fails on some buildbots.

llvm-svn: 325227
2018-02-15 09:45:59 +00:00
Serge Pavlov 431502a675 Report fatal error in the case of out of memory
Analysis of fails in the case of out of memory errors can be tricky on
Windows. Such error emerges at the point where memory allocation function
fails, but manifests itself when null pointer is used. These two points
may be distant from each other. Besides, next runs may not exhibit
allocation error.

Usual programming practice does not require checking result of 'operator
new' because it throws 'std::bad_alloc' in the case of allocation error.
However, LLVM is usually built with exceptions turned off, so 'new' can
return null pointer. This change installs custom new handler, which causes
fatal error in the case of out of memory. The handler is installed
automatically prior to call to 'main' during construction of a static
object defined in 'lib/Support/ErrorHandling.cpp'. If the application does
not use this file, the handler may be installed manually by a call to
'llvm::install_out_of_memory_new_handler', declared in
'include/llvm/Support/ErrorHandling.h".

There are calls to C allocation functions, malloc, calloc and realloc.
They are used for interoperability with C code, when allocated object has
variable size and when it is necessary to avoid call of constructors. In
many calls the result is not checked against null pointer. To simplify
checks, new functions are defined in the namespace 'llvm' with the
same names as these C function. These functions produce fatal error if
allocation fails. User should use 'llvm::malloc' instead of 'std::malloc'
in order to use the safe variant. This change replaces 'std::malloc'
in the cases when the result of allocation function is not checked against
null pointer.

Finally, there are plain C code, that uses malloc and similar functions. If
the result is not checked, assert statements are added.

Differential Revision: https://reviews.llvm.org/D43010

llvm-svn: 325224
2018-02-15 09:20:26 +00:00
Lang Hames 1cd3dd0bd8 [ORC] Consolidate RTDyldObjectLinkingLayer GetMemMgr and GetResolver into a
unified GetResources callback.

Having a single 'GetResources' callback will simplify adding new resources in
the future.

llvm-svn: 325180
2018-02-14 22:13:02 +00:00
Lang Hames e833fe8ec3 [ORC] Switch to shared_ptr ownership for AsynchronousSymbolQueries.
Queries need to stay alive until each owner has set the values they are
responsible for.

llvm-svn: 325179
2018-02-14 22:12:56 +00:00
Lang Hames 0976cee8e9 [ORC] Remove Layer handles from the layer concept.
Handles were returned by addModule and used as keys for removeModule,
findSymbolIn, and emitAndFinalize. Their job is now subsumed by VModuleKeys,
which simplify resource management by providing a consistent handle across all
layers.

llvm-svn: 324700
2018-02-09 02:30:40 +00:00
Lang Hames 0ccef9ee16 [ORC] Use explicit constructor calls to fix a builder error at
http://lab.llvm.org:8011/builders/lld-x86_64-darwin13/builds/17627

llvm-svn: 324411
2018-02-06 22:17:09 +00:00
Lang Hames 4b546c9145 [ORC] Start migrating ORC layers to use the new ORC Core.h APIs.
In particular this patch switches RTDyldObjectLinkingLayer to use
orc::SymbolResolver and threads the requried changse (ExecutionSession
references and VModuleKeys) through the existing layer APIs.

The purpose of the new resolver interface is to improve query performance and
better support parallelism, both in JIT'd code and within the compiler itself.

The most visibile change is switch of the <Layer>::addModule signatures from:

Expected<Handle> addModule(std::shared_ptr<ModuleType> Mod,
                           std::shared_ptr<JITSymbolResolver> Resolver)

to:

Expected<Handle> addModule(VModuleKey K, std::shared_ptr<ModuleType> Mod);

Typical usage of addModule will now look like:

auto K = ES.allocateVModuleKey();
Resolvers[K] = createSymbolResolver(...);
Layer.addModule(K, std::move(Mod));

See the BuildingAJIT tutorial code for example usage.

llvm-svn: 324405
2018-02-06 21:25:11 +00:00
Lang Hames 371412b1ec [ORC] Rename NullResolver to NullLegacyResolver.
This resolver conforms to the LegacyJITSymbolResolver interface, and will be
replaced with a null-returning resolver conforming to the newer
orc::SymbolResolver interface in the near future. This patch renames the class
to avoid a clash.

llvm-svn: 324175
2018-02-03 16:52:48 +00:00
Rafael Espindola d5f76ad37f Move getPlatformFlags to ELFObjectFileBase and simplify.
This removes a few std::error_code results that were ignored on every
call.

llvm-svn: 323674
2018-01-29 18:27:30 +00:00
Lang Hames c8a74a0448 [ORC] Refactor the various lookupFlags methods to return the flags map via the
first argument.

This makes lookupFlags more consistent with lookup (which takes the query as the
first argument) and composes better in practice, since lookups are usually
linearly chained: Each lookupFlags can populate the result map based on the
symbols not found in the previous lookup. (If the maps were returned rather than
passed by reference there would have to be a merge step at the end).

llvm-svn: 323398
2018-01-25 01:43:00 +00:00
Rafael Espindola fc16f76edb Handle R_386_PLT32 in RuntimeDyldELF.
This should fix the 32 bit buildbots.

llvm-svn: 323344
2018-01-24 17:36:08 +00:00
Lang Hames 635fd9092b [ORC] Add orc::SymbolResolver, a Orc/Legacy API interop header, and an
orc::SymbolResolver to JITSymbolResolver adapter.

The new orc::SymbolResolver interface uses asynchronous queries for better
performance. (Asynchronous queries with bulk lookup minimize RPC/IPC overhead,
support parallel incoming queries, and expose more available work for
distribution). Existing ORC layers will soon be updated to use the
orc::SymbolResolver API rather than the legacy llvm::JITSymbolResolver API.

Because RuntimeDyld still uses JITSymbolResolver, this patch also includes an
adapter that wraps an orc::SymbolResolver with a JITSymbolResolver API.

llvm-svn: 323073
2018-01-22 03:00:31 +00:00
Lang Hames 5ff5a30bee [ORC] Add a lookupFlags method to VSO.
lookupFlags returns a SymbolFlagsMap for the requested symbols, along with a
set containing the SymbolStringPtr for any symbol not found in the VSO.

The JITSymbolFlags for each symbol will have been stripped of its transient
JIT-state flags (i.e. NotMaterialized, Materializing).

Calling lookupFlags does not trigger symbol materialization.

llvm-svn: 323060
2018-01-21 03:20:39 +00:00
Lang Hames 86edf53664 [ORC] More cleanup. NFC.
llvm-svn: 323059
2018-01-21 03:20:36 +00:00
Lang Hames 0d7d442699 [ORC] Cleanup. NFC.
llvm-svn: 323057
2018-01-21 02:24:45 +00:00
Rui Ueyama 20b49240b8 Fix -Wunused-variable.
llvm-svn: 323004
2018-01-19 22:56:04 +00:00
Lang Hames b72f48452c [ORC] Re-apply r322913 with a fix for a read-after-free error.
ExternalSymbolMap now stores the string key (rather than using a StringRef),
as the object file backing the key may be removed at any time.

llvm-svn: 323001
2018-01-19 22:24:13 +00:00
Lang Hames 44efd042a2 [ORC] Revert r322913 while I investigate an ASan failure.
llvm-svn: 322914
2018-01-19 01:40:26 +00:00
Lang Hames 817df9fa0c [ORC] Redesign the JITSymbolResolver interface to support bulk queries.
Bulk queries reduce IPC/RPC overhead for cross-process JITing and expose
opportunities for parallel compilation.

The two new query methods are lookupFlags, which finds the flags for each of a
set of symbols; and lookup, which finds the address and flags for each of a
set of symbols. (See doxygen comments for more details.)

The existing JITSymbolResolver class is renamed LegacyJITSymbolResolver, and
modified to extend the new JITSymbolResolver class using the following scheme:

- lookupFlags is implemented by calling findSymbolInLogicalDylib for each of the
symbols, then returning the result of calling getFlags() on each of these
symbols. (Importantly: lookupFlags does NOT call getAddress on the returned
symbols, so lookupFlags will never trigger materialization, and lookupFlags will
never call findSymbol, so only symbols that are part of the logical dylib will
return results.)

- lookup is implemented by calling findSymbolInLogicalDylib for each symbol and
falling back to findSymbol if findSymbolInLogicalDylib returns a null result.
Assuming a symbol is found its getAddress method is called to materialize it and
the result (if getAddress succeeds) is stored in the result map, or the error
(if getAddress fails) is returned immediately from lookup. If any symbol is not
found then lookup returns immediately with an error.

This change will break any out-of-tree derivatives of JITSymbolResolver. This
can be fixed by updating those classes to derive from LegacyJITSymbolResolver
instead.

llvm-svn: 322913
2018-01-19 01:12:40 +00:00
Lang Hames 4a793c0667 [ExecutionEngine] Rename JITSymbol::isStrongDefinition to isStrong.
For symmetry with isWeak, isCommon.

llvm-svn: 322594
2018-01-16 20:39:51 +00:00
Rui Ueyama fc63551082 Remove ELFDataTypeTypedefHelper class.
Differential Revision: https://reviews.llvm.org/D41973

llvm-svn: 322395
2018-01-12 19:59:43 +00:00
Rui Ueyama 478d635156 Instead of ELFFile<ELFT>::Type, use ELFT::Type. NFC.
llvm-svn: 322346
2018-01-12 02:28:31 +00:00
Lang Hames 4628880167 [ORC] Add a stub ExecutionSession and VModuleKey type.
ExecutionSession will represent a running JIT program.

VModuleKey is a unique key assigned to each module added as part of
an ExecutionSession. The Layer concept will be updated in future to
require a VModuleKey when a module is added.

llvm-svn: 322336
2018-01-12 00:22:05 +00:00
Lang Hames 8e328d6c48 [ExecutionEngine] Remove an unused variable.
Patch by Evgeniy Tyurin. Thanks Evgeniy!

Review: https://reviews.llvm.org/D41431
llvm-svn: 322158
2018-01-10 03:43:14 +00:00
Lang Hames 9510447a66 [ORC] Re-apply r321838 again with a workaround for a bug present in the libcxx
version being used on some of the green dragon builders (plus a clang-format).

Workaround: AsynchronousSymbolQuery and VSO want to work with
JITEvaluatedSymbols anyway, so just use them (instead of JITSymbol, which
happens to tickle the bug).

The libcxx bug being worked around was fixed in r276003, and there are plans to
update the offending builders.

llvm-svn: 322140
2018-01-10 00:09:38 +00:00
Lang Hames 0b93cd7351 [ORC] Remove AsynchronousSymbolQuery while I debug an issue on one of the
builders.

llvm-svn: 321941
2018-01-06 20:14:22 +00:00
Lang Hames 4b6cae190d [ORC] Yet more debugging output to diagnose test failures.
llvm-svn: 321927
2018-01-06 05:19:07 +00:00
Lang Hames 0f74d273b0 [ORC] Temporarily adding some redundant asserts / debug output to aid in
debugging a tester failure.

llvm-svn: 321920
2018-01-06 01:06:07 +00:00
Lang Hames 1097dc47eb [ORC] Re-apply just the AsynchronousSymbolLookup class from r321838 while I
investigate builder / test failures.

llvm-svn: 321910
2018-01-05 22:50:43 +00:00
Lang Hames 5d4a74a320 [ORC] Re-revert r321838: Tests are still failing.
llvm-svn: 321858
2018-01-05 03:10:15 +00:00
Lang Hames 33b89c5713 [ORC] Re-apply r321838 - Addition of new ORC core APIs.
The original commit broke the builders due to a think-o in an assertion:
AsynchronousSymbolQuery's constructor needs to check the callback member
variables, not the constructor arguments.

llvm-svn: 321853
2018-01-05 02:21:02 +00:00
Lang Hames 0429ebfabc Revert r321838 -- It broke some of the builders.
llvm-svn: 321842
2018-01-05 00:29:37 +00:00
Lang Hames 2d3bc98f78 [ORC] Add new core ORC APIs (Core.h/Core.cpp): VSO, AsynchronousSymbolQuery and
SymbolSource.

These new APIs are a first stab at tackling some current shortcomings of ORC,
especially in performance and threading support.

VSO (Virtual Shared Object) is a symbol table representing the symbol
definitions of a set of modules that behave as if they had been statically
linked together into a shared object or dylib. Symbol definitions, either
pre-defined addresses or lazy definitions, can be added and queries for symbol
addresses made. The table applies the same linkage strength rules that static
linkers do when constructing a dylib or shared object: duplicate definitions
result in errors, strong definitions override weak or common ones. This class
should improve symbol lookup speed by providing centralized symbol tables (as
compared to the findSymbol implementation in the in-tree ORC layers, which
maintain one symbol table per object file / module added).

AsynchronousSymbolQuery is a query for the addresses of a set of symbols.
Query results are returned via a callback once they become available. Querying
for a set of symbols, rather than one symbol at a time (as the current lookup
scheme does) the JIT has the opportunity to make better use of available
resources (e.g. by spawning multiple jobs to materialize the requested symbols
if possible). Returning results via a callback makes queries asynchronous, so
queries from multiple threads of JIT'd code can proceed simultaneously.

SymbolSource represents a source of symbol definitions. It is used when
adding lazy symbol definitions to a VSO. Symbol definitions can be materialized
when needed or discarded if a stronger definition is found. Materializing on
demand via SymbolSources should (eventually) allow us to remove the lazy
materializers from JITSymbol, which will in turn allow the removal of many
current error checks and reduce the number of RPC round-trips involved in
materializing remote symbols. Adding a discard function allows sources to
discard symbol definitions (or mark them as available_externally), reducing the
amount of redundant code generated by the JIT for ODR symbols.

llvm-svn: 321838
2018-01-05 00:04:16 +00:00
Michael Zolotukhin a44d5fe333 Remove redundant includes from lib/ExecutionEngine.
llvm-svn: 320621
2017-12-13 21:30:50 +00:00
Simon Pilgrim 89d31658e5 Fix 'not all control paths return a value' warning on MSVC builds
llvm-svn: 317790
2017-11-09 14:56:17 +00:00
Sanjoy Das e3992c6328 [SectionMemoryManager] Abstract out mmap, munmap, mprotect even more ; NFC
Summary:
This will let ORC JIT clients plug in custom logic for the mmap, munmap and
mprotect paths.

Reviewers: loladiro, dblaikie

Subscribers: mcrosier, llvm-commits

Differential Revision: https://reviews.llvm.org/D39300

llvm-svn: 317770
2017-11-09 06:31:33 +00:00
Saleem Abdulrasool 9e802eaf60 ExecutionEngine: make COFF Thumb2 assertions non-tautological
The overflow detection assertions were tautological due to truncation.
Adjust them to no longer be tautological.

Patch by Alex Langford!

llvm-svn: 316303
2017-10-22 20:51:25 +00:00
Nitesh Jain 757f74c2d3 [mips] Adds support for R_MIPS_26, HIGHER, HIGHEST relocations in RuntimeDyld.
Reviewers: sdardis

Subscribers: jaydeep, bhushan, llvm-commits

Differential Revision: https://reviews.llvm.org/D38314

llvm-svn: 316287
2017-10-22 09:47:41 +00:00
Lang Hames 8eec91e96d [ExecutionEngine] After a heroic dev-meeting hack session, the JIT supports TLS.
Turns on EmulatedTLS support by default in EngineBuilder. ;)

llvm-svn: 316200
2017-10-20 00:53:16 +00:00
Saleem Abdulrasool 1261151912 ExecutionEngine: adjust COFF i386 tautological asserts
Modify static_casts to not be tautological in some COFF i386
relocations.

Patch by Alex Langford!

llvm-svn: 316169
2017-10-19 16:57:40 +00:00
Shoaib Meenai a42f60e7f7 [ExecutionEngine] Correct the size of a write in a COFF i386 relocation
We want to be writing a 32bit value, so we should be writing 4 bytes
instead of 2.

Patch by Alex Langford <apl@fb.com>.

Differential Revision: https://reviews.llvm.org/D38872

llvm-svn: 315964
2017-10-17 01:41:14 +00:00
Rafael Espindola 1a0e5a1933 Convert an ErrorOr to Expected.
getRelocationAddend should never be called on non SHT_RELA sections,
but changing that requires changing RelocVisitor.h.

llvm-svn: 315473
2017-10-11 16:56:33 +00:00
Rafael Espindola ef421f9c18 Make the ELFObjectFile constructor private.
This forces every user to use the new create method that returns an
Expected. This in turn propagates better error messages.

llvm-svn: 315371
2017-10-10 21:21:16 +00:00
Rafael Espindola 8ed1198774 Try to make gcc happy.
llvm-svn: 315349
2017-10-10 19:27:51 +00:00
Rafael Espindola ff9f5f3372 Return Expected from createRTDyldELFObject.
No functionality change, it just makes it easier to use Expected in
Object.

llvm-svn: 315348
2017-10-10 19:14:30 +00:00
Rafael Espindola 563ede149a Simplify. NFC.
llvm-svn: 315347
2017-10-10 19:07:10 +00:00
Lang Hames 705db63ce1 [ORC] Fix the type of RTDyldObjectLinkingLayer::NotifyLoadedFtor.
Bug found by Stefan Granitz. Thanks Stefan!

llvm-svn: 314436
2017-09-28 17:43:07 +00:00
Saleem Abdulrasool 562630a1fe Revert "Revert "ExecutionEngine: add R_AARCH64_ABS{16,32}""
This reverts commit SVN r313668.  The original test case attempted to
write a pointer value into 16-bits, although the value may exceed the
range representable in 16-bits.  Ensure that the symbol is located in
the address space such that its absolute address is representable in
16-bits.  This should fix the assertion failure that was seen on the
Windows hosts.

llvm-svn: 313822
2017-09-20 21:32:44 +00:00
Saleem Abdulrasool 5c37d57e15 Revert "ExecutionEngine: add R_AARCH64_ABS{16,32}"
This reverts commit SVN r313654.  Seems that it is triggering an
assertion on Windows specifically.  Revert until I can build on Windows
and look into what is happening there.

llvm-svn: 313668
2017-09-19 20:35:25 +00:00
Saleem Abdulrasool 6567ecd741 ExecutionEngine: add R_AARCH64_ABS{16,32}
Add support for the R_AARCH64_ABS{16,32} relocations in the execution
engine.  This is primarily used for DWARF debug information relocations
and needed by the LLVM JIT to support JITing for lldb.

Patch by Alex Langford!

llvm-svn: 313654
2017-09-19 18:00:50 +00:00
Lang Hames ec30063ff3 [ORC] Hook up the LLVMOrcAddObjectFile function in the Orc C Bindings.
This can be used to add a relocatable object to the JIT session.

llvm-svn: 313474
2017-09-17 03:25:03 +00:00
Lang Hames d4d6a1aa9b [ORC] Fix a typo.
llvm-svn: 313346
2017-09-15 06:50:19 +00:00
Lang Hames 617fc35637 [ORC] Add a pair of ORC layers that forward object-layer operations via RPC.
This patch introduces RemoteObjectClientLayer and RemoteObjectServerLayer,
which can be used to forward ORC object-layer operations from a JIT stack in
the client to a JIT stack (consisting only of object-layers) in the server.

This is a new way to support remote-JITing in LLVM. The previous approach
(supported by OrcRemoteTargetClient and OrcRemoteTargetServer) used a
remote-mapping memory manager that sat "beneath" the JIT stack and sent
fully-relocated binary blobs to the server. The main advantage of the new
approach is that relocatable objects can be cached on the server and re-used
(if the code that they represent hasn't changed), whereas fully-relocated blobs
can not (since the addresses they have been permanently bound to will change
from run to run).

llvm-svn: 312511
2017-09-05 03:34:09 +00:00
Lang Hames 8a6bab78f6 [ORC] Add an Error return to the JITCompileCallbackManager::grow method.
Calling grow may result in an error if, for example, this is a callback
manager for a remote target. We need to be able to return this error to the
callee.

llvm-svn: 312429
2017-09-03 00:50:42 +00:00
Lang Hames 1aad27e17b [Orc] Add a comment about member variable dependencies to OrcMCJITReplacement.
The comment explains the reason behind the change in member variable order in
r312086.

Thanks to Philip Reames for the suggestion.

llvm-svn: 312205
2017-08-31 01:09:56 +00:00
Lang Hames 5409d95d5b [Orc] Fix member variable ordering issue in OrcMCJITReplacement.
https://reviews.llvm.org/D36888

From that review description:

When an OrcMCJITReplacement object gets destructed, LazyEmitLayer may still
contain a shared_ptr of a module, which requires ShouldDelete in the deleter.
But ShouldDelete gets destructed before LazyEmitLayer due to the order of
declaration in OrcMCJITReplacement, which leads to a crash, when the destructor
of LazyEmitLayer is executed.  Changing the order of declaration fixes this.

Patch by Moritz Kroll. Thanks Moritz!

llvm-svn: 312086
2017-08-30 00:47:42 +00:00
NAKAMURA Takumi a1e97a77f5 Untabify.
llvm-svn: 311875
2017-08-28 06:47:47 +00:00
Lang Hames 359983bdd3 [ORC] Add case statements for AArch64 to the local stub and callback manager
creation functions.

This should allow lli to lazily execute code using OrcLazyJIT on AArch64.

llvm-svn: 310938
2017-08-15 18:10:19 +00:00
Michal Gorny 348240536f [cmake] Expose the dependencies of ExecutionEngine as PUBLIC
Expose the dependencies of LLVMExecutionEngine library as PUBLIC rather
than PRIVATE when building a shared library. This is necessary because
the library is not contained but exposes API of other LLVM libraries via
its headers.

This causes other libraries to fail to link if the linker verifies for
correctness of -l flags (i.e. fails on indirect dependencies). This e.g.
happens when building LLDB against shared LLVM:

  lib64/liblldbExpression.a(IRExecutionUnit.cpp.o):(.data.rel.ro._ZTIN4llvm18MCJITMemoryManagerE[_ZTIN4llvm18MCJITMemoryManagerE]+0x10): undefined reference to `typeinfo for llvm::RuntimeDyld::MemoryManager'
  lib64/liblldbExpression.a(IRExecutionUnit.cpp.o):(.data.rel.ro._ZTVN4llvm18MCJITMemoryManagerE[_ZTVN4llvm18MCJITMemoryManagerE]+0x60): undefined reference to `llvm::RuntimeDyld::MemoryManager::anchor()'
  lib64/liblldbExpression.a(IRExecutionUnit.cpp.o):(.data.rel.ro._ZTVN12lldb_private15IRExecutionUnit13MemoryManagerE[_ZTVN12lldb_private15IRExecutionUnit13MemoryManagerE]+0x48): undefined reference to `llvm::RTDyldMemoryManager::deregisterEHFrames()'
  lib64/liblldbExpression.a(IRExecutionUnit.cpp.o):(.data.rel.ro._ZTVN12lldb_private15IRExecutionUnit13MemoryManagerE[_ZTVN12lldb_private15IRExecutionUnit13MemoryManagerE]+0x60): undefined reference to `llvm::RuntimeDyld::MemoryManager::anchor()'
  lib64/liblldbExpression.a(IRExecutionUnit.cpp.o):(.data.rel.ro._ZTVN12lldb_private15IRExecutionUnit13MemoryManagerE[_ZTVN12lldb_private15IRExecutionUnit13MemoryManagerE]+0xd0): undefined reference to `llvm::JITSymbolResolver::anchor()'
  collect2: error: ld returned 1 exit status

Declaring the dependencies as PUBLIC guarantees that any package using
the ExecutionEngine library will also get explicit -l flags for
the dependent libraries guaranteeing that the symbols exposed in headers
could be resolved.

Patch originally written by NAKAMURA Takumi.

Differential Revision: https://reviews.llvm.org/D36211

llvm-svn: 310712
2017-08-11 13:25:20 +00:00
Lang Hames 14a22a442d [RuntimeDyld][ORC] Add support for Thumb mode to RuntimeDyldMachOARM.
This patch adds support for thumb relocations to RuntimeDyldMachOARM, and adds
a target-specific flags field to JITSymbolFlags (so that on ARM we can record
whether each symbol is Thumb-mode code).

RuntimeDyldImpl::emitSection is modified to ensure that stubs memory is
correctly aligned based on the size returned by getStubAlignment().

llvm-svn: 310517
2017-08-09 20:19:27 +00:00
Rafael Espindola 79e238afee Delete Default and JITDefault code models
IMHO it is an antipattern to have a enum value that is Default.

At any given piece of code it is not clear if we have to handle
Default or if has already been mapped to a concrete value. In this
case in particular, only the target can do the mapping and it is nice
to make sure it is always done.

This deletes the two default enum values of CodeModel and uses an
explicit Optional<CodeModel> when it is possible that it is
unspecified.

llvm-svn: 309911
2017-08-03 02:16:21 +00:00
NAKAMURA Takumi 4c29ca4b9b RuntimeDyldELF.cpp: Prune unused "TargetRegistry.h"
llvm-svn: 308846
2017-07-23 11:47:22 +00:00
Rafael Espindola 3ee9e11acb Remove some leftover DWARFContextInMemory.
Not sure how I missed these on the previous commit.

llvm-svn: 308550
2017-07-19 23:34:59 +00:00
Lang Hames 2306f9c5d2 [RuntimeDyld][MachO/ARM] Don't add a redundant relocation entry.
We only need to add this entry once for it to be fixed up.

llvm-svn: 308375
2017-07-18 21:12:03 +00:00
Lang Hames 4ce98662e7 [ORC] Errorize the ORC APIs.
This patch updates the ORC layers and utilities to return and propagate
llvm::Errors where appropriate. This is necessary to allow ORC to safely handle
error cases in cross-process and remote JITing.

llvm-svn: 307350
2017-07-07 02:59:13 +00:00
David Blaikie 8713157747 DebugInfo: Generalize LoadedObjectInfoHelper from RuntimeDyld
Make it usable by any class derived (even indirectly) from
LoadedObjectInfo by allowing a custom base class to be specified and
perfect forwarding to the ctor.

llvm-svn: 307166
2017-07-05 15:23:56 +00:00
Lang Hames 5b51816020 [Orc] Remove the memory manager argument to addModule, and de-templatize the
symbol resolver argument.

De-templatizing the symbol resolver is part of the ongoing simplification of
ORC layer API.

Removing the memory management argument (and delegating construction of memory
managers for RTDyldObjectLinkingLayer to a functor passed in to the constructor)
allows us to build JITs whose base object layers need not be compatible with
RTDyldObjectLinkingLayer's memory mangement scheme. For example, a 'remote
object layer' that sends fully relocatable objects directly to the remote does
not need a memory management scheme at all (that will be handled by the remote).

llvm-svn: 307058
2017-07-04 04:42:30 +00:00
Reid Kleckner f03096b3c3 Attempt to fix Orc JIT test timeouts
I think there are some destruction ordering issues here. The
ShouldDelete map seems to be getting destroyed before the shared_ptr
deleter lambda accesses it. In any case, this avoids inserting elements
into the map during shutdown.

llvm-svn: 306736
2017-06-29 20:15:08 +00:00
Sam Clegg 3d65030c45 Remove `inline` keyword from inline `classof` methods
The style guide states that the explicit `inline`
should not be used with inline methods.  classof is
very common inline method with a fair amount on
inconsistency:

$ git grep classof ./include | grep inline | wc -l
230
$ git grep classof ./include | grep -v inline | wc -l
257

I chose to target this method rather the larger change
since this method is easily cargo-culted (I did it at
least once).  I considered doing the larger change and
removing all occurrences but that would be a much larger
change.

Differential Revision: https://reviews.llvm.org/D33906

llvm-svn: 306731
2017-06-29 19:35:17 +00:00
Lang Hames cd9d49b605 [ORC] Re-apply r306166 and r306168 with fix for regression test.
llvm-svn: 306182
2017-06-23 23:25:28 +00:00
Rafael Espindola f6242c3e90 This reverts commit r306166 and r306168.
Revert "[ORC] Remove redundant semicolons from DEFINE_SIMPLE_CONVERSION_FUNCTIONS uses."
Revert "[ORC] Move ORC IR layer interface from addModuleSet to addModule and fix the module type as std::shared_ptr<Module>."

They broke ExecutionEngine/OrcMCJIT/test-global-ctors.ll on linux.

llvm-svn: 306176
2017-06-23 22:50:24 +00:00
Lang Hames eabde9306d [ORC] Remove redundant semicolons from DEFINE_SIMPLE_CONVERSION_FUNCTIONS uses.
llvm-svn: 306168
2017-06-23 21:56:09 +00:00
Lang Hames 2c19c1be56 [ORC] Move ORC IR layer interface from addModuleSet to addModule and fix the
module type as std::shared_ptr<Module>.

llvm-svn: 306166
2017-06-23 21:45:29 +00:00
Lang Hames 266202236f [ORC] Switch the object layer API from addObjectSet to addObject (singular), and
move the ObjectCache from the IRCompileLayer to SimpleCompiler.

This is the first in a series of patches aimed at cleaning up and improving the
robustness and performance of the ORC APIs.

llvm-svn: 306058
2017-06-22 21:06:54 +00:00
Sagar Thakur 15126308c8 Revert [mips] Adds support for R_MIPS_26, HIGHER, HIGHEST relocations in RuntimeDyld
Reverting due to build bot failures

llvm-svn: 306000
2017-06-22 12:48:04 +00:00
Sagar Thakur f8858d0979 [mips] Adds support for R_MIPS_26, HIGHER, HIGHEST relocations in RuntimeDyld
After the N64 static relocation model support was added to llvm it is required to add its support in RuntimeDyld also because lldb uses ExecutionEngine for evaluating expressions.

Reviewed by sdardis
Differential: D31649

llvm-svn: 305997
2017-06-22 11:49:19 +00:00
Eugene Zelenko f292a2feca [ExecutionEngine] Fix some Clang-tidy modernize-use-using and Include What You Use warnings; other minor fixes (NFC).
llvm-svn: 305760
2017-06-19 23:37:52 +00:00
Zachary Turner 264b5d9e88 Move Object format code to lib/BinaryFormat.
This creates a new library called BinaryFormat that has all of
the headers from llvm/Support containing structure and layout
definitions for various types of binary formats like dwarf, coff,
elf, etc as well as the code for identifying a file from its
magic.

Differential Revision: https://reviews.llvm.org/D33843

llvm-svn: 304864
2017-06-07 03:48:56 +00:00
Chandler Carruth 6bda14b313 Sort the remaining #include lines in include/... and lib/....
I did this a long time ago with a janky python script, but now
clang-format has built-in support for this. I fed clang-format every
line with a #include and let it re-sort things according to the precise
LLVM rules for include ordering baked into clang-format these days.

I've reverted a number of files where the results of sorting includes
isn't healthy. Either places where we have legacy code relying on
particular include ordering (where possible, I'll fix these separately)
or where we have particular formatting around #include lines that
I didn't want to disturb in this patch.

This patch is *entirely* mechanical. If you get merge conflicts or
anything, just ignore the changes in this patch and run clang-format
over your #include lines in the files.

Sorry for any noise here, but it is important to keep these things
stable. I was seeing an increasing number of patches with irrelevant
re-ordering of #include lines because clang-format was used. This patch
at least isolates that churn, makes it easy to skip when resolving
conflicts, and gets us to a clean baseline (again).

llvm-svn: 304787
2017-06-06 11:49:48 +00:00
Ulrich Weigand fe9fcb8af3 [RuntimeDyld, PowerPC] Fix regression from r303637
Actually, to identify external symbols, we need to check for
*either* non-null Value.SymbolName *or* a SymType of
Symbol::ST_Unknown.

The former may happen for symbols not known to the JIT at all
(e.g. defined in a native library), while the latter happens
for symbols known to the JIT, but defined in a different module.

Fixed several regressions on big-endian ppc64.

llvm-svn: 303655
2017-05-23 17:03:23 +00:00
Ulrich Weigand 7f02d67fce [RuntimeDyld, PowerPC] Fix check for external symbols when detecting reloction overflow
The PowerPC part of processRelocationRef currently assumes that external
symbols can be identified by checking for SymType == SymbolRef::ST_Unknown.
This is actually incorrect in some cases, causing relocation overflows to
be mis-detected. The correct check is to test whether Value.SymbolName
is null.

Includes test case. Note that it is a bit tricky to replicate the exact
condition that triggers the bug in a test case. The one included here
seems to fail reliably (before the fix) across different operating
system versions on Power, but it still makes a few assumptions (called
out in the test case comments).

Also add ppc64le platform name to the supported list in the lit.local.cfg
files for the MCJIT and OrcMCJIT directories, since those tests were
currently not run at all.

Fixes PR32650.

Reviewer: hfinkel

Differential Revision: https://reviews.llvm.org/D33402

llvm-svn: 303637
2017-05-23 14:51:18 +00:00
Ulrich Weigand b6d40cceee [RuntimeDyld, PowerPC] Fix relocation detection overflow
Code in RuntimeDyldELF currently uses 32-bit temporaries to detect
whether a PPC64 relocation target is out of range. This is incorrect,
and can mis-detect overflow where the distance between relocation site
and target is close to a multiple of 4GB. Fixed by using 64-bit
temporaries.

Noticed while debugging PR32650.

Reviewer: hfinkel

Differential Revision: https://reviews.llvm.org/D33403

llvm-svn: 303632
2017-05-23 12:43:57 +00:00
Pavel Labath 859d302349 [RuntimeDyld] Fix debug section relocation (pr20457)
Summary:
Debug info sections, (or non-SHF_ALLOC sections in general) should be
linked as if their load address was zero to emulate the behavior of the
static linker.

This bug was discovered because it was breaking lldb expression evaluation on
linux.

Reviewers: lhames

Subscribers: aprantl, eugene, clayborg, lldb-commits, llvm-commits

Differential Revision: https://reviews.llvm.org/D32899

llvm-svn: 303239
2017-05-17 08:47:28 +00:00
Lang Hames c936ac7f37 [ExecutionEngine] Make RuntimeDyld::MemoryManager responsible for tracking EH
frames.

RuntimeDyld was previously responsible for tracking allocated EH frames, but it
makes more sense to have the RuntimeDyld::MemoryManager track them (since the
frames are allocated through the memory manager, and written to memory owned by
the memory manager). This patch moves the frame tracking into
RTDyldMemoryManager, and changes the deregisterFrames method on
RuntimeDyld::MemoryManager from:

void deregisterEHFrames(uint8_t *Addr, uint64_t LoadAddr, size_t Size);

to:

void deregisterEHFrames();

Separating this responsibility will allow ORC to continue to throw the
RuntimeDyld instances away post-link (saving a few dozen bytes per lazy
function) while properly deregistering frames when modules are unloaded.

This patch also updates ORC to call deregisterEHFrames when modules are
unloaded. This fixes a bug where an exception that tears down the JIT can then
unwind through dangling EH frames that have been deallocated but not
deregistered, resulting in UB.

For people using SectionMemoryManager this should be pretty much a no-op. For
people with custom allocators that override registerEHFrames/deregisterEHFrames,
you will now be responsible for tracking allocated EH frames.

Reviewed in https://reviews.llvm.org/D32829

llvm-svn: 302589
2017-05-09 21:32:18 +00:00