This reverts commit r342939.
MSVC's promise/future implementation does not like types that are not default
constructible. Reverting while I figure out a solution.
llvm-svn: 342941
Asynchronous resolution (where the caller receives a callback once the requested
set of symbols are resolved) is a core part of the new concurrent ORC APIs. This
change extends the asynchronous resolution model down to RuntimeDyld, which is
necessary to prevent deadlocks when compiling/linking on a fixed number of
threads: If RuntimeDyld's linking process were a blocking operation, then any
complete K-graph in a program will require at least K threads to link in the
worst case, as each thread would block waiting for all the others to complete.
Using callbacks instead allows the work to be passed between dependent threads
until it is complete.
For backwards compatibility, all existing RuntimeDyld functions will continue
to operate in blocking mode as before. This change will enable the introduction
of a new async finalization process in a subsequent patch to enable asynchronous
JIT linking.
llvm-svn: 342939
The DEBUG() macro is very generic so it might clash with other projects.
The renaming was done as follows:
- git grep -l 'DEBUG' | xargs sed -i 's/\bDEBUG\s\?(/LLVM_DEBUG(/g'
- git diff -U0 master | ../clang/tools/clang-format/clang-format-diff.py -i -p1 -style LLVM
- Manual change to APInt
- Manually chage DOCS as regex doesn't match it.
In the transition period the DEBUG() macro is still present and aliased
to the LLVM_DEBUG() one.
Differential Revision: https://reviews.llvm.org/D43624
llvm-svn: 332240
orc::SymbolResolver to JITSymbolResolver adapter.
The new orc::SymbolResolver interface uses asynchronous queries for better
performance. (Asynchronous queries with bulk lookup minimize RPC/IPC overhead,
support parallel incoming queries, and expose more available work for
distribution). Existing ORC layers will soon be updated to use the
orc::SymbolResolver API rather than the legacy llvm::JITSymbolResolver API.
Because RuntimeDyld still uses JITSymbolResolver, this patch also includes an
adapter that wraps an orc::SymbolResolver with a JITSymbolResolver API.
llvm-svn: 323073
ExternalSymbolMap now stores the string key (rather than using a StringRef),
as the object file backing the key may be removed at any time.
llvm-svn: 323001
Bulk queries reduce IPC/RPC overhead for cross-process JITing and expose
opportunities for parallel compilation.
The two new query methods are lookupFlags, which finds the flags for each of a
set of symbols; and lookup, which finds the address and flags for each of a
set of symbols. (See doxygen comments for more details.)
The existing JITSymbolResolver class is renamed LegacyJITSymbolResolver, and
modified to extend the new JITSymbolResolver class using the following scheme:
- lookupFlags is implemented by calling findSymbolInLogicalDylib for each of the
symbols, then returning the result of calling getFlags() on each of these
symbols. (Importantly: lookupFlags does NOT call getAddress on the returned
symbols, so lookupFlags will never trigger materialization, and lookupFlags will
never call findSymbol, so only symbols that are part of the logical dylib will
return results.)
- lookup is implemented by calling findSymbolInLogicalDylib for each symbol and
falling back to findSymbol if findSymbolInLogicalDylib returns a null result.
Assuming a symbol is found its getAddress method is called to materialize it and
the result (if getAddress succeeds) is stored in the result map, or the error
(if getAddress fails) is returned immediately from lookup. If any symbol is not
found then lookup returns immediately with an error.
This change will break any out-of-tree derivatives of JITSymbolResolver. This
can be fixed by updating those classes to derive from LegacyJITSymbolResolver
instead.
llvm-svn: 322913
This patch updates the ORC layers and utilities to return and propagate
llvm::Errors where appropriate. This is necessary to allow ORC to safely handle
error cases in cross-process and remote JITing.
llvm-svn: 307350
Currently llvm-rtdyld in -check mode will map sections to back-to-back 4k
aligned slabs starting at 0x1000. Automatically remapping sections by default is
helpful because it quickly exposes relocation bugs due to use of local addresses
rather than load addresses (these would silently pass if the load address was
not remapped). These mappings can be explicitly overridden on a per-section
basis using llvm-rtdlyd's -map-section option. This patch extends this scheme to
also preserve any mappings made by RuntimeDyld itself. Preserving RuntimeDyld's
automatic mappings allows us to write test cases to verify that these automatic
mappings have been applied.
This will allow the fix in https://reviews.llvm.org/D32899 to be tested with
llvm-rtdyld -check.
llvm-svn: 302372
Summary:
Remove naked access to the data members in `SectionEntry` and route
accesses through accessor functions. This makes it obvious how the
instances of the class are used, and will also facilitate adding bounds
checking to `advanceStubOffset` in a later change.
Reviewers: lhames, loladiro, andrew.w.kaylor
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D14674
llvm-svn: 253918
Requested by Eugene Rozenfeld of the LLILC team, this feature allows JIT
clients to skip relocations for selected external symbols by returning ~0ULL
from their symbol resolver. If this value is returned for a given symbol,
RuntimeDyld will skip all relocations for that symbol. The client will be
responsible for applying the skipped relocations manually before the code
is executed.
llvm-svn: 241383
The patch is generated using this command:
tools/clang/tools/extra/clang-tidy/tool/run-clang-tidy.py -fix \
-checks=-*,llvm-namespace-comment -header-filter='llvm/.*|clang/.*' \
llvm/lib/
Thanks to Eugene Kosov for the original patch!
llvm-svn: 240137
MCJIT.
This patch decouples the two responsibilities of the RTDyldMemoryManager class,
memory management and symbol resolution, into two new classes:
RuntimeDyld::MemoryManager and RuntimeDyld::SymbolResolver.
The symbol resolution interface is modified slightly, from:
uint64_t getSymbolAddress(const std::string &Name);
to:
RuntimeDyld::SymbolInfo findSymbol(const std::string &Name);
The latter passes symbol flags along with symbol addresses, allowing RuntimeDyld
and others to reason about non-strong/non-exported symbols.
The memory management interface removes the following method:
void notifyObjectLoaded(ExecutionEngine *EE,
const object::ObjectFile &) {}
as it is not related to memory management. (Note: Backwards compatibility *is*
maintained for this method in MCJIT and OrcMCJITReplacement, see below).
The RTDyldMemoryManager class remains in-tree for backwards compatibility.
It inherits directly from RuntimeDyld::SymbolResolver, and indirectly from
RuntimeDyld::MemoryManager via the new MCJITMemoryManager class, which
just subclasses RuntimeDyld::MemoryManager and reintroduces the
notifyObjectLoaded method for backwards compatibility).
The EngineBuilder class retains the existing method:
EngineBuilder&
setMCJITMemoryManager(std::unique_ptr<RTDyldMemoryManager> mcjmm);
and includes two new methods:
EngineBuilder&
setMemoryManager(std::unique_ptr<MCJITMemoryManager> MM);
EngineBuilder&
setSymbolResolver(std::unique_ptr<RuntimeDyld::SymbolResolver> SR);
Clients should use EITHER:
A single call to setMCJITMemoryManager with an RTDyldMemoryManager.
OR (exclusive)
One call each to each of setMemoryManager and setSymbolResolver.
This patch should be fully compatible with existing uses of RTDyldMemoryManager.
If it is not it should be considered a bug, and the patch either fixed or
reverted.
If clients find the new API to be an improvement the goal will be to deprecate
and eventually remove the RTDyldMemoryManager class in favor of the new classes.
llvm-svn: 233509
Author: Lang Hames <lhames@gmail.com>
Date: Mon Mar 9 23:51:09 2015 +0000
[Orc][MCJIT][RuntimeDyld] Add header that was accidentally left out of r231724.
Author: Lang Hames <lhames@gmail.com>
Date: Mon Mar 9 23:44:13 2015 +0000
[Orc][MCJIT][RuntimeDyld] Add symbol flags to symbols in RuntimeDyld. Thread the
new types through MCJIT and Orc.
In particular, add a 'weak' flag. When plumbed through RTDyldMemoryManager, this
will allow us to distinguish between weak and strong definitions and find the
right ones during symbol resolution.
llvm-svn: 231731
new types through MCJIT and Orc.
In particular, add a 'weak' flag. When plumbed through RTDyldMemoryManager, this
will allow us to distinguish between weak and strong definitions and find the
right ones during symbol resolution.
llvm-svn: 231724
RuntimeDyld symbol info previously consisted of just a Section/Offset pair. This
patch replaces that pair type with a SymbolInfo class that also tracks symbol
visibility. A new method, RuntimeDyld::getExportedSymbolLoadAddress, is
introduced which only returns a non-zero result for exported symbols. For
non-exported or non-existant symbols this method will return zero. The
RuntimeDyld::getSymbolAddress method retains its current behavior, returning
non-zero results for all symbols regardless of visibility.
No in-tree clients of RuntimeDyld are changed. The newly introduced
functionality will be used by the Orc APIs.
No test case: Since this patch doesn't modify the behavior for any in-tree
clients we don't have a good tool to test this with yet. Once Orc is in we can
use it to write regression tests that test these changes.
llvm-svn: 226341
utils/sort_includes.py.
I clearly haven't done this in a while, so more changed than usual. This
even uncovered a missing include from the InstrProf library that I've
added. No functionality changed here, just mechanical cleanup of the
include order.
llvm-svn: 225974
With this patch MCDisassembler::getInstruction takes an ArrayRef<uint8_t>
instead of a MemoryObject.
Even on X86 there is a maximum size an instruction can have. Given
that, it seems way simpler and more efficient to just pass an ArrayRef
to the disassembler instead of a MemoryObject and have it do a virtual
call every time it wants some extra bytes.
llvm-svn: 221751
A "stub found found" diagnostic is emitted when RuntimeDyldChecker's stub lookup
logic fails to find the requested stub. The obvious reason for the failure is
that no such stub has been created, but it can also fail for internal symbols if
the symbol offset is not computed correctly (E.g. due to a mangled relocation
addend). This patch adds a comment about the latter case so that it's not
overlooked.
Inspired by confusion experienced during test case construction for r217635.
llvm-svn: 217643
field of RelocationValueRef, rather than the 'Addend' field.
This is consistent with RuntimeDyldELF's use of RelocationValueRef, and more
consistent with the semantics of the data being stored (the offset from the
start of a section or symbol).
llvm-svn: 217328
The syntax of the new builtin is 'section_addr(<filename>, <section-name>)'
(similar to the stub_addr builtin, but without a symbol name). It returns the
base address of the given section in the given object file. This builtin makes
it possible to refer to the contents of sections that cannot contain symbols,
e.g. sections added by the linker itself, like __eh_frame.
llvm-svn: 217010
use in -verify mode.
This patch adds three hidden command line options to llvm-rtdyld:
-target-addr-start <start-addr> : Specify the start of the virtual address
space on the phony target.
-target-addr-end <end-addr> : Specify the end of the virtual address space
on the phony target.
-target-section-sep <sep> : Specify the separation (in bytes) between the
end of one section and the start of the next.
These options automatically default to sane values for the target platform. In
particular, they allow narrow (e.g. 32-bit, 16-bit) targets to be tested from
wider (e.g. 64-bit, 32-bit) hosts without overflowing pointers.
The section separation option defaults to zero, but can be set to a large number
(e.g. 1 << 32) to force large separations between sections in order to
stress-test large-code-model code.
llvm-svn: 214255
full paths for its first argument.
This allows us to remove the annoying sed lines in the test cases, and write
direct references to file names in stub_addr calls (rather than <filename>
placeholders).
llvm-svn: 214211
RuntimeDyldChecker had been testing isalpha(Expr[0]) to recognise symbol tokens,
and throwing unrecognized token errors when it hit symbols with leading
underscores. This fixes that.
llvm-svn: 213706
This patch introduces a 'stub_addr' builtin that can be used to find the address
of the stub for a given (<file>, <section>, <symbol>) tuple. This address can be
used both to verify the contents of stubs (by loading from the returned address)
and to verify references to stubs (by comparing against the returned address).
Example (1) - Verifying stub contents:
Load 8 bytes (assuming a 64-bit target) from the stub for 'x' in the __text
section of f.o, and compare that value against the addres of 'x'.
# rtdyld-check: *{8}(stub_addr(f.o, __text, x) = x
Example (2) - Verifying references to stubs:
Decode the immediate of the instruction at label 'l', and verify that it's
equal to the offset from the next instruction's PC to the stub for 'y' in the
__text section of f.o (i.e. it's the correct PC-rel difference).
# rtdyld-check: decode_operand(l, 4) = stub_addr(f.o, __text, y) - next_pc(l)
l:
movq y@GOTPCREL(%rip), %rax
Since stub inspection requires cooperation with RuntimeDyldImpl this patch
pimpl-ifies RuntimeDyldChecker. Its implementation is moved in to a new class,
RuntimeDyldCheckerImpl, that has access to the definition of RuntimeDyldImpl.
llvm-svn: 213698
The compiler often emits assembler-local labels (beginning with 'L') for use in
relocation expressions, however these aren't included in the object files.
Teach RuntimeDyldChecker to warn the user if they try to use one of these in an
expression, since it will never work.
llvm-svn: 212777
Lang Hames