This reverts commit 6b2a96285b.
The ccache builders are still failing. Looks like they need to be updated to
get the llvm-zorg config change in 490633945677656ba75d42ff1ca9d4a400b7b243.
I'll re-apply this as soon as the builders are updated.
This reapplies commit a7733e9556 ("Re-apply
[ORC][ORC-RT] Add initial native-TLV support to MachOPlatform."), and
d4abdefc99 ("[ORC-RT] Rename macho_tlv.x86-64.s
to macho_tlv.x86-64.S (uppercase suffix)").
These patches were reverted in 48aa82cacb while I
investigated bot failures (e.g.
https://lab.llvm.org/buildbot/#/builders/109/builds/18981). The fix was to
disable building of the ORC runtime on buliders using ccache (which is the same
fix used for other compiler-rt projects containing assembly code). This fix was
commited to llvm-zorg in 490633945677656ba75d42ff1ca9d4a400b7b243.
This reverts commit d4abdefc99 ("[ORC-RT] Rename
macho_tlv.x86-64.s to macho_tlv.x86-64.S (uppercase suffix)", and
a7733e9556 ("Re-apply "[ORC][ORC-RT] Add initial
native-TLV support to MachOPlatform."), while I investigate failures on
ccache builders (e.g. https://lab.llvm.org/buildbot/#/builders/109/builds/18981)
Reapplies fe1fa43f16, which was reverted in
6d8c63946c, with fixes:
1. Remove .subsections_via_symbols directive from macho_tlv.x86-64.s (it's
not needed here anyway).
2. Return error from pthread_key_create to the MachOPlatform to silence unused
variable warning.
Adds code to LLVM (MachOPlatform) and the ORC runtime to support native MachO
thread local variables. Adding new TLVs to a JITDylib at runtime is supported.
On the LLVM side MachOPlatform is updated to:
1. Identify thread local variables in the LinkGraph and lower them to GOT
accesses to data in the __thread_data or __thread_bss sections.
2. Merge and report the address range of __thread_data and thread_bss sections
to the runtime.
On the ORC runtime a MachOTLVManager class introduced which records the address
range of thread data/bss sections, and creates thread-local instances from the
initial data on demand. An orc-runtime specific tlv_get_addr implementation is
included which saves all register state then calls the MachOTLVManager to get
the address of the requested variable for the current thread.
This member is now only used when storage is heap-allocated so it does not
need to be const. Dropping 'const' eliminates cast warnings on many builders.
Adds support for MachO static initializers/deinitializers and eh-frame
registration via the ORC runtime.
This commit introduces cooperative support code into the ORC runtime and ORC
LLVM libraries (especially the MachOPlatform class) to support macho runtime
features for JIT'd code. This commit introduces support for static
initializers, static destructors (via cxa_atexit interposition), and eh-frame
registration. Near-future commits will add support for MachO native
thread-local variables, and language runtime registration (e.g. for Objective-C
and Swift).
The llvm-jitlink tool is updated to use the ORC runtime where available, and
regression tests for the new MachOPlatform support are added to compiler-rt.
Notable changes on the ORC runtime side:
1. The new macho_platform.h / macho_platform.cpp files contain the bulk of the
runtime-side support. This includes eh-frame registration; jit versions of
dlopen, dlsym, and dlclose; a cxa_atexit interpose to record static destructors,
and an '__orc_rt_macho_run_program' function that defines running a JIT'd MachO
program in terms of the jit- dlopen/dlsym/dlclose functions.
2. Replaces JITTargetAddress (and casting operations) with ExecutorAddress
(copied from LLVM) to improve type-safety of address management.
3. Adds serialization support for ExecutorAddress and unordered_map types to
the runtime-side Simple Packed Serialization code.
4. Adds orc-runtime regression tests to ensure that static initializers and
cxa-atexit interposes work as expected.
Notable changes on the LLVM side:
1. The MachOPlatform class is updated to:
1.1. Load the ORC runtime into the ExecutionSession.
1.2. Set up standard aliases for macho-specific runtime functions. E.g.
___cxa_atexit -> ___orc_rt_macho_cxa_atexit.
1.3. Install the MachOPlatformPlugin to scrape LinkGraphs for information
needed to support MachO features (e.g. eh-frames, mod-inits), and
communicate this information to the runtime.
1.4. Provide entry-points that the runtime can call to request initializers,
perform symbol lookup, and request deinitialiers (the latter is
implemented as an empty placeholder as macho object deinits are rarely
used).
1.5. Create a MachO header object for each JITDylib (defining the __mh_header
and __dso_handle symbols).
2. The llvm-jitlink tool (and llvm-jitlink-executor) are updated to use the
runtime when available.
3. A `lookupInitSymbolsAsync` method is added to the Platform base class. This
can be used to issue an async lookup for initializer symbols. The existing
`lookupInitSymbols` method is retained (the GenericIRPlatform code is still
using it), but is deprecated and will be removed soon.
4. JIT-dispatch support code is added to ExecutorProcessControl.
The JIT-dispatch system allows handlers in the JIT process to be associated with
'tag' symbols in the executor, and allows the executor to make remote procedure
calls back to the JIT process (via __orc_rt_jit_dispatch) using those tags.
The primary use case is ORC runtime code that needs to call bakc to handlers in
orc::Platform subclasses. E.g. __orc_rt_macho_jit_dlopen calling back to
MachOPlatform::rt_getInitializers using __orc_rt_macho_get_initializers_tag.
(The system is generic however, and could be used by non-runtime code).
The new ExecutorProcessControl::JITDispatchInfo struct provides the address
(in the executor) of the jit-dispatch function and a jit-dispatch context
object, and implementations of the dispatch function are added to
SelfExecutorProcessControl and OrcRPCExecutorProcessControl.
5. OrcRPCTPCServer is updated to support JIT-dispatch calls over ORC-RPC.
6. Serialization support for StringMap is added to the LLVM-side Simple Packed
Serialization code.
7. A JITLink::allocateBuffer operation is introduced to allocate writable memory
attached to the graph. This is used by the MachO header synthesis code, and will
be generically useful for other clients who want to create new graph content
from scratch.
This will simplify integration of this code into LLVM -- The
Simple-Packed-Serialization code can be copied near-verbatim, but
WrapperFunctionResult will require more adaptation.
WrapperFunctionResult is a C++ wrapper for __orc_rt_CWrapperFunctionResult
that automatically manages the underlying struct.
The Simple Packed Serialization (SPS) utilities support a simple serialization
scheme for wrapper function argument and result buffers:
Primitive typess (bool, char, int8_t, and uint8_t, int16_t, uint16_t, int32_t,
uint32_t, int64_t, uint64_t) are serialized in little-endian form.
SPSTuples are serialized by serializing each of the tuple members in order
without padding.
SPSSequences are serialized by serializing a sequence length (as a uint64_t)
followed by each of the elements of the sequence in order without padding.
Serialization/deserialization always involves a pair of SPS type tag (a tag
representing the serialized format to use, e.g. uint32_t, or
SPSTuple<bool, SPSString>) and a concrete type to be serialized from or
deserialized to (uint32_t, std::pair<bool, std::string>). Serialization for new
types can be implemented by specializing the SPSSerializationTraits type.
OrcRTCWrapperFunctionResult is a C struct that can be used to return serialized
results from "wrapper functions" -- functions that deserialize an argument
buffer, call through to an actual implementation function, then serialize and
return the result of that function. Wrapper functions allow calls between ORC
and the ORC Runtime to be written using a single signature,
WrapperFunctionResult(const char *ArgData, size_t ArgSize), and without coupling
either side to a particular transport mechanism (in-memory, TCP, IPC, ... the
actual mechanism will be determined by the TargetProcessControl implementation).
OrcRTCWrapperFunctionResult is designed to allow small serialized buffers to
be returned by value, with larger serialized results stored on the heap. They
also provide an error state to report failures in serialization/deserialization.
These will be used for error propagation and handling in the ORC runtime.
The implementations of these types are cut-down versions of the error
support in llvm/Support/Error.h. Most advice on llvm::Error and llvm::Expected
(e.g. from the LLVM Programmer's manual) applies equally to __orc_rt::Error
and __orc_rt::Expected. The primary difference is the mechanism for testing
and handling error types: The ORC runtime uses a new 'error_cast' operation
to replace the handleErrors family of functions. See error_cast comments in
error.h.
This is a substitute for std::apply, which we can't use until we move to c++17.
apply_tuple will be used in upcoming the upcoming wrapper-function utils code.
Add unit test infrastructure for the ORC runtime, plus a cut-down
extensible_rtti system and extensible_rtti unit test.
Removes the placeholder.cpp source file.
Differential Revision: https://reviews.llvm.org/D102080
This reapplies 1e1d75b190, which was reverted in ce1a4d5323 due to build
failures.
The unconditional dependencies on clang and llvm-jitlink in
compiler-rt/test/orc/CMakeLists.txt have been removed -- they don't appear to
be necessary, and I suspect they're the cause of the build failures seen
earlier.
Some builders failed with a missing clang dependency. E.g.
CMake Error at /Users/buildslave/jenkins/workspace/clang-stage1-RA/clang-build \
/lib/cmake/llvm/AddLLVM.cmake:1786 (add_dependencies):
The dependency target "clang" of target "check-compiler-rt" does not exist.
Reverting while I investigate.
This reverts commit 1e1d75b190.
Lang Hames