Summary:
Enables JIT-linking by RuntimeDyld of COFF objects that contain references to
dllimport symbols. This is done by recognizing symbols that start with the
reserved "__imp_" prefix and building a pointer entry to the target symbol in
the stubs area of the section. References to the "__imp_" symbol are updated to
point to this pointer.
Work in progress: The generic code is in place, but only RuntimeDyldCOFFX86_64
and RuntimeDyldCOFFI386 have been updated to look for and update references to
dllimport symbols.
Reviewers: compnerd
Subscribers: hiraditya, ributzka, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D75884
This patch allows rtdyld-check / jitlink-check expressions to be extended over
multiple lines by terminating each line with a '\'. E.g.
# llvm-rtdyld: *{8}X = \
# llvm-rtdyld: Y
X:
.quad Y
This will be used to break up some long lines in upcoming test cases.
Summary: A function that creates JITSymbolFlags from a GlobalValueSummary. Similar functions exist: fromGlobalValue(), fromObjectSymbol()
Reviewers: lhames
Reviewed By: lhames
Subscribers: hiraditya, steven_wu, dexonsmith, arphaman, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D75082
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
The argument is llvm::null() everywhere except llvm::errs() in
llvm-objdump in -DLLVM_ENABLE_ASSERTIONS=On builds. It is used by no
target but X86 in -DLLVM_ENABLE_ASSERTIONS=On builds.
If we ever have the needs to add verbose log to disassemblers, we can
record log with a member function, instead of passing it around as an
argument.
Summary:
Most libraries are defined in the lib/ directory but there are also a
few libraries defined in tools/ e.g. libLLVM, libLTO. I'm defining
"Component Libraries" as libraries defined in lib/ that may be included in
libLLVM.so. Explicitly marking the libraries in lib/ as component
libraries allows us to remove some fragile checks that attempt to
differentiate between lib/ libraries and tools/ libraires:
1. In tools/llvm-shlib, because
llvm_map_components_to_libnames(LIB_NAMES "all") returned a list of
all libraries defined in the whole project, there was custom code
needed to filter out libraries defined in tools/, none of which should
be included in libLLVM.so. This code assumed that any library
defined as static was from lib/ and everything else should be
excluded.
With this change, llvm_map_components_to_libnames(LIB_NAMES, "all")
only returns libraries that have been added to the LLVM_COMPONENT_LIBS
global cmake property, so this custom filtering logic can be removed.
Doing this also fixes the build with BUILD_SHARED_LIBS=ON
and LLVM_BUILD_LLVM_DYLIB=ON.
2. There was some code in llvm_add_library that assumed that
libraries defined in lib/ would not have LLVM_LINK_COMPONENTS or
ARG_LINK_COMPONENTS set. This is only true because libraries
defined lib lib/ use LLVMBuild.txt and don't set these values.
This code has been fixed now to check if the library has been
explicitly marked as a component library, which should now make it
easier to remove LLVMBuild at some point in the future.
I have tested this patch on Windows, MacOS and Linux with release builds
and the following combinations of CMake options:
- "" (No options)
- -DLLVM_BUILD_LLVM_DYLIB=ON
- -DLLVM_LINK_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_BUILD_LLVM_DYLIB=ON
- -DBUILD_SHARED_LIBS=ON -DLLVM_LINK_LLVM_DYLIB=ON
Reviewers: beanz, smeenai, compnerd, phosek
Reviewed By: beanz
Subscribers: wuzish, jholewinski, arsenm, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, nhaehnle, mgorny, mehdi_amini, sbc100, jgravelle-google, hiraditya, aheejin, fedor.sergeev, asb, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, steven_wu, rogfer01, MartinMosbeck, brucehoult, the_o, dexonsmith, PkmX, jocewei, jsji, dang, Jim, lenary, s.egerton, pzheng, sameer.abuasal, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D70179
It returns just a section_iterator currently and have a report_fatal_error call inside.
This change adds a way to return errors and handle them on caller sides.
The patch also changes/improves current users and adds test cases.
Differential revision: https://reviews.llvm.org/D69167
llvm-svn: 375408
Summary:
This is patch is part of a series to introduce an Alignment type.
See this thread for context: http://lists.llvm.org/pipermail/llvm-dev/2019-July/133851.html
See this patch for the introduction of the type: https://reviews.llvm.org/D64790
Reviewers: courbet, JDevlieghere, alexshap, rupprecht, jhenderson
Subscribers: sdardis, nemanjai, hiraditya, kbarton, jakehehrlich, jrtc27, MaskRay, atanasyan, jsji, seiya, cfe-commits, llvm-commits
Tags: #clang, #llvm
Differential Revision: https://reviews.llvm.org/D67499
llvm-svn: 371742
This is the main CodeGen patch to support the arm64_32 watchOS ABI in LLVM.
FastISel is mostly disabled for now since it would generate incorrect code for
ILP32.
llvm-svn: 371722
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
llvm-svn: 369013
Changes: no changes. A fix for the clang code will be landed right on top.
Original commit message:
SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.
For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)
This patch makes a change for all users to switch to Expected<> version.
Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).
Differential revision: https://reviews.llvm.org/D66089
llvm-svn: 368826
SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.
For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)
This patch makes a change for all users to switch to Expected<> version.
Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).
Differential revision: https://reviews.llvm.org/D66089
llvm-svn: 368812
Summary:
This was flagged in https://www.viva64.com/en/b/0629/ under "Snippet No.
33".
It seems that this statement is doing the standard bitwise trick for
adjusting a value to have a specific alignment.
The issue is that getStubAlignment() returns an unsigned, while DataSize
is declared a uint64_t. The right hand side of the expression is not
extended to 64b before bitwise negation, resulting in the top half of
the mask being 0s, which is not correct for realignment.
Reviewers: lhames, MaskRay
Reviewed By: MaskRay
Subscribers: RKSimon, MaskRay, hiraditya, llvm-commits, srhines
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62227
llvm-svn: 362286
increase the MachO/x86-64 stub alignment to 8.
Stub alignment should be guaranteed for any section containing RuntimeDyld
stubs/GOT-entries. To do this we should pad and align all sections containing
stubs, not just code sections.
This commit also bumps the MachO/x86-64 stub alignment to 8, so that GOT entries
will be aligned.
llvm-svn: 362139
Summary:
EH Frames aren't supported on AIX with the system compiler, but the definition of HAVE_EHTABLE_SUPPORT misses this which causes linking problems on AIX. This patch updates the definition of HAVE_EHTABLE_SUPPORT in both JITLink and RuntimeDyld.
Author: daltenty
Reviewers: sfertile, xingxue, hubert.reinterpretcase
Reviewed By: xingxue
Subscribers: hiraditya, jsji, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D62203
llvm-svn: 361410
r360876 didn't fix 2 call sites in clang.
Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.
Follow-up of D61781.
llvm-svn: 360892
It broke the Clang build, see llvm-commits thread.
> Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.
>
> Follow-up of D61781.
llvm-svn: 360878
Also updates RuntimeDyldChecker and llvm-rtdyld to support zero-fill tests by
returning a content address of zero (but no error) for zero-fill atoms, and
treating loads from zero as returning zero.
llvm-svn: 360547
Summary:
JITLink is a jit-linker that performs the same high-level task as RuntimeDyld:
it parses relocatable object files and makes their contents runnable in a target
process.
JITLink aims to improve on RuntimeDyld in several ways:
(1) A clear design intended to maximize code-sharing while minimizing coupling.
RuntimeDyld has been developed in an ad-hoc fashion for a number of years and
this had led to intermingling of code for multiple architectures (e.g. in
RuntimeDyldELF::processRelocationRef) in a way that makes the code more
difficult to read, reason about, extend. JITLink is designed to isolate
format and architecture specific code, while still sharing generic code.
(2) Support for native code models.
RuntimeDyld required the use of large code models (where calls to external
functions are made indirectly via registers) for many of platforms due to its
restrictive model for stub generation (one "stub" per symbol). JITLink allows
arbitrary mutation of the atom graph, allowing both GOT and PLT atoms to be
added naturally.
(3) Native support for asynchronous linking.
JITLink uses asynchronous calls for symbol resolution and finalization: these
callbacks are passed a continuation function that they must call to complete the
linker's work. This allows for cleaner interoperation with the new concurrent
ORC JIT APIs, while still being easily implementable in synchronous style if
asynchrony is not needed.
To maximise sharing, the design has a hierarchy of common code:
(1) Generic atom-graph data structure and algorithms (e.g. dead stripping and
| memory allocation) that are intended to be shared by all architectures.
|
+ -- (2) Shared per-format code that utilizes (1), e.g. Generic MachO to
| atom-graph parsing.
|
+ -- (3) Architecture specific code that uses (1) and (2). E.g.
JITLinkerMachO_x86_64, which adds x86-64 specific relocation
support to (2) to build and patch up the atom graph.
To support asynchronous symbol resolution and finalization, the callbacks for
these operations take continuations as arguments:
using JITLinkAsyncLookupContinuation =
std::function<void(Expected<AsyncLookupResult> LR)>;
using JITLinkAsyncLookupFunction =
std::function<void(const DenseSet<StringRef> &Symbols,
JITLinkAsyncLookupContinuation LookupContinuation)>;
using FinalizeContinuation = std::function<void(Error)>;
virtual void finalizeAsync(FinalizeContinuation OnFinalize);
In addition to its headline features, JITLink also makes other improvements:
- Dead stripping support: symbols that are not used (e.g. redundant ODR
definitions) are discarded, and take up no memory in the target process
(In contrast, RuntimeDyld supported pointer equality for weak definitions,
but the redundant definitions stayed resident in memory).
- Improved exception handling support. JITLink provides a much more extensive
eh-frame parser than RuntimeDyld, and is able to correctly fix up many
eh-frame sections that RuntimeDyld currently (silently) fails on.
- More extensive validation and error handling throughout.
This initial patch supports linking MachO/x86-64 only. Work on support for
other architectures and formats will happen in-tree.
Differential Revision: https://reviews.llvm.org/D58704
llvm-svn: 358818
This patch reduces the number of functions in the interface between RuntimeDyld
and RuntimeDyldChecker by combining "GetXAddress" and "GetXContent" functions
into "GetXInfo" functions that return a struct describing both the address and
content. The GetStubOffset function is also replaced with a pair of utilities,
GetStubInfo and GetGOTInfo, that fit the new scheme. For RuntimeDyld both of
these functions will return the same result, but for the new JITLink linker
(https://reviews.llvm.org/D58704) these will provide the addresses of PLT stubs
and GOT entries respectively.
For JITLink's use, a 'got_addr' utility has been added to the rtdyld-check
language, and the syntax of 'got_addr' and 'stub_addr' has been changed: both
functions now take two arguments, a 'stub container name' and a target symbol
name. For llvm-rtdyld/RuntimeDyld the stub container name is the object file
name and section name, separated by a slash. E.g.:
rtdyld-check: *{8}(stub_addr(foo.o/__text, y)) = y
For the upcoming llvm-jitlink utility, which creates stubs on a per-file basis
rather than a per-section basis, the container name is just the file name. E.g.:
jitlink-check: *{8}(got_addr(foo.o, y)) = y
llvm-svn: 358295
ELF sections allow 0 for the alignment, which is specified to
be the same as 1. However many clients do not expect this and
will behave poorly in the presence of a 0-aligned section (for
example by trying to modulo something by the section alignment).
We can be more polite by making sure that we always pass a
non-zero value to clients.
Differential Revision: https://reviews.llvm.org/D57482
llvm-svn: 352694
This patch tried to address the following use case.
. bcc (https://github.com/iovisor/bcc) utilizes llvm JIT to
compile for BTF target.
. with -g, .BTF and .BTF.ext sections (BPF debug info)
will be generated by LLVM.
. .BTF does not have relocations and .BTF.ext has some
relocations.
. With ProcessAllSections, .BTF.ext is loaded by JIT dynamic linker
and is available to application. But .BTF is not loaded.
The bcc application needs both .BTF.ext and .BTF for debugging
purpose, and .BTF is not loaded. This patch addressed this issue
by iterating over all sections and loading any missing
sections, after symbol/relocation processing in loadObjectImpl().
Signed-off-by: Yonghong Song <yhs@fb.com>
Differential Revision: https://reviews.llvm.org/D55943
llvm-svn: 352432
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
lldb on Windows uses the ExecutionEngine for expression evaluation
and hits the llvm_unreachable due to this relocation. Thus, implement
the relocation and add a test to verify it's function.
llvm-svn: 348904
In a lot of places an empty string was passed as the ErrorBanner to
logAllUnhandledErrors. This patch makes that argument optional to
simplify the call sites.
llvm-svn: 346604
Non-loaded sections (whose unused load-address defaults to zero) should not
be taken into account when calculating ImageBase, or ImageBase will be
incorrectly set to 0.
Patch by Andrew Scheidecker. Thanks Andrew!
https://reviews.llvm.org/D51343
+ // The Sections list may contain sections that weren't loaded for
+ // whatever reason: they may be debug sections, and ProcessAllSections
+ // is false, or they may be sections that contain 0 bytes. If the
+ // section isn't loaded, the load address will be 0, and it should not
+ // be included in the ImageBase calculation.
llvm-svn: 344995
CompileOnDemandLayer2 now supports user-supplied partition functions (the
original CompileOnDemandLayer already supported these).
Partition functions are called with the list of requested global values
(i.e. global values that currently have queries waiting on them) and have an
opportunity to select extra global values to materialize at the same time.
Also adds testing infrastructure for the new feature to lli.
llvm-svn: 343396
switch RTDyldObjectLinkingLayer2 to use it.
RuntimeDyld::loadObject is currently a blocking operation. This means that any
JIT'd code whose call-graph contains an embedded complete K graph will require
at least K threads to link, which precludes the use of a fixed sized thread
pool for concurrent JITing of arbitrary code (whatever K the thread-pool is set
at, any code with a K+1 complete subgraph will deadlock at JIT-link time).
To address this issue, this commmit introduces a function called jitLinkForORC
that uses continuation-passing style to pass the fix-up and finalization steps
to the asynchronous symbol resolver interface so that linking can be performed
without blocking.
llvm-svn: 343043
This reverts commit r342939.
MSVC's promise/future implementation does not like types that are not default
constructible. Reverting while I figure out a solution.
llvm-svn: 342941
Asynchronous resolution (where the caller receives a callback once the requested
set of symbols are resolved) is a core part of the new concurrent ORC APIs. This
change extends the asynchronous resolution model down to RuntimeDyld, which is
necessary to prevent deadlocks when compiling/linking on a fixed number of
threads: If RuntimeDyld's linking process were a blocking operation, then any
complete K-graph in a program will require at least K threads to link in the
worst case, as each thread would block waiting for all the others to complete.
Using callbacks instead allows the work to be passed between dependent threads
until it is complete.
For backwards compatibility, all existing RuntimeDyld functions will continue
to operate in blocking mode as before. This change will enable the introduction
of a new async finalization process in a subsequent patch to enable asynchronous
JIT linking.
llvm-svn: 342939
Lang Hames