Commit Graph

2568 Commits

Author SHA1 Message Date
Lang Hames e00585c77c [ORC] Fix a FIXME: Propagate errors to dependencies.
When symbols are failed (via MaterializationResponsibility::failMaterialization)
any symbols depending on them will now be moved to an error state. Attempting
to resolve or emit a symbol in the error state (via the notifyResolved or
notifyEmitted methods on MaterializationResponsibility) will result in an error.
If notifyResolved or notifyEmitted return an error due to failure of a
dependence then the caller should log or discard the error and call
failMaterialization to propagate the failure to any queries waiting on the
symbols being resolved/emitted (plus their dependencies).

llvm-svn: 369808
2019-08-23 20:37:31 +00:00
Hubert Tong 71974b5175 [cmake] Link in LLVMPasses due to dependency by LLVMOrcJIT; NFC
Summary:
rL367756 (f5c40cb) increases the dependency of LLVMOrcJIT on LLVMPasses.
In particular, symbols defined in LLVMPasses that are referenced by the
destructor of `PassBuilder` are now referenced by LLVMOrcJIT through
`Speculation.cpp.o`.

We believe that referencing symbols defined in LLVMPasses in the
destructor of `PassBuilder` is valid, and that adding to the set of such
symbols is legitimate. To support such cases, this patch adds LLVMPasses
to the set of libraries being linked when linking in LLVMOrcJIT causes
such symbols from LLVMPasses to be referenced.

Reviewers: Whitney, anhtuyen, pree-jackie

Reviewed By: pree-jackie

Subscribers: mgorny, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66441

llvm-svn: 369310
2019-08-19 23:12:48 +00:00
Matthias Gehre 5b3275e56f [ORC] fix use-after-free detected by -Wreturn-stack-address
Summary:
llvm/lib/ExecutionEngine/Orc/Layer.cpp:53:12: warning: returning address of local temporary object [-Wreturn-stack-address]

In
```
StringRef IRMaterializationUnit::getName() const {
[...]
     return TSM.withModuleDo(
        [](const Module &M) { return M.getModuleIdentifier(); });
```
`getModuleIdentifier()` returns a `const std::string &`, but the implicit return type
of the lambda is `std::string` by value, and thus the returned `StringRef` refers
to a temporary `std::string`.

Detect by annotating `llvm::StringRef` with `[[gsl::Pointer]]`.

Reviewers: lhames, sgraenitz

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D66440

llvm-svn: 369306
2019-08-19 21:59:44 +00:00
Lang Hames 9bb9a0c10b [ORC] Remove some stray debugging output accidentally left in r368707
llvm-svn: 369141
2019-08-16 19:33:37 +00:00
Jonas Devlieghere 0eaee545ee [llvm] Migrate llvm::make_unique to std::make_unique
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.

llvm-svn: 369013
2019-08-15 15:54:37 +00:00
George Rimar bcc00e1afb Recommit r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"
Changes: no changes. A fix for the clang code will be landed right on top.

Original commit message:

SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368826
2019-08-14 11:10:11 +00:00
George Rimar 468919e182 Revert r368812 "[llvm/Object] - Convert SectionRef::getName() to return Expected<>"
It broke clang BB: http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/16455

llvm-svn: 368813
2019-08-14 08:56:55 +00:00
George Rimar a0c6a35714 [llvm/Object] - Convert SectionRef::getName() to return Expected<>
SectionRef::getName() returns std::error_code now.
Returning Expected<> instead has multiple benefits.

For example, it forces user to check the error returned.
Also Expected<> may keep a valuable string error message,
what is more useful than having a error code.
(Object\invalid.test was updated to show the new messages printed.)

This patch makes a change for all users to switch to Expected<> version.

Note: in a few places the error returned was ignored before my changes.
In such places I left them ignored. My intention was to convert the interface
used, and not to improve and/or the existent users in this patch.
(Though I think this is good idea for a follow-ups to revisit such places
and either remove consumeError calls or comment each of them to clarify why
it is OK to have them).

Differential revision: https://reviews.llvm.org/D66089

llvm-svn: 368812
2019-08-14 08:46:54 +00:00
Lang Hames 52a34a78d9 [ORC] Refactor definition-generation, add a generator for static libraries.
This patch replaces the JITDylib::DefinitionGenerator typedef with a class of
the same name, and adds support for attaching a sequence of DefinitionGeneration
objects to a JITDylib.

This patch also adds a new definition generator,
StaticLibraryDefinitionGenerator, that can be used to add symbols fom a static
library to a JITDylib. An object from the static library will be added (via
a supplied ObjectLayer reference) whenever a symbol from that object is
referenced.

To enable testing, lli is updated to add support for the --extra-archive option
when running in -jit-kind=orc-lazy mode.

llvm-svn: 368707
2019-08-13 16:05:18 +00:00
Eric Christopher 4acb4ee767 Move findBBwithCalls to the file it's used in to avoid unused function
warnings.

llvm-svn: 368636
2019-08-13 00:05:01 +00:00
Benjamin Kramer 3d5360a439 Replace llvm::MutexGuard/UniqueLock with their standard equivalents
All supported platforms have <mutex> now, so we don't need our own
copies any longer. No functionality change intended.

llvm-svn: 368149
2019-08-07 10:57:25 +00:00
Diego Caballero 8bac17709e Re-land D65760/r367944
Fixed most vexing parse ambiguation.

llvm-svn: 368055
2019-08-06 16:24:17 +00:00
Puyan Lotfi 37fe40c330 Reverting D65760/r367944 due to buildbot failure.
http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/15952/steps/build/logs/stdio

JITTargetMachineBuilder.cpp fails to build.

llvm-svn: 367954
2019-08-05 23:47:07 +00:00
Diego Caballero 1647758882 [ORC] Add CPU name and sub-target features to detectHost
This commit adds host CPU name and sub-target features to the
`JITTargetMachineBuilder` created by `JITTargetMachineBuilder::detectHost()`.

Differential Revision: https://reviews.llvm.org/D65760

llvm-svn: 367944
2019-08-05 23:02:12 +00:00
Lang Hames 1707735fa4 [ORC] Work around broken GCC/libstdc++ by adding an explicit conversion.
This should fix the bots that have been failing due to r367712.

llvm-svn: 367921
2019-08-05 20:30:35 +00:00
Lang Hames 3daccaac8a [JITLink] Add support for MachO/x86-64 UNSIGNED relocs with length=2.
MachO/x86-64 UNSIGNED relocs are almost always 64-bit (length=3), but UNSIGNED
relocs of length=2 are allowed if the target resides in the low 32-bits. This
patch adds support for such relocations in JITLink (previously they would have
triggered an unsupported relocation error).

llvm-svn: 367764
2019-08-03 20:17:10 +00:00
Lang Hames b31229af4f [JITLink] Fix error message formatting.
llvm-svn: 367763
2019-08-03 20:17:08 +00:00
Praveen Velliengiri f5c40cb900 Speculative Compilation
[ORC] Remove Speculator Variants for Different Program Representations

[ORC] Block Freq Analysis

Speculative Compilation with Naive Block Frequency

Add Applications to OrcSpeculation

ORC v2 with Block Freq Query & Example

Deleted BenchMark Programs

Signed-off-by: preejackie <praveenvelliengiri@gmail.com>

ORCv2 comments resolved

[ORCV2] NFC

ORCv2 NFC

[ORCv2] Speculative compilation - CFGWalkQuery

ORCv2 Adapting IRSpeculationLayer to new locking scheme

llvm-svn: 367756
2019-08-03 14:42:13 +00:00
Lang Hames 10430f4174 [ORC] Remove a dead method.
llvm-svn: 367716
2019-08-02 20:09:30 +00:00
Lang Hames cb391279b4 [ORC] Turn on symbol-flags overrides for LLJIT on Windows by default.
libObject does not apply the Exported flag to symbols in COFF object files,
which can lead to assertions when the symbol flags initially derived from
IR added to the JIT clash with the flags seen by the JIT linker. Both
RTDyldObjectLinkingLayer and ObjectLinkingLayer have a workaround for this:
they can be told to override the flags seen by the linker with the flags
attached to the materialization responsibility object that was passed down
to the linker. This patch modifies LLJIT's setup code to enable this override
by default on platforms where COFF is the default object format.

llvm-svn: 367712
2019-08-02 19:43:20 +00:00
Lang Hames 809e9d1efa [ORC] Change the locking scheme for ThreadSafeModule.
ThreadSafeModule/ThreadSafeContext are used to manage lifetimes and locking
for LLVMContexts in ORCv2. Prior to this patch contexts were locked as soon
as an associated Module was emitted (to be compiled and linked), and were not
unlocked until the emit call returned. This could lead to deadlocks if
interdependent modules that shared contexts were compiled on different threads:
when, during emission of the first module, the dependence was discovered the
second module (which would provide the required symbol) could not be emitted as
the thread emitting the first module still held the lock.

This patch eliminates this possibility by moving to a finer-grained locking
scheme. Each client holds the module lock only while they are actively operating
on it. To make this finer grained locking simpler/safer to implement this patch
removes the explicit lock method, 'getContextLock', from ThreadSafeModule and
replaces it with a new method, 'withModuleDo', that implicitly locks the context,
calls a user-supplied function object to operate on the Module, then implicitly
unlocks the context before returning the result.

ThreadSafeModule TSM = getModule(...);
size_t NumFunctions = TSM.withModuleDo(
    [](Module &M) { // <- context locked before entry to lambda.
      return M.size();
    });

Existing ORCv2 layers that operate on ThreadSafeModules are updated to use the
new method.

This method is used to introduce Module locking into each of the existing
layers.

llvm-svn: 367686
2019-08-02 15:21:37 +00:00
Lang Hames 9e52d0576a [ORC] Suppress an ORCv1 deprecation warning.
llvm-svn: 366485
2019-07-18 19:55:42 +00:00
Lang Hames 1716454027 [ORC] Add deprecation warnings to ORCv1 layers and utilities.
Summary:
ORCv1 is deprecated. The current aim is to remove it before the LLVM 10.0
release. This patch adds deprecation attributes to the ORCv1 layers and
utilities to warn clients of the change.

Reviewers: dblaikie, sgraenitz, AlexDenisov

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D64609

llvm-svn: 366344
2019-07-17 16:40:52 +00:00
Lang Hames 843f198a83 [ORC] Add custom IR compiler configuration to LLJITBuilder to enable obj caches.
LLJITBuilder now has a setCompileFunctionCreator method which can be used to
construct a CompileFunction for the LLJIT instance being created. The motivating
use-case for this is supporting ObjectCaches, which can now be set up at
compile-function construction time. To demonstrate this an example project,
LLJITWithObjectCache, is included.

llvm-svn: 365671
2019-07-10 17:24:24 +00:00
Lang Hames f5a885fddd [JITLink][ORC] Add EHFrameRegistrar interface, use in EHFrameRegistrationPlugin.
Replaces direct calls to eh-frame registration with calls to methods on an
EHFrameRegistrar instance. This allows clients to substitute a registrar that
registers frames in a remote process via IPC/RPC.

llvm-svn: 365098
2019-07-04 00:05:12 +00:00
Erik Pilkington eee944e7f9 [C++2a] Add __builtin_bit_cast, used to implement std::bit_cast
This commit adds a new builtin, __builtin_bit_cast(T, v), which performs a
bit_cast from a value v to a type T. This expression can be evaluated at
compile time under specific circumstances.

The compile time evaluation currently doesn't support bit-fields, but I'm
planning on fixing this in a follow up (some of the logic for figuring this out
is in CodeGen). I'm also planning follow-ups for supporting some more esoteric
types that the constexpr evaluator supports, as well as extending
__builtin_memcpy constexpr evaluation to use the same infrastructure.

rdar://44987528

Differential revision: https://reviews.llvm.org/D62825

llvm-svn: 364954
2019-07-02 18:28:13 +00:00
Lang Hames 1b091540d2 [JITLink] Move JITLinkMemoryManager into its own header.
llvm-svn: 363444
2019-06-14 19:41:21 +00:00
Lang Hames 2f8c6f9362 [ORC] Rename MaterializationResponsibility resolve and emit methods to
notifyResolved/notifyEmitted.

The 'notify' prefix better describes what these methods do: they update the JIT
symbol states and notify any pending queries that the 'resolved' and 'emitted'
states have been reached (rather than actually performing the resolution or
emission themselves). Since new states are going to be introduced in the near
future (to track symbol registration/initialization) it's worth changing the
convention pre-emptively to avoid further confusion.

llvm-svn: 363322
2019-06-13 20:11:23 +00:00
Cameron McInally 670d0f478b [ExecutionEngine] Fix rL362941: Add UnaryOperator visitor to the interpreter
Missed break statements. This was D62881.

llvm-svn: 362958
2019-06-10 16:05:25 +00:00
Cameron McInally ce49e2231b [ExecutionEngine] Add UnaryOperator visitor to the interpreter
This is to support the unary FNeg instruction.

Differential Revision: https://reviews.llvm.org/D62881

llvm-svn: 362941
2019-06-10 14:38:48 +00:00
Lang Hames d4a8089f03 [ORC] Update symbol lookup to use a single callback with a required symbol state
rather than two callbacks.

The asynchronous lookup API (which the synchronous lookup API wraps for
convenience) used to take two callbacks: OnResolved (called once all requested
symbols had an address assigned) and OnReady to be called once all requested
symbols were safe to access). This patch updates the asynchronous lookup API to
take a single 'OnComplete' callback and a required state (SymbolState) to
determine when the callback should be made. This simplifies the common use case
(where the client is interested in a specific state) and will generalize neatly
as new states are introduced to track runtime initialization of symbols.

Clients who were making use of both callbacks in a single query will now need to
issue two queries (one for SymbolState::Resolved and another for
SymbolState::Ready). Synchronous lookup API clients who were explicitly passing
the WaitOnReady argument will now need neeed to pass a SymbolState instead (for
'WaitOnReady == true' use SymbolState::Ready, for 'WaitOnReady == false' use
SymbolState::Resolved). Synchronous lookup API clients who were using default
arugment values should see no change.

llvm-svn: 362832
2019-06-07 19:33:51 +00:00
Nick Desaulniers b380846a12 [RuntimeDyld] fix too-small-bitmask error
Summary:
This was flagged in https://www.viva64.com/en/b/0629/ under "Snippet No.
33".

It seems that this statement is doing the standard bitwise trick for
adjusting a value to have a specific alignment.

The issue is that getStubAlignment() returns an unsigned, while DataSize
is declared a uint64_t. The right hand side of the expression is not
extended to 64b before bitwise negation, resulting in the top half of
the mask being 0s, which is not correct for realignment.

Reviewers: lhames, MaskRay

Reviewed By: MaskRay

Subscribers: RKSimon, MaskRay, hiraditya, llvm-commits, srhines

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62227

llvm-svn: 362286
2019-06-01 04:51:26 +00:00
Lang Hames a100042b27 [RuntimeDyld] Update reserveAllocationSpace to account for stub padding.
This should fix the buildbot failures caused by r362139.

llvm-svn: 362151
2019-05-30 20:58:28 +00:00
Lang Hames 0e124b37bd [RuntimeDyld] Apply padding and alignment bumps to all sections with stubs, and
increase the MachO/x86-64 stub alignment to 8.

Stub alignment should be guaranteed for any section containing RuntimeDyld
stubs/GOT-entries. To do this we should pad and align all sections containing
stubs, not just code sections.

This commit also bumps the MachO/x86-64 stub alignment to 8, so that GOT entries
will be aligned.

llvm-svn: 362139
2019-05-30 19:59:20 +00:00
Lang Hames eb5ee3004f [ORC] Track JIT symbol states more explicitly.
Prior to this patch, JITDylibs inferred symbol states (whether a symbol was
newly added, materializing, resolved, or ready to run) via a combination of (1)
bits in the JITSymbolFlags member, and (2) the state of some internal JITDylib
data structures. This patch explicitly tracks symbol states by adding a new
SymbolState member to the symbol table entries, and removing the 'Lazy' and
'Materializing' bits from JITSymbolFlags. This is a first step towards adding
additional states representing initialization phases (e.g. eh-frame registration,
registration with the language runtime, and static initialization).

llvm-svn: 361899
2019-05-28 23:35:44 +00:00
Lang Hames 23343c5d90 [RuntimeDyld][ARM] Fix an incorrect assertion condition.
Fixes https://llvm.org/PR42036

llvm-svn: 361782
2019-05-27 21:34:31 +00:00
Galina Kistanova ed49f6d8e6 Reverted r361134 because of a failing test left unattended for a long time.
http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/17792/steps/test-check-all/logs/stdio
Failing Tests (1):
    LLVM :: CodeGen/AMDGPU/regbank-reassign.mir

llvm-svn: 361430
2019-05-22 20:42:56 +00:00
Xing Xue 4246b75295 Disable EHFrameSupport in JITLink/RuntimeDyld on AIX
Summary:
EH Frames aren't supported on AIX with the system compiler, but the definition of HAVE_EHTABLE_SUPPORT misses this which causes linking problems on AIX. This patch updates the definition of HAVE_EHTABLE_SUPPORT in both JITLink and RuntimeDyld.

Author: daltenty

Reviewers: sfertile, xingxue, hubert.reinterpretcase

Reviewed By: xingxue

Subscribers: hiraditya, jsji, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62203

llvm-svn: 361410
2019-05-22 17:41:27 +00:00
Lang Hames f088e195cc [ORC] Assert that JITDylibs have unique names.
Patch by Praveen Velliengiri. Thanks Praveen!

Differential Revision: https://reviews.llvm.org/D62139

llvm-svn: 361215
2019-05-21 03:23:08 +00:00
Nick Desaulniers 28e351af2a [ORC] fix use-after-move. NFC
Summary:
scan-build flagged a potential use-after-move in debug builds.  It's not
safe that a moved from value contains anything but garbage.  Manually
DRY up these repeated expressions.

Reviewers: lhames

Reviewed By: lhames

Subscribers: hiraditya, llvm-commits, srhines

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62112

llvm-svn: 361203
2019-05-20 22:17:43 +00:00
Lang Hames 0dcf69eb82 [ORC] Remove some unreachable code.
Fixes http://llvm.org/PR41662.

llvm-svn: 361199
2019-05-20 21:30:33 +00:00
Lang Hames 93d2bdda6b [Support] Renamed member 'Size' to 'AllocatedSize' in MemoryBlock and OwningMemoryBlock.
Rename member 'Size' to 'AllocatedSize' in order to provide a hint that the
allocated size may be different than the requested size. Comments are added to
clarify this point.  Updated the InMemoryBuffer in FileOutputBuffer.cpp to track
the requested buffer size.

Patch by Machiel van Hooren. Thanks Machiel!

https://reviews.llvm.org/D61599

llvm-svn: 361195
2019-05-20 20:53:05 +00:00
Fangrui Song 68774edcd6 Use llvm::sort. NFC
llvm-svn: 361134
2019-05-20 10:18:35 +00:00
Lang Hames c97b50e224 [ORC] Change handling for SymbolStringPtr tombstones and empty keys.
SymbolStringPtr used to use nullptr as its empty value and (since it performed
ref-count operations on any non-nullptr) a pointer to a special pool-entry
instance as its tombstone.

This commit changes the scheme to use two invalid pointer values as the empty
and tombstone values, and broadens the ref-count guard to prevent ref-counting
operations from being performed on these pointers. This should improve the
performance of SymbolStringPtrs used in DenseMaps/DenseSets, as ref counting
operations will no longer be performed on the tombstone.

llvm-svn: 360925
2019-05-16 18:29:34 +00:00
Fangrui Song e183340c29 Recommit [Object] Change object::SectionRef::getContents() to return Expected<StringRef>
r360876 didn't fix 2 call sites in clang.

Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360892
2019-05-16 13:24:04 +00:00
Hans Wennborg 4da9ff9fcf Revert r360876 "[Object] Change object::SectionRef::getContents() to return Expected<StringRef>"
It broke the Clang build, see llvm-commits thread.

> Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.
>
> Follow-up of D61781.

llvm-svn: 360878
2019-05-16 12:08:34 +00:00
Fangrui Song a076ec54be [Object] Change object::SectionRef::getContents() to return Expected<StringRef>
Expected<ArrayRef<uint8_t>> may be better but use Expected<StringRef> for now.

Follow-up of D61781.

llvm-svn: 360876
2019-05-16 11:33:48 +00:00
Lang Hames c2fb896522 [JITLink][MachO] Use getSymbol64TableEntry for 64-bit MachO files.
Fixes a think-o. No test case: The nlist and nlist64 data structures happen to
line up for this field, so there's no way to construct a failing test case.

llvm-svn: 360830
2019-05-16 00:21:07 +00:00
Lang Hames 56baade10d [JITLink][MachO] Honor the no-dead-strip flag on nlist entries.
llvm-svn: 360618
2019-05-13 20:52:30 +00:00
Lang Hames 4513929094 [JITLink] Track section alignment and make sure it is respected during layout.
Previously we had only honored alignments on individual atoms, but
tools/runtimes may assume that the section alignment is respected too.

llvm-svn: 360555
2019-05-13 04:51:31 +00:00
Lang Hames 23085ec36d [JITLink] Add a test for zero-filled content.
Also updates RuntimeDyldChecker and llvm-rtdyld to support zero-fill tests by
returning a content address of zero (but no error) for zero-fill atoms, and
treating loads from zero as returning zero.

llvm-svn: 360547
2019-05-12 22:26:33 +00:00
Lang Hames b3d6073b3c [ORC] Make a narrowing-cast explicit to silence a compiler warning.
llvm-svn: 360478
2019-05-10 22:51:03 +00:00
Lang Hames b0cecfc907 [JITLink][MachO] Mark atoms in sections 'no-dead-strip' set live by default.
If a MachO section has the no-dead-strip attribute set then its atoms should
be preserved, regardless of whether they're public or referenced elsewhere in
the object.

llvm-svn: 360477
2019-05-10 22:24:37 +00:00
Lang Hames 112967833e [JITLink] Fixed a signedness bug when processing X86_64_RELOC_SUBTRACTOR.
Subtractor relocation addends are signed, so we need to read them via signed
int pointers. Accidentally treating 32-bit addends as unsigned leads to
out-of-range errors when we try to add very large (>INT32_MAX) bogus addends.

llvm-svn: 360392
2019-05-09 23:17:41 +00:00
Lang Hames 5e332f1992 [ORC] Simplify logic for updating edges when should-discard atoms are pruned.
llvm-svn: 360384
2019-05-09 22:03:58 +00:00
Lang Hames dd61274f77 [JITLink] Improve/fix some JITLink debugging output.
Adds full edge details (rather than just edge targets) when out-of-range errors
are generated. Also fixes a bug where debugging output accessed an invalidated
DenseMap iterator by moving the debugging output above the invalidation point.

llvm-svn: 360383
2019-05-09 22:03:57 +00:00
Lang Hames 5fa4e9d990 [ORC] Fix a formatting bug.
llvm-svn: 360382
2019-05-09 22:03:53 +00:00
Sven van Haastregt ad9c7e0789 Fix LLVM_USE_PERF build after getPageSize change
Commit r360221 ("[Support] Add error handling to
sys::Process::getPageSize().", 2019-05-08) seems to have missed these
uses of getPageSize().  Update them to getPageSizeEstimate().

llvm-svn: 360322
2019-05-09 10:10:44 +00:00
Lang Hames e4b4ab6d26 [Support] Add error handling to sys::Process::getPageSize().
This patch changes the return type of sys::Process::getPageSize to
Expected<unsigned> to account for the fact that the underlying syscalls used to
obtain the page size may fail (see below).

For clients who use the page size as an optimization only this patch adds a new
method, getPageSizeEstimate, which calls through to getPageSize but discards
any error returned and substitues a "reasonable" page size estimate estimate
instead. All existing LLVM clients are updated to call getPageSizeEstimate
rather than getPageSize.

On Unix, sys::Process::getPageSize is implemented in terms of getpagesize or
sysconf, depending on which macros are set. The sysconf call is documented to
return -1 on failure. On Darwin getpagesize is implemented in terms of sysconf
and may also fail (though the manpage documentation does not mention this).
These failures have been observed in practice when highly restrictive sandbox
permissions have been applied. Without this patch, the result is that
getPageSize returns -1, which wreaks havoc on any subsequent code that was
assuming a sane page size value.

<rdar://problem/41654857>

Reviewers: dblaikie, echristo

Subscribers: kristina, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59107

llvm-svn: 360221
2019-05-08 02:11:07 +00:00
Lang Hames 0d8ae1e343 Reapply r360194 "[JITLink] Add support for MachO .alt_entry atoms." with fixes.
This patch modifies MachOAtomGraphBuilder to use setLayoutNext rather than
addEdge, and fixes a bug in the section layout algorithm that could result in
atoms appearing more than once in the section ordering (which resulted in those
atoms being assigned invalid addresses during layout).

llvm-svn: 360205
2019-05-07 22:56:40 +00:00
Lang Hames 1a10101e21 Revert r360194 "[JITLink] Add support for MachO .alt_entry atoms."
The testcase is asserting on some bots - reverting while I investigate.

llvm-svn: 360200
2019-05-07 22:19:29 +00:00
Lang Hames 2b09b25e48 [JITLink] Add support for MachO .alt_entry atoms.
The MachO .alt_entry directive is applied to a symbol to indicate that it is
locked (in terms of address layout and liveness) to its predecessor atom. I.e.
it is an alternate entry point, at a fixed offset, for the previous atom.

This patch updates MachOAtomGraphBuilder to check for the .alt_entry flag on
symbols and add a corresponding LayoutNext edge to the atom-graph. It also
updates MachOAtomGraphBuilder_x86_64 to generalize handling of the
X86_64_RELOC_SUBTRACTOR relocation: previously either the minuend or
subtrahend of the subtraction had to be the same as the atom being fixed up,
now it is only necessary for the minuend or subtrahend to be locked (via any
chain of alt_entry directives) to the atom being fixed up.

llvm-svn: 360194
2019-05-07 21:35:14 +00:00
Lang Hames ce8255f3e2 [JITLink] Add two useful Section operations: find by name, get address range.
These operations were already used in eh-frame registration, and are likely to
be used in other runtime registrations, so this commit moves them into a header
where they can be re-used.

llvm-svn: 359950
2019-05-04 00:23:09 +00:00
Lang Hames 42a3b4ff0e [ORC] Pass object buffer ownership back in NotifyEmitted.
Clients who want to regain ownership of object buffers after they have been
linked may now use the NotifyEmitted callback for this purpose.

Note: Currently NotifyEmitted is only called if linking succeeds. If linking
fails the buffer is always discarded.

llvm-svn: 359735
2019-05-01 22:40:23 +00:00
Lang Hames 3181b87cb6 [JITLink] Make sure we explicitly deallocate memory on failure.
JITLinkGeneric phases 2 and 3 (focused on applying fixups and finalizing memory,
respectively) may fail for various reasons. If this happens, we need to
explicitly de-allocate the memory allocated in phase 1 (explicitly, because
deallocation may also fail and so is implemented as a method returning error).

No testcase yet: I am still trying to decide on the right way to test totally
platform agnostic code like this.

llvm-svn: 359643
2019-05-01 02:43:52 +00:00
Lang Hames 4637e15844 [ORC] Move SimpleCompiler/ConcurrentIRCompiler definitions into a .cpp file.
SimpleCompiler is no longer templated, so there's no reason for this code to be
in a header any more.

llvm-svn: 359626
2019-04-30 22:42:01 +00:00
Lang Hames d407b4b980 [JITLink] Add debugging output to print resolved external atoms.
llvm-svn: 359614
2019-04-30 21:28:07 +00:00
Lang Hames 88816bdd2f [ORC][JITLink] Name in-memory compiled objects after their source modules.
In-memory compiled object buffer identifiers will now be derived from the
identifiers of their source IR modules. This makes it easier to connect
in-memory objects with their source modules in debugging output.

llvm-svn: 359613
2019-04-30 21:27:56 +00:00
Lang Hames b12867230c [ORC] Allow JITDylib definition generators to return Errors.
Background: A definition generator can be attached to a JITDylib to generate
new definitions in response to queries. For example: a generator that forwards
calls to dlsym can map symbols from a dynamic library into the JIT process on
demand.

If definition generation fails then the generator should be able to return an
error. This allows the JIT API to distinguish between the case where a
generator does not provide a definition, and the case where it was not able to
determine whether it provided a definition due to an error.

The immediate motivation for this is cross-process symbol lookups: If the
remote-lookup generator is attached to a JITDylib early in the search list, and
if a generator failure is misinterpreted as "no definition in this JITDylib" then
lookup may continue and bind to a different definition in a later JITDylib, which
is a bug.

llvm-svn: 359521
2019-04-30 00:03:26 +00:00
Lang Hames eb14dc7585 [ORC] Replace the LLJIT/LLLazyJIT Create methods with Builder utilities.
LLJITBuilder and LLLazyJITBuilder construct LLJIT and LLLazyJIT instances
respectively. Over time these will allow more configurable options to be
added while remaining easy to use in the default case, which for default
in-process JITing is now:

auto J = ExitOnErr(LLJITBuilder.create());

llvm-svn: 359511
2019-04-29 22:37:27 +00:00
Lang Hames a9fdf375b3 [ORC] Add a 'plugin' interface to ObjectLinkingLayer for events/configuration.
ObjectLinkingLayer::Plugin provides event notifications when objects are loaded,
emitted, and removed. It also provides a modifyPassConfig callback that allows
plugins to modify the JITLink pass configuration.

This patch moves eh-frame registration into its own plugin, and teaches
llvm-jitlink to only add that plugin when performing execution runs on
non-Windows platforms. This should allow us to re-enable the test case that was
removed in r359198.

llvm-svn: 359357
2019-04-26 22:58:39 +00:00
Lang Hames 4f71049a39 [ORC] Remove symbols from dependency lists when failing materialization.
When failing materialization of a symbol X, remove X from the dependants list
of any of X's dependencies. This ensures that when X's dependencies are
emitted (or fail themselves) they do not try to access the no-longer-existing
MaterializationInfo for X.

llvm-svn: 359252
2019-04-25 23:31:33 +00:00
Lang Hames b1ba4d8a8a [JITLink] Refer to FDE's CIE (not the most recent CIE) when parsing eh-frame.
Frame Descriptor Entries (FDEs) have a pointer back to a Common Information
Entry (CIE) that describes how the rest FDE should be parsed. JITLink had been
assuming that FDEs always referred to the most recent CIE encountered, but the
spec allows them to point back to any previously encountered CIE. This patch
fixes JITLink to look up the correct CIE for the FDE.

The testcase is a MachO binary with an FDE that refers to a CIE that is not the
one immediately proceeding it (the layout can be viewed wit
'dwarfdump --eh-frame <testcase>'. This test case had to be a binary as llvm-mc
now sorts FDEs (as of r356216) to ensure FDEs *do* point to the most recent CIE.

llvm-svn: 359105
2019-04-24 15:15:55 +00:00
Simon Pilgrim ddd225d1a9 Fix MSVC "32-bit shift implicitly converted to 64 bits" warning. NFCI.
llvm-svn: 358970
2019-04-23 11:16:16 +00:00
Lang Hames 1233c15be5 [JITLink] Remove a lot of reduntant 'JITLink_' prefixes. NFC.
llvm-svn: 358869
2019-04-22 03:03:09 +00:00
Lang Hames d3dac47aa2 [JITLink] Fix section start address calculation in eh-frame recorder.
Section atoms are not sorted, so we need to scan the whole section to find the
start address.

No test case: Found by inspection, and any reproduction would depend on pointer
ordering.

llvm-svn: 358865
2019-04-22 01:35:16 +00:00
Lang Hames a97032e947 [JITLink] Remove an overly strict error check in JITLink's eh-frame parser.
The error check required FDEs to refer to the most recent CIE, but the eh-frame
spec allows them to refer to any previously seen CIE. This patch removes the
offending check.

llvm-svn: 358840
2019-04-21 04:48:32 +00:00
Lang Hames 0191531a76 [JITLink] Factor basic common GOT and stub creation code into its own class.
llvm-svn: 358838
2019-04-21 03:14:42 +00:00
Lang Hames 65e1ddd713 [JITLink] Add yet more detail to MachO/x86-64 unsupported relocation errors.
Knowing the address/symbolnum field values makes it easier to identify the
unsupported relocation, and provides enough information for the full bit
pattern of the relocation to be reconstructed.

llvm-svn: 358833
2019-04-20 22:59:43 +00:00
Lang Hames 5004abcd86 [JITLink][ORC] Add JITLink to the list of dependencies for ORC.
The new ObjectLinkingLayer in ORC depends on JITLink.

This should fix the build error at
http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/9621

llvm-svn: 358832
2019-04-20 22:15:57 +00:00
Lang Hames 7f77a231fa [JITLink] Fix a bad formatv format string.
llvm-svn: 358831
2019-04-20 22:06:12 +00:00
Lang Hames daed9b10f1 [JITLink] Add BinaryFormat to JITLink's dependencies.
Hopefully this will fix the missing dependence on llvm::identify_magic that is
showing up on some PPC bots. E.g.

http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/9617

llvm-svn: 358827
2019-04-20 19:48:45 +00:00
Lang Hames c283fc5ebb [JITLink] Add more detail to MachO/x86-64 "unsupported relocation" errors.
The extra information here will be helpful in diagnosing errors, like the
ones currently occuring on the PPC big-endian bots. :)

llvm-svn: 358826
2019-04-20 18:50:13 +00:00
Lang Hames dfc3a4f6ff [JITLink] Silence some MSVC implicit cast warnings.
llvm-svn: 358824
2019-04-20 18:30:16 +00:00
Lang Hames b39109585a [JITLink] Use memset instead of bzero.
llvm-svn: 358822
2019-04-20 17:49:58 +00:00
Lang Hames 68b0b8c192 [JITLink] Fix a missing header and bad prototype.
llvm-svn: 358819
2019-04-20 17:29:57 +00:00
Lang Hames 11c8dfa583 Initial implementation of JITLink - A replacement for RuntimeDyld.
Summary:

JITLink is a jit-linker that performs the same high-level task as RuntimeDyld:
it parses relocatable object files and makes their contents runnable in a target
process.

JITLink aims to improve on RuntimeDyld in several ways:

(1) A clear design intended to maximize code-sharing while minimizing coupling.

RuntimeDyld has been developed in an ad-hoc fashion for a number of years and
this had led to intermingling of code for multiple architectures (e.g. in
RuntimeDyldELF::processRelocationRef) in a way that makes the code more
difficult to read, reason about, extend. JITLink is designed to isolate
format and architecture specific code, while still sharing generic code.

(2) Support for native code models.

RuntimeDyld required the use of large code models (where calls to external
functions are made indirectly via registers) for many of platforms due to its
restrictive model for stub generation (one "stub" per symbol). JITLink allows
arbitrary mutation of the atom graph, allowing both GOT and PLT atoms to be
added naturally.

(3) Native support for asynchronous linking.

JITLink uses asynchronous calls for symbol resolution and finalization: these
callbacks are passed a continuation function that they must call to complete the
linker's work. This allows for cleaner interoperation with the new concurrent
ORC JIT APIs, while still being easily implementable in synchronous style if
asynchrony is not needed.

To maximise sharing, the design has a hierarchy of common code:

(1) Generic atom-graph data structure and algorithms (e.g. dead stripping and
 |  memory allocation) that are intended to be shared by all architectures.
 |
 + -- (2) Shared per-format code that utilizes (1), e.g. Generic MachO to
       |  atom-graph parsing.
       |
       + -- (3) Architecture specific code that uses (1) and (2). E.g.
                JITLinkerMachO_x86_64, which adds x86-64 specific relocation
                support to (2) to build and patch up the atom graph.

To support asynchronous symbol resolution and finalization, the callbacks for
these operations take continuations as arguments:

  using JITLinkAsyncLookupContinuation =
      std::function<void(Expected<AsyncLookupResult> LR)>;

  using JITLinkAsyncLookupFunction =
      std::function<void(const DenseSet<StringRef> &Symbols,
                         JITLinkAsyncLookupContinuation LookupContinuation)>;

  using FinalizeContinuation = std::function<void(Error)>;

  virtual void finalizeAsync(FinalizeContinuation OnFinalize);

In addition to its headline features, JITLink also makes other improvements:

  - Dead stripping support: symbols that are not used (e.g. redundant ODR
    definitions) are discarded, and take up no memory in the target process
    (In contrast, RuntimeDyld supported pointer equality for weak definitions,
    but the redundant definitions stayed resident in memory).

  - Improved exception handling support. JITLink provides a much more extensive
    eh-frame parser than RuntimeDyld, and is able to correctly fix up many
    eh-frame sections that RuntimeDyld currently (silently) fails on.

  - More extensive validation and error handling throughout.

This initial patch supports linking MachO/x86-64 only. Work on support for
other architectures and formats will happen in-tree.

Differential Revision: https://reviews.llvm.org/D58704

llvm-svn: 358818
2019-04-20 17:10:34 +00:00
Lang Hames c7c1f21525 Simplify decoupling between RuntimeDyld/RuntimeDyldChecker, add 'got_addr' util.
This patch reduces the number of functions in the interface between RuntimeDyld
and RuntimeDyldChecker by combining "GetXAddress" and "GetXContent" functions
into "GetXInfo" functions that return a struct describing both the address and
content. The GetStubOffset function is also replaced with a pair of utilities,
GetStubInfo and GetGOTInfo, that fit the new scheme. For RuntimeDyld both of
these functions will return the same result, but for the new JITLink linker
(https://reviews.llvm.org/D58704) these will provide the addresses of PLT stubs
and GOT entries respectively.

For JITLink's use, a 'got_addr' utility has been added to the rtdyld-check
language, and the syntax of 'got_addr' and 'stub_addr' has been changed: both
functions now take two arguments, a 'stub container name' and a target symbol
name. For llvm-rtdyld/RuntimeDyld the stub container name is the object file
name and section name, separated by a slash. E.g.:

rtdyld-check: *{8}(stub_addr(foo.o/__text, y)) = y

For the upcoming llvm-jitlink utility, which creates stubs on a per-file basis
rather than a per-section basis, the container name is just the file name. E.g.:

jitlink-check: *{8}(got_addr(foo.o, y)) = y
llvm-svn: 358295
2019-04-12 18:07:28 +00:00
Nico Weber af5834596b gn build: Fix Windows builds after r357797
llvm-svn: 358004
2019-04-09 14:02:02 +00:00
Lang Hames d250238abd [RuntimeDyld] Fix an ambiguous make_unique call.
llvm-svn: 357950
2019-04-08 22:19:05 +00:00
Lang Hames 941f247d30 [RuntimeDyld] Decouple RuntimeDyldChecker from RuntimeDyld.
This will allow RuntimeDyldChecker (and rtdyld-check tests) to test a new JIT
linker: JITLink (https://reviews.llvm.org/D58704).

llvm-svn: 357947
2019-04-08 21:50:48 +00:00
Fangrui Song 6a0746a92f Change some StringRef::data() reinterpret_cast to bytes_begin() or arrayRefFromStringRef()
llvm-svn: 357852
2019-04-07 03:58:42 +00:00
Brock Wyma bc92c8c863 [DebugInfo] IntelJitEventListener follow up for "add SectionedAddress ..."
Following r354972 the Intel JIT Listener would not report line table
information because the section indices did not match. There was
a similar issue with the PerfJitEventListener. This change performs
the section index lookup when building the object address used to
query the line table information.

Differential Revision: https://reviews.llvm.org/D59490

llvm-svn: 356895
2019-03-25 13:50:26 +00:00
Sylvestre Ledru ba92e9bb11 [perf][DebugInfo] follow up for "add SectionedAddress to DebugInfo interfaces"
Summary: Fix the build failure when perf jit is enabled

Reviewers: avl, dblaikie

Reviewed By: avl

Subscribers: modocache, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D59189

llvm-svn: 356542
2019-03-20 10:02:18 +00:00
Kristof Beyls 730ecf8fd5 Add newline to interpreter debugging output
When running lli --debug --force-interpreter=true the executed instructions are
printed but are missing newlines. This commit adds the missing newlines.

Patch by Andrew Brown.

Differential Revision: https://reviews.llvm.org/D57806

llvm-svn: 355587
2019-03-07 10:14:38 +00:00
Alexey Lapshin d89d638055 Attempt to fix buildbot after r354972 [#1]. NFCI.
llvm-svn: 355013
2019-02-27 18:36:46 +00:00
James Y Knight 14359ef1b6 [opaque pointer types] Pass value type to LoadInst creation.
This cleans up all LoadInst creation in LLVM to explicitly pass the
value type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57172

llvm-svn: 352911
2019-02-01 20:44:24 +00:00
James Y Knight 7976eb5838 [opaque pointer types] Pass function types to CallInst creation.
This cleans up all CallInst creation in LLVM to explicitly pass a
function type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57170

llvm-svn: 352909
2019-02-01 20:43:25 +00:00
Richard Trieu 108b892939 Add namespace to some types.
llvm-svn: 352725
2019-01-31 04:33:11 +00:00
Zachary Turner 3c35f774de [RuntimeDyld] Don't try to allocate sections with align 0.
ELF sections allow 0 for the alignment, which is specified to
be the same as 1.  However many clients do not expect this and
will behave poorly in the presence of a 0-aligned section (for
example by trying to modulo something by the section alignment).
We can be more polite by making sure that we always pass a
non-zero value to clients.

Differential Revision: https://reviews.llvm.org/D57482

llvm-svn: 352694
2019-01-30 23:52:32 +00:00