Commit Graph

2424 Commits

Author SHA1 Message Date
Kristof Beyls 730ecf8fd5 Add newline to interpreter debugging output
When running lli --debug --force-interpreter=true the executed instructions are
printed but are missing newlines. This commit adds the missing newlines.

Patch by Andrew Brown.

Differential Revision: https://reviews.llvm.org/D57806

llvm-svn: 355587
2019-03-07 10:14:38 +00:00
Alexey Lapshin d89d638055 Attempt to fix buildbot after r354972 [#1]. NFCI.
llvm-svn: 355013
2019-02-27 18:36:46 +00:00
James Y Knight 14359ef1b6 [opaque pointer types] Pass value type to LoadInst creation.
This cleans up all LoadInst creation in LLVM to explicitly pass the
value type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57172

llvm-svn: 352911
2019-02-01 20:44:24 +00:00
James Y Knight 7976eb5838 [opaque pointer types] Pass function types to CallInst creation.
This cleans up all CallInst creation in LLVM to explicitly pass a
function type rather than deriving it from the pointer's element-type.

Differential Revision: https://reviews.llvm.org/D57170

llvm-svn: 352909
2019-02-01 20:43:25 +00:00
Richard Trieu 108b892939 Add namespace to some types.
llvm-svn: 352725
2019-01-31 04:33:11 +00:00
Zachary Turner 3c35f774de [RuntimeDyld] Don't try to allocate sections with align 0.
ELF sections allow 0 for the alignment, which is specified to
be the same as 1.  However many clients do not expect this and
will behave poorly in the presence of a 0-aligned section (for
example by trying to modulo something by the section alignment).
We can be more polite by making sure that we always pass a
non-zero value to clients.

Differential Revision: https://reviews.llvm.org/D57482

llvm-svn: 352694
2019-01-30 23:52:32 +00:00
Yonghong Song 61bc1d7ed5 [RuntimeDyld] load all sections with ProcessAllSections
This patch tried to address the following use case.
  . bcc (https://github.com/iovisor/bcc) utilizes llvm JIT to
    compile for BTF target.
  . with -g, .BTF and .BTF.ext sections (BPF debug info)
    will be generated by LLVM.
  . .BTF does not have relocations and .BTF.ext has some
    relocations.
  . With ProcessAllSections, .BTF.ext is loaded by JIT dynamic linker
    and is available to application. But .BTF is not loaded.

The bcc application needs both .BTF.ext and .BTF for debugging
purpose, and .BTF is not loaded. This patch addressed this issue
by iterating over all sections and loading any missing
sections, after symbol/relocation processing in loadObjectImpl().

Signed-off-by: Yonghong Song <yhs@fb.com>

Differential Revision: https://reviews.llvm.org/D55943

llvm-svn: 352432
2019-01-28 21:35:23 +00:00
Rui Ueyama 7c8fc8142e MemoryBlock: Do not automatically extend a given size to a multiple of page size.
Previously, MemoryBlock automatically extends a requested buffer size to a
multiple of page size because (I believe) doing it was thought to be harmless
and with that you could get more memory (on average 2KiB on 4KiB-page systems)
"for free".

That programming interface turned out to be error-prone. If you request N
bytes, you usually expect that a resulting object returns N for `size()`.
That's not the case for MemoryBlock.

Looks like there is only one place where we take the advantage of
allocating more memory than the requested size. So, with this patch, I
simply removed the automatic size expansion feature from MemoryBlock
and do it on the caller side when needed. MemoryBlock now always
returns a buffer whose size is equal to the requested size.

Differential Revision: https://reviews.llvm.org/D56941

llvm-svn: 351916
2019-01-23 02:03:26 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Lang Hames 199a00c3a2 Revert r351138 "[ORC] Move ORC Core symbol map and set types into their own
header: CoreTypes.h."

This commit broke some bots. Reverting while I investigate.

llvm-svn: 351195
2019-01-15 15:21:13 +00:00
Lang Hames ed2df18a48 [ORC] Move ORC Core symbol map and set types into their own header: CoreTypes.h.
This will allow other utilities (including a future RuntimeDyld replacement) to
use these types without pulling in the major Core types (JITDylib, etc.).

llvm-svn: 351138
2019-01-14 23:49:13 +00:00
Simon Atanasyan 789f4154db [ORC][MIPS] Fill delay-slot after `jr` instruction
MIPS `jr` instruction uses a delay-slot. To escape execution of
arbitrary instruction we should either fill the delay-slot by `nop`
instruction or swap `jr` instruction and logically preceding
instruction. This fix implements the second method to generate a bit
more effective code.

llvm-svn: 351001
2019-01-12 11:12:08 +00:00
Simon Atanasyan f903f782e7 [ORC][MIPS] Setup t9 register and call function through this register
MIPS ABI states that every function must be called through jalr $t9. In
other words, a function expect that t9 register points to the beginning
of its code. A function uses this register to calculate offset to the
Global Offset Table and save it to the `gp` register.
```
lui   $gp, %hi(_gp_disp)
addiu $gp, %lo(_gp_disp)
addu  $gp, $gp, $t9
```

If `t9` and as a result `$gp` point to the wrong place the following code
loads incorrect value from GOT and passes control to invalid code.
```
lw    $v0,%call16(foo)($gp)
jalr  $t9
```

OrcMips32 and OrcMips64 writeResolverCode methods pass control to the
resolved address, but do not setup `$t9` before the call. The `t9` holds
value of the beginning of `resolver` code so any attempts to call
routines via GOT failed.

This change fixes the problem. The `OrcLazy/hidden-visibility.ll` test
starts to pass correctly. Before the change it fails on MIPS because the
`exitOnLazyCallThroughFailure` called from the resolver code could not
call libc routine `exit` via GOT.

Differential Revision: http://reviews.llvm.org/D56058

llvm-svn: 351000
2019-01-12 11:12:04 +00:00
James Y Knight 62df5eed16 [opaque pointer types] Remove some calls to generic Type subtype accessors.
That is, remove many of the calls to Type::getNumContainedTypes(),
Type::subtypes(), and Type::getContainedType(N).

I'm not intending to remove these accessors -- they are
useful/necessary in some cases. However, removing the pointee type
from pointers would potentially break some uses, and reducing the
number of calls makes it easier to audit.

llvm-svn: 350835
2019-01-10 16:07:20 +00:00
Simon Atanasyan 4553cdafe1 [ORC] Rename register in the OrcMips64 resolver code comments. NFC
The `fp` and `s8` register names are synonyms. But `fp` better reflects
a purpose of the register.

llvm-svn: 350023
2018-12-23 12:05:04 +00:00
Simon Atanasyan 15b68b87d5 [ORC] clang-format OrcMips32 and OrcMips64 code. NFC
llvm-svn: 350022
2018-12-23 12:05:00 +00:00
Simon Atanasyan da29981707 [ORC] Remove redundant instruction from MIPS resolver code. NFC
It's redundant to restore the `$a3` register twice.

llvm-svn: 350021
2018-12-23 12:04:55 +00:00
Nathan Lanza 893083ae5e Implement IMAGE_REL_AMD64_SECREL for RuntimeDyldCOFFX86_64
lldb on Windows uses the ExecutionEngine for expression evaluation
and hits the llvm_unreachable due to this relocation. Thus, implement
the relocation and add a test to verify it's function.

llvm-svn: 348904
2018-12-12 00:04:06 +00:00
Lang Hames eca6d4b638 [ExecutionEngine] Change NotifyObjectEmitted/NotifyObjectFreed API.
This patch renames both methods (NotifyObjectEmitted -> notifyObjectLoaded, and
NotifyObjectFreed -> notifyObjectFreed), adds an abstract "ObjectKey" (uint64_t)
parameter to notifyObjectLoaded, and replaces the ObjectFile parameter for
notifyObjectFreed with an ObjectKey. Using an ObjectKey to track identify
events, rather than a reference to the ObjectFile, allows us to free the
ObjectFile after notifyObjectLoaded is called, saving memory.

https://reviews.llvm.org/D53773

llvm-svn: 348223
2018-12-04 00:55:15 +00:00
Lang Hames a4d5b34f73 [ExecutionEngine][Interpreter] Fix out-of-bounds array access.
If args is empty then accesing element 0 is illegal.

https://reviews.llvm.org/D53556

Patch by Eugene Sharygin. Thanks Eugene!

llvm-svn: 347281
2018-11-20 01:01:26 +00:00
Lang Hames 575515fddf [BuildingAJIT] Update chapter 2 to use the ORCv2 APIs.
llvm-svn: 346726
2018-11-13 01:25:34 +00:00
Jonas Devlieghere 45eb84f340 [Support] Make error banner optional in logAllUnhandledErrors
In a lot of places an empty string was passed as the ErrorBanner to
logAllUnhandledErrors. This patch makes that argument optional to
simplify the call sites.

llvm-svn: 346604
2018-11-11 01:46:03 +00:00
Lang Hames 91449355f5 [ORC] Fix hex printing of uint64_t values.
A plain "%x" format string will drop the high 32-bits. Use the PRIx64 macro
instead.

llvm-svn: 345696
2018-10-31 05:16:14 +00:00
Matthias Braun 9fd397b423 ADT/STLExtras: Introduce llvm::empty; NFC
This is modeled after C++17 std::empty().

Differential Revision: https://reviews.llvm.org/D53909

llvm-svn: 345679
2018-10-31 00:23:23 +00:00
Lang Hames 23cb2e7f77 [ORC] Re-apply r345077 with fixes to remove ambiguity in lookup calls.
llvm-svn: 345098
2018-10-23 23:01:39 +00:00
Reid Kleckner db367e952e Revert r345077 "[ORC] Change how non-exported symbols are matched during lookup."
Doesn't build on Windows. The call to 'lookup' is ambiguous. Clang and
MSVC agree, anyway.

http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/787
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\unittests\ExecutionEngine\Orc\CoreAPIsTest.cpp(315): error C2668: 'llvm::orc::ExecutionSession::lookup': ambiguous call to overloaded function
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\include\llvm/ExecutionEngine/Orc/Core.h(823): note: could be 'llvm::Expected<llvm::JITEvaluatedSymbol> llvm::orc::ExecutionSession::lookup(llvm::ArrayRef<llvm::orc::JITDylib *>,llvm::orc::SymbolStringPtr)'
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\include\llvm/ExecutionEngine/Orc/Core.h(817): note: or       'llvm::Expected<llvm::JITEvaluatedSymbol> llvm::orc::ExecutionSession::lookup(const llvm::orc::JITDylibSearchList &,llvm::orc::SymbolStringPtr)'
C:\b\slave\clang-x64-windows-msvc\build\llvm.src\unittests\ExecutionEngine\Orc\CoreAPIsTest.cpp(315): note: while trying to match the argument list '(initializer list, llvm::orc::SymbolStringPtr)'

llvm-svn: 345078
2018-10-23 20:54:43 +00:00
Lang Hames 841796decd [ORC] Change how non-exported symbols are matched during lookup.
In the new scheme the client passes a list of (JITDylib&, bool) pairs, rather
than a list of JITDylibs. For each JITDylib the boolean indicates whether or not
to match against non-exported symbols (true means that they should be found,
false means that they should not). The MatchNonExportedInJD and MatchNonExported
parameters on lookup are removed.

The new scheme is more flexible, and easier to understand.

This patch also updates JITDylib search orders to be lists of (JITDylib&, bool)
pairs to match the new lookup scheme. Error handling is also plumbed through
the LLJIT class to allow regression tests to fail predictably when a lookup from
a lazy call-through fails.

llvm-svn: 345077
2018-10-23 20:20:22 +00:00
Lang Hames 776f1d50c8 [RuntimeDyld][COFF] Skip non-loaded sections when calculating ImageBase.
Non-loaded sections (whose unused load-address defaults to zero) should not
be taken into account when calculating ImageBase, or ImageBase will be
incorrectly set to 0.

Patch by Andrew Scheidecker. Thanks Andrew!

https://reviews.llvm.org/D51343

+        // The Sections list may contain sections that weren't loaded for
+        // whatever reason: they may be debug sections, and ProcessAllSections
+        // is false, or they may be sections that contain 0 bytes. If the
+        // section isn't loaded, the load address will be 0, and it should not
+        // be included in the ImageBase calculation.

llvm-svn: 344995
2018-10-23 01:36:33 +00:00
Lang Hames 3d16af69cf [ORC] Show JITDylib search order in JITDylib::dump.
This can be helpful in debugging search-order related failures.

llvm-svn: 344994
2018-10-23 01:36:32 +00:00
Lang Hames 1aa3292a43 [ORC] Dump flags for JITDylib symbol table entries.
This can help when debugging flag-specific symbol table issues.

llvm-svn: 344993
2018-10-23 01:36:31 +00:00
Lang Hames 95abadec0b [ORC] Guard access to the MemMgrs vector in RTDyldObjectLinkingLayer.
Otherwise we can end up with a data-race when linking concurrently.

This should fix an intermittent failure in the multiple-compile-threads-basic.ll
testcase.

llvm-svn: 344956
2018-10-22 21:17:56 +00:00
Lang Hames 8b94274f22 [ORC] Make the VModuleKey optional, propagate it via MaterializationUnit and
MaterializationResponsibility.

VModuleKeys are intended to enable selective removal of modules from a JIT
session, however for a wide variety of use cases selective removal is not
needed and introduces unnecessary overhead. As of this commit, the default
constructed VModuleKey value is reserved as a "do not track" value, and
becomes the default when adding a new module to the JIT.

This commit also changes the propagation of VModuleKeys. They were passed
alongside the MaterializationResponsibity instance in XXLayer::emit methods,
but are now propagated as part of the MaterializationResponsibility instance
itself (and as part of MaterializationUnit when stored in a JITDylib).
Associating VModuleKeys with MaterializationUnits in this way should allow
for a thread-safe module removal mechanism in the future, even when a module
is in the process of being compiled, by having the
MaterializationResponsibility object check in on its VModuleKey's state
before commiting its results to the JITDylib.

llvm-svn: 344643
2018-10-16 20:13:06 +00:00
Lang Hames 079df9ab2c [ORC] Rename ORC layers to make the "new" ORC layers the default.
This commit adds a 'Legacy' prefix to old ORC layers and utilities, and removes
the '2' suffix from the new ORC layers. If you wish to continue using the old
ORC layers you will need to add a 'Legacy' prefix to your classes. If you were
already using the new ORC layers you will need to drop the '2' suffix.

The legacy layers will remain in-tree until the new layers reach feature
parity with them. This will involve adding support for removing code from the
new layers, and ensuring that performance is comperable.

llvm-svn: 344572
2018-10-15 22:56:10 +00:00
Lang Hames 9f342d9038 [ORC] Rename MultiThreadedSimpleCompiler to ConcurrentIRCompiler.
The new name is a better fit: This class does not actually spawn any new
threads for compilation, it is just safe to call from multiple threads
concurrently.

The "Simple" part of the name did not convey much either, so it was
dropped.

llvm-svn: 344567
2018-10-15 22:36:22 +00:00
Lang Hames 6e66b2a6a3 [ORC] Switch to DenseMap/DenseSet for ORC symbol map/set types.
llvm-svn: 344565
2018-10-15 22:27:02 +00:00
Lang Hames a5157d6f4b [ORC] Simplify naming for JITDylib definition generators.
Renames:
  JITDylib's setFallbackDefinitionGenerator method to setGenerator.
  DynamicLibraryFallbackGenerator class to DynamicLibrarySearchGenerator.
  ReexportsFallbackDefinitionGenerator to ReexportsGenerator.

llvm-svn: 344489
2018-10-15 05:07:54 +00:00
Lang Hames 7899ccbcca [ORC] During lookup, do not match against hidden symbols in other JITDylibs.
This adds two arguments to the main ExecutionSession::lookup method:
MatchNonExportedInJD, and MatchNonExported. These control whether and where
hidden symbols should be matched when searching a list of JITDylibs.

A similar effect could have been achieved by filtering search results, but
this would have involved materializing symbol definitions (since materialization
is triggered on lookup) only to throw the results away, among other issues.

llvm-svn: 344467
2018-10-13 21:53:40 +00:00
Lang Hames bf6603e918 [ORC] Promote and rename private symbols inside the CompileOnDemand layer,
rather than require them to have been promoted before being passed in.

Dropping this precondition is better for layer composition (CompileOnDemandLayer
was the only one that placed pre-conditions on the modules that could be added).
It also means that the promoted private symbols do not show up in the target
JITDylib's symbol table. Instead, they are confined to the hidden implementation
dylib that contains the actual definitions.

For the 403.gcc testcase this cut down the public symbol table size from ~15,000
symbols to ~4000, substantially reducing symbol dependence tracking costs.

llvm-svn: 344078
2018-10-09 20:44:32 +00:00
Lang Hames 535448e08d [ORC] Add a 'remove' method to JITDylib to remove symbols.
Symbols can be removed provided that all are present in the JITDylib and none
are currently in the materializing state. On success all requested symbols are
removed. On failure an error is returned and no symbols are removed.

llvm-svn: 343928
2018-10-06 23:03:59 +00:00
Lang Hames cb5702c3fd [ORC] Pass symbol name to discard by const reference.
This saves some unnecessary atomic ref-counting operations.

llvm-svn: 343927
2018-10-06 23:02:06 +00:00
Lang Hames deb3640d95 [ORC] Pass Symbols to ExecutionSession::lookup by value, potentially saving a
copy.

llvm-svn: 343442
2018-10-01 04:59:10 +00:00
Lang Hames 47d0a37704 [ORC] Add convenience methods for creating DynamicLibraryFallbackGenerators for
libraries on disk, and for the current process.

Avoids more boilerplate during JIT construction.

llvm-svn: 343430
2018-10-01 00:59:28 +00:00
Lang Hames 71d781c434 [ORC] Add an 'intern' method to ExecutionEngine for interning symbol names.
This cuts down on boilerplate by reducing 'ES.getSymbolStringPool().intern(...)'
to 'ES.intern(...)'.

llvm-svn: 343427
2018-09-30 23:18:24 +00:00
Lang Hames d435ce4343 [ORC] Extract and tidy up JITTargetMachineBuilder, add unit test.
(1) Adds comments for the API.

(2) Removes the setArch method: This is redundant: the setArchStr method on the
    triple should be used instead.

(3) Turns EmulatedTLS on by default. This matches EngineBuilder's behavior.

llvm-svn: 343423
2018-09-30 19:12:23 +00:00
Lang Hames 98440293fb [ORC] Add partitioning support to CompileOnDemandLayer2.
CompileOnDemandLayer2 now supports user-supplied partition functions (the
original CompileOnDemandLayer already supported these).

Partition functions are called with the list of requested global values
(i.e. global values that currently have queries waiting on them) and have an
opportunity to select extra global values to materialize at the same time.

Also adds testing infrastructure for the new feature to lli.

llvm-svn: 343396
2018-09-29 23:49:57 +00:00
Lang Hames c3053e41bf [ORC] Clear SymbolToDefinitionMap when materializing a MaterializationUnit.
The map is inaccessible at this point, so we may as well reclaim the memory
early.

llvm-svn: 343395
2018-09-29 23:49:56 +00:00
Lang Hames 2afc22ed9e [ORC] Make MaterializationResponsibility::getRequestedSymbols() const.
This makes it available for use in IRTransformLayer2::TransformFunction
instances (since a const MaterializationResponsibility& parameter was
added in r343365).

llvm-svn: 343367
2018-09-28 22:03:17 +00:00
Lang Hames 3e709d5f78 [ORC] Add more utilities to aid debugging output.
(1) A const accessor for the LLVMContext held by a ThreadSafeContext.

(2) A const accessor for the ThreadSafeModules held by an IRMaterializationUnit.

(3) A const MaterializationResponsibility reference to IRTransformLayer2's
    transform function. This makes IRTransformLayer2 useful for JIT debugging
    (since it can inspect JIT state through the responsibility argument) as well
    as program transformations.

llvm-svn: 343365
2018-09-28 21:49:53 +00:00
Lang Hames b62f73420b [ORC] Narrow a cast: the block guarded by the condition only handles
GlobalVariables, not all GlobalValues.

llvm-svn: 343358
2018-09-28 20:16:16 +00:00
Lang Hames 604769943a [ORC] Remove some dead code.
llvm-svn: 343327
2018-09-28 15:13:41 +00:00