Commit Graph

7091 Commits

Author SHA1 Message Date
Lang Hames 8a36750236 [ORC] Handle void and no-argument async wrapper calls. 2021-08-19 12:20:31 +10:00
Rafael Auler d8bbfe8a48 [DWARF] Expose raw bytes in DWARFExpression
This information is necessary for clients of DebugInfo that
do not want to process a DWARF expression, but just treat it as a blob
of data. In BOLT, for example, we need to read these expressions in
CFIs and write them back to the binary, unchanged, so having access to
the original expression encoding is a shortcut to avoid the need to
re-encode the entire expression when re-writing exception handling
info (CFIs).

This patch is an alternative to https://reviews.llvm.org/D98301, in
which we implement the support to re-encode these expressions. But
since we don't really need to change anything in these expressions,
we can just copy their bytes.

Reviewed By: aprantl

Differential Revision: https://reviews.llvm.org/D107515
2021-08-18 14:41:20 -07:00
Nikita Popov 3dd8c9176b [LICM] Remove AST-based implementation
MSSA-based LICM has been enabled by default for a few years now.
This drops the old AST-based implementation. Using loop(licm) will
result in a fatal error, the use of loop-mssa(licm) is required
(or just licm, which defaults to loop-mssa).

Note that the core canSinkOrHoistInst() logic has to retain AST
support for now, because it is shared with LoopSink.

Differential Revision: https://reviews.llvm.org/D108244
2021-08-18 20:21:53 +02:00
Arthur Eubanks 7557d6c896 [NFC] Cleanup calls to CallBase::getAttribute() 2021-08-18 09:39:33 -07:00
Arthur Eubanks 3f4d00bc3b [NFC] More get/removeAttribute() cleanup 2021-08-17 21:05:41 -07:00
Arthur Eubanks de0ae9e89e [NFC] Cleanup more AttributeList::addAttribute() 2021-08-17 21:05:41 -07:00
Arthur Eubanks ad727ab7d9 [NFC] Migrate some callers away from Function/AttributeLists methods that take an index
These methods can be confusing.
2021-08-17 21:05:40 -07:00
Mark Danial 4018d25da8 LoopNest Analysis expansion to return instructions that prevent a Loop
Nest from being perfect

Expand LoopNestAnalysis to return the full list of instructions that
cause a loop nest to be imperfect. This is useful for other passes to
know if they should continue for in the inner loops.
Added New function getInterveningInstructions
that returns a small vector with the instructions that prevent a loop
for being perfect. Also added a couple of helper functions to reduce
code duplication.

Reviewed By: Whitney

Differential Revision: https://reviews.llvm.org/D107773
2021-08-17 22:25:49 +00:00
Fraser Cormack f3e9047249 [VP] Add vector-predicated reduction intrinsics
This patch adds vector-predicated ("VP") reduction intrinsics corresponding to
each of the existing unpredicated `llvm.vector.reduce.*` versions. Unlike the
unpredicated reductions, all VP reductions have a start value. This start value
is returned when the no vector element is active.

Support for expansion on targets without native vector-predication support is
included.

This patch is based on the ["reduction
slice"](https://reviews.llvm.org/D57504#1732277) of the LLVM-VP reference patch
(https://reviews.llvm.org/D57504).

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D104308
2021-08-17 17:56:35 +01:00
Nikita Popov 735a590471 [MemorySSA] Remove -enable-mssa-loop-dependency option
This option has been enabled by default for quite a while now.
The practical impact of removing the option is that MSSA use
cannot be disabled in default pipelines (both LPM and NPM) and
in manual LPM invocations. NPM can still choose to enable/disable
MSSA using loop vs loop-mssa.

The next step will be to require MSSA for LICM and drop the
AST-based implementation entirely.

Differential Revision: https://reviews.llvm.org/D108075
2021-08-16 20:59:37 +02:00
Christian Sigg 93c55d5ea2 Reset all options in cl::ResetCommandLineParser()
Reset cl::Positional, cl::Sink and cl::ConsumeAfter options as well in cl::ResetCommandLineParser().

Reviewed By: rriddle, sammccall

Differential Revision: https://reviews.llvm.org/D103356
2021-08-16 09:56:22 +02:00
David Blaikie 2af4db7d5c Migrate DWARFVerifier tests to lit-based yaml instead of gtest with embedded yaml
Improves maintainability (edit/modify the tests without recompiling) and
error messages (previously the failure would be a gtest failure
mentioning nothing of the input or desired text) and the option to
improve tests with more checks.

(maybe these tests shouldn't all be in separate files - we could
probably have DWARF yaml that contains multiple errors while still being
fairly maintainable - the various invalid offsets (ref_addr, rnglists,
ranges, etc) could probably be all in one test, but for the simple sake
of the migration I just did the mechanical thing here)
2021-08-13 19:09:41 -07:00
Arthur Eubanks d7593ebaee [NFC] Clean up users of AttributeList::hasAttribute()
AttributeList::hasAttribute() is confusing, use clearer methods like
hasParamAttr()/hasRetAttr().

Add hasRetAttr() since it was missing from AttributeList.
2021-08-13 11:59:18 -07:00
Arthur Eubanks a9831cce1e [NFC] Remove public uses of AttributeList::getAttributes()
Use methods that better convey the intent.
2021-08-13 11:38:12 -07:00
Arthur Eubanks 80ea2bb574 [NFC] Rename AttributeList::getParam/Ret/FnAttributes() -> get*Attributes()
This is more consistent with similar methods.
2021-08-13 11:16:52 -07:00
Arthur Eubanks 92ce6db9ee [NFC] Rename AttributeList::hasFnAttribute() -> hasFnAttr()
This is more consistent with similar methods.
2021-08-13 11:09:18 -07:00
Arthur Eubanks a0c42ca56c [NFC] Remove AttributeList::hasParamAttribute()
It's the same as AttributeList::hasParamAttr().
2021-08-13 10:58:21 -07:00
Michael Kruse b1de32d6dd [OMPIRBuilder] Clarify CanonicalLoopInfo. NFC.
Add in-source documentation on how CanonicalLoopInfo is intended to be used. In particular, clarify what parts of a CanonicalLoopInfo is considered part of the loop, that those parts must be side-effect free, and that InsertPoints to instructions outside those parts can be expected to be preserved after method calls implementing loop-associated directives.

CanonicalLoopInfo are now invalidated after it does not describe canonical loop anymore and asserts when trying to use it afterwards.

In addition, rename `createXYZWorkshareLoop` to `applyXYZWorkshareLoop` and remove the update location to avoid that the impression that they insert something from scratch at that location where in reality its InsertPoint is ignored. createStaticWorkshareLoop does not return a CanonicalLoopInfo anymore. First, it was not a canonical loop in the clarified sense (containing side-effects in form of calls to the OpenMP runtime). Second, it is ambiguous which of the two possible canonical loops it should actually return. It will not be needed before a feature expected to be introduced in OpenMP 6.0

Also see discussion in D105706.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D107540
2021-08-12 21:02:19 -05:00
Christopher Di Bella c874dd5362 [llvm][clang][NFC] updates inline licence info
Some files still contained the old University of Illinois Open Source
Licence header. This patch replaces that with the Apache 2 with LLVM
Exception licence.

Differential Revision: https://reviews.llvm.org/D107528
2021-08-11 02:48:53 +00:00
Konstantin Schwarz 64bef13f08 [GlobalISel] Look through truncs and extends in narrowScalarShift
If a G_SHL is fed by a G_CONSTANT, the lower and upper bits of the source can be
shifted individually by the constant shift amount.

However in case the shift amount came from a G_TRUNC(G_CONSTANT), the generic shift legalization
code was used, producing intermediate shifts that are potentially illegal on some targets.

This change teaches narrowScalarShift to look through G_TRUNCs and G_*EXTs.

Reviewed By: paquette

Differential Revision: https://reviews.llvm.org/D89100
2021-08-10 13:49:22 +02:00
Jon Roelofs eae4a44c1d [GlobalISel][KnownBits] Implement G_CTPOP
Implementation copied almost verbatim from ValueTracking.

Differential revision: https://reviews.llvm.org/D107606
2021-08-06 09:48:39 -07:00
Jay Foad 57b9107e3f [GlobalISel] Improve widening of cttz/cttz_zero_undef
Differential Revision: https://reviews.llvm.org/D107631
2021-08-06 14:25:56 +01:00
Mircea Trofin ae1a2a09e4 [NFC][MLGO] Make logging more robust
1) add some self-diagnosis (when asserts are enabled) to check that all
features have the same nr of entries

2) avoid storing pointers to mutable fields because the proto API
contract doesn't actually guarantee those stay fixed even if no further
mutation of the object occurs.

Differential Revision: https://reviews.llvm.org/D107594
2021-08-06 04:44:52 -07:00
Jay Foad cd2594e1c6 [GlobalISel] Improve legalization of narrow CTTZ
Differential Revision: https://reviews.llvm.org/D107457
2021-08-06 09:40:48 +01:00
Jon Roelofs 5fc7b1a260 Revert "[GlobalISel][KnownBits] Implement G_CTPOP"
This reverts commit ce6eb4f15a.

It's broken on the windows bots: https://reviews.llvm.org/D107606#2930121
2021-08-05 17:47:47 -07:00
Jon Roelofs ce6eb4f15a [GlobalISel][KnownBits] Implement G_CTPOP
Implementation copied almost verbatim from ValueTracking.

Differential revision: https://reviews.llvm.org/D107606
2021-08-05 17:17:29 -07:00
Paul Robinson 75aa3d520d Add a DIExpression const-folder to prevent silly expressions.
It's entirely possible (because it actually happened) for a bool
variable to end up with a 256-bit DW_AT_const_value.  This came about
when a local bool variable was initialized from a bitfield in a
32-byte struct of bitfields, and after inlining and constant
propagation, the variable did have a constant value. The sequence of
optimizations had it carrying "i256" values around, but once the
constant made it into the llvm.dbg.value, no further IR changes could
affect it.

Technically the llvm.dbg.value did have a DIExpression to reduce it
back down to 8 bits, but the compiler is in no way ready to emit an
oversized constant *and* a DWARF expression to manipulate it.
Depending on the circumstances, we had either just the very fat bool
value, or an expression with no starting value.

The sequence of optimizations that led to this state did seem pretty
reasonable, so the solution I came up with was to invent a DWARF
constant expression folder.  Currently it only does convert ops, but
there's no reason it couldn't do other ops if that became useful.

This broke three tests that depended on having convert ops survive
into the DWARF, so I added an operator that would abort the folder to
each of those tests.

Differential Revision: https://reviews.llvm.org/D106915
2021-08-05 06:14:40 -07:00
Senran Zhang 486b6013f9 [Support] Initialize common options in `getRegisteredOptions`
This allows users accessing options in libSupport before invoking
`cl::ParseCommandLineOptions`, and also matches the behavior before
D105959.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D106334
2021-08-03 23:59:10 -07:00
Chang-Sun Lin, Jr b58eda39eb [ValueTracking] Fix computeConstantRange to use "may" instead of "always" semantics for llvm.assume
ValueTracking should allow for value ranges that may satisfy
llvm.assume, instead of restricting the ranges only to values that
will always satisfy the condition.

Differential Revision: https://reviews.llvm.org/D107298
2021-08-02 22:20:17 +02:00
Alexander Yermolovich 5a865b0b1e [DWARF] Don't process .debug_info relocations for DWO Context
When we build with split dwarf in single mode the .o files that contain both "normal" debug sections and dwo sections, along with relocaiton sections for "normal" debug sections.
When we create DWARF context in DWARFObjInMemory we process relocations and store them in the map for .debug_info, etc section.
For DWO Context we also do it for non dwo dwarf sections. Which I believe is not necessary. This leads to a lot of memory being wasted. We observed 70GB extra memory being used.

I went with context sensitive approach, flag is passed in. I am not sure if it's always safe not to process relocations for regular debug sections if Obj contains .dwo sections.
If it is alternatvie might be just to scan, in constructor, sections and if there are .dwo sections not to process regular debug ones.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D106624
2021-08-02 10:41:47 -07:00
Alex Zinenko aa426c372c [OMPIRBuilder] add minimalist reduction support
This introduces a builder function for emitting IR performing reductions in
OpenMP. Reduction variable privatization and initialization to the
reduction-neutral value is expected to be handled separately. The caller
provides the reduction functions. Further commits can provide implementation of
reduction functions for the reduction operators defined in the OpenMP
specification.

This implementation was tested on an MLIR fork targeting OpenMP from C and
produced correct executable code.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D104928
2021-07-30 13:58:26 +02:00
Sander de Smalen 84a4caeb84 [InstSimplify] Don't assume parent function when simplifying llvm.vscale.
D106850 introduced a simplification for llvm.vscale by looking at the
surrounding function's vscale_range attributes. The call that's being
simplified may not yet have been inserted into the IR. This happens for
example during function cloning.

This patch fixes the issue by checking if the instruction is in a
parent basic block.
2021-07-29 20:08:08 +01:00
Mehdi Amini b70de61f48 Add `all_of_zip` to STLExtras
This takes two ranges and invokes a predicate on the element-wise pair in the
ranges. It returns true if all the pairs are matching the predicate and the ranges
have the same size.
It is useful with containers that aren't random iterator where we can't check the
sizes in O(1).

Differential Revision: https://reviews.llvm.org/D106605
2021-07-29 05:00:35 +00:00
Valentin Clement fe7ca1a9fc [mlir][openacc] Initial translation for DataOp to LLVM IR
Add basic translation of acc.data to LLVM IR with runtime calls.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D104301
2021-07-27 22:04:04 -04:00
Lang Hames 2487db1f28 [ORC] Require ExecutorProcessControl when constructing an ExecutionSession.
Wrapper function call and dispatch handler helpers are moved to
ExecutionSession, and existing EPC-based tools are re-written to take an
ExecutionSession argument instead.

Requiring an ExecutorProcessControl instance simplifies existing EPC based
utilities (which only need to take an ES now), and should encourage more
utilities to use the EPC interface. It also simplifies process termination,
since the session can automatically call ExecutorProcessControl::disconnect
(previously this had to be done manually, and carefully ordered with the
rest of JIT tear-down to work correctly).
2021-07-27 16:53:49 +10:00
Tom Stellard dbefcde6da Merge all the llvm-exegesis unit tests into a single binary
These tests access private symbols in the backends, so they cannot link
against libLLVM.so and must be statically linked.  Linking these tests
can be slow and with debug builds the resulting binaries use a lot of
disk space.

By merging them into a single test binary means we now only need to
statically link 1 test instead of 6, which helps reduce the build
times and saves disk space.

Reviewed By: courbet

Differential Revision: https://reviews.llvm.org/D106464
2021-07-26 22:07:07 -07:00
Johannes Doerfert 25a3130d89 [Local] Do not introduce a new `llvm.trap` before `unreachable`
This is the second attempt to remove the `llvm.trap` insertion after
https://reviews.llvm.org/rGe14e7bc4b889dfaffb7180d176a03311df2d4ae6
reverted the first one. It is not clear what the exact issue was back
then and it might already be gone by now, it has been >5 years after
all.

Replaces D106299.

Differential Revision: https://reviews.llvm.org/D106308
2021-07-26 23:33:36 -05:00
Ulrich Weigand 8cd8120a7b [SystemZ] Add support for new cpu architecture - arch14
This patch adds support for the next-generation arch14
CPU architecture to the SystemZ backend.

This includes:
- Basic support for the new processor and its features.
- Detection of arch14 as host processor.
- Assembler/disassembler support for new instructions.
- New LLVM intrinsics for certain new instructions.
- Support for low-level builtins mapped to new LLVM intrinsics.
- New high-level intrinsics in vecintrin.h.
- Indicate support by defining  __VEC__ == 10304.

Note: No currently available Z system supports the arch14
architecture.  Once new systems become available, the
official system name will be added as supported -march name.
2021-07-26 16:57:28 +02:00
Paul Walker 8a8d01d58c [NFC] Change VFShape so it contains an ElementCount rather than seperate VF and IsScalable properties.
Differential Revision: https://reviews.llvm.org/D106750
2021-07-26 12:25:46 +01:00
Kuter Dinel 0cd964ff25 [Attributor][FIX] checkForAllInstructions, correctly handle declarations
checkForAllInstructions was not handling declarations correctly.
It should have been returning false when it gets called on a declaration

The patch also fixes a test case for AAFunctionReachability for it to be able
to pass after the changes to the checkForAllinstructions.

Differential Revision: https://reviews.llvm.org/D106625
2021-07-24 02:21:29 +03:00
Mircea Trofin 55e12f7080 [NFC][MLGO] Just use the underlying protobuf object for logging
Avoid buffering just to copy the buffered data, in 'development
mode', when logging. Instead, just populate the underlying protobuf.

Differential Revision: https://reviews.llvm.org/D106592
2021-07-23 10:56:48 -07:00
Dylan Fleming 80e0bd1496 [SVE][IR] Fix Binary op matching in PatternMatch::m_VScale
Reviewed By: sdesmalen

Differential Revision: https://reviews.llvm.org/D105978
2021-07-23 11:39:13 +01:00
Paulo Matos 842e718b66 Add support for zero-sized Scalars as a LowLevelType
Opaque values (of zero size) can be stored in memory with the
implemention of reference types in the WebAssembly backend. Since
MachineMemOperand uses LLTs we need to be able to support
zero-sized scalars types in LLTs.

Differential Revision: https://reviews.llvm.org/D105423
2021-07-22 13:47:19 +02:00
Joseph Huber 754eb1c210 [OpenMP] Change `__kmpc_free_shared` to include the paired allocation size
This patch changes `__kmpc_free_shared` to take an additional argument
corresponding to the associated allocation's size. This makes it easier to
implement the allocator in the runtime.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D106496
2021-07-21 20:56:21 -04:00
John McCall 326a5a2658 Fix a bug in OptimizedStructLayout when filling gaps before
fixed fields with highly-aligned flexible fields.

The code was not considering the possibility that aligning
the current offset to the alignment of a queue might push
us past the end of the gap.  Subtracting the offsets to
figure out the maximum field size for the gap then overflowed,
making us think that we had nearly unbounded space to fill.

Fixes PR 51131.
2021-07-21 15:47:18 -04:00
Jakub Kuderski 3c3165cfa0 [ADT] Add initializer_list constructor to SmallDenseMap
Make it easier to initialize small maps inline. Note that DenseMap already has an initializer_list constructor.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D106363
2021-07-21 09:32:16 -04:00
Guillaume Chatelet d6da02d952 [llvm] Add enum iteration to Sequence
This patch allows iterating typed enum via the ADT/Sequence utility.

It also changes the original design to better separate concerns:
 - `StrongInt` only deals with safe `intmax_t` operations,
 - `SafeIntIterator` presents the iterator and reverse iterator
 interface but only deals with safe `StrongInt` internally.
 - `iota_range` only deals with `SafeIntIterator` internally.

 This design ensures that operations are always valid. In particular,
 "Out of bounds" assertions fire when:
  - the `value_type` is not representable as an `intmax_t`
  - iterator operations make internal computation underflow/overflow
  - the internal representation cannot be converted back to `value_type`

Differential Revision: https://reviews.llvm.org/D106279
2021-07-21 12:48:53 +00:00
Lang Hames a876d09bc7 [JITLink] Add support for moving blocks and symbols between sections.
LinkGraph::transferBlock can be used to move a block and all associated symbols
from one section to another.

LinkGraph::mergeSections moves all blocks and sections from a source section to
a destination section.
2021-07-21 09:10:09 +10:00
Sterling Augustine edfdb5fcd1 Consolidate string types into ptr and length representations.
After rGbbbc4f110e35ac709b943efaa1c4c99ec073da30, we can move
any string type that has convenient pointer and length fields
into the PtrAndLengthKind, reducing the amount of code.

Differential Revision: https://reviews.llvm.org/D106381
2021-07-20 13:29:57 -07:00
Nikita Popov a7f183afe7 [Orc] Fix sret/byval attributes in test (NFC)
This was placing sret/byval attributes without type argument on
non-pointer arguments. Make this valid IR by using pointer
arguments and passing the corresponding attribute type argument.
2021-07-20 20:47:15 +02:00
Sterling Augustine bbbc4f110e Avoid keeping internal string_views in Twine.
This is a follow-up to https://reviews.llvm.org/D103935

A Twine's internal layout should not depend on which version of the
C++ standard is in use. Dynamically linking binaries compiled with two
different layouts (eg, --std=c++14 vs --std=c++17) ends up
problematic.

This change avoids that issue by immediately converting a
string_view to a pointer-and-length at the cost of an extra eight-bytes
in Twine.

Differential Revision: https://reviews.llvm.org/D106186
2021-07-20 08:46:53 -07:00
Anirudh Prasad 0977f31cec [SystemZ][z/OS] Add GOFF support to file magic identification
- This patch adds in the GOFF format to the file magic identification logic in LLVM
- Currently, for the object file support, GOFF is marked as having as an error
- However, this is only temporary until https://reviews.llvm.org/D98437 is merged in

Reviewed By: abhina.sreeskantharajan

Differential Revision: https://reviews.llvm.org/D105993
2021-07-20 10:50:47 -04:00
Ulrich Weigand e04c05e823 [SystemZ] Fix invalid assumption in getCPUNameFromS390Model
Code in getCPUNameFromS390Model currently assumes that the
numerical value of the model number always increases with
future hardware.  While this has happened to be the case
with the last few machines, it is not guaranteed -- that
assumption was violated with (much) older machines, and
it can be violated again with future machines.

Fix by explicitly listing model numbers for all supported
machine models.
2021-07-20 13:39:22 +02:00
Mircea Trofin 55e2d2060a [MLGO] Use binary protobufs for improved training performance.
It turns out that during training, the time required to parse the
textual protobuf of a training log is about the same as the time it
takes to compile the module generating that log. Using binary protobufs
instead elides that cost almost completely.

Differential Revision: https://reviews.llvm.org/D106157
2021-07-19 13:59:28 -07:00
Haowei Wu 6103fdfab4 [ifs][elfabi] Merge llvm-ifs/elfabi tools
This change merges llvm-elfabi and llvm-ifs tools.

Differential Revision: https://reviews.llvm.org/D100139
2021-07-19 11:23:19 -07:00
Haowei Wu 61fa9afe4c [ifs] Prepare llvm-ifs for elfabi/ifs merging.
This diff changes llvm-ifs to use unified IFS file format
and perform other renaming changes in preparation for the
merging between elfabi/ifs.

Differential Revision: https://reviews.llvm.org/D99810
2021-07-19 11:23:00 -07:00
Haowei Wu 8b4acb067f [elfabi] Prepare elfabi/ifs merging.
This change implements unified text stub format and command line
interface proposed in the elfabi/ifs merge plan.

Differential Revision: https://reviews.llvm.org/D99399
2021-07-19 11:22:43 -07:00
Lang Hames bb5f97e3ad [ORC][ORC-RT] Introduce ORC-runtime based MachO-Platform.
Adds support for MachO static initializers/deinitializers and eh-frame
registration via the ORC runtime.

This commit introduces cooperative support code into the ORC runtime and ORC
LLVM libraries (especially the MachOPlatform class) to support macho runtime
features for JIT'd code. This commit introduces support for static
initializers, static destructors (via cxa_atexit interposition), and eh-frame
registration. Near-future commits will add support for MachO native
thread-local variables, and language runtime registration (e.g. for Objective-C
and Swift).

The llvm-jitlink tool is updated to use the ORC runtime where available, and
regression tests for the new MachOPlatform support are added to compiler-rt.

Notable changes on the ORC runtime side:

1. The new macho_platform.h / macho_platform.cpp files contain the bulk of the
runtime-side support. This includes eh-frame registration; jit versions of
dlopen, dlsym, and dlclose; a cxa_atexit interpose to record static destructors,
and an '__orc_rt_macho_run_program' function that defines running a JIT'd MachO
program in terms of the jit- dlopen/dlsym/dlclose functions.

2. Replaces JITTargetAddress (and casting operations) with ExecutorAddress
(copied from LLVM) to improve type-safety of address management.

3. Adds serialization support for ExecutorAddress and unordered_map types to
the runtime-side Simple Packed Serialization code.

4. Adds orc-runtime regression tests to ensure that static initializers and
cxa-atexit interposes work as expected.

Notable changes on the LLVM side:

1. The MachOPlatform class is updated to:

  1.1. Load the ORC runtime into the ExecutionSession.
  1.2. Set up standard aliases for macho-specific runtime functions. E.g.
       ___cxa_atexit -> ___orc_rt_macho_cxa_atexit.
  1.3. Install the MachOPlatformPlugin to scrape LinkGraphs for information
       needed to support MachO features (e.g. eh-frames, mod-inits), and
       communicate this information to the runtime.
  1.4. Provide entry-points that the runtime can call to request initializers,
       perform symbol lookup, and request deinitialiers (the latter is
       implemented as an empty placeholder as macho object deinits are rarely
       used).
  1.5. Create a MachO header object for each JITDylib (defining the __mh_header
       and __dso_handle symbols).

2. The llvm-jitlink tool (and llvm-jitlink-executor) are updated to use the
runtime when available.

3. A `lookupInitSymbolsAsync` method is added to the Platform base class. This
can be used to issue an async lookup for initializer symbols. The existing
`lookupInitSymbols` method is retained (the GenericIRPlatform code is still
using it), but is deprecated and will be removed soon.

4. JIT-dispatch support code is added to ExecutorProcessControl.

The JIT-dispatch system allows handlers in the JIT process to be associated with
'tag' symbols in the executor, and allows the executor to make remote procedure
calls back to the JIT process (via __orc_rt_jit_dispatch) using those tags.

The primary use case is ORC runtime code that needs to call bakc to handlers in
orc::Platform subclasses. E.g. __orc_rt_macho_jit_dlopen calling back to
MachOPlatform::rt_getInitializers using __orc_rt_macho_get_initializers_tag.
(The system is generic however, and could be used by non-runtime code).

The new ExecutorProcessControl::JITDispatchInfo struct provides the address
(in the executor) of the jit-dispatch function and a jit-dispatch context
object, and implementations of the dispatch function are added to
SelfExecutorProcessControl and OrcRPCExecutorProcessControl.

5. OrcRPCTPCServer is updated to support JIT-dispatch calls over ORC-RPC.

6. Serialization support for StringMap is added to the LLVM-side Simple Packed
Serialization code.

7. A JITLink::allocateBuffer operation is introduced to allocate writable memory
attached to the graph. This is used by the MachO header synthesis code, and will
be generically useful for other clients who want to create new graph content
from scratch.
2021-07-19 19:50:16 +10:00
Valentin Churavy 3321f4821e
Reland [Orc] Add verylazy example for C-bindings
This patch relands https://reviews.llvm.org/D104799, but fixes the
memory handling causing leak sanitizer failures.

This reverts commit a56fe117e0.
2021-07-18 21:17:49 +02:00
Valentin Churavy a56fe117e0
Revert "[Orc] Add verylazy example for C-bindings"
Broke ASAN buildbot, will reland with fixes

This reverts commit b5a6ad8c89.
2021-07-18 16:21:37 +02:00
Simon Pilgrim 367ec7755f [Orc] Remove unnecessary <string> include dependency from Orc headers. NFC.
At most these use the StringRef/Twine wrappers and don't have any implicit uses of std::string.

Move the include down to any cpp implementation where std::string is actually used.
2021-07-18 12:31:13 +01:00
Valentin Churavy b5a6ad8c89
[Orc] Add verylazy example for C-bindings
Still WIP, based on the Kaleidoscope/BuildingAJIT/Chapter4.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D104799
2021-07-18 12:07:16 +02:00
Amara Emerson 4c55cdb00a [GlobalISel] Fix known bits for G_BSWAP and B_BITREVERSE not doing anything.
llvm::KnownBits::byteSwap() and reverse() don't modify in-place, so
we weren't actually computing anything. This was causing a miscompile on an
arm64 stage2 bootstrap clang build.
2021-07-17 23:07:16 -07:00
Mehdi Amini 76374573ce Use ManagedStatic and lazy initialization of cl::opt in libSupport to make it free of global initializer
We can build it with -Werror=global-constructors now. This helps
in situation where libSupport is embedded as a shared library,
potential with dlopen/dlclose scenario, and when command-line
parsing or other facilities may not be involved. Avoiding the
implicit construction of these cl::opt can avoid double-registration
issues and other kind of behavior.

Reviewed By: lattner, jpienaar

Differential Revision: https://reviews.llvm.org/D105959
2021-07-16 07:38:16 +00:00
Mehdi Amini 8d051d8546 Revert "Use ManagedStatic and lazy initialization of cl::opt in libSupport to make it free of global initializer"
This reverts commit af9321739b.
Still some specific config broken in some way that requires more
investigation.
2021-07-16 07:35:13 +00:00
Mehdi Amini af9321739b Use ManagedStatic and lazy initialization of cl::opt in libSupport to make it free of global initializer
We can build it with -Werror=global-constructors now. This helps
in situation where libSupport is embedded as a shared library,
potential with dlopen/dlclose scenario, and when command-line
parsing or other facilities may not be involved. Avoiding the
implicit construction of these cl::opt can avoid double-registration
issues and other kind of behavior.

Reviewed By: lattner, jpienaar

Differential Revision: https://reviews.llvm.org/D105959
2021-07-16 06:54:26 +00:00
Mehdi Amini 16b5e9d6a2 Revert "Use ManagedStatic and lazy initialization of cl::opt in libSupport to make it free of global initializer"
This reverts commit 42f588f39c.
Broke some buildbots
2021-07-16 03:46:53 +00:00
Mehdi Amini 42f588f39c Use ManagedStatic and lazy initialization of cl::opt in libSupport to make it free of global initializer
We can build it with -Werror=global-constructors now. This helps
in situation where libSupport is embedded as a shared library,
potential with dlopen/dlclose scenario, and when command-line
parsing or other facilities may not be involved. Avoiding the
implicit construction of these cl::opt can avoid double-registration
issues and other kind of behavior.

Reviewed By: lattner, jpienaar

Differential Revision: https://reviews.llvm.org/D105959
2021-07-16 03:33:20 +00:00
Philip Reames b980d2f54b [unittest] Exercise SCEV's udiv and udiv ceiling routines
The ceiling variant was recently added (due to the work towards D105216), and we're spending a lot of time trying to find optimizations for the expression. This patch brute forces the space of i8 unsigned divides and checks that we get a correct (well consistent with APInt) result for both udiv and udiv ceiling.

(This is basically what I've been doing locally in a hand rolled C++ program, and I realized there no good reason not to check it in as a unit test which directly exercises the logic on constants.)

Differential Revision: https://reviews.llvm.org/D106083
2021-07-15 11:55:00 -07:00
Fraser Cormack 5cbd5c62be [VP][NFC] Correct formatting in unit test 2021-07-15 12:38:47 +01:00
Steven Wu 483df57313 [Support] Turn on SupportTest for Apple Silicon
Follow up for D106012, turn on unittest for Host on Apple Silicon.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D106020
2021-07-14 15:24:56 -07:00
Cullen Rhodes c08dabb0f4 [AArch64][SME] Add matrix register definitions and parsing support
SME introduces the ZA array, a new piece of architectural register state
consisting of a matrix of [SVLb x SVLb] bytes, where SVL is the
implementation defined Streaming SVE vector length and SVLb is the
number of 8-bit elements in a vector of SVL bits.

SME instructions consist of three types of matrix operands:

  * Tiles: a ZA tile is a square, two-dimensional sub-array of elements
  within the ZA array. These tiles make up the larger accumulator array
  and the granularity varies based on the element size, i.e.
    - ZAQ0..ZAQ15 (smallest tile granule)
    - ZAD0..ZAD7
    - ZAS0..ZAS3
    - ZAH0..ZAH1
    or ZAB0       (largest tile granule, single tile)
  * Tile vectors: similar to regular tiles, but have an extra 'h' or 'v'
  to tell how the vector at [reg+offset] is layed out in the tile,
  horizontally or vertically. E.g. za1h.h or za15v.q, which corresponds
  to vectors in registers ZAH1 and ZAQ15, respectively.
  * Accumulator matrix: this is the entire accumulator array ZA.

This patch adds the register classes and related operands and parsing
for SME instructions operating on the accumulator array.

The ADDHA and ADDVA instructions which operate on tiles are also added
in this patch to make some use of the code added, later patches will
make use of the other operands introduced here.

The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2021-06

Co-authored by: Sander de Smalen (@sdesmalen)

Reviewed By: david-arm

Differential Revision: https://reviews.llvm.org/D105570
2021-07-14 08:25:49 +00:00
Guillaume Chatelet 2c47b8847e Revert "[llvm] Add enum iteration to Sequence"
This reverts commit a006af5d6e.
2021-07-13 16:44:42 +00:00
Guillaume Chatelet a006af5d6e [llvm] Add enum iteration to Sequence
This patch allows iterating typed enum via the ADT/Sequence utility.

Differential Revision: https://reviews.llvm.org/D103900
2021-07-13 16:22:19 +00:00
Tim Northover 85cb4f9904 Support: reduce stack used in default size test.
When the sanitizers aren't enabled they can use more than 1KB of stack, causing
an overflow where there shouldn't be.

Should fix Green Dragon test.
2021-07-13 11:24:12 +01:00
Nikita Popov 7ed3e87825 [Attributes] Determine attribute properties from TableGen data
Continuing from D105763, this allows placing certain properties
about attributes in the TableGen definition. In particular, we
store whether an attribute applies to fn/param/ret (or a combination
thereof). This information is used by the Verifier, as well as the
ForceFunctionAttrs pass. I also plan to use this in LLParser,
which also duplicates info on which attributes are valid where.

This keeps metadata about attributes in one place, and makes it
more likely that it stays in sync, rather than in various
functions spread across the codebase.

Differential Revision: https://reviews.llvm.org/D105780
2021-07-12 22:13:38 +02:00
Cullen Rhodes 9e42675103 [AArch64] Add target features for Armv9-A Scalable Matrix Extension (SME)
First patch in a series adding MC layer support for the Arm Scalable
Matrix Extension.

This patch adds the following features:

    sme, sme-i64, sme-f64

The sme-i64 and sme-f64 flags are for the optional I16I64 and F64F64
features.

If a target supports I16I64 then the following instructions are
implemented:

  * 64-bit integer ADDHA and ADDVA variants (D105570).
  * SMOPA, SMOPS, SUMOPA, SUMOPS, UMOPA, UMOPS, USMOPA, and USMOPS
    instructions that accumulate 16-bit integer outer products into 64-bit
    integer tiles.

If a target supports F64F64 then the FMOPA and FMOPS instructions that
accumulate double-precision floating-point outer products into
double-precision tiles are implemented.

Outer products are implemented in D105571.

The reference can be found here:
https://developer.arm.com/documentation/ddi0602/2021-06

Reviewed By: CarolineConcatto

Differential Revision: https://reviews.llvm.org/D105569
2021-07-12 13:28:10 +00:00
Sander de Smalen 41b6057641 [InstructionCost] Add saturation support.
This patch makes the operations on InstructionCost saturate, so that when
costs are accumulated they saturate to <max value>.

One of the compelling reasons for wanting to have saturation support
is because in various places, arbitrary values are used to represent
a 'high' cost, but when accumulating the cost of some set of operations
or a loop, overflow is not taken into account, which may lead to unexpected
results. By defining the operations to saturate, we can express the cost
of something 'very expensive' as InstructionCost::getMax().

Reviewed By: kparzysz, dmgreen

Differential Revision: https://reviews.llvm.org/D105108
2021-07-10 11:28:42 +01:00
Amara Emerson 97c426394a [AArch64][GlobalISel] Implement moreElements legalization for G_SHUFFLE_VECTOR.
Differential Revision: https://reviews.llvm.org/D103301
2021-07-10 00:25:26 -07:00
Eli Friedman 9c4baf5101 [ScalarEvolution] Strictly enforce pointer/int type rules.
Rules:

1. SCEVUnknown is a pointer if and only if the LLVM IR value is a
   pointer.
2. SCEVPtrToInt is never a pointer.
3. If any other SCEV expression has no pointer operands, the result is
   an integer.
4. If a SCEVAddExpr has exactly one pointer operand, the result is a
   pointer.
5. If a SCEVAddRecExpr's first operand is a pointer, and it has no other
   pointer operands, the result is a pointer.
6. If every operand of a SCEVMinMaxExpr is a pointer, the result is a
   pointer.
7. Otherwise, the SCEV expression is invalid.

I'm not sure how useful rule 6 is in practice.  If we exclude it, we can
guarantee that ScalarEvolution::getPointerBase always returns a
SCEVUnknown, which might be a helpful property. Anyway, I'll leave that
for a followup.

This is basically mop-up at this point; all the changes with significant
functional effects have landed.  Some of the remaining changes could be
split off, but I don't see much point.

Differential Revision: https://reviews.llvm.org/D105510
2021-07-09 17:29:26 -07:00
Arthur Eubanks 4e6013250d [NFC][OpaquePtr] Use GlobalValue::getValueType() more
Instead of getType()->getElementType().
2021-07-09 09:55:41 -07:00
David Blaikie 1def2579e1 PR51018: Remove explicit conversions from SmallString to StringRef to future-proof against C++23
C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible with C++23 (& a follow-up change I've made
will make this ambiguous/invalid even in <C++23 so we don't regress
this & it generally improves the code anyway)
2021-07-08 13:37:57 -07:00
Matt Arsenault 9b057f647d GlobalISel: Track original argument index in ArgInfo
SelectionDAG's equivalents in ISD::InputArg/OutputArg track the
original argument index. Mips relies on this, and its currently
reinventing its own parallel CallLowering infrastructure which tracks
these indexes on the side. Add this to help move towards deleting the
custom mips handling.
2021-07-08 13:39:02 -04:00
Tim Northover 48c68a630e Recommit: Support: add llvm::thread class that supports specifying stack size.
This adds a new llvm::thread class with the same interface as std::thread
except there is an extra constructor that allows us to set the new thread's
stack size. On Darwin even the default size is boosted to 8MB to match the main
thread.

It also switches all users of the older C-style `llvm_execute_on_thread` API
family over to `llvm::thread` followed by either a `detach` or `join` call and
removes the old API.

Moved definition of DefaultStackSize into the .cpp file to hopefully
fix the build on some (GCC-6?) machines.
2021-07-08 16:22:26 +01:00
Tim Northover 2bf5e8d953 Revert "Support: add llvm::thread class that supports specifying stack size."
It's causing build failures because DefaultStackSize isn't defined everywhere
it should be and I need time to investigate.
2021-07-08 14:59:47 +01:00
Tim Northover 727e1c9be3 Support: add llvm::thread class that supports specifying stack size.
This adds a new llvm::thread class with the same interface as std::thread
except there is an extra constructor that allows us to set the new thread's
stack size. On Darwin even the default size is boosted to 8MB to match the main
thread.

It also switches all users of the older C-style `llvm_execute_on_thread` API
family over to `llvm::thread` followed by either a `detach` or `join` call and
removes the old API.
2021-07-08 14:51:53 +01:00
xndcn 7445f1e4dc [NFC] Mark Expected<T>::assertIsChecked() as const
Some const methods of Expected<T> invoke assertIsChecked(),
so we should mark it as const too.

Differential Revision: https://reviews.llvm.org/D105292
2021-07-08 21:30:23 +08:00
Arthur Eubanks d338e79a4c [OpaquePtr] Remove checking pointee type for byval/preallocated type
These currently always require a type parameter. The bitcode reader
already upgrades old bitcode without the type parameter to use the
pointee type.

In cases where the caller does not have byval but the callee does, we
need to follow CallBase::paramHasAttr() and also look at the callee for
the byval type so that CallBase::isByValArgument() and
CallBase::getParamByValType() are in sync. Do the same for preallocated.

While we're here add a corresponding version for inalloca since we'll
need it soon.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D104663
2021-07-07 14:28:55 -07:00
Tom Stellard 2c03d92ee6 unittests: Fix library dependency name
Corrects the library name of LLVMAsmParser added in
2e4ec3e5d6.
2021-07-07 06:56:15 -07:00
Tom Stellard 2e4ec3e5d6 unittests: Fix build with LLVM_LINK_LLVM_DYLIB=ON
The build system was linking the PluginsTests unittest against libLLVM.so
and LLVMAsmParser which was causing the test to fail with this error:

LLVM ERROR: inconsistency in registered CommandLine options

We need to add llvm libraries to LLVM_LINK_COMPONENTS so that
they are dropped from the linker arguments when linking with
LLVM_LINK_LLVM_DYLIB=ON

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D105523
2021-07-07 06:50:45 -07:00
Jacob Hegna 99f00635d7 Unpack the CostEstimate feature in ML inlining models.
This change yields an additional 2% size reduction on an internal search
binary, and an additional 0.5% size reduction on fuchsia.

Differential Revision: https://reviews.llvm.org/D104751
2021-07-02 16:57:16 +00:00
Scott Linder 83887df155 [ADT] Follow up to fix bug in "Add makeVisitor to STLExtras.h"
Address mistakenly comparing the pointer values of two C-style strings
rather than comparing their contents in the unit tests for makeVisitor,
added in 6d6f35eb7b
2021-07-01 18:24:49 +00:00
Hussain Kadhem d21a35ac0a [VP] Implementation of intrinsic and SDNode definitions for VP load, store, gather, scatter.
This patch adds intrinsic definitions and SDNodes for predicated
load/store/gather/scatter, based on the work done in D57504.

Reviewed By: simoll, craig.topper

Differential Revision: https://reviews.llvm.org/D99355
2021-07-01 13:34:44 +02:00
Lang Hames 39f64c4c83 [ORC] Add wrapper-function support methods to ExecutorProcessControl.
Adds support for both synchronous and asynchronous calls to wrapper functions
using SPS (Simple Packed Serialization). Also adds support for wrapping
functions on the JIT side in SPS-based wrappers that can be called from the
executor.

These new methods simplify calls between the JIT and Executor, and will be used
in upcoming ORC runtime patches to enable communication between ORC and the
runtime.
2021-07-01 18:21:49 +10:00
Matt Arsenault 28f2f66200 GlobalISel: Use LLT in memory legality queries
This enables proper lowering of non-byte sized loads. We still aren't
faithfully preserving memory types everywhere, so the legality checks
still only consider the size.
2021-06-30 17:44:13 -04:00
Matt Arsenault fae05692a3 CodeGen: Print/parse LLTs in MachineMemOperands
This will currently accept the old number of bytes syntax, and convert
it to a scalar. This should be removed in the near future (I think I
converted all of the tests already, but likely missed a few).

Not sure what the exact syntax and policy should be. We can continue
printing the number of bytes for non-generic instructions to avoid
test churn and only allow non-scalar types for generic instructions.

This will currently print the LLT in parentheses, but accept parsing
the existing integers and implicitly converting to scalar. The
parentheses are a bit ugly, but the parser logic seems unable to deal
without either parentheses or some keyword to indicate the start of a
type.
2021-06-30 16:54:13 -04:00
Michael Liao 4339d3bd84 Fix shared build. 2021-06-30 14:04:16 -04:00
Tomas Matheson f617ab1044 [NPM] Resolve llvmGetPassPluginInfo to the plugin being loaded
Dynamically loaded plugins for the new pass manager are initialised by
calling llvmGetPassPluginInfo. This is defined as a weak symbol so that
it is continually redefined by each plugin that is loaded. When loading
a plugin from a shared library, the intention is that
llvmGetPassPluginInfo will be resolved to the definition in the most
recent plugin. However, using a global search for this resolution can
fail in situations where multiple plugins are loaded.

Currently:

* If a plugin does not define llvmGetPassPluginInfo, then it will be
  silently resolved to the previous plugin's definition.

* If loading the same plugin twice with another in between, e.g. plugin
  A/plugin B/plugin A, then the second load of plugin A will resolve to
  llvmGetPassPluginInfo in plugin B.

* The previous case can also occur when a dynamic library defines both
  NPM and legacy plugins; the legacy plugins are loaded first and then
  with `-fplugin=A -fpass-plugin=B -fpass-plugin=A`: A will be loaded as
  a legacy plugin and define llvmGetPassPluginInfo; B will be loaded
  and redefine it; and finally when A is loaded as an NPM plugin it will
  be resolved to the definition from B.

Instead of searching globally, restrict the symbol lookup to the library
that is currently being loaded.

Differential Revision: https://reviews.llvm.org/D104916
2021-06-30 18:11:28 +01:00
Greg Clayton ab546ead3b Fix a case where multiple symbols with zero size would cause duplicate entries in gsym files.
Symbol tables can have symbols with no size in mach-o files that were failing to get combined into a single entry. This resulted in many duplicate entries for the same address and made gsym files larger.

Differential Revision: https://reviews.llvm.org/D105068
2021-06-28 18:26:26 -07:00
Scott Linder 6d6f35eb7b [ADT] Add makeVisitor to STLExtras.h
Relands patch reverted by 61242c0add
The original patch mistakenly included unrelated tests.

Adds a utility to combine multiple Callables into a single Callable.
This is useful to make constructing a visitor for `std::visit`-like
functions more natural; functions like this will be added in future
patches.

Intended to supercede https://reviews.llvm.org/D99560 by
perfectly-forwarding the combined Callables.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D100670
2021-06-28 20:07:11 +00:00