Commit Graph

155473 Commits

Author SHA1 Message Date
zhongyunde 064b2a6dc6 [DAGCombiner][AArch64] Enhance to fold CSNEG into CSINC instruction
Perform the scalar expression combine in the form of:
  CSNEG(1, c, cc) + b  =>  cc  ? b+1 : b-c => CSINC(b-c, b, !cc)
  CSNEG(c, -1, cc) + b =>  cc  ? b+c : b+1 => CSINC(b+c, b, cc)

Fix https://github.com/llvm/llvm-project/issues/53071

Reviewed By: dmgreen

Differential Revision: https://reviews.llvm.org/D119105
2022-02-16 09:39:38 +08:00
Jan Korous 4a794d848c [Bitstream] Improve error propagation from ReadVBR
Differential Revision: https://reviews.llvm.org/D119307
2022-02-15 17:12:38 -08:00
Mubariz Afzal ea0676f97d [SystemZ][z/OS] Fix f32 variadic argument assertion
The tablegen lines that specify the XPLINK64 calling convention for promoting an f32 vararg to an f64 are effectively overwritten by the following tablegen line which bitcast an f64 vararg to an i64 (so that it can be used in the GPRs). It becomes a bitcast from f32 to i64.

Since we don't handle a bitcast for f32s this caused an assertion.
2022-02-15 18:11:57 -05:00
Florian Mayer a650bb58c0 [NFC] [MTE] only do one pass over allocas for stack tagging.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D119801
2022-02-15 13:09:24 -08:00
Stanislav Mekhanoshin 29a0e0a9e5 [AMDGPU] Do not define GET_INSTRINFO_SCHED_ENUM
Autogenerated names are too long and break compilation on Windows,
while we do not need this enum at all.

Differential Revision: https://reviews.llvm.org/D119869
2022-02-15 13:00:54 -08:00
Philip Reames 2e50760775 [SLP] Add assert that entities are scheduled as expected
Requested in D118538
2022-02-15 12:21:49 -08:00
Florian Mayer 59e7de26aa [HWASan] remove replacement of DbgVariableIntrinsics.
This code was dead because we AI->replaceUsesWithIf above. I verified
this doesn't actually get run by applying
https://gist.github.com/fmayer/aea7cbb4700cfe2c9d932591ae1073c3 to the
Android toolchain and building AOSP, without any crash.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D119802
2022-02-15 11:40:58 -08:00
Mircea Trofin c62eefb886 [nfc][codegen] Move RegisterBank[Info].cpp under CodeGen
Layering-wise, it seems RegisterBank stuff fits under CodeGen, like
other target abstraction.
In particular, TargetSubtargetInfo has a getRegBankInfo member, but
using that object requires making sure GlobalISel is linked, which is
not always the case (e.g. llvm-jitlink doesn't).

Differential Revision: https://reviews.llvm.org/D119053
2022-02-15 11:27:15 -08:00
minglotus-6 3940f1e237 [ProfData] Change type of options from int to uint64_t.
- Reader uses option values to override uint64_t values.

Differential Revision: https://reviews.llvm.org/D119810
2022-02-15 10:59:06 -08:00
Simon Moll de42307e44 [VE] Fix breakage after D118981
VE backend code expected all VP SDNode to have a mask parameter.  This
is not the case with vp.select|merge after D118981.
2022-02-15 18:56:20 +01:00
Simon Moll 03e83cc8eb [VP] vp.fptosi cast intrinsic and docs
Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D119535
2022-02-15 18:17:19 +01:00
Craig Topper ab6e02dded [RISCV] Match vwmulsu_vx with scalar splat input.
This is a more generic version of D119110 that uses MaskedValueIsZero
to do the matching and SimplifyDemandedBits to remove any unneeded
AND instructions.

Tests were taken from D119110.

Reviewed By: Chenbing.Zheng

Differential Revision: https://reviews.llvm.org/D119622
2022-02-15 08:45:21 -08:00
Craig Topper d132b47bb9 [RISCV] Replace llvm_unreachable with report_fatal_error.
Parsing errors aren't handled earlier in all cases. A simple
example is llc -mtriple=riscv64 -mattr=+zve32f. If F or Finx is
not also specified, this will hit a parse error.

Use a fatal_error so that the error is conveyed to the user.
2022-02-15 08:40:37 -08:00
Sanjay Patel 7cc0a29b3f [Analysis] propagate poison through add/sub saturate intrinsics
A more general enhancement needs to add tests and make sure
that intrinsics that return structs are correct. There are also
target-specific intrinsics, and I'm not sure what behavior is
expected for those.
2022-02-15 10:45:32 -05:00
Sanjay Patel 00218c188b [Analysis] propagate poison through integer min/max intrinsics
A more general enhancement needs to add tests and make sure
that intrinsics that return structs are correct. There are also
target-specific intrinsics, and I'm not sure what behavior is
expected for those.
2022-02-15 10:45:32 -05:00
Matt Devereau 7dce12de68 [AArch64] Suggest b.nfrst if the user tries b.nfirst.
Differential Revision: https://reviews.llvm.org/D119453

Co-authored-by: George Steed <george.steed@arm.com>
2022-02-15 15:06:04 +00:00
Max Kazantsev bfc1217119 [NFC] Introduce option to switch off compatible invokes merge
Does not affect default behavior (transform is on).
2022-02-15 21:51:03 +07:00
Amy Kwan ac5a5a9cfe [PowerPC] Add default handling for single element vectors, and split/promote vNi1 vectors.
This patch updates the handling of vectors in getPreferredVectorAction():

For single-element and scalable vectors, fall back to default vector legalization
handling. For vNi1 vectors, add handling to either split or promote them in
order to prevent the production of wide v256i1/v512i1 types.

The following assertion is fixed by this patch, as we ended up producing the
wide vector types (that are used for MMA) in the backend prior to this fix.

```
Assertion failed: VT.getSizeInBits() == Operand.getValueSizeInBits() &&
"Cannot BITCAST between types of different sizes!"
```

Differential Revision: https://reviews.llvm.org/D119521
2022-02-15 08:44:08 -06:00
Simon Pilgrim 2808743cbd [X86] LowerVSETCC - always split 512-bit vectors before lowering to PCMPEQ/GT (PR53842)
Extend the existing split where we already do this for v32i16/v64i8

We can end up trying to use PCMPEQ/GT if the result needs to be sign-extended (typically due to the DAGCombiner::foldSextSetcc fold).

Fixes #53842
2022-02-15 14:21:12 +00:00
Alexander Potapenko 05ee1f4af8 Revert "[asan] Add support for disable_sanitizer_instrumentation attribute"
This reverts commit dd145f953d.

https://reviews.llvm.org/D119726, like https://reviews.llvm.org/D114421,
still causes TSan to fail, see https://lab.llvm.org/buildbot/#/builders/70/builds/18020

Differential Revision: https://reviews.llvm.org/D119838
2022-02-15 15:04:53 +01:00
Sanjay Patel 6357ccf57f [InstCombine] reassociate min/max intrinsics with constant operands
Integer min/max operations are associative:
  max (max X, C0), C1 --> max X, (max C0, C1) --> max X, NewC

https://alive2.llvm.org/ce/z/wW5HVM

This would avoid a regression when we canonicalize to min/max intrinsics
(see D98152 ).

Differential Revision: https://reviews.llvm.org/D119754
2022-02-15 08:31:23 -05:00
Simon Pilgrim 9606c69087 [InstCombine] Fold sub(Y,and(lshr(X,C),1)) --> add(ashr(shl(X,(BW-1)-C),BW-1),Y) (PR53610)
As noted on PR53610, we can fold a 'bit splat' negation of a shifted bitmask pattern into a pair of shifts.

https://alive2.llvm.org/ce/z/eGrsoN

Differential Revision: https://reviews.llvm.org/D119715
2022-02-15 13:24:20 +00:00
Anton Afanasyev b7574b092a [SLP] Don't try to vectorize pair with insertelement
Particularly this breaks vectorization of insertelements where some of
intermediate (i.e. not last) insertelements are used externally.

Fixes PR52275
Fixes #51617

Differential Revision: https://reviews.llvm.org/D119679
2022-02-15 16:12:59 +03:00
Alexander Potapenko dd145f953d [asan] Add support for disable_sanitizer_instrumentation attribute
For ASan this will effectively serve as a synonym for
__attribute__((no_sanitize("address")))

This is a reland of https://reviews.llvm.org/D114421

Reviewed By: melver, eugenis

Differential Revision: https://reviews.llvm.org/D119726
2022-02-15 14:06:12 +01:00
Nikita Popov 2460a2ce47 [DSE] Extract a common PDT check (NFC) 2022-02-15 13:05:45 +01:00
Simon Moll 53efbc15cb [VE] v256i1 broadcast isel and tests
Reviewed By: kaz7

Differential Revision: https://reviews.llvm.org/D119241
2022-02-15 12:40:51 +01:00
Markus Böck 78c27a3cee [X86][Win64] Avoid statepoints in trailing call position
The "avoid trailing call pass" makes sure that no function ends with a call instruction for the purpose of the unwinder.
It starts of by skipping over any non real instruction, which is approximated via the Pseudo and Meta property. This sadly leads to issues when the last machine instruction is a STATEPOINT, as it is skipped despite it lowering to a call.

This patch fixes the use of a statepoint in the trailing call position by making sure call instructions are not skipped.

Differential Revision: https://reviews.llvm.org/D119644
2022-02-15 12:17:19 +01:00
Nikita Popov f35af77573 [InstSimplify] Strip offsets once in computePointerICmp()
Instead of doing an inbounds strip first and another non-inbounds
strip afterward for equality comparisons, directly do a single
inbounds or non-inbounds strip based on whether we have an equality
predicate or not.

This is NFC-ish in that the alloca equality codepath is the only
part that sees additional non-inbounds offsets now, and for that
codepath it doesn't matter whether or not the GEP is inbounds, as
it does a stronger check itself. InstCombine would infer inbounds
for such GEPs.
2022-02-15 12:04:24 +01:00
Jay Foad a65b9dd049 [AMDGPU] Divergence-driven instruction selection for bfm patterns
Differential Revision: https://reviews.llvm.org/D119706
2022-02-15 10:49:18 +00:00
David Green 655d0d86f9 [DAGCombine] Move AVG combine to SimplifyDemandBits
This moves the matching of AVGFloor and AVGCeil into a place where
demand bit are available, so that it can detect more cases for more
folds. It changes the transform to start from a shift, not from a
truncate. We match the pattern shr(add(ext(A), ext(B)), 1), transforming
to ext(hadd(A, B)).

For signed values, because only the bottom bits are demanded llvm will
transform the above to use a lshr too, as opposed to ashr. In order to
correctly detect the hadd we need to know the demanded bits to turn it
back. Depending on whether the shift is signed (ashr) or logical (lshr),
and the extensions are signed or unsigned we can create different nodes.
If the shift is signed:
  Needs >= 2 sign bits. https://alive2.llvm.org/ce/z/h4gQAW generating signed rhadd.
  Needs >= 2 zero bits. https://alive2.llvm.org/ce/z/B64DUA generating unsigned rhadd.
If the shift is unsigned:
  Needs >= 1 zero bits. https://alive2.llvm.org/ce/z/ByD8sj generating unsigned rhadd.
  Needs 1 demanded bit zero and >= 2 sign bits https://alive2.llvm.org/ce/z/hvPGxX and
    https://alive2.llvm.org/ce/z/32P5n1 generating signed rhadd.

Differential Revision: https://reviews.llvm.org/D119072
2022-02-15 10:17:02 +00:00
Jay Foad f72d8897ac [AMDGPU] Honor !invariant.load metadata on load-like intrinsics
Differential Revision: https://reviews.llvm.org/D119739
2022-02-15 09:16:57 +00:00
Nikita Popov 093e9489d5 [BitcodeReader] Change order of assignValue() arguments (NFC)
Other methods in ValueList generally pass Idx first, and it is
more convention for assignment methods to have the target on the
LHS rather than RHS.
2022-02-15 10:11:01 +01:00
Nikita Popov 8f55cd0178 [Support] Fix build on illumos
D116366 added a call to madvise() in Path.inc. Unfortunately,
Illumos does not declare this function if _XOPEN_SOURCE is defined
(which it is by default) and we need to provide the declaration
manually. This is the same workaround used in sanitizers:
ee423d93ea/compiler-rt/lib/sanitizer_common/sanitizer_solaris.cpp (L77-L85)

Differential Revision: https://reviews.llvm.org/D119695
2022-02-15 09:42:40 +01:00
Nikita Popov 1c456a8220 [Bitcode] Improve support for opaque-pointer bitcode upgrade
This is step two of supporting autoupgrade of old bitcode to opaque
pointers. Rather than tracking the element type ID of pointers in
particular, track all type IDs that a type contains. This allows us
to recover the element type in more complex situations, e.g. when
we need to determine the pointer element type of a vector element
or function type parameter.

Differential Revision: https://reviews.llvm.org/D119339
2022-02-15 09:39:48 +01:00
serge-sans-paille 290e482342 Cleanup LLVMDWARFDebugInfo
As usual with that header cleanup series, some implicit dependencies now need to
be explicit:

llvm/DebugInfo/DWARF/DWARFContext.h no longer includes:
- "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
- "llvm/DebugInfo/DWARF/DWARFCompileUnit.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAbbrev.h"
- "llvm/DebugInfo/DWARF/DWARFDebugAranges.h"
- "llvm/DebugInfo/DWARF/DWARFDebugFrame.h"
- "llvm/DebugInfo/DWARF/DWARFDebugLoc.h"
- "llvm/DebugInfo/DWARF/DWARFDebugMacro.h"
- "llvm/DebugInfo/DWARF/DWARFGdbIndex.h"
- "llvm/DebugInfo/DWARF/DWARFSection.h"
- "llvm/DebugInfo/DWARF/DWARFTypeUnit.h"
- "llvm/DebugInfo/DWARF/DWARFUnitIndex.h"

Plus llvm/Support/Errc.h not included by a bunch of llvm/DebugInfo/DWARF/DWARF*.h files

Preprocessed lines to build llvm on my setup:
after: 1065629059
before: 1066621848

Which is a great diff!

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D119723
2022-02-15 09:16:03 +01:00
Serguei Katkov cd16836ce2 [Safepoint Verifier] Add a missed comment to previous commit. 2022-02-15 12:21:33 +07:00
Serguei Katkov 57092d4f4f [Safepoint Verifier] gc.relocate does not change the constant property.
Add traverse through gc.relocate in determining whether base is
isExclusivelyDerivedFromNull OR ExclusivelyNull.

Reviewers: reames, anna
Reviewed By: reames, anna
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D119712
2022-02-15 12:18:46 +07:00
Min-Yih Hsu b99365a7f4 [TableGen] Add a new `encoder` directive into VarLenCodeEmitterGen
The new encoder directive can be used to specify custom encoder for a
single operand or slice. This is different from the EncoderMethod field
within an Operand, which affects every operands in the target.

In addition, this patch also changes the function signature of the
encoder method -- a new argument, InsertPost, is added to both the
default one (i.e. getMachineValue) and the custom one. This argument
provides the bit position where the operand will eventually be inserted.

Differential Revision: https://reviews.llvm.org/D119100
2022-02-14 20:41:15 -08:00
Yonghong Song f419029fcd [BPF] Fix a bug in BTF_KIND_TYPE_TAG generation
Kumar Kartikeya Dwivedi reported a bug ([1]) where BTF_KIND_TYPE_TAG types
are not generated.

Currently, BPF backend only generates BTF types which are used by
the program, e.g., global variables, functions and some builtin functions.
For example, suppose we have
  struct task_struct {
    ...
    struct task_group               *sched_task_group;
    struct mm_struct                *mm;
    ...
    pid_t                           pid;
    pid_t                           tgid;
    ...
  }
If BPF program intends to access task_struct->pid and task_struct->tgid,
there really no need to generate BTF types for struct task_group
and mm_struct.

In BPF backend, during BTF generation, when generating BTF for struct
task_struct, if types for task_group and mm_struct have not been generated
yet, a Fixup structure will be created, which will be reexamined later
to instantiate into either a full type or a forward type.

In current implementation, if we have something like
  struct foo {
     struct bar  __tag1    *f;
  };
and when generating types for struct foo, struct bar type
has not been generated, the __tag1 will be lost during later
Fixup instantiation. This patch fixed this issue by properly
handling btf_type_tag's during Fixup instantiation stage.

  [1] https://lore.kernel.org/bpf/20220210232411.pmhzj7v5uptqby7r@apollo.legion/

Differential Revision: https://reviews.llvm.org/D119799
2022-02-14 19:43:57 -08:00
Snehasish Kumar 50713461d4 Reland "[memprof] Introduce a wrapper around MemInfoBlock."
This reverts commit e6999040f5.

Update test to fix signed int comparison warning, fix whitespace in
compiler-rt MIBEntryDef.inc file.

Differential Revision: https://reviews.llvm.org/D117256
2022-02-14 19:04:36 -08:00
Hongtao Yu 62ef77ca63 [CSSPGO] Do not merge a context that is already duplicated into the base profile.
Do not merge a context that is already duplicated into the base profile.

Also fixing a typo caused by previous refactoring.

Reviewed By: wenlei

Differential Revision: https://reviews.llvm.org/D119735
2022-02-14 18:07:11 -08:00
Jonas Devlieghere 409c515f3f [dsymutil] Add the ability to run the DWARF verifier on the input
Currently you can run the DWARF verifier on the linked dsymutil output.
This patch extends this functionality and makes it possible to
run the DWARF verifier on the input as well.

A new option --verify-dwarf allows you to specify input, output, all and
none. The existing --verify flag remains unchanged and acts and alias
for --verify-dwarf=output.

Input verification issues do not result in a non-zero exit code because
dsymutil is capable of taking invalid DWARF as input and producing valid
DWARF as output.

Differential revision: https://reviews.llvm.org/D89216
2022-02-14 16:14:07 -08:00
Florian Mayer 8de457eafc [HWASAN] use common alignAndPadAlloca
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D119614
2022-02-14 15:28:32 -08:00
Stefan Pintilie a601db30c6 [PowerPC] Remove the LDMX instruction.
The LDMX instruction was to be potentially added in P9 but it was never added
in either ISA 3.0 or ISA 3.1. This patch removes that instruction as it is
currently still an invalid instruction.

Reviewed By: lei

Differential Revision: https://reviews.llvm.org/D118074
2022-02-14 17:03:48 -06:00
Florian Mayer 205308de6b [NFC] [MTE] Move alignAndPadAlloca to MemoryTaggingSupport.
Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D119610
2022-02-14 14:54:04 -08:00
Dmitry Vassiliev 885140171a [NVPTX] Fix NVPTXReplaceImageHandles for multiple uses of a texref
The texsurf_handle is removed by NVPTXReplaceImageHandles.cpp. There are more than one uses of the texsurf_handle, one of them is a regular function call, and one of them is a texture intrinsic.
The current hacky logic in NVPTXReplaceImageHandles.cpp for CUDA cannot handle such a mixed use. This patch fixes this issue.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D119635
2022-02-15 01:30:13 +03:00
Dmitry Vassiliev 6645bfa8f5 [NVPTX] Fix bug with int_nvvm_rotate_b64 when operand immediate
Need to subract from 64, not 32.

Reviewed By: tra

Differential Revision: https://reviews.llvm.org/D119639
2022-02-15 01:23:11 +03:00
Sam Clegg 37f422f4ac [WebAssembly] Use GeneralDynamic TLS for exception handling builtins.
These global TLS symbols are shared across all shared libraries and
therefor should not be assumed to be local to the current module.

Also add new error in the linker when TLS relocations are used against
undefined symbols.  TLS relocations are offsets into the current modules
tls data segment, and don't make sense for undefined symbols which are
modeled as global imports.

Fixes: https://github.com/emscripten-core/emscripten/issues/13398

Differential Revision: https://reviews.llvm.org/D119630
2022-02-14 14:08:32 -08:00
Nick Desaulniers 9dcb006165 [funcattrs] check reachability to improve noreturn
There was a fixme in the code pertaining to attributing functions as
noreturn.  By using reachability, if none of the blocks that are
reachable from the entry return, then the function is noreturn.

Previously, the code only checked if any blocks returned. If they're
unreachable, then they don't matter.

This improves codegen for the Linux kernel.

Fixes: https://github.com/ClangBuiltLinux/linux/issues/1563

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D119571
2022-02-14 14:01:59 -08:00
Fangrui Song a00ae86ab2 Revert D119669 "[NVPTX] Prefix "$L__" for branch label names"
This reverts commit cccef32109.

Broke clang-cuda-t4

```
/buildbot/cuda-t4-0/work/clang-cuda-t4/clang/bin/clang++  -DNDEBUG  -O3 -DNDEBUG   -w -Werror=date-time -UNDEBUG --cuda-path=/buildbot/cuda-t4-0/work/clang-cuda-t4/external/cuda/cuda-11.0 -I/buildbot/cuda-t4-0/work/clang-cuda-t4/external/cuda/cuda-11.0/include --cuda-gpu-arch=sm_75 -std=c++20 -stdlib=libstdc++ --gcc-toolchain=/buildbot/cuda-t4-0/work/clang-cuda-t4/external/cuda/gcc-8 -DSTDLIB_VERSION=2014 -MD -MT External/CUDA/CMakeFiles/complex-cuda-11.0-c++20-libstdc++-8.dir/complex.cu.o -MF External/CUDA/CMakeFiles/complex-cuda-11.0-c++20-libstdc++-8.dir/complex.cu.o.d -o External/CUDA/CMakeFiles/complex-cuda-11.0-c++20-libstdc++-8.dir/complex.cu.o -c /buildbot/cuda-t4-0/work/clang-cuda-t4/llvm-test-suite/External/CUDA/complex.cu
ptxas /tmp/complex-cfa050/complex-sm_75.s, line 250; fatal   : Parsing error near '$L__BB6_2': syntax error
ptxas fatal   : Ptx assembly aborted due to errors
```
2022-02-14 13:23:22 -08:00