Commit Graph

20386 Commits

Author SHA1 Message Date
Craig Topper a58b62b4a2 [IR] Replace all uses of CallBase::getCalledValue() with getCalledOperand().
This method has been commented as deprecated for a while. Remove
it and replace all uses with the equivalent getCalledOperand().

I also made a few cleanups in here. For example, to removes use
of getElementType on a pointer when we could just use getFunctionType
from the call.

Differential Revision: https://reviews.llvm.org/D78882
2020-04-27 22:17:03 -07:00
Nick Desaulniers bc7f3240e6 [X86] remove derived method w/ same impl as base
Summary:
While looking into issues with IfConverter, I noticed that
X86InstrInfo::isUnpredicatedTerminator matched its overriden
implementation in TargetInstrInfo::isUnpredicatedTerminator.

Reviewers: craig.topper, hfinkel, MaskRay, echristo

Reviewed By: MaskRay, echristo

Subscribers: hiraditya, llvm-commits, srhines

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D62749
2020-04-27 17:41:00 -07:00
Craig Topper 37ec709233 [X86][CostModel] Update truncate costs for some narrow vector cases to match their wider version.
This updates v4i16->v4i8 with sse2 to match v8i16->v8i8.
Update v2i16->v2i8 and v4i16->v4i8 with sse 4.1 to match v8i16->v8i8.
2020-04-27 13:47:48 -07:00
Wei Mi 68d2301e12 Recommit "Generate Callee Saved Register (CSR) related cfi directives
like .cfi_restore"

Insert .cfi_offset/.cfi_register when IncomingCSRSaved of current block
is larger than OutgoingCSRSaved of its previous block.

Original commit message:
https://reviews.llvm.org/D42848 only handled CFA related cfi directives but
didn't handle CSR related cfi. The patch adds the CSR part. Basically it reuses
the framework created in D42848. For each basicblock, the patch tracks which
CSR set have been saved at its CFG predecessors's exits, and compare the CSR
set with the set at its previous basicblock's exit (The previous block is the
block laid before the current block). If the saved CSR set at its previous
basicblock's exit is larger, .cfi_restore will be inserted.

The patch also generates proper .cfi_restore in epilogue to make sure the
saved CSR set is consistent for the incoming edges of each block.

Differential Revision: https://reviews.llvm.org/D74303
2020-04-27 12:46:58 -07:00
Craig Topper bdbbed115f [X86][CostModel] Update costs for vector truncate with avx512f/avx512bw.
All avx512 truncate instructions except vXi64->vXi32 are 2 uops
on port 5. So raise their costs to 2. Except when we have an
earlier faster sequence like pshufb for 128 bit input vectors.

Add a lower cost of 3 v16i16->v16i8 with avx512f where we can
extend to v16i32 then truncate. And a cost of 2 for avx512bw with
and without avx512vl. There we can use vpmovwb with either a ymm
or zmm input. Both of these beat masking, splitting, and using
packuswb which is our avx/avx2 codegen.
2020-04-27 12:00:24 -07:00
Craig Topper 5eff75d86a [X86][CostModel] Improve costs for fp_to_uint/fp_to_sint for vXi8/vXi16/v2i32 results.
Differential Revision: https://reviews.llvm.org/D78893
2020-04-27 10:35:15 -07:00
Simon Pilgrim d9e174dbf7 [X86][SSE] getFauxShuffle - account for PEXTW/PEXTB implicit zero-extension
The insert(truncate/extend(extract(vec0,c0)),vec1,c1) case in rGacbc5ede99 wasn't combining the 'mineltsize' with the src vector elt size which may be smaller due to implicit extension during extraction.

Reduced from test case provided by @mstorsjo
2020-04-27 12:46:50 +01:00
QingShan Zhang 2957fa0cd1 [NFC][DAGCombine] Adding three helper functions and change the getNegatedExpression to negateExpression
This is a NFC patch for D77319. The idea is to hide the getNegatibleCost inside the getNegatedExpression()
to have it return null if the cost is expensive, and add some helper function for easy to use. And
rename the old getNegatedExpression to negateExpression to avoid the semantic conflict.

Reviewed By: RKSimon

Differential revision: https://reviews.llvm.org/D78291
2020-04-27 04:11:42 +00:00
Craig Topper fc02d9f3c6 [X86] Add cost table entry for v2i32->v2f64 fp_to_uint with avx512.
We're currently getting this from the default implementation. But
I don't like how the cost model came to this answer and I might
be making some changes there.
2020-04-26 19:59:01 -07:00
Simon Pilgrim acbc5ede99 [X86][SSE] getFauxShuffle - support insert(truncate/extend(extract(vec0,c0)),vec1,c1) shuffle patterns at the byte level
Followup to the PR45604 fix at rGe71dd7c011a3 where we disabled most of these cases.

By creating the shuffle at the byte level we can handle any extension/truncation as long as we track how small the scalar got and assume that the upper bytes will need to be zero.
2020-04-26 15:31:01 +01:00
Simon Pilgrim 33f043cc9f X86ISelDAGToDAG.cpp - remove unnecessary includes. NFC.
The X86 specific headers have to include these so we don't need to duplicate.
2020-04-26 14:50:53 +01:00
Simon Pilgrim a90d939030 X86MCTargetDesc.h - remove unused DataType.h include. NFC. 2020-04-26 14:50:52 +01:00
Simon Pilgrim 5cc84d095e X86MCTargetDesc.cpp - remove MSVC intrin.h include. NFC.
This was needed when the file called cpuid but that was removed at rL233170.
2020-04-26 14:50:52 +01:00
Simon Pilgrim fd283ddb9b X86MacroFusion.h - reduce MachineScheduler.h include. NFC.
We only need a ScheduleDAGMutation forward declaration.
2020-04-26 14:50:52 +01:00
Simon Pilgrim e4196b1cae X86Operand.h - remove unnecessary includes. NFC. 2020-04-26 12:12:22 +01:00
Craig Topper b9de62c2b6 [X86] Fix the cost of v16i1->v16i16 sext/zext on avx targets.
Previously we were hitting the scalarization case in the default
implementation.
2020-04-25 23:16:20 -07:00
Craig Topper 19cb26f517 [X86][CostModel] Improve costs for vXi1 sign_extend/zero_extend with avx512.
With avx512 vXi1 is legal and uses k-registers with many custom cases
for extending.
2020-04-25 23:16:20 -07:00
Fangrui Song 2cb48d620f [TableGen] Drop deprecated leading # operation (NOP) and replace ## with # 2020-04-25 16:26:45 -07:00
Craig Topper c1cb733db6 [X86] Improve lowering of v16i8->v16i1 truncate under prefer-vector-width=256. 2020-04-25 15:20:33 -07:00
Simon Pilgrim 4425751317 X86ISelLowering.h - remove unnecessary includes. NFC.
Fixed implicit MachineFrameInfo.h dependency in X86SelectionDAGInfo.cpp
2020-04-25 20:07:34 +01:00
Sanjay Patel 7f4ff782d4 [x86] use vector instructions to lower even more FP->int->FP casts
This is another enhancement to D77895/D78362
to avoid a round-trip from XMM->GPR->XMM.
This time we handle the case of starting/ending with different FP types
but always with signed i32 as the intermediate value.
I think this covers all of the faux vector optimization possibilities
for pre-AVX512.

There is at least 1 other transform mentioned in PR36617:
https://bugs.llvm.org/show_bug.cgi?id=36617#c19
...where we fold an 'fpext' into a preceding 'sitofp'. I think we will
want to handle that earlier (DAGCombiner or instcombine) because that's
a target-independent optimization.

Differential Revision: https://reviews.llvm.org/D78758
2020-04-25 11:38:54 -04:00
Craig Topper 7664a0d282 [X86] Improve accuracy of cost for v16i64->v16i8 truncate with avx512.
The 2 vpmovqds are only 1 uop each.
2020-04-24 19:13:55 -07:00
Craig Topper e4a9190ad7 [X86][ArgumentPromotion] Allow Argument Promotion if caller and callee disagree on 512-bit vectors support if the arguments are scalar.
If one of caller/callee has disabled ZMM registers due to
prefer-vector-width=256, we were previously
disabling argument promotion as the ABI might be incompatible since
one side will split 512-bit vectors in this case.

But if we can see that the types are all scalar this shouldn't be
a problem.

This patch assumes that pointer element type reflects the type that
the argument will be promoted to.

Differential Revision: https://reviews.llvm.org/D78770
2020-04-24 15:47:02 -07:00
Simon Pilgrim c741dfe325 X86MCTargetDesc.h - replace FormattedStream.h include with forward declaration. NFC. 2020-04-23 17:42:51 +01:00
Simon Pilgrim 90c956318b X86TargetObjectFile.h - remove unnecessary TargetLoweringObjectFile.h include. NFC.
We already include TargetLoweringObjectFileImpl.h which includes it and we only use its types as part of TargetLoweringObjectFile* overridden methods.
2020-04-23 17:42:50 +01:00
Kazuaki Ishizaki 0312b9f550 [llvm] NFC: Fix trivial typo in rst and td files
Differential Revision: https://reviews.llvm.org/D77469
2020-04-23 14:26:32 +09:00
Vedant Kumar 10ce1bc8d0 [MachineBasicBlock] Add helpers for skipping debug instructions [1/14]
Summary:
These helpers are exercised by follow-up commits in this patch series,
which is all about removing CodeGen differences with vs. without debug
info in the AArch64 backend.

Reviewers: fhahn, aprantl, jpaquette, paquette

Subscribers: kristof.beyls, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78260
2020-04-22 17:03:39 -07:00
Simon Pilgrim 09ba6f9e69 X86TargetMachine.h - remove unused X86RegisterBankInfo forward declaration. NFC. 2020-04-22 14:01:51 +01:00
aartbik 8387bee94d [llvm] [X86] Fixed type bug in vselect for AVX masked load
Summary:
Bugzilla issue 45563
https://bugs.llvm.org/show_bug.cgi?id=45563

Reviewers: nicolasvasilache, mehdi_amini, craig.topper

Reviewed By: craig.topper

Subscribers: RKSimon, hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78527
2020-04-21 11:11:35 -07:00
Fangrui Song 5771c98562 [XRay] Change xray_instr_map sled addresses from absolute to PC relative for x86-64
xray_instr_map contains absolute addresses of sleds, which are relocated
by `R_*_RELATIVE` when linked in -pie or -shared mode.

By making these addresses relative to PC, we can avoid the dynamic
relocations and remove the SHF_WRITE flag from xray_instr_map.  We can
thus save VM pages containg xray_instr_map (because they are not
modified).

This patch changes x86-64 and bumps the sled version to 2. Subsequent
changes will change powerpc64le and AArch64.

Reviewed By: dberris, ianlevesque

Differential Revision: https://reviews.llvm.org/D78082
2020-04-21 09:36:09 -07:00
Simon Pilgrim c74acd8fc9 X86ISelLowering.cpp - clang-format to fix col80 limit. NFC. 2020-04-21 15:18:23 +01:00
Shengchen Kan c031378ce0 [MC][NFC] Use camelCase style for functions in MCObjectStreamer 2020-04-20 20:09:20 -07:00
Shengchen Kan 8bb059ab63 [MC][Bugfix] Remove redundant parameter for relaxInstruction
Summary:
Before this patch, `relaxInstruction` takes three arguments, the first
argument refers to the instruction before relaxation and the third
argument is the output instruction after relaxation. There are two quite
strange things:
  1) The first argument's type is `const MCInst &`, the third
  argument's type is `MCInst &`, but they may be aliased to the same
  variable
  2) The backends of ARM, AMDGPU, RISC-V, Hexagon assume that the third
  argument is a fresh uninitialized `MCInst` even if `relaxInstruction`
  may be called like `relaxInstruction(Relaxed, STI, Relaxed)` in a
  loop.

In this patch, we drop the thrid argument, and let `relaxInstruction`
directly modify the given instruction. Also, this patch fixes the bug https://bugs.llvm.org/show_bug.cgi?id=45580, which is introduced by D77851, and
breaks the assumption of ARM, AMDGPU, RISC-V, Hexagon.

Reviewers: Razer6, MaskRay, jyknight, asb, luismarques, enderby, rtaylor, colinl, bcain

Reviewed By: Razer6, MaskRay, bcain

Subscribers: bcain, nickdesaulniers, nathanchance, wuzish, annita.zhang, arsenm, dschuff, jyknight, dylanmckay, sdardis, nemanjai, jvesely, nhaehnle, tpr, sbc100, jgravelle-google, kristof.beyls, hiraditya, aheejin, kbarton, fedor.sergeev, asb, rbar, johnrusso, simoncook, sabuasal, niosHD, jrtc27, MaskRay, zzheng, edward-jones, atanasyan, rogfer01, MartinMosbeck, brucehoult, the_o, PkmX, jocewei, Jim, lenary, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, kerbowa, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78364
2020-04-21 11:06:55 +08:00
Simon Pilgrim 21bd3767c8 X86MacroFusion.cpp - ensure X86MacroFusion.h module header is included first. NFC. 2020-04-20 14:25:15 +01:00
Simon Pilgrim 4ba7ae85da X86Subtarget.h - remove unused includes. NFC.
Replace with forward declarations.
2020-04-20 12:10:45 +01:00
Simon Pilgrim 2cfcbc52c3 X86Subtarget.cpp - sort includes. NFC
Ensure X86Subtarget.h module header is at the top, and sort the remaining includes.
2020-04-20 11:54:04 +01:00
Simon Pilgrim 179dced13b X86MCTargetDesc.h - remove unnecessary MCStreamer.h include. NFC.
We don't need all of MCStreamer.h, just FormattedStream.h. The rest can be replaced with forward declarations.
X86WinAllocaExpander.cpp had an implicit dependency on MapVector.h which I've added locally.
2020-04-20 11:39:38 +01:00
Simon Pilgrim 44cf9b85ad X86MCAsmInfo.h - remove unnecessary MCAsmInfo.h include. NFC.
We only use the COFF/Darwin/ELF classes directly.
2020-04-20 11:39:38 +01:00
Simon Pilgrim da3bf811be X86InstrFoldTables.h - remove unnecessary include. NFC.
We don't need the limits defines, just the sized integer types so use cstdint system header directly.
2020-04-20 11:39:38 +01:00
Sam Parker e3056ae9a0 [NFC][TTI] Explicit use of VectorType
The API for shuffles and reductions uses generic Type parameters,
instead of VectorType, and so assertions and casts are used a lot.
This patch makes those types explicit, which means that the clients
can't be lazy, but results in less ambiguity, and that can only be a
good thing.

Bugzilla: https://bugs.llvm.org/show_bug.cgi?id=45562

Differential Revision: https://reviews.llvm.org/D78357
2020-04-20 09:16:52 +01:00
Xiang1 Zhang 0980038a5e Handle CET for -exception-model sjlj
Summary:
In SjLj exception mode, the old landingpad BB will create a new landingpad BB and use indirect branch jump to the old landingpad BB in lowering.
So we should add 2 endbr for this exception model.

Reviewers: hjl.tools, craig.topper, annita.zhang, LuoYuanke, pengfei, efriedma

Reviewed By: LuoYuanke

Subscribers: hiraditya, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77124
2020-04-20 11:13:40 +08:00
Shengchen Kan b78c3c89c2 [X86][MC][NFC] Reduce the parameters of functions in X86MCCodeEmitter(Part III)
Summary:
When we encode an instruction, we need to know the number of bytes being
emitted to determine the fixups in `X86MCCodeEmitter::emitImmediate`.
There are only two callers for `emitImmediate`: `emitMemModRMByte` and
`encodeInstruction`.

Before this patch, we kept track of the current byte being emitted
by passing a reference parameter `CurByte` across all the `emit*`
funtions, which is ugly and unnecessary. For example, we don't have any
fixups when emitting prefixes, so we don't need to track this value.

In this patch, we use `StartByte` to record the initial status of the
streamer, and use `OS.tell()` to get the current status of the streamer
when we need to know the number of bytes being emitted. On one hand,
this eliminates the parameter `CurByte` for most `emit*` functions, on
the other hand, this make things clear: Only pass the parameter when we
really need it.

Reviewers: craig.topper, pengfei, MaskRay

Reviewed By: craig.topper, MaskRay

Subscribers: hiraditya, llvm-commits, annita.zhang

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78419
2020-04-20 10:03:41 +08:00
Craig Topper d7e2d937bc [X86] Add X86ISD nodes for PDEP and PEXT.
This will allow use to add DAG combines for these instructions.
2020-04-19 16:14:13 -07:00
Simon Pilgrim a938c7b9ed X86CallLowering.h - remove unnecessary ArrayRef.h include. NFC. 2020-04-19 21:25:10 +01:00
Simon Pilgrim 8859c7f6eb X86MachineFunctionInfo.h - remove unused include. NFC. 2020-04-19 16:58:59 +01:00
Simon Pilgrim c27fdc84df X86InstrInfo.h - remove unused forward declarations. NFC. 2020-04-19 16:58:59 +01:00
Simon Pilgrim a156646443 X86DisassemblerDecoder.h - remove unused forward declaration. NFC. 2020-04-19 16:58:58 +01:00
Sanjay Patel 720015e537 [x86] avoid build warning for enum mismatch; NFC
gcc may warn here because X86ISD::NodeType is specified as "unsigned",
but ISD::NodeType is a naked C enum (although passed as an "unsigned"
throughout SDAG).
2020-04-19 10:22:11 -04:00
Simon Pilgrim 60765e911d X86MCTargetDesc.h - remove unnecessary includes and forward declarations. NFC. 2020-04-19 14:29:35 +01:00
Simon Pilgrim 18bf42a86c X86.h - remove unused forward declarations. NFC. 2020-04-19 14:28:52 +01:00