Commit Graph

42886 Commits

Author SHA1 Message Date
Chad Rosier 6db9ff64a8 [AArch64] Prefer Bcc to CBZ/CBNZ/TBZ/TBNZ when NZCV flags can be set for "free".
This patch contains a pass that transforms CBZ/CBNZ/TBZ/TBNZ instructions into a
conditional branch (Bcc), when the NZCV flags can be set for "free". This is
preferred on targets that have more flexibility when scheduling Bcc
instructions as compared to CBZ/CBNZ/TBZ/TBNZ (assuming all other variables are
equal). This can reduce register pressure and is also the default behavior for
GCC.

A few examples:

 add w8, w0, w1  -> cmn w0, w1             ; CMN is an alias of ADDS.
 cbz w8, .LBB_2  -> b.eq .LBB0_2           ; single def/use of w8 removed.

 add w8, w0, w1  -> adds w8, w0, w1        ; w8 has multiple uses.
 cbz w8, .LBB1_2 -> b.eq .LBB1_2

 sub w8, w0, w1       -> subs w8, w0, w1   ; w8 has multiple uses.
 tbz w8, #31, .LBB6_2 -> b.ge .LBB6_2

In looking at all current sub-target machine descriptions, this transformation
appears to be either positive or neutral.

Differential Revision: https://reviews.llvm.org/D34220.

llvm-svn: 306144
2017-06-23 19:20:12 +00:00
whitequark 00ede4dcc1 [X86] Fix SP adjustment in stack probes emitted on 32-bit Windows.
Commit r306010 adjusted the condition as follows:

-  if (Is64Bit) {
+  if (!STI.isTargetWin32()) {

The intent was to preserve the behavior on all Windows platforms
but extend the behavior on 64-bit Windows platforms to every
other one. (Before r306010, emitStackProbeCall only ever executed
when emitting code for Windows triples.)

Unfortunately,
  if (Is64Bit && STI.isOSWindows())
is not the same as
  if (!STI.isTargetWin32())
because of the way isTargetWin32() is defined:

  bool isTargetWin32() const {
    return !In64BitMode && (isTargetCygMing() ||
                            isTargetKnownWindowsMSVC());
  }

In practice this broke the JIT tests on 32-bit Windows, which did not
satisfy the new condition:

    LLVM :: ExecutionEngine/MCJIT/2003-01-15-AlignmentTest.ll
    LLVM :: ExecutionEngine/MCJIT/2003-08-15-AllocaAssertion.ll
    LLVM :: ExecutionEngine/MCJIT/2003-08-23-RegisterAllocatePhysReg.ll
    LLVM :: ExecutionEngine/MCJIT/test-loadstore.ll
    LLVM :: ExecutionEngine/OrcMCJIT/2003-01-15-AlignmentTest.ll
    LLVM :: ExecutionEngine/OrcMCJIT/2003-08-15-AllocaAssertion.ll
    LLVM :: ExecutionEngine/OrcMCJIT/2003-08-23-RegisterAllocatePhysReg.ll
    LLVM :: ExecutionEngine/OrcMCJIT/test-loadstore.ll

because %esp was not updated correctly. The failures are only visible
on a MSVC 2017 Debug build, for which we do not have bots.

llvm-svn: 306142
2017-06-23 18:58:10 +00:00
Krzysztof Parzyszek c0a102f505 [Hexagon] Remove call to printAndVerify from HexagonPassConfig
It causes an extra pass of the machine verifier to be added to the pass
manager, and causes test/CodeGen/Generic/llc-start-stop.ll to fail.

llvm-svn: 306140
2017-06-23 18:47:55 +00:00
Sanjay Patel 3de6bad65f [x86] fix value types for SBB transform (PR33560)
I'm not sure yet why this wouldn't fail in the simple case,
but clearly I used the wrong value type with:
https://reviews.llvm.org/rL306040

...and the bug manifests with:
https://bugs.llvm.org/show_bug.cgi?id=33560

llvm-svn: 306139
2017-06-23 18:42:15 +00:00
Krzysztof Parzyszek bb2fcd1921 [Hexagon] Handle decreasing of stack alignment in frame lowering
llvm-svn: 306124
2017-06-23 16:53:59 +00:00
Simon Pilgrim 6e85e92b6c Remove trailing whitespace. NFCI.
llvm-svn: 306121
2017-06-23 16:35:32 +00:00
Tim Northover 4b4eec7009 GlobalISel: remove G_SEQUENCE instruction.
It was trying to do too many things. The basic lumping together of values for
legalization purposes is now handled by G_MERGE_VALUES. More complex things
involving gaps and odd sizes are handled by G_INSERT sequences.

llvm-svn: 306120
2017-06-23 16:15:55 +00:00
Ulrich Weigand eaf0051ba3 [SystemZ] Remove unnecessary serialization before volatile loads
This reverts the use of TargetLowering::prepareVolatileOrAtomicLoad
introduced by r196905.  Nothing in the semantics of the "volatile"
keyword or the definition of the z/Architecture actually requires
that volatile loads are preceded by a serialization operation, and
no other compiler on the platform actually implements this.

Since we've now seen a use case where this additional serialization
causes noticable performance degradation, this patch removes it.

The patch still leaves in the serialization before atomic loads,
which is now implemented directly in lowerATOMIC_LOAD.  (This also
seems overkill, but that can be addressed separately.)

llvm-svn: 306117
2017-06-23 15:56:14 +00:00
Tom Stellard af552dc352 AMDGPU/GlobalISel: Mark 32-bit G_AND as legal
Reviewers: arsenm

Reviewed By: arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, rovka, kristof.beyls, igorb, dstuttard, tpr, t-tye, llvm-commits

Differential Revision: https://reviews.llvm.org/D34349

llvm-svn: 306112
2017-06-23 15:17:17 +00:00
Jonas Paulsson 82f15a7168 [SystemZ] Fix trap issue and enable expensive checks.
The isBarrier/isTerminator flags have been removed from the SystemZ trap
instructions, so that tests do not fail with EXPENSIVE_CHECKS. This was just
an issue at -O0 and did not affect code output on benchmarks.

(Like Eli pointed out: "targets are split over whether they consider their
"trap" a terminator; x86, AArch64, and NVPTX don't, but ARM, MIPS, PPC, and
SystemZ do. We should probably try to be consistent here.". This is still the
case, although SystemZ has switched sides).

SystemZ now returns true in isMachineVerifierClean() :-)

These Generic tests have been modified so that they can be run with or without
EXPENSIVE_CHECKS: CodeGen/Generic/llc-start-stop.ll and
CodeGen/Generic/print-machineinstrs.ll

Review: Ulrich Weigand, Simon Pilgrim, Eli Friedman
https://bugs.llvm.org/show_bug.cgi?id=33047
https://reviews.llvm.org/D34143

llvm-svn: 306106
2017-06-23 14:30:46 +00:00
Petar Jovanovic 78811c2c07 Revert r306095: [mips] Fix reg positions in the aui/daui instructions
ELF/mips-plt-r6.s in lld-test is failing. Reverting the change.

Original commit message:

  [mips] Fix register positions in the aui/daui instructions

  Swapped the position of the rt and rs register in the aut/daui
  instructions for mips32r6 and mips64r6. With this change, the format of
  the generated instructions complies with specifications and GCC.
  Patch by Milos Stojanovic.

llvm-svn: 306099
2017-06-23 13:33:46 +00:00
Petar Jovanovic d5f7711ebb [mips] Fix register positions in the aui/daui instructions
Swapped the position of the rt and rs register in the aut/daui instructions
for mips32r6 and mips64r6. With this change, the format of the generated
instructions complies with specifications and GCC.

Patch by Milos Stojanovic.

Differential Revision: https://reviews.llvm.org/D33988

llvm-svn: 306095
2017-06-23 12:47:18 +00:00
Stefan Maksimovic b794c0a5ca [mips][msa] Splat.d endianness check
Before this change, it was always the first element of a vector that got splatted since the lower 6 bits of vshf.d $wd were always zero for little endian.
Additionally, masking has been performed for vshf via which splat.d is created.

Vshf has a property where if its first operand's elements have either bit 6 or 7 set, destination element is set to zero.
Initially masked with 63 to avoid this property, which would result in generation of and.v + vshf.d in all cases.
Masking with one results in generating a single splati.d instruction when possible.

Differential Revision: https://reviews.llvm.org/D32216

llvm-svn: 306090
2017-06-23 09:09:31 +00:00
Rafael Espindola 58173b9720 COFF: Produce an error on invalid pcrel relocs.
X86_64 COFF only has support for 32 bit pcrel relocations. Produce an
error on all others.

Note that gnu as has extended the relocation values to support
this. It is not clear if we should support the gnu extension.

llvm-svn: 306082
2017-06-23 04:07:44 +00:00
Farhana Aleen 9bd593e0d7 Fixed a (product) build error that was due to an unused variable
Details: There was a use but it was in the assert which was not
         exercised during product build.

Reviewers: Andrew Kaylor

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32658

llvm-svn: 306073
2017-06-22 23:56:31 +00:00
Sanjay Patel 359ae44fb4 [x86] add/sub (X==0) --> sbb(cmp X, 1)
This is very similar to the transform in:
https://reviews.llvm.org/rL306040
...but in this case, we use cmp X, 1 to set the carry bit as needed.

Again, we can show that all of these are logically equivalent (although
InstCombine currently canonicalizes to a form not seen here), and if
we believe IACA, then this is the smallest/fastest code. Eg, with SNB:

| Num Of |              Ports pressure in cycles               |    |
|  Uops  |  0  - DV  |  1  |  2  -  D  |  3  -  D  |  4  |  5  |    |
---------------------------------------------------------------------
|   1    | 1.0       |     |           |           |     |     |    | cmp edi, 0x1
|   2    |           | 1.0 |           |           |     | 1.0 | CP | sbb eax, eax


The larger motivation is to clean up all select-of-constants combining/lowering 
because we're missing some common cases.

llvm-svn: 306072
2017-06-22 23:47:15 +00:00
Farhana Aleen 4b652a5335 Supported lowerInterleavedStore() in X86InterleavedAccess.
Reviewers: RKSimon, DavidKreitzer

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D32658

llvm-svn: 306068
2017-06-22 22:59:04 +00:00
Jacob Gravelle a31ec61c46 [WebAssembly] WebAssemblyFastISel getelementptr variable index support
Summary:
Previously -fast-isel getelementptr would constant-fold non-constant i8
load/stores.

Reviewers: sunfish

Subscribers: jfb, dschuff, sbc100, llvm-commits

Differential Revision: https://reviews.llvm.org/D34044

llvm-svn: 306060
2017-06-22 21:26:08 +00:00
Krzysztof Parzyszek 9b7c1d2dcf [Hexagon] Properly update kill flags in HexagonNewValueJump
The feeder instruction will be moved to right before the compare, so
the updating code should not be looking for kills past the compare.

llvm-svn: 306059
2017-06-22 21:11:44 +00:00
Krzysztof Parzyszek 1a0da8d5a3 [Hexagon] Use LivePhysRegs to fix up kills in HexagonGenMux
Remove the previous, manual shuffling of the kill flags. 

llvm-svn: 306054
2017-06-22 20:43:02 +00:00
Craig Topper 792fc92be2 [AVX-512] Remove and autoupgrade the masked integer compare intrinsics
Summary:
These intrinsics aren't used by clang and haven't been for a while.

There's some really terrible codegen in the 32-bit target for avx512bw due to i64 not being legal. But as I said these intrinsics aren't used by clang even before this patch so this codegen reflects our clang behavior today.

Reviewers: spatel, RKSimon, zvi, igorb

Reviewed By: RKSimon

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D34389

llvm-svn: 306047
2017-06-22 20:11:01 +00:00
Sanjay Patel 41a34e4111 [x86] add/sub (X==0) --> sbb(neg X)
Our handling of select-of-constants is lumpy in IR (https://reviews.llvm.org/D24480),
lumpy in DAGCombiner, and lumpy in X86ISelLowering. That's why we only had the 'sbb'
codegen in 1 out of the 4 tests. This is a step towards smoothing that out.

First, show that all of these IR forms are equivalent:
http://rise4fun.com/Alive/mx

Second, show that the 'sbb' version is faster/smaller. IACA output for SandyBridge
(later Intel and AMD chips are similar based on Agner's tables):

This is the "obvious" x86 codegen (what gcc appears to produce currently):

| Num Of |              Ports pressure in cycles               |    |
|  Uops  |  0  - DV  |  1  |  2  -  D  |  3  -  D  |  4  |  5  |    |
---------------------------------------------------------------------
|   1*   |           |     |           |           |     |     |    | xor eax, eax
|   1    | 1.0       |     |           |           |     |     | CP | test edi, edi
|   1    |           |     |           |           |     | 1.0 | CP | setnz al
|   1    |           | 1.0 |           |           |     |     | CP | neg eax


This is the adc version:
|   1*   |           |     |           |           |     |     |    | xor eax, eax
|   1    | 1.0       |     |           |           |     |     | CP | cmp edi, 0x1
|   2    |           | 1.0 |           |           |     | 1.0 | CP | adc eax, 0xffffffff


And this is sbb:
|   1    | 1.0       |     |           |           |     |     |    | neg edi
|   2    |           | 1.0 |           |           |     | 1.0 | CP | sbb eax, eax

If IACA is trustworthy, then sbb became a single uop in Broadwell, so this will be
clearly better than the alternatives going forward.

llvm-svn: 306040
2017-06-22 18:11:19 +00:00
Rafael Espindola 8a261c2565 Add a common error checking for some invalid expressions.
This refactors a bit of duplicated code and fixes an assertion failure
on ELF.

llvm-svn: 306035
2017-06-22 17:25:35 +00:00
David Stuttard f677966e2e [AMDGPU] Add intrinsics for tbuffer load and store - build error fix
Variable was unused in non-debug build (used in assert) causing compile time
warning and eventual build failure

llvm-svn: 306034
2017-06-22 17:15:49 +00:00
David Stuttard 70e8bc1bf3 [AMDGPU] Add intrinsics for tbuffer load and store
Intrinsic already existed for llvm.SI.tbuffer.store

Needed tbuffer.load and also re-implementing the intrinsic as llvm.amdgcn.tbuffer.*

Added CodeGen tests for the 2 new variants added.
Left the original llvm.SI.tbuffer.store implementation to avoid issues with existing code

Subscribers: arsenm, kzhuravl, wdng, nhaehnle, yaxunl, tony-tye, tpr

Differential Revision: https://reviews.llvm.org/D30687

llvm-svn: 306031
2017-06-22 16:29:22 +00:00
Krzysztof Parzyszek f63ad39e7d [Hexagon] Handle a global operand to A2_addi when creating duplexes
llvm-svn: 306012
2017-06-22 15:53:31 +00:00
whitequark cebe8241ca [X86] Add support for "probe-stack" attribute
This commit adds prologue code emission for stack probe function
calls.

Reviewed By: majnemer

Differential Revision: https://reviews.llvm.org/D34387

llvm-svn: 306010
2017-06-22 15:42:53 +00:00
Florian Hahn 5991b5be74 [ARM] Create relocations for beq.w branches to ARM function syms.
Summary:
The ARM ELF ABI requires the linker to do interworking for wide
conditional branches from Thumb code to ARM code. 

That was pointed out by @peter.smith in the comments for D33436.

Reviewers: rafael, peter.smith, echristo

Reviewed By: peter.smith

Subscribers: aemerson, javed.absar, kristof.beyls, llvm-commits, peter.smith

Differential Revision: https://reviews.llvm.org/D34447

llvm-svn: 306009
2017-06-22 15:32:41 +00:00
Petar Jovanovic 636851b845 [mips] Allow $AT to be used as a register name
This patch allows $AT to be used as a register name in assembly files.
Currently only $at is recognized as a valid register name.

Patch by Stanislav Ocovaj.

Differential Revision: https://reviews.llvm.org/D34348

llvm-svn: 306007
2017-06-22 15:24:16 +00:00
Krzysztof Parzyszek 69ffba4595 [Hexagon] Recognize potential offset overflow for store-imm to stack
Reserve an extra scavenging stack slot if the offset field in store-
-immediate instructions may overflow.

llvm-svn: 306004
2017-06-22 14:11:23 +00:00
Sam Kolton ca5a30ed74 [AMDGPU] SDWA: remove support for VOP2 instructions that have only 64-bit encoding
Summary:
Despite that this instructions are listed in VOP2, they are treated as VOP3 in specs. They should not support SDWA.
There are no real instructions for them, but there are pseudo instructions.

Reviewers: arsenm, vpykhtin, cfang

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye

Differential Revision: https://reviews.llvm.org/D34403

llvm-svn: 305999
2017-06-22 12:42:14 +00:00
Kristof Beyls 9665249fd8 Don't conditionalize Neon instructions, even in IT blocks.
This has been deprecated since ARMARM v7-AR, release C.b, published back
in 2012.

This also removes test/CodeGen/Thumb2/ifcvt-neon.ll that originally was
introduced to check that conditionalization of Neon instructions did
happen when generating Thumb2. However, the test had evolved and was no
longer testing that. Rather than trying to adapt that test, this commit
introduces test/CodeGen/Thumb2/ifcvt-neon-deprecated.mir, since we can
now use the MIR framework to write nicer/more maintainable tests.

llvm-svn: 305998
2017-06-22 12:11:38 +00:00
Simon Dardis 1c73fcc131 [mips] Implement the ".rdata" MIPS assembly directive.
Rather than creating a separate ".rdata" section distinct from the
customary ".rodata" in ELF, ".rdata" switches to the ".rodata" section.

This patch relands r305949 and r305950 with the correct commit message
and addresses nit raised during review.

Patch By: John Baldwin!

Differential Revision: https://reviews.llvm.org/D34452

llvm-svn: 305995
2017-06-22 10:41:51 +00:00
John Brawn ed78aaf093 [ARM] Add .w aliases of MOV with shifted operand
These appear to have been simply missing.

Differential Revision: https://reviews.llvm.org/D34461

llvm-svn: 305993
2017-06-22 10:30:53 +00:00
John Brawn 192f74a84d [ARM] Clean up choice of narrow instructions in ARMAsmParser, NFC
This patch makes a couple of changes to how we decide whether to use the narrow
or wide encoding of thumb2 instructions:
 * Common out the detection of the .w qualifier
 * Check for the CPSR operand in a consistent way

Differential Revision: https://reviews.llvm.org/D34460

llvm-svn: 305992
2017-06-22 10:29:31 +00:00
Igor Breger 1c29be7e4f [GlobalISel][X86] Support vector type G_INSERT legalization/selection.
Summary:
Support vector type G_INSERT legalization/selection.
Split from https://reviews.llvm.org/D33665

Reviewers: qcolombet, t.p.northover, zvi, guyblank

Reviewed By: guyblank

Subscribers: guyblank, rovka, llvm-commits, kristof.beyls

Differential Revision: https://reviews.llvm.org/D33956

llvm-svn: 305989
2017-06-22 09:43:35 +00:00
Florian Hahn b489e56ae2 [ARM] Add macro fusion for AES instructions.
Summary:
This patch adds a macro fusion using CodeGen/MacroFusion.cpp to pair AES
instructions back to back and adds FeatureFuseAES to enable the feature.

Reviewers: evandro, javed.absar, rengolin, t.p.northover

Reviewed By: javed.absar

Subscribers: aemerson, mgorny, kristof.beyls, llvm-commits

Differential Revision: https://reviews.llvm.org/D34142

llvm-svn: 305988
2017-06-22 09:39:36 +00:00
Elena Demikhovsky 2dac0b4d58 AVX-512: Lowering Masked Gather intrinsic - fixed a bug
Masked gather for vector length 2 is lowered incorrectly for element type i32.
The type <2 x i32> was automatically extended to <2 x i64> and we generated VPGATHERQQ instead of VPGATHERQD.
The type <2 x float> is extended to <4 x float>, so there is no bug for this type, but the sequence may be more optimal.

In this patch I'm fixing <2 x i32>bug and optimizing <2 x float> sequence for GATHERs only. The same fix should be done for Scatters as well.

Differential revision: https://reviews.llvm.org/D34343

llvm-svn: 305987
2017-06-22 06:47:41 +00:00
Sam Kolton 3c4933fcc6 [AMDGPU] SDWA: add support for GFX9 in peephole pass
Summary:
Added support based on merged SDWA pseudo instructions. Now peephole allow one scalar operand, omod and clamp modifiers.
Added several subtarget features for GFX9 SDWA.
This diff also contains changes from D34026.
Depends D34026

Reviewers: vpykhtin, rampitec, arsenm

Subscribers: kzhuravl, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye

Differential Revision: https://reviews.llvm.org/D34241

llvm-svn: 305986
2017-06-22 06:26:41 +00:00
Hiroshi Inoue 1d5693c915 [PowerPC] fix potential verification errors
This patch fixes trivial mishandling of 32-bit/64-bit instructions that may cause verification errors with -verify-machineinstrs.

llvm-svn: 305984
2017-06-22 04:33:44 +00:00
Reid Kleckner ef5817579b [wasm] Fix WebAssembly asm backend after r305968
llvm-svn: 305978
2017-06-22 01:07:05 +00:00
Davide Italiano 7a6c5c12ad Revert "[Target] Implement the ".rdata" MIPS assembly directive."
This reverts commit r305949 and r305950 as they didn't have the
correct commit message.

llvm-svn: 305973
2017-06-22 00:11:41 +00:00
Stanislav Mekhanoshin 3ed38c601a [AMDGPU] Add FP_CLASS to the add/setcc combine
This is one of the nodes which also compile as v_cmp_*.

Differential Revision: https://reviews.llvm.org/D34485

llvm-svn: 305970
2017-06-21 23:46:22 +00:00
Rafael Espindola 88d9e37ec8 Use a MutableArrayRef. NFC.
llvm-svn: 305968
2017-06-21 23:06:53 +00:00
Rafael Espindola 6da25f4fc4 Fix build.
llvm-svn: 305967
2017-06-21 23:02:57 +00:00
Stanislav Mekhanoshin a8b26936d0 [AMDGPU] Combine add and adde, sub and sube
If one of the arguments of adde/sube is zero we can fold another
add/sub into it.

Differential Revision: https://reviews.llvm.org/D34374

llvm-svn: 305964
2017-06-21 22:30:01 +00:00
Sam Clegg 705f798bff Mark dump() methods as const. NFC
Add const qualifier to any dump() method where adding one
was trivial.

Differential Revision: https://reviews.llvm.org/D34481

llvm-svn: 305963
2017-06-21 22:19:17 +00:00
Stanislav Mekhanoshin e3eb42cef6 [AMDGPU] simplify add x, *ext (setcc) => addc|subb x, 0, setcc
This simplification allows to avoid generating v_cndmask_b32
to serialize condition code between compare and use.

Differential Revision: https://reviews.llvm.org/D34300

llvm-svn: 305962
2017-06-21 22:05:06 +00:00
Krzysztof Parzyszek 5b933fee3c [Hexagon] Use MachineInstrBuilder instead of changing instruction in place
llvm-svn: 305953
2017-06-21 21:03:34 +00:00
Davide Italiano 75ed943def [Target] Implement the ".rdata" MIPS assembly directive.
Patch by John Baldwin < jhb at freebsd dot org >!

Differential Revision:  https://reviews.llvm.org/D34452

llvm-svn: 305949
2017-06-21 20:40:27 +00:00