Commit Graph

77761 Commits

Author SHA1 Message Date
Ahmed Bougacha fab5892f8b [AArch64] Avoid going through GPRs for across-vector instructions.
This adds new node types for each intrinsic.
For instance, for addv, we have AArch64ISD::UADDV, such that:
  (v4i32 (uaddv ...))
is the same as
  (v4i32 (scalar_to_vector (i32 (int_aarch64_neon_uaddv ...))))
that is,
  (v4i32 (INSERT_SUBREG (v4i32 (IMPLICIT_DEF)),
           (i32 (int_aarch64_neon_uaddv ...)), ssub)

In a combine, we transform all such across-vector-lanes intrinsics to:

  (i32 (extract_vector_elt (uaddv ...), 0))

This has one big advantage: by making the extract_element explicit, we
enable the existing patterns for lane-aware instructions to fire.
This lets us avoid needlessly going through the GPRs.  Consider:

    uint32x4_t test_mul(uint32x4_t a, uint32x4_t b) {
        return vmulq_n_u32(a, vaddvq_u32(b));
    }

We now generate:
    addv.4s  s1, v1
    mul.4s   v0, v0, v1[0]
instead of the previous:
    addv.4s  s1, v1
    fmov     w8, s1
    dup.4s   v1, w8
    mul.4s   v0, v1, v0

rdar://20044838

llvm-svn: 231840
2015-03-10 20:45:38 +00:00
Ahmed Bougacha 8f6a115de9 [AArch64] Remove integer INSvi*lane patterns. NFCI.
Most are redundant, and they never seem to fire.

The V128 integer patterns already exist in the INS multiclass.
The duplicates only fire when the vector index type isn't i64,
because they accept "imm" instead of an explicit "i64", as the
instruction definition patterns do.

TLI::getVectorIdxTy is i64 on AArch64, so this should never happen.
Also, one of them had a typo: for i64, INSvi32lane was used.
I noticed because I mistakenly used an explicit i32 as the idx type,
and got ins.s for an i64 vector_insert.

The V64 patterns also don't seem to ever fire, as V64 vector
extract/insert are legalized to V128.

The equivalent float patterns are unique and useful, so keep them.

No functional change intended;  none exhibited on the LIT and LNT tests.

llvm-svn: 231838
2015-03-10 20:37:19 +00:00
Chad Rosier 99fb8d17ec Don't evaluate rend() on every iteration of the loop.
llvm-svn: 231837
2015-03-10 20:29:59 +00:00
David Majnemer d388e930ce LoopAccessAnalysis: Silence -Wreturn-type diagnostic from GCC
llvm-svn: 231836
2015-03-10 20:23:29 +00:00
Benjamin Kramer d58792f38b Don't use LLVM_LIBRARY_VISIBILITY in cpp files.
llvm-svn: 231831
2015-03-10 20:07:44 +00:00
Bruno Cardoso Lopes b3a58b4c3c [AsmPrinter][TLOF] Reintroduce AArch64 test
Follow up from r231505.

Fix the non-determinism by using a MapVector and reintroduce the AArch64
testcase. Defer deleting the got candidates up to the end and remove
them in a bulk, avoiding linear time removal of each element.

Thanks to Renato Golin for trying it out on other platforms.

llvm-svn: 231830
2015-03-10 20:05:23 +00:00
Colin LeMahieu 60a99e66a0 [Hexagon] Adding nodes for PIC support.
llvm-svn: 231829
2015-03-10 20:04:44 +00:00
Colin LeMahieu 092d9c18a8 [Hexagon] Adding DuplexInst instruction format and duplex class defs.
llvm-svn: 231828
2015-03-10 19:53:14 +00:00
Kit Barton 20d3981e15 Change the generation of the vmuluwm instruction to be based on the MUL opcode.
Phabricator review: http://reviews.llvm.org/D8185

llvm-svn: 231827
2015-03-10 19:49:38 +00:00
Sanjay Patel 0fdb437b25 remove function names from comments; NFC
llvm-svn: 231826
2015-03-10 19:42:57 +00:00
Colin LeMahieu 3b6747df13 [Hexagon] Adding nodes for vector insert/extract lowering.
llvm-svn: 231825
2015-03-10 19:40:03 +00:00
Colin LeMahieu ee776452f9 [Hexagon] Renaming HexagonJT to JT and adding CP for constantpool.
llvm-svn: 231824
2015-03-10 19:29:53 +00:00
Adrian Prantl 51233680ae Change the datatype of DwarfExpression::Emit(Un)Signed to (u)int64_t
so it matches the one used by ByteStreamer::Emit(U|S)LEB128.

llvm-svn: 231823
2015-03-10 19:23:37 +00:00
Benjamin Kramer 414c0964c3 NVPTX: move NVPTXAllocaHoisting into the cpp file
Also initialize without using static initialization.

llvm-svn: 231822
2015-03-10 19:20:52 +00:00
Adam Nemet 949e91a6fa [LAA-memchecks] Comment improvement
I forgot to roll this into r231816.  It was requested by Hal in D8122.

llvm-svn: 231821
2015-03-10 19:12:41 +00:00
Michael Zolotukhin 267e12f714 Enable loop-rotate before loop-vectorize by default
llvm-svn: 231820
2015-03-10 19:07:41 +00:00
Adam Nemet ec1e2bb6a4 [LAA-memchecks 3/3] Introduce pointer partitions for memchecks
This is the final patch that actually introduces the new parameter of
partition mapping to RuntimePointerCheck::needsChecking.

Another API (LAI::getInstructionsForAccess) is also exposed that helps
to map pointers to instructions because ultimately we partition
instructions.

The WIP version of the Loop Distribution pass in D6930 has been adapted
to use all this.  See for example, how
InstrPartitionContainer::computePartitionSetForPointers sets up the
partitions using the above API and then calls to LAI::addRuntimeCheck
with the pointer partitions.

llvm-svn: 231818
2015-03-10 18:54:26 +00:00
Adam Nemet 98c4c5dd78 [LAA-memchecks 2/3] Move number of memcheck threshold checking to LV
Now the analysis won't "fail" if the memchecks exceed the threshold.  It
is the transform pass' responsibility to perform the check.

This allows the transform pass to further analyze/eliminate the
memchecks.  E.g. in Loop distribution we only need to check pointers
that end up in different partitions.

Note that there is a slight change of functionality here.  The logic in
analyzeLoop is that if dependence checking fails due to non-constant
distance between the pointers, another attempt is made to prove safety
of the dependences purely using run-time checks.

Before this patch we could fail the loop due to exceeding the memcheck
threshold after the first step, now we only check the threshold in the
client after the full analysis.  There is no measurable compile-time
effect but I wanted to record this here.

llvm-svn: 231817
2015-03-10 18:54:23 +00:00
Adam Nemet b6dc76ffe5 [LAA-memchecks 1/3] Split out NumComparisons checks. NFC
The check for the number of memchecks will be moved to the client of
this analysis.  Besides allowing for transform-specific thresholds, this
also lets Loop Distribution post-process the memchecks; Loop
Distribution only needs memchecks between pointers of different
partitions.

The motivation for this first patch is to untangle the CanDoRT check
from the NumComparison check before moving the NumComparison part.
CanDoRT means that we couldn't determine the bounds for the pointer.
Note that NumComparison is set independent of this flag.

llvm-svn: 231816
2015-03-10 18:54:19 +00:00
Sanjay Patel abf7023c63 remove names from comments; NFC
llvm-svn: 231813
2015-03-10 18:41:22 +00:00
Sanjay Patel 51bd9421ac fix typos; NFC
llvm-svn: 231812
2015-03-10 18:37:05 +00:00
Benjamin Kramer 42a31e1f5e NVPTX: Remove copy of LLVMInitializeNVPTXAsmPrinter.
If anyone is using this for some strange reason,
LLVMInitializeNVPTXAsmPrinter does exactly the same thing and is what
other LLVM tools are calling.

llvm-svn: 231810
2015-03-10 18:19:24 +00:00
Benjamin Kramer 9d1f2dfec7 Hexagon: Remove unused InstrMapping.
llvm-svn: 231809
2015-03-10 18:19:16 +00:00
Adam Nemet 58913d65ad [LoopAccesses 3/3] Print the dependences with -analyze
The dependences are now expose through the new getInterestingDependences
API so we can use that with -analyze too and fix the FIXME.

This lets us remove the test that relied on -debug to check the
dependences.

llvm-svn: 231807
2015-03-10 17:40:43 +00:00
Adam Nemet 9c92657971 [LoopAccesses 2/3] Allow querying of interesting dependences
Gather an array of interesting dependences rather than just failing
after the first unsafe one and regarding the loop unsafe.  Loop
Distribution needs to be able to collect all dependences in order to
isolate the dependence cycles into their own partition.

Since the dependence checking algorithm is quadratic in terms of
accesses sharing the same underlying pointer, I am applying a cut-off
threshold (MaxInterestingDependence).  Exceeding that, the logic reverts
back to the original approach deeming the loop unsafe upon encountering
the first unsafe dependence.

The main idea of the patch is to split isDepedent from directly
answering the question whether the dep is safe for vectorization to
return a dependence type which then gets mapped to old boolean result
using Dependence::isSafeForVectorization.

Tested that this was compile-time neutral on SpecINT2006 LTO bitcode
inputs.  No assembly change on the testsuite including external.

llvm-svn: 231806
2015-03-10 17:40:37 +00:00
Adam Nemet dee666bc63 [LoopAccesses 1/3] Expose MemoryDepChecker to LAA users
LoopDistribution needs to query various results of the dependence
analysis.  This series will expose some more APIs and state of the
dependence checker.

This patch is a simple one to just expose the DepChecker instance.  The
set is compile-time neutral measured with LTO bitcode files of
SpecINT2006.  Also there is no assembly change on the testsuite.

llvm-svn: 231805
2015-03-10 17:40:34 +00:00
Rafael Espindola 063d725fd7 Store an optional section start label in MCSection.
This makes code that uses section relative expressions (debug info) simpler and
less brittle.

This is still a bit awkward as the symbol is created late and has to be
stored in a mutable field.

I will move the symbol creation earlier in the next patch.

llvm-svn: 231802
2015-03-10 16:58:10 +00:00
Sanjay Patel f1b0db1545 remove function names from comments; NFC
llvm-svn: 231801
2015-03-10 16:42:24 +00:00
Igor Laevsky 85f7f727d3 Teach lowering to correctly handle invoke statepoint and gc results tied to them. Note that we still can not lower gc.relocates for invoke statepoints.
Also it extracts getCopyFromRegs helper function in SelectionDAGBuilder as we need to be able to customize type of the register exported from basic block during lowering of the gc.result.
(Resubmitting this change after not being able to reproduce buildbot failure)

Differential Revision: http://reviews.llvm.org/D7760

llvm-svn: 231800
2015-03-10 16:26:48 +00:00
Chad Rosier 3b67c8d0f7 [BranchFolding] Remove MMOs during tail merge to preserve dependencies.
When tail merging it may be necessary to remove MMOs from memory operations to
ensures later passes (e.g., MI sched) conservatively compute dependencies.
Currently, we only remove the MMO from the common tail if the MMO doesn't match
with the relative instruction in the non-common tail(s).

A more robust solution would be to add multiple MMOs from the duplicate MIs to
the new MI. Currently ScheduleDAGInstrs.cpp ignores all MMOs on instructions
with multiple MMOs, so this solution is equivalent for the time being.

No test case included as this is incredibly difficult to reproduce.

Patch was a collaborative effort between Ana Pazos and myself.
Phabricator: http://reviews.llvm.org/D7769

llvm-svn: 231799
2015-03-10 16:22:52 +00:00
Tom Stellard a14b0114a3 R600/SI: Add _IDXEN and _BOTHEN variants for buffer_store
llvm-svn: 231798
2015-03-10 16:16:51 +00:00
Tom Stellard c229baa94c R600/SI: Re-order MUBUF operands to match asm strings.
llvm-svn: 231797
2015-03-10 16:16:49 +00:00
Tom Stellard 4d6c99ddf2 R600/SI: Move kill flag to second instruction when splitting SMRD
This fixes a machine verifier error in the salu-to-valu.ll, which
would have been exposed by a future commit.

llvm-svn: 231796
2015-03-10 16:16:48 +00:00
Tom Stellard 5224df36c8 R600/SI: Add 32-bit encoding of v_cndmask_b32
This was done by refactoring the v_cndmask_b32 tablegen definition
to use inherit from VOP2Inst.

llvm-svn: 231795
2015-03-10 16:16:44 +00:00
Sanjay Patel 19792fb270 [X86, AVX] replace vinsertf128 intrinsics with generic shuffles
We want to replace as much custom x86 shuffling via intrinsics
as possible because pushing the code down the generic shuffle
optimization path allows for better codegen and less complexity
in LLVM.

This is the sibling patch for the Clang half of this change:
http://reviews.llvm.org/D8088

Differential Revision: http://reviews.llvm.org/D8086

llvm-svn: 231794
2015-03-10 16:08:36 +00:00
Benjamin Kramer 4683395808 Hexagon: Remove pass that does nothing at all
llvm-svn: 231791
2015-03-10 15:06:38 +00:00
Rafael Espindola 3c066d1da3 Remove effectively dead code.
Switching back and forth between sections does nothing (other than producing
larger .s files).

llvm-svn: 231790
2015-03-10 14:48:01 +00:00
Karthik Bhat 8d7f7eda14 Fix a memory corruption in Dependency Analysis.
This crash occurs due to memory corruption when trying to update dependency
direction based on Constraints.

This crash was observed during lnt regression of Polybench benchmark test case dynprog.

Review: http://reviews.llvm.org/D8059
llvm-svn: 231788
2015-03-10 14:32:02 +00:00
Aaron Ballman f00fd1f087 Removing dead code to silence warning C4060: switch statement contains no 'case' or 'default' labels; NFC.
llvm-svn: 231785
2015-03-10 13:56:28 +00:00
Karthik Bhat 8d0099bdab Fix a crash in Dependency Analysis.
This crash in Dependency analysis is because we assume here that in case of UsefulGEP
both source and destination have the same number of operands which may not be true.
This incorrect assumption results in crash while populating Pairs. Fix the same.

This crash was observed during lnt regression for code such as-
  struct s{
    int A[10][10];
    int C[10][10][10]; 
  } S;
  void dep_constraint_crash_test(int k,int N)  {
     for( int i=0;i<N;i++)
       for( int j=0;j<N;j++)
         S.A[0][0] = S.C[0][0][k];
  }
Review: http://reviews.llvm.org/D8162

llvm-svn: 231784
2015-03-10 13:31:03 +00:00
Daniel Sanders 2db94ba0bc The operand flag word used in ISD::INLINEASM is an i32 not a pointer. NFC.
Summary:
This is part of the work to support memory constraints that behave
differently to 'm'. The subsequent patches will expand on the existing
encoding (which is a 32-bit int) and as a result in some flag words will no
longer fit into an i16. This problem only affected the MSP430 target which
appears to have 16-bit pointers.

Reviewers: hfinkel

Reviewed By: hfinkel

Subscribers: hfinkel, llvm-commits

Differential Revision: http://reviews.llvm.org/D8168

llvm-svn: 231783
2015-03-10 10:42:59 +00:00
Yaron Keren 09fb7c6e7a Teach raw_ostream to accept SmallString.
Saves adding .str() call to any raw_ostream << SmallString usage
and a small step towards making .str() consistent in the ADTs by
removing one of the SmallString::str() use cases, discussion at

http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20141013/240026.html

I'll update the Phabricator patch http://reviews.llvm.org/D6372
for review of the Twine SmallString support, it's more complex
than this one.

llvm-svn: 231763
2015-03-10 07:33:23 +00:00
Owen Anderson 58364dc4da Fix a crash in InstCombine where we could try to truncate a switch comparison to zero width.
llvm-svn: 231761
2015-03-10 06:51:39 +00:00
Owen Anderson e90f992b21 Fix a stack overflow in the assembler when checking that GEPs must be over sized types.
We failed to use a marking set to properly handle recursive types, which caused use
to recurse infinitely and eventually overflow the stack.

llvm-svn: 231760
2015-03-10 06:34:57 +00:00
Owen Anderson 3e7e67b5ed Fix an issue in the verifier where we could try to read information out of a malformed statepoint intrinsic.
In this situation we would always have already flagged an error on the statepoint intrinsic,
but then we carry on to parse other, related GC intrinsics, and could end up crashing during that
verification when they try to access data from the malformed statepoint.

llvm-svn: 231759
2015-03-10 05:58:21 +00:00
Owen Anderson 51b75b8c34 Fix an infinite loop in InstCombine when an instruction with no users and side effects can be constant folded.
ReplaceInstUsesWith needs to return nullptr when the input has no users,
because in that case it does not mutate the program.  Otherwise, we can
get stuck in an infinite loop of repeatedly attempting to constant fold
and instruction with no users.

llvm-svn: 231755
2015-03-10 05:13:47 +00:00
Rafael Espindola c63a98a662 Move variable into assert to fix -Asserts builds.
llvm-svn: 231753
2015-03-10 04:28:09 +00:00
Rafael Espindola 760bf9520a Remove incredibly confusing isBaseAddressKnownZero.
When referring to a symbol in a dwarf section on ELF we should use

.long foo

instead of

.long foo - .debug_something

because ELF is unaware of the content of the sections and therefore needs
relocations. This has nothing to do with optimizing a -0.

llvm-svn: 231751
2015-03-10 04:11:52 +00:00
Rafael Espindola fcc2821882 Use a better name for compile unit labels.
They mark the start of a compile unit, so name them .Lcu_*. Using
Section->getLabelBeginName() makes it looks like they mark the start of the
section.

While at it, switch to createTempSymbol to avoid collisions with labels
created in inline assembly. Not sure if a "don't crash" test is worth it.

With this getLabelBeginName is dead, delete it.

llvm-svn: 231750
2015-03-10 03:58:36 +00:00
Sanjay Patel d45a3f1a03 removed function names from comments; NFC
llvm-svn: 231749
2015-03-10 03:48:14 +00:00