Commit Graph

1023 Commits

Author SHA1 Message Date
Sanjoy Das a19edc4d15 Fix a bug in SCEV's poison value propagation
The worklist algorithm introduced in rL271151 didn't check to see if the
direct users of the post-inc add recurrence propagates poison.  This
change fixes the problem and makes the code structure more obvious.

Note for release managers: correctness wise, this bug wasn't a
regression introduced by rL271151 -- the behavior of SCEV around
post-inc add recurrences was strictly improved (in terms of correctness)
in rL271151.

llvm-svn: 272179
2016-06-08 17:48:31 +00:00
George Burgess IV a1f9a2daeb [CFLAA] Add AttrEscaped, remove bit twiddling functions.
This patch does a few things:

- Unifies AttrAll and AttrUnknown (since they were used for more or less
  the same purpose anyway).

- Introduces AttrEscaped, an attribute that notes that a value escapes
  our analysis for a given set, but not that an unknown value flows into
  said set.

- Removes functions that take bit indices, since we also had functions
  that took bitsets, and the use of both (with similar names) was
  unclear and bug-prone.

Patch by Jia Chen.

Differential Revision: http://reviews.llvm.org/D21000

llvm-svn: 272040
2016-06-07 18:35:37 +00:00
Andrey Turetskiy 9f02c58670 [LAA] Improve non-wrapping pointer detection by handling loop-invariant case.
This fixes PR26314. This patch adds new helper “isNoWrap” with detection of
loop-invariant pointer case.

Patch by Roman Shirokiy.

Ref: https://llvm.org/bugs/show_bug.cgi?id=26314

Differential Revision: http://reviews.llvm.org/D17268

llvm-svn: 272014
2016-06-07 14:55:27 +00:00
Easwaran Raman 019e0bf592 Reapply r271728 after adding move cobstructor for ProfileSummaryInfo
llvm-svn: 271745
2016-06-03 22:54:26 +00:00
Easwaran Raman 94edaaaefb Revert r271728 as it breaks Windows build
llvm-svn: 271738
2016-06-03 21:14:26 +00:00
Easwaran Raman d142050f3a Analysis pass to access profile summary info
Differential Revision: http://reviews.llvm.org/D20648

llvm-svn: 271728
2016-06-03 20:37:19 +00:00
Daniel Berlin 73694bb92b Revert "Claim NoAlias if two GEPs index different fields of the same struct"
This reverts commit 2d5d6493f43eb68493a3852b8c226ac9fafdc7eb.

llvm-svn: 271422
2016-06-01 18:55:32 +00:00
George Burgess IV 18b83fe6cf [CFLAA] Recognize builtin allocation functions.
This patch extends CFLAA to recognize allocation functions such as
malloc, free, etc, so we can treat them more aggressively.

Patch by Jia Chen.

Differential Revision: http://reviews.llvm.org/D20776

llvm-svn: 271421
2016-06-01 18:39:54 +00:00
Daniel Berlin e846c9dc52 Claim NoAlias if two GEPs index different fields of the same struct
Patch by Taewook Oh

Summary: Patch for Bug 27478. Make BasicAliasAnalysis claims NoAlias if two GEPs index different fields of the same structure.

Reviewers: hfinkel, dberlin

Subscribers: dberlin, mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D20665

llvm-svn: 271415
2016-06-01 18:12:01 +00:00
Sanjoy Das 10df497a1f Reduce dependence on pointee types when deducing dereferenceability
Summary:
Change some of the internal interfaces in Loads.cpp to keep track of the
number of bytes we're trying to prove dereferenceable using an explicit
`Size` parameter.

Before this, the `Size` parameter was implicitly inferred from the
pointee type of the pointer whose dereferenceability we were trying to
prove, causing us to be conservative around bitcasts. This was
unfortunate since bitcast instructions are no-ops and should never
break optimizations.  With an explicit `Size` parameter, we're more
precise (as shown in the test cases), and the code is simpler.

We should eventually move towards a `DerefQuery` struct that groups
together a base pointer, an offset, a size and an alignment; but this
patch is a first step.

Reviewers: apilipenko, dblaikie, hfinkel, reames

Subscribers: mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D20764

llvm-svn: 271406
2016-06-01 16:47:45 +00:00
George Burgess IV a880146925 [CFLAA] Don't link GEP pointers to GEP indices.
Code like the following is considered broken, and doesn't need to be
supported by our AA magicks:

void getFoo(int *P) {
  int *PAlias = (int *)((char *)NULL + (uintptr_t)P);
}

This patch makes CFLAA drop support for code like this.

Patch by Jia Chen.

Differential Revision: http://reviews.llvm.org/D20775

llvm-svn: 271322
2016-05-31 19:55:05 +00:00
Sanjoy Das f49ca52b9d [SCEV] See through op.with.overflow intrinsics (re-apply)
Summary:
This change teaches SCEV to see reduce `(extractvalue
0 (op.with.overflow X Y))` into `op X Y` (with a no-wrap tag if
possible).

This was first checked in at r265912 but reverted in r265950 because it
exposed some issues around how SCEV handled post-inc add recurrences.
Those issues have now been fixed.

Reviewers: atrick, regehr

Subscribers: mcrosier, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D18684

llvm-svn: 271152
2016-05-29 00:34:42 +00:00
Sanjoy Das 7e4a64167d [SCEV] Don't always add no-wrap flags to post-inc add recs
Fixes PR27315.

The post-inc version of an add recurrence needs to "follow the same
rules" as a normal add or subtract expression.  Otherwise we miscompile
programs like

```
int main() {
  int a = 0;
  unsigned a_u = 0;
  volatile long last_value;
  do {
    a_u += 3;
    last_value = (long) ((int) a_u);
    if (will_add_overflow(a, 3)) {
      // Leave, and don't actually do the increment, so no UB.
      printf("last_value = %ld\n", last_value);
      exit(0);
    }
    a += 3;
  } while (a != 46);
  return 0;
}
```

This patch changes SCEV to put no-wrap flags on post-inc add recurrences
only when the poison from a potential overflow will go ahead to cause
undefined behavior.

To avoid regressing performance too much, I've assumed infinite loops
without side effects is undefined behavior to prove poison<->UB
equivalence in more cases.  This isn't ideal, but is not new to LLVM as
a whole, and far better than the situation I'm trying to fix.

llvm-svn: 271151
2016-05-29 00:32:17 +00:00
Sanjoy Das 70c2bbd29c [ValueTracking] ICmp instructions propagate poison
This is a stripped down version of D19211, leaving out the questionable
"branching in poison is UB" bit.

llvm-svn: 271150
2016-05-29 00:31:18 +00:00
Michael Kuperstein ae21491819 [BasicAA] Extend inbound GEP negative offset logic to GlobalVariables
r270777 improved the precision of alloca vs. inbounbds GEP alias queries: if
we have (a) an inbounds GEP and (b) a pointer based on an alloca, and the
beginning of the object the GEP points to would have a negative offset with
respect to the alloca, then the GEP can not alias pointer (b).

This makes the same logic fire when (b) is based on a GlobalVariable instead
of an alloca.

Differential Revision: http://reviews.llvm.org/D20652

llvm-svn: 270893
2016-05-26 19:30:49 +00:00
Michael Kuperstein 82069c44ca [BasicAA] Improve precision of alloca vs. inbounds GEP alias queries
If a we have (a) a GEP and (b) a pointer based on an alloca, and the
beginning of the object the GEP points would have a negative offset with
repsect to the alloca, then the GEP can not alias pointer (b).

For example, consider code like:

struct { int f0, int f1, ...} foo;
...
foo alloca;
foo *random = bar(alloca);
int *f0 = &alloca.f0
int *f1 = &random->f1;

Which is lowered, approximately, to:
%alloca = alloca %struct.foo
%random = call %struct.foo* @random(%struct.foo* %alloca)
%f0 = getelementptr inbounds %struct, %struct.foo* %alloca, i32 0, i32 0
%f1 = getelementptr inbounds %struct, %struct.foo* %random, i32 0, i32 1

Assume %f1 and %f0 alias. Then %f1 would point into the object allocated
by %alloca. Since the %f1 GEP is inbounds, that means %random must also
point into the same object. But since %f0 points to the beginning of %alloca,
the highest %f1 can be is (%alloca + 3). This means %random can not be higher
than (%alloca - 1), and so is not inbounds, a contradiction.

Differential Revision: http://reviews.llvm.org/D20495

llvm-svn: 270777
2016-05-25 22:23:08 +00:00
Oleg Ranevskyy eb4eccae5c [SCEV] No-wrap flags are not propagated when folding "{S,+,X}+T ==> {S+T,+,X}"
Summary:
**Description**

This makes `WidenIV::widenIVUse` (IndVarSimplify.cpp) fail to widen narrow IV uses in some cases. The latter affects IndVarSimplify which may not eliminate narrow IV's when there actually exists such a possibility, thereby producing ineffective code.

When `WidenIV::widenIVUse` gets a NarrowUse such as `{(-2 + %inc.lcssa),+,1}<nsw><%for.body3>`, it first tries to get a wide recurrence for it via the `getWideRecurrence` call.
`getWideRecurrence` returns recurrence like this: `{(sext i32 (-2 + %inc.lcssa) to i64),+,1}<nsw><%for.body3>`.

Then a wide use operation is generated by `cloneIVUser`. The generated wide use is evaluated to `{(-2 + (sext i32 %inc.lcssa to i64))<nsw>,+,1}<nsw><%for.body3>`, which is different from the `getWideRecurrence` result. `cloneIVUser` sees the difference and returns nullptr.

This patch also fixes the broken LLVM tests by adding missing <nsw> entries introduced by the correction.

**Minimal reproducer:**
```
int foo(int a, int b, int c);
int baz();

void bar()
{
   int arr[20];
   int i = 0;

   for (i = 0; i < 4; ++i)
     arr[i] = baz();

   for (; i < 20; ++i)
     arr[i] = foo(arr[i - 4], arr[i - 3], arr[i - 2]);
}
```

**Clang command line:**
```
clang++ -mllvm -debug -S -emit-llvm -O3 --target=aarch64-linux-elf test.cpp -o test.ir
```

**Expected result:**
The ` -mllvm -debug` log shows that all the IV's for the second `for` loop have been eliminated.

Reviewers: sanjoy

Subscribers: atrick, asl, aemerson, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D20058

llvm-svn: 270695
2016-05-25 13:01:33 +00:00
Simon Pilgrim 14000b3cea [CostModel][X86][XOP] Added XOP costmodel for BITREVERSE
Now that we have a nice fast VPPERM solution. Added framework for future intrinsic costs as well.

llvm-svn: 270537
2016-05-24 08:17:50 +00:00
Matthew Simpson 6feebe9847 [LAA] Check independence of strided accesses before forward case
This patch changes the order in which we attempt to prove the independence of
strided accesses. We previously did this after we knew the dependence distance
was positive. With this change, we check for independence before handling the
negative distance case. The patch prevents LAA from reporting forward
dependences for independent strided accesses.

This change was requested in the review of D19984.

llvm-svn: 270072
2016-05-19 15:37:19 +00:00
Sanjoy Das f5d40d5350 [SCEV] Be more aggressive in proving NUW
... for AddRec's in loops for which SCEV is unable to compute a max
tripcount.  This is the NUW variant of r269211 and fixes PR27691.

(Note: PR27691 is not a correct or stability bug, it was created to
track a pending task).

llvm-svn: 269790
2016-05-17 17:51:14 +00:00
Simon Pilgrim 2ea513847c [CostModel][X86] Tidied up checks
llvm-svn: 269770
2016-05-17 14:43:41 +00:00
Simon Pilgrim 6e9898f362 [CostModel][X86] Added scalar bitreverse tests
llvm-svn: 269594
2016-05-15 17:40:48 +00:00
Adam Nemet c62e554e9a [LAA] Include MaxSafeDepDistBytes in the analysis print-out
llvm-svn: 269508
2016-05-13 22:49:13 +00:00
Sanjoy Das 4e8c80382f [SCEVExpander] Fix a failed cast<> assertion
SCEVExpander::replaceCongruentIVs assumes the backedge value of an
SCEV-analysable PHI to always be an instruction, when this is not
necessarily true.  For now address this by bailing out of the
optimization if the backedge value of the PHI is a non-Instruction.

llvm-svn: 269213
2016-05-11 17:41:41 +00:00
Sanjoy Das abb7b93eb9 [SCEVExpander] Don't break SSA in replaceCongruentIVs
`SCEVExpander::replaceCongruentIVs` bypasses `hoistIVInc` if both the
original and the isomorphic increments are PHI nodes.  Doing this can
break SSA if the isomorphic increment is not dominated by the original
increment.  Get rid of the bypass, and let `hoistIVInc` do the right
thing.

Fixes PR27232 (compile time crash/hang).

llvm-svn: 269212
2016-05-11 17:41:34 +00:00
Sanjoy Das 787c2460c2 [SCEV] Be more aggressive around proving no-wrap
... for AddRec's in loops for which SCEV is unable to compute a max
tripcount.  This is not a problem for "normal" loops[0] that don't have
guards or assumes, but helps in cases where we have guards or assumes in
the loop that can be used to constrain incoming values over the backedge.

This partially fixes PR27691 (we still don't handle the NUW case).

[0]: for "normal" loops, in the cases where we'd be able to prove
no-wrap via isKnownPredicate, we'd also be able to compute a max
tripcount.

llvm-svn: 269211
2016-05-11 17:41:26 +00:00
Vedant Kumar ee20294af5 [BasicAA] Compare GEP indices based on value (Fix PR27418)
Equivalent GEP indices with different types are treated as different
indices altogether, leading to an incorrect AA result. Fix the issue
by comparing indices based on their values.

Thanks to Mikael Holmén for reporting the issue!

Differential Revision: http://reviews.llvm.org/D19935

llvm-svn: 269197
2016-05-11 15:45:43 +00:00
Sanjoy Das d47f42435a [BasicAA] Guard intrinsics don't write to memory
Summary:
The idea is very close to what we do for assume intrinsics: we mark the
guard intrinsics as writing to arbitrary memory to maintain control
dependence, but under the covers we teach AA that they do not mod any
particular memory location.

Reviewers: chandlerc, hfinkel, gbiv, reames

Subscribers: george.burgess.iv, mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D19575

llvm-svn: 269007
2016-05-10 02:35:41 +00:00
Sanjoy Das 2512d0c837 [SCEV] Use guards to prove predicates
We can use calls to @llvm.experimental.guard to prove predicates,
relying on the fact that in all locations domianted by a call to
@llvm.experimental.guard the predicate it is guarding is known to be
true.

llvm-svn: 268997
2016-05-10 00:31:49 +00:00
Simon Pilgrim eec3a95f95 [X86][SSE] Improve cost model for i64 vector comparisons on pre-SSE42 targets
As discussed on PR24888, until SSE42 we don't have access to PCMPGTQ for v2i64 comparisons, but the cost models don't reflect this, resulting in over-optimistic vectorizaton.

This patch adds SSE2 'base level' costs that match what a typical target is capable of and only reduces the v2i64 costs at SSE42.

Technically SSE41 provides a PCMPEQQ v2i64 equality test, but as getCmpSelInstrCost doesn't give us a way to discriminate between comparison test types we can't easily make use of this, otherwise we could split the cost of integer equality and greater-than tests to give better costings of each.

Differential Revision: http://reviews.llvm.org/D20057

llvm-svn: 268972
2016-05-09 21:14:38 +00:00
Matt Arsenault 1af53a91c0 DivergenceAnalysis: Fix crash with no return blocks
The post dominator tree does not have a root node in this case.

llvm-svn: 268933
2016-05-09 16:57:08 +00:00
Simon Pilgrim 4a9d32c5ba [CostModel][X86] Extended comparison instruction cost model tests to include SSE2/SSE3/SSSE3/SSE41/SSE42 targets
llvm-svn: 268877
2016-05-08 15:24:53 +00:00
Simon Pilgrim 420852e8d4 [CostModel][X86] Split BSWAP/BITREVERSE cost tests from CTPOP/CTLZ/CTTZ 'bit count' cost tests
llvm-svn: 268859
2016-05-07 16:34:16 +00:00
Simon Pilgrim b3f5cb7a65 [CostModel][X86] Tweak 'SSE2-only' test CPU as it was only disabling SSE41 not SSE3/SSSE3 etc.
llvm-svn: 268763
2016-05-06 17:50:07 +00:00
Simon Pilgrim 93d9b96bdb [CostModel][X86] Added ctlz/cttz undef-zero costmodel tests
llvm-svn: 268761
2016-05-06 17:48:35 +00:00
Simon Pilgrim 5122c64fd8 [CostModel][X86] Added costmodel tests for vector ctpop/ctlz/cttz/bitreverse/bswap
llvm-svn: 268738
2016-05-06 14:38:14 +00:00
Xinliang David Li 28a932742c [PM] port Branch Frequency Analaysis pass to new PM
llvm-svn: 268687
2016-05-05 21:13:27 +00:00
Xinliang David Li 6e5dd41481 [PM] Port Branch Probability Analysis pass to the new pass manager.
Differential Revision: http://reviews.llvm.org/D19839

llvm-svn: 268601
2016-05-05 02:59:57 +00:00
Sanjoy Das 013a4ac4aa [SCEV] Tweak the output format and content of -analyze
In the "LoopDispositions:" section:

 - Instead of printing out a list, print out a "dictionary" to make it
   obvious by inspection which disposition is for which loop.  This is
   just a cosmetic change.

 - Print dispositions for parent _and_ sibling loops.  I will use this
   to write a test case.

llvm-svn: 268405
2016-05-03 17:49:57 +00:00
Nicolai Haehnle 119d3d80cb AMDGPU: llvm.SI.fs.constant is a source of divergence
Summary:
This intrinsic is used to get flat-shaded fragment shader inputs. Those are
uniform across a primitive, but a fragment shader wave may process pixels from
multiple primitives (as indicated by the prim_mask), and so that's where
divergence can arise.

Reviewers: arsenm, tstellarAMD

Subscribers: arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D19747

llvm-svn: 268259
2016-05-02 17:37:01 +00:00
Sanjoy Das f2f00fb11a [SCEV] When printing via -analysis, dump loop disposition
There are currently some bugs in tree around SCEV caching an incorrect
loop disposition.  Printing out loop dispositions will let us write
whitebox tests as those are fixed.

The dispositions are printed as a list in "inside out" order,
i.e. innermost loop first.

llvm-svn: 268177
2016-05-01 04:51:05 +00:00
Matt Arsenault 790eb1c490 DivergenceAnalysis: Fix crash with unreachable blocks
Unreachable blocks may not be in the dominator tree,
so don't crash on them.

llvm-svn: 268001
2016-04-29 06:17:47 +00:00
Artur Pilipenko c97eac6555 Use DL preferred alignment for alloca in Value::getPointerAlignment
Teach Value::getPointerAlignment that allocas with no explicit alignment are aligned to preferred alignment of the allocated type.

Reviewed By: hfinkel

Differential Revision: http://reviews.llvm.org/D17569

llvm-svn: 267689
2016-04-27 10:42:29 +00:00
Silviu Baranga 795c629ec9 [SCEV] Improve the run-time checking of the NoWrap predicate
Summary:
This implements a new method of run-time checking the NoWrap
SCEV predicates, which should be easier to optimize and nicer
for targets that don't correctly handle multiplication/addition
of large integer types (like i128).

If the AddRec is {a,+,b} and the backedge taken count is c,
the idea is to check that |b| * c doesn't have unsigned overflow,
and depending on the sign of b, that:

   a + |b| * c >= a (b >= 0) or
   a - |b| * c <= a (b <= 0)

where the comparisons above are signed or unsigned, depending on
the flag that we're checking.

The advantage of doing this is that we avoid extending to a larger
type and we avoid the multiplication of large types (multiplying
i128 can be expensive).

Reviewers: sanjoy

Subscribers: llvm-commits, mzolotukhin

Differential Revision: http://reviews.llvm.org/D19266

llvm-svn: 267389
2016-04-25 09:27:16 +00:00
Sanjoy Das a6155b659a Have isKnownNotFullPoison be smarter around control flow
Summary:
(... while still not using a PostDomTree)

The way we use isKnownNotFullPoison from SCEV today, the new CFG walking
logic will not trigger for any realistic cases -- it will kick in only
for situations where we could have merged the contiguous basic blocks
anyway[0], since the poison generating instruction dominates all of its
non-PHI uses (which are the only uses we consider right now).

However, having this change in place will allow a later bugfix to break
fewer llvm-lit tests.

[0]: i.e. cases where block A branches to block B and B is A's only
successor and A is B's only predecessor.

Reviewers: broune, bjarke.roune

Subscribers: mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D19212

llvm-svn: 267175
2016-04-22 17:41:06 +00:00
Ashutosh Nema 468558a061 [X86]: Changing cost for “TRUNCATE v16i32 to v16i8” in SSE4.1 mode.
Summary:
rL256194 transforms truncations between vectors of integers into PACKUS/PACKSS
operations during DAG combine. This generates better code for truncate, so cost
of truncate needs to be changed but looks like it got changed only in SSE2 table
Whereas this change is also applicable for SSE4.1, so the cost of truncate needs
to be changed for that as well. Cost of “TRUNCATE v16i32 to v16i8” & “TRUNCATE 
v16i16 to v16i8” should be same in SSE4.1 & SSE2 table. Removing their cost from
SSE4.1, so it will fall back to SSE2.

Reviewers: Simon Pilgrim
llvm-svn: 267123
2016-04-22 08:34:05 +00:00
Michael Kuperstein de16b44f74 Port DemandedBits to the new pass manager.
Differential Revision: http://reviews.llvm.org/D18679

llvm-svn: 266699
2016-04-18 23:55:01 +00:00
Sanjoy Das 432c1c3fb3 [BPI] Consider deoptimize calls as "unreachable"
Summary:
Calls to @llvm.experimental.deoptimize are expected to "never execute",
so optimize them as such.

Reviewers: chandlerc

Subscribers: junbuml, mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D19095

llvm-svn: 266654
2016-04-18 19:01:28 +00:00
Nicolai Haehnle 13d90f324c [DivergenceAnalysis] Treat PHI with incoming undef as constant
Summary:
If a PHI has an incoming undef, we can pretend that it is equal to one
non-undef, non-self incoming value.

This is particularly relevant in combination with the StructurizeCFG
pass, which introduces PHI nodes with undefs. Previously, this lead to
branch conditions that were uniform before StructurizeCFG to become
non-uniform afterwards, which confused the SIAnnotateControlFlow
pass.

This fixes a crash when Mesa radeonsi compiles a shader from
dEQP-GLES3.functional.shaders.switch.switch_in_for_loop_dynamic_vertex

Reviewers: arsenm, tstellarAMD, jingyue

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19013

llvm-svn: 266347
2016-04-14 17:42:47 +00:00
Silviu Baranga b77365b595 [SCEV][LAA] Add tests for SCEV expression transformations performed during LAA
Summary:
Add a print method to Predicated Scalar Evolution which prints all interesting
transformations done by PSE.

Loop Access Analysis will now print this as part of the analysis output.
We now use this to check the exact expression transformations that were done
by PSE in LAA.

The additional checking also acts as white-box testing for the getAsAddRec method.

Reviewers: anemet, sanjoy

Subscribers: sanjoy, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D18792

llvm-svn: 266334
2016-04-14 16:08:45 +00:00
Adam Nemet 7aab648831 Revert "Support arbitrary addrspace pointers in masked load/store intrinsics"
This reverts commit r266086.

It breaks the LTO build of gcc in SPEC2000.

llvm-svn: 266282
2016-04-14 08:47:17 +00:00
Matt Arsenault b34eea9cb5 AMDGPU: Remove leftover ShaderType attributes in tests
llvm-svn: 266155
2016-04-13 00:39:48 +00:00
Artur Pilipenko dbe0bc8df4 Support arbitrary addrspace pointers in masked load/store intrinsics
This is a resubmittion of 263158 change.

This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace.

The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics.

Reviewed By: reames

Differential Revision: http://reviews.llvm.org/D17270

llvm-svn: 266086
2016-04-12 15:58:04 +00:00
Sanjoy Das f9d88e650b This reverts commit r265913 and r265912
See PR27315

r265913: "[IndVars] Eliminate op.with.overflow when possible"

r265912: "[SCEV] See through op.with.overflow intrinsics"
llvm-svn: 265950
2016-04-11 15:26:18 +00:00
Sanjoy Das 3c529a40ca [SCEV] See through op.with.overflow intrinsics
Summary:
This change teaches SCEV to see reduce `(extractvalue
0 (op.with.overflow X Y))` into `op X Y` (with a no-wrap tag if
possible).

Reviewers: atrick, regehr

Subscribers: mcrosier, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D18684

llvm-svn: 265912
2016-04-10 22:50:26 +00:00
Silviu Baranga 6f444dfd55 Re-commit [SCEV] Introduce a guarded backedge taken count and use it in LAA and LV
This re-commits r265535 which was reverted in r265541 because it
broke the windows bots. The problem was that we had a PointerIntPair
which took a pointer to a struct allocated with new. The problem
was that new doesn't provide sufficient alignment guarantees.
This pattern was already present before r265535 and it just happened
to work. To fix this, we now separate the PointerToIntPair from the
ExitNotTakenInfo struct into a pointer and a bool.

Original commit message:

Summary:
When the backedge taken codition is computed from an icmp, SCEV can
deduce the backedge taken count only if one of the sides of the icmp
is an AddRecExpr. However, due to sign/zero extensions, we sometimes
end up with something that is not an AddRecExpr.

However, we can use SCEV predicates to produce a 'guarded' expression.
This change adds a method to SCEV to get this expression, and the
SCEV predicate associated with it.

In HowManyGreaterThans and HowManyLessThans we will now add a SCEV
predicate associated with the guarded backedge taken count when the
analyzed SCEV expression is not an AddRecExpr. Note that we only do
this as an alternative to returning a 'CouldNotCompute'.

We use new feature in Loop Access Analysis and LoopVectorize to analyze
and transform more loops.

Reviewers: anemet, mzolotukhin, hfinkel, sanjoy

Subscribers: flyingforyou, mcrosier, atrick, mssimpso, sanjoy, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D17201

llvm-svn: 265786
2016-04-08 14:29:09 +00:00
Sanjoy Das 5ce3272833 Don't IPO over functions that can be de-refined
Summary:
Fixes PR26774.

If you're aware of the issue, feel free to skip the "Motivation"
section and jump directly to "This patch".

Motivation:

I define "refinement" as discarding behaviors from a program that the
optimizer has license to discard.  So transforming:

```
void f(unsigned x) {
  unsigned t = 5 / x;
  (void)t;
}
```

to

```
void f(unsigned x) { }
```

is refinement, since the behavior went from "if x == 0 then undefined
else nothing" to "nothing" (the optimizer has license to discard
undefined behavior).

Refinement is a fundamental aspect of many mid-level optimizations done
by LLVM.  For instance, transforming `x == (x + 1)` to `false` also
involves refinement since the expression's value went from "if x is
`undef` then { `true` or `false` } else { `false` }" to "`false`" (by
definition, the optimizer has license to fold `undef` to any non-`undef`
value).

Unfortunately, refinement implies that the optimizer cannot assume
that the implementation of a function it can see has all of the
behavior an unoptimized or a differently optimized version of the same
function can have.  This is a problem for functions with comdat
linkage, where a function can be replaced by an unoptimized or a
differently optimized version of the same source level function.

For instance, FunctionAttrs cannot assume a comdat function is
actually `readnone` even if it does not have any loads or stores in
it; since there may have been loads and stores in the "original
function" that were refined out in the currently visible variant, and
at the link step the linker may in fact choose an implementation with
a load or a store.  As an example, consider a function that does two
atomic loads from the same memory location, and writes to memory only
if the two values are not equal.  The optimizer is allowed to refine
this function by first CSE'ing the two loads, and the folding the
comparision to always report that the two values are equal.  Such a
refined variant will look like it is `readonly`.  However, the
unoptimized version of the function can still write to memory (since
the two loads //can// result in different values), and selecting the
unoptimized version at link time will retroactively invalidate
transforms we may have done under the assumption that the function
does not write to memory.

Note: this is not just a problem with atomics or with linking
differently optimized object files.  See PR26774 for more realistic
examples that involved neither.

This patch:

This change introduces a new set of linkage types, predicated as
`GlobalValue::mayBeDerefined` that returns true if the linkage type
allows a function to be replaced by a differently optimized variant at
link time.  It then changes a set of IPO passes to bail out if they see
such a function.

Reviewers: chandlerc, hfinkel, dexonsmith, joker.eph, rnk

Subscribers: mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D18634

llvm-svn: 265762
2016-04-08 00:48:30 +00:00
Nicolai Haehnle df3a20cd80 AMDGPU: Add a shader calling convention
This makes it possible to distinguish between mesa shaders
and other kernels even in the presence of compute shaders.

Patch By: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>

Differential Revision: http://reviews.llvm.org/D18559

llvm-svn: 265589
2016-04-06 19:40:20 +00:00
Silviu Baranga a393baf1fd Revert r265535 until we know how we can fix the bots
llvm-svn: 265541
2016-04-06 14:06:32 +00:00
Silviu Baranga 72b4a4a330 [SCEV] Introduce a guarded backedge taken count and use it in LAA and LV
Summary:
When the backedge taken codition is computed from an icmp, SCEV can
deduce the backedge taken count only if one of the sides of the icmp
is an AddRecExpr. However, due to sign/zero extensions, we sometimes
end up with something that is not an AddRecExpr.

However, we can use SCEV predicates to produce a 'guarded' expression.
This change adds a method to SCEV to get this expression, and the
SCEV predicate associated with it.

In HowManyGreaterThans and HowManyLessThans we will now add a SCEV
predicate associated with the guarded backedge taken count when the
analyzed SCEV expression is not an AddRecExpr. Note that we only do
this as an alternative to returning a 'CouldNotCompute'.

We use new feature in Loop Access Analysis and LoopVectorize to analyze
and transform more loops.

Reviewers: anemet, mzolotukhin, hfinkel, sanjoy

Subscribers: flyingforyou, mcrosier, atrick, mssimpso, sanjoy, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D17201

llvm-svn: 265535
2016-04-06 13:18:26 +00:00
George Burgess IV 7e5404cc20 [CFLAA] Fix PR27213; incorrect tagging of args/globals
Prior to this patch, CFLAA wouldn't tag arguments/globals properly if
it didn't find any "interesting" edges on them. This means that, if all
you do is store constants to a global or argument, we would never
actually treat it as a global/argument.

Test case:

define void @foo(i32* %A, i32* %B) #0 {
entry:
  store i32 0, i32* %A, align 4
  store i32 0, i32* %B, align 4
  ret void
}

CFLAA would say that %A can't alias %B, because neither pointer was
used in an interesting way. This patch makes us note whether something
is an argument, global, ... regardless of how interesting CFLAA thinks
its uses are.

(For the record, using a value in an interesting way means loading
from it, using it in a GEP, ...)

llvm-svn: 265474
2016-04-05 21:40:45 +00:00
Brendon Cahoon 86f783e315 [DependenceAnalysis] Check if result of getConstantPart is null
A seg-fault occurs due to a reference of a null pointer, which is
the value returned by getConstantPart. This function returns
null if the constant part is not found. The code that calls this
function needs to check for the null return value.

Differential Revision: http://reviews.llvm.org/D18718

llvm-svn: 265319
2016-04-04 18:13:18 +00:00
Benjamin Kramer cad9a8a6bb [TTI] Let the cost model estimate ctpop costs based on legality
PPC has a vector popcount, this lets the vectorizer use the correct cost
for it. Tweak X86 test to use an intrinsic that's actually scalarized (we
have a somewhat efficient lowering for vector popcount using SSE, the
cost model finds that now).

llvm-svn: 265005
2016-03-31 10:42:40 +00:00
Matt Arsenault 8c8fcb2585 AMDGPU: Cost model for basic integer operations
This resolves bug 21148 by preventing promotion to
i64 induction variables.

llvm-svn: 264376
2016-03-25 01:16:40 +00:00
Matt Arsenault 9651813ee0 AMDGPU: Partially implement getArithmeticInstrCost for FP ops
llvm-svn: 264374
2016-03-25 01:00:32 +00:00
Matt Arsenault 51d702812d TTI: Report 0 cost for free addrspacecasts
llvm-svn: 264369
2016-03-25 00:26:29 +00:00
Matt Arsenault 8e9aa0acc8 TTI: Use 0 for cost of fabs if free
Ideally this would also happen for fneg, but that
isn't a distinct operation in the IR.

llvm-svn: 264368
2016-03-25 00:26:22 +00:00
Matt Arsenault 59767cea79 AMDGPU: TTI: Make insertelement free.
We don't want to have a cost to scalarizing operations.

llvm-svn: 264364
2016-03-25 00:14:11 +00:00
Adam Nemet 279784ffc4 [LAA] Support memchecks involving loop-invariant addresses
We used to only allow SCEVAddRecExpr for pointer expressions in order to
be able to compute the bounds.  However this is also trivially possible
for loop-invariant addresses (scUnknown) since then the bounds are the
address itself.

Interestingly, we used allow this for the special case when the
loop-invariant address happens to also be an SCEVAddRecExpr (in an outer
loop).

There are a couple more loops that are vectorized in SPEC after this.
My guess is that the main reason we don't see more because for example a
loop-invariant load is vectorized into a splat vector with several
vector-inserts.  This is likely to make the vectorization unprofitable.
I.e. we don't notice that a later LICM will move all of this out of the
loop so the cost estimate should really be 0.

llvm-svn: 264243
2016-03-24 04:28:47 +00:00
Matthias Braun 68bb2931cc Revert "Support arbitrary addrspace pointers in masked load/store intrinsics"
This commit broke LTO builds. Reverting it to unbreak the bots while the
issue is investigated. See also:

http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20160321/341002.html

This reverts r263158

llvm-svn: 264088
2016-03-22 20:24:34 +00:00
Nicolai Haehnle ad63638f6d AMDGPU/SI: Add llvm.amdgcn.buffer.atomic.* intrinsics
Summary:
These intrinsics expose the BUFFER_ATOMIC_* instructions and will be used
by Mesa to implement atomics with buffer semantics. The intrinsic interface
matches that of buffer.load.format and buffer.store.format, except that the
GLC bit is not exposed (it is automatically deduced based on whether the
return value is used).

The change of hasSideEffects is required for TableGen to accept the pattern
that matches the intrinsic.

Reviewers: tstellarAMD, arsenm

Subscribers: arsenm, rivanvx, llvm-commits

Differential Revision: http://reviews.llvm.org/D18151

llvm-svn: 263791
2016-03-18 16:24:31 +00:00
Nicolai Haehnle 79cad857a0 AMDGPU: mark atomic instructions as sources of divergence
Summary:
As explained by the comment, threads will typically see different values
returned by atomic instructions even if the arguments are equal.

Reviewers: arsenm, tstellarAMD

Subscribers: arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D18156

llvm-svn: 263719
2016-03-17 16:21:59 +00:00
Nicolai Haehnle 74127fe8d7 AMDGPU: mark llvm.amdgcn.image.atomic.* as a source of divergence
Summary:
When multiple threads perform an atomic op with the same arguments, they
will usually see different return values.

Reviewers: arsenm, tstellarAMD

Subscribers: arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D18101

llvm-svn: 263440
2016-03-14 15:37:18 +00:00
Chandler Carruth 45a9c203a0 [PM/AA] Teach the AAManager how to handle module analyses in addition to
function analyses, and use it to wire up globals-aa to the new pass
manager.

llvm-svn: 263211
2016-03-11 09:15:11 +00:00
Artur Pilipenko 3c8fc57e16 Support arbitrary addrspace pointers in masked load/store intrinsics
This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace.

The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics.

Reviewed By: reames

Differential Revision: http://reviews.llvm.org/D17270

llvm-svn: 263158
2016-03-10 20:39:22 +00:00
Chandler Carruth 4c660f7087 [CG] Add a new pass manager printer pass for the old call graph and
actually finish wiring up the old call graph.

There were bugs in the old call graph that hadn't been caught because it
wasn't being tested. It wasn't being tested because it wasn't in the
pipeline system and we didn't have a printing pass to run in tests. This
fixes all of that.

As for why I'm still keeping the old call graph alive its so that I can
port GlobalsAA to the new pass manager with out forking it to work with
the lazy call graph. That's clearly the right eventual design, but it
seems pragmatic to defer that until its necessary. The old call graph
works just fine for GlobalsAA.

llvm-svn: 263104
2016-03-10 11:24:11 +00:00
Chandler Carruth b95def7491 [LCG] Spell the printing pass pipeline name for the lazy call graph
'lcg' instead of just 'cg'.

This makes it consistent with the analysis name of 'lcg'.

No functionality changed.

llvm-svn: 263103
2016-03-10 11:24:06 +00:00
Matthias Braun c31032d607 InstCombine: Restrict computeKnownBits() on all Values to OptLevel > 2
As part of r251146 InstCombine was extended to call computeKnownBits on
every value in the function to determine whether it happens to be
constant. This increases typical compiletime by 1-3% (5% in irgen+opt
time) in my measurements. On the other hand this case did not trigger
once in the whole llvm-testsuite.

This patch introduces the notion of ExpensiveCombines which are only
enabled for OptLevel > 2. I removed the check in InstructionSimplify as
that is called from various places where the OptLevel is not known but
given the rarity of the situation I think a check in InstCombine is
enough.

Differential Revision: http://reviews.llvm.org/D16835

llvm-svn: 263047
2016-03-09 18:47:11 +00:00
Sanjoy Das 84216672da Remove trailing newline from test case; NFC
llvm-svn: 262980
2016-03-09 01:51:44 +00:00
Sanjoy Das 97d19bd95f [SCEV] Slightly generalize getRangeViaFactoring
Building on the previous change, this generalizes
ScalarEvolution::getRangeViaFactoring to work with
{Ext(C?A:B)+k0,+,Ext(C?A:B)+k1} where Ext can be a zero extend, sign
extend or truncate operation, and k0 and k1 are constants.

llvm-svn: 262979
2016-03-09 01:51:02 +00:00
Sanjoy Das d3488c6060 [SCEV] Slightly generalize getRangeViaFactoring
This change generalizes ScalarEvolution::getRangeViaFactoring to work
with {Ext(C?A:B),+,Ext(C?A:B)} where Ext can be a zero extend, sign
extend or truncate operation.

llvm-svn: 262978
2016-03-09 01:50:57 +00:00
Adam Nemet 4896c7a82a [ScopedNoAliasAA] Make test basic.ll less confusing
Summary:
This testcase had me confused.  It made me believe that you can use
alias scopes and alias scopes list interchangeably with alias.scope and
noalias.  Both langref and the other testcase use scope lists so I went
looking.

Turns out using scope directly only happens to work by chance.  When
ScopedNoAliasAAResult::mayAliasInScopes traverses this as a scope list:

!1 = !{!1, !0, !"some scope"}

, the first entry is in fact a scope but only because the scope is
happened to be defined self-referentially to make it unique globally.

The remaining elements in the tuple (!0, !"some scope") are considered
as scopes but AliasScopeNode::getDomain will just bail on those without
any error.

This change avoids this ambiguity in the test but I've also been
wondering if we should issue some sort of a diagnostics.

Reviewers: dexonsmith, hfinkel

Subscribers: mssimpso, llvm-commits

Differential Revision: http://reviews.llvm.org/D16670

llvm-svn: 262841
2016-03-07 17:49:10 +00:00
Philip Reames 146307eb52 [ValueTracking] Remove dead code from an old experiment
This experiment was originally about trying to use facts implied dominating conditions to infer more precise known bits.  While the compile time was found to be acceptable on several large code bases, we never found sufficiently profitable examples to justify turning on the code by default.  Given this, it's time to abandon the experiment.  

Several folks have commented that they've found this useful for experimentation, but nothing has come of those experiments.  Given how easy the patch is to apply, there's no reason to leave the code in tree.  

For anyone interested in further investigation in this area, I recommend finding the summary email I sent on one of the original review threads.  In particular, I now believe the use-list based approach is strictly worse than the dom-tree-walking approach.  

llvm-svn: 262646
2016-03-03 19:44:06 +00:00
Sanjoy Das 724f5cf278 [SCEV] Prove no-overflow via constant ranges
Exploit ScalarEvolution::getRange's newly acquired smartness (since
r262438) by using that to infer nsw and nuw when possible.

llvm-svn: 262639
2016-03-03 18:31:29 +00:00
Sanjoy Das 11ef606f1d [SCEV] Be less eager about demoting zexts to sexts
After r262438 we can have provably positive NSW SCEV expressions whose
zero extensions cannot be simplified (since r262438 makes SCEV better at
computing constant ranges).  This means demoting sexts of positive add
recurrences eagerly can result in an unsimplified zero extension where
we could have had a simplified sign extension.  This change fixes the
issue by teaching SCEV to demote sext of a positive SCEV expression to a
zext only if the sext could not be simplified.

llvm-svn: 262638
2016-03-03 18:31:23 +00:00
Sanjoy Das bf73098472 [SCEV] Make getRange smarter around selects
Have ScalarEvolution::getRange re-consider cases like "{C?A:B,+,C?P:Q}"
by factoring out "C" and computing RangeOf{A,+,P} union RangeOf({B,+,Q})
instead.

The latter can be easier to compute precisely in cases like
"{C?0:N,+,C?1:-1}" N is the backedge taken count of the loop; since in
such cases the latter form simplifies to [0,N+1) union [0,N+1).

llvm-svn: 262438
2016-03-02 00:57:54 +00:00
Hongbin Zheng b8bb0d8813 Another fix the testcase introduced by r261903 - Add the missing matches
llvm-svn: 261971
2016-02-26 03:41:47 +00:00
Hongbin Zheng 8c70ab75a0 Use regex in testcase, do not fail windows bots
llvm-svn: 261922
2016-02-25 19:16:40 +00:00
Hongbin Zheng bc53977a0d Introduce RegionInfoAnalysis, which compute Region Tree in the new PassManager. NFC
Differential Revision: http://reviews.llvm.org/D17571

llvm-svn: 261904
2016-02-25 17:54:25 +00:00
Hongbin Zheng 751337faa7 Introduce DominanceFrontierAnalysis to the new PassManager to compute DominanceFrontier. NFC
Differential Revision: http://reviews.llvm.org/D17570

llvm-svn: 261903
2016-02-25 17:54:15 +00:00
Hongbin Zheng 3f97840721 Introduce analysis pass to compute PostDominators in the new pass manager. NFC
Differential Revision: http://reviews.llvm.org/D17537

llvm-svn: 261902
2016-02-25 17:54:07 +00:00
Hongbin Zheng 66b19fbc4e Revert "Introduce analysis pass to compute PostDominators in the new pass manager. NFC"
This reverts commit a3e5cc6a51ab5ad88d1760c63284294a4e34c018.

llvm-svn: 261891
2016-02-25 16:45:53 +00:00
Hongbin Zheng ad782ce3f7 Revert "Introduce DominanceFrontierAnalysis to the new PassManager to compute DominanceFrontier. NFC"
This reverts commit 109c38b2226a87b0be73fa7a0a8c1a81df20aeb2.

llvm-svn: 261890
2016-02-25 16:45:46 +00:00
Hongbin Zheng 921fabf34b Revert "Introduce RegionInfoAnalysis, which compute Region Tree in the new PassManager. NFC"
This reverts commit 8228b4d374edeb4cc0c5fddf6e1ab876918ee126.

llvm-svn: 261889
2016-02-25 16:45:37 +00:00
Hongbin Zheng 2fa386fd6c Introduce RegionInfoAnalysis, which compute Region Tree in the new PassManager. NFC
Differential Revision: http://reviews.llvm.org/D17571

llvm-svn: 261884
2016-02-25 16:33:26 +00:00
Hongbin Zheng 237197ba63 Introduce DominanceFrontierAnalysis to the new PassManager to compute DominanceFrontier. NFC
Differential Revision: http://reviews.llvm.org/D17570

llvm-svn: 261883
2016-02-25 16:33:15 +00:00
Hongbin Zheng a0273a04f5 Introduce analysis pass to compute PostDominators in the new pass manager. NFC
Differential Revision: http://reviews.llvm.org/D17537

llvm-svn: 261882
2016-02-25 16:33:06 +00:00
Chandler Carruth c1dc384b54 [PM/AA] Wire up TBAA to the new pass manager's registry and test it.
llvm-svn: 261411
2016-02-20 04:04:52 +00:00
Chandler Carruth d6091a0344 [PM/AA] Wire up the scoped-no-alias AA to the new pass manager's
registry and test it.

llvm-svn: 261410
2016-02-20 04:03:06 +00:00
Chandler Carruth 2b3d0446f4 [PM/AA] Wire up SCEVAA to the new pass manager's registry and test it.
llvm-svn: 261409
2016-02-20 04:01:45 +00:00