Commit Graph

10844 Commits

Author SHA1 Message Date
Hal Finkel 1a61f621da BBVectorize: Add initial stores to the write set when tracking uses
When computing the use set of a store, we need to add the store to the write
set prior to iterating over later instructions. Otherwise, if there is a later
aliasing load of that store, that load will not be tagged as a use, and bad
things will happen.

trackUsesOfI still adds later dependent stores of an instruction to that
instruction's write set, but it never sees the original instruction, and so
when tracking uses of a store, the store must be added to the write set by the
caller.

Fixes PR16834.

llvm-svn: 188329
2013-08-13 23:34:32 +00:00
Nick Lewycky c7776f737f Revert r187191, which broke opt -mem2reg on the testcases included in PR16867.
However, opt -O2 doesn't run mem2reg directly so nobody noticed until r188146
when SROA started sending more things directly down the PromoteMemToReg path.

In order to revert r187191, I also revert dependent revisions r187296, r187322
and r188146. Fixes PR16867. Does not add the testcases from that PR, but both
of them should get added for both mem2reg and sroa when this revert gets
unreverted.

llvm-svn: 188327
2013-08-13 22:51:58 +00:00
Dmitry Vyukov 96a7084620 dfsan: fix lint warnings
llvm-svn: 188293
2013-08-13 16:52:41 +00:00
Arnold Schwaighofer 124ccf3ad1 Also remove logic in LateVectorize
llvm-svn: 188285
2013-08-13 16:12:04 +00:00
Arnold Schwaighofer c14b59d1a1 Remove logic that decides whether to vectorize or not depending on O-levels
I have moved this logic into clang and opt.

llvm-svn: 188281
2013-08-13 15:51:25 +00:00
Peter Collingbourne 8d642de169 Reapply r188119 now that the bug it exposed is fixed.
llvm-svn: 188217
2013-08-12 22:38:43 +00:00
Peter Collingbourne fb3a2b4f97 DataFlowSanitizer: fix a use-after-free. Spotted by libgmalloc.
llvm-svn: 188216
2013-08-12 22:38:39 +00:00
Bill Wendling e1eaecd528 Move stack protector names to the same place.
llvm-svn: 188198
2013-08-12 20:09:37 +00:00
Nadav Rotem e23147bbd4 Fix PR16797 - Support PHINodes with multiple inputs from the same basic block.
Do not generate new vector values for the same entries because we know that the incoming values
from the same block must be identical.

llvm-svn: 188185
2013-08-12 17:46:44 +00:00
Alexey Samsonov 15dc0af78b Remove unused SpecialCaseList constructors
llvm-svn: 188171
2013-08-12 11:50:44 +00:00
Alexey Samsonov e4b5fb8851 Add SpecialCaseList::createOrDie() factory and use it in sanitizer passes
llvm-svn: 188169
2013-08-12 11:46:09 +00:00
Alexey Samsonov 9e4fdd2656 Introduce factory methods for SpecialCaseList
Summary:
Doing work in constructors is bad: this change suggests to
call SpecialCaseList::create(Path, Error) instead of
"new SpecialCaseList(Path)". Currently the latter may crash with
report_fatal_error, which is undesirable - sometimes we want to report
the error to user gracefully - for example, if he provides an incorrect
file as an argument of Clang's -fsanitize-blacklist flag.

Reviewers: pcc

Reviewed By: pcc

CC: llvm-commits

Differential Revision: http://llvm-reviews.chandlerc.com/D1327

llvm-svn: 188156
2013-08-12 07:49:36 +00:00
Richard Sandiford feb34713d5 Fix big-endian handling of integer-to-vector bitcasts in InstCombine
These functions used to assume that the lsb of an integer corresponds
to vector element 0, whereas for big-endian it's the other way around:
the msb is in the first element and the lsb is in the last element.

Fixes MultiSource/Benchmarks/mediabench/gsm/toast for z.

llvm-svn: 188155
2013-08-12 07:26:09 +00:00
Chandler Carruth d7cd7e367e Re-instate r187323 which fast-tracks promotable allocas as soon as the
SROA-based analysis has enough information. This should work now that
both mem2reg *and* the SSAUpdater-based AllocaPromoter have been updated
to be able to promote the types of allocas that the SROA analysis
detects.

I've included tests for the AllocaPromoter that were only possible to
write once we fast-tracked promotable allocas without rewriting them.
This includes a test both for r187347 and r188145.

Original commit log for r187323:
"""
Now that mem2reg understands how to cope with a slightly wider set of uses of
an alloca, we can pre-compute promotability while analyzing an alloca for
splitting in SROA. That lets us short-circuit the common case of a bunch of
trivially promotable allocas. This cuts 20% to 30% off the run time of SROA for
typical frontend-generated IR sequneces I'm seeing. It gets the new SROA to
within 20% of ScalarRepl for such code. My current benchmark for these numbers
is PR15412, but it fits the general pattern of IR emitted by Clang so it should
be widely applicable.
"""

llvm-svn: 188146
2013-08-11 02:17:11 +00:00
Chandler Carruth c17283b407 Finish fixing the SSAUpdater-based AllocaPromoter strategy in SROA to cope with
the more general set of patterns that are now handled by mem2reg and that we
can detect quickly while doing SROA's initial analysis. Notably, this allows it
to promote through no-op bitcast and GEP sequences. A core part of the
SSAUpdater approach is the ability to test whether a particular instruction is
part of the set being promoted. Testing this becomes significantly more complex
in the world where the operand to every load and store isn't the alloca itself.
I ended up using the approach of walking up the def-chain until we find the
alloca. I benchmarked this against keeping a set of pointer operands and
keeping a set of the loads and stores we care about, and this one seemed faster
although the difference was very small.

No test case yet because currently the rewriting always "fixes" the inputs to
not require this. The next patch which re-enables early promotion of easy cases
in SROA will include a test case that specifically exercises this aspect of the
alloca promoter.

llvm-svn: 188145
2013-08-11 01:56:15 +00:00
Chandler Carruth 45b136f4cf Reformat some bits of AllocaPromoter and simplify the name and type of
our visiting datastructures in the AllocaPromoter/SSAUpdater path of
SROA. Also shift the order if clears around to be more consistent.

No functionality changed here, this is just a cleanup.

llvm-svn: 188144
2013-08-11 01:03:18 +00:00
Arnold Schwaighofer 3dcdb89d69 Revert r188119 "Kill some duplicated code for removing unreachable BBs."
It is breaking builbots with libgmalloc enabled on Mac OS X.

$ cd llvm ; mkdir release ; cd release
$ ../configure --enable-optimized —prefix=$PWD/install
$ make
$ make check
$ Release+Asserts/bin/llvm-lit -v --param use_gmalloc=1 --param \
  gmalloc_path=/usr/lib/libgmalloc.dylib \
  ../test/Instrumentation/DataFlowSanitizer/args-unreachable-bb.ll

llvm-svn: 188142
2013-08-10 20:16:06 +00:00
Michael Gottesman d6ce6cbdac [objc-arc] Track if we encountered an additive overflow while computing {TopDown,BottomUp}PathCounts and do nothing if it occurred.
I fixed the aforementioned problems that came up on some of the linux boxes.
Major thanks to Nick Lewycky for his help debugging!

rdar://14590914

llvm-svn: 188122
2013-08-09 23:22:27 +00:00
Peter Collingbourne 32090aba06 Kill some duplicated code for removing unreachable BBs.
This moves removeUnreachableBlocksFromFn from SimplifyCFGPass.cpp
to Utils/Local.cpp and uses it to replace the implementation of
llvm::removeUnreachableBlocks, which appears to do a strict subset
of what removeUnreachableBlocksFromFn does.

Differential Revision: http://llvm-reviews.chandlerc.com/D1334

llvm-svn: 188119
2013-08-09 22:47:24 +00:00
Peter Collingbourne ae66d57bcf DataFlowSanitizer: Remove unreachable BBs so IR continues to verify
under the args ABI.

Differential Revision: http://llvm-reviews.chandlerc.com/D1316

llvm-svn: 188113
2013-08-09 21:42:53 +00:00
Jakub Staszak 23ec6a97d1 Mark obviously const methods. Also use reference for parameters when possible.
llvm-svn: 188103
2013-08-09 20:53:48 +00:00
Michael Gottesman 6663c7d5fc Revert "[objc-arc] Track if we encountered an additive overflow while computing {TopDown,BottomUp}PathCounts and do nothing if it occured."
This reverts commit r187941.

The commit was passing on my os x box, but it is failing on some non-osx
platforms. I do not have time to look into it now, so I am reverting and will
recommit after I figure this out.

llvm-svn: 187946
2013-08-08 00:41:18 +00:00
Peter Collingbourne a5689e69af Fix ARM build.
llvm-svn: 187944
2013-08-08 00:15:27 +00:00
Michael Gottesman ddc89fcccd [objc-arc] Track if we encountered an additive overflow while computing {TopDown,BottomUp}PathCounts and do nothing if it occured.
rdar://14590914

llvm-svn: 187941
2013-08-07 23:56:41 +00:00
Michael Gottesman 0fecf98955 [objc-arc] Change 4 iterator methods which return const_iterators to be const methods.
llvm-svn: 187940
2013-08-07 23:56:34 +00:00
Hal Finkel 171817ee8a Add ISD::FROUND for libm round()
All libm floating-point rounding functions, except for round(), had their own
ISD nodes. Recent PowerPC cores have an instruction for round(), and so here I'm
adding ISD::FROUND so that round() can be custom lowered as well.

For the most part, this is straightforward. I've added an intrinsic
and a matching ISD node just like those for nearbyint() and friends. The
SelectionDAG pattern I've named frnd (because ISD::FP_ROUND has already claimed
fround).

This will be used by the PowerPC backend in a follow-up commit.

llvm-svn: 187926
2013-08-07 22:49:12 +00:00
Peter Collingbourne e5d5b0c71e DataFlowSanitizer; LLVM changes.
DataFlowSanitizer is a generalised dynamic data flow analysis.

Unlike other Sanitizer tools, this tool is not designed to detect a
specific class of bugs on its own.  Instead, it provides a generic
dynamic data flow analysis framework to be used by clients to help
detect application-specific issues within their own code.

Differential Revision: http://llvm-reviews.chandlerc.com/D965

llvm-svn: 187923
2013-08-07 22:47:18 +00:00
Benjamin Kramer 6a4976d3e0 JumpThreading: Turn a select instruction into branching if it allows to thread one half of the select.
This is a common pattern coming out of simplifycfg generating gross code.

a:                                       ; preds = %entry
  %sel = select i1 %cmp1, double %add, double 0.000000e+00
  br label %b

b:
  %cond5 = phi double [ %sel, %a ], [ %sub, %entry ]
  %cmp6 = fcmp oeq double %cond5, 0.000000e+00
  br i1 %cmp6, label %if.then, label %if.end

becomes

a:
  br i1 %cmp1, label %b, label %if.then

b:
  %cond5 = phi double [ %sub, %entry ], [ %add, %a ]
  %cmp6 = fcmp oeq double %cond5, 0.000000e+00
  br i1 %cmp6, label %if.then, label %if.end

Skipping block b completely if possible.

llvm-svn: 187880
2013-08-07 10:29:38 +00:00
Bill Wendling 58f8cef83b Change the linkage of these global values to 'internal'.
The globals being generated here were given the 'private' linkage type. However,
this caused them to end up in different sections with the wrong prefix. E.g.,
they would be in the __TEXT,__const section with an 'L' prefix instead of an 'l'
(lowercase ell) prefix.

The problem is that the linker will eat a literal label with 'L'. If a weak
symbol is then placed into the __TEXT,__const section near that literal, then it
cannot distinguish between the literal and the weak symbol.

Part of the problems here was introduced because the address sanitizer converted
some C strings into constant initializers with trailing nuls. (Thus putting them
in the __const section with the wrong prefix.) The others were variables that
the address sanitizer created but simply had the wrong linkage type.

llvm-svn: 187827
2013-08-06 22:52:42 +00:00
Arnold Schwaighofer a7cd6bf3bb LoopVectorize: Allow vectorization of loops with lifetime markers
Patch by Marc Jessome!

llvm-svn: 187825
2013-08-06 22:37:52 +00:00
Jakub Staszak 27da123d66 Adjust file to the coding standard.
llvm-svn: 187808
2013-08-06 17:03:42 +00:00
Serge Pavlov 71044cbe16 Unbreak Debug build on Windows
llvm-svn: 187786
2013-08-06 08:44:18 +00:00
Tom Stellard aa664d9b92 Factor FlattenCFG out from SimplifyCFG
Patch by: Mei Ye

llvm-svn: 187764
2013-08-06 02:43:45 +00:00
Matt Arsenault ff7dc7248e Fix missing -*- C++ -*-s
llvm-svn: 187758
2013-08-06 00:16:21 +00:00
Peter Collingbourne bace606657 Introduce an optimisation for special case lists with large numbers of literal entries.
Our internal regex implementation does not cope with large numbers
of anchors very efficiently.  Given a ~3600-entry special case list,
regex compilation can take on the order of seconds.  This patch solves
the problem for the special case of patterns matching literal global
names (i.e. patterns with no regex metacharacters).  Rather than
forming regexes from literal global name patterns, add them to
a StringSet which is checked before matching against the regex.
This reduces regex compilation time by an order of roughly thousands
when reading the aforementioned special case list, according to a
completely unscientific study.

No test cases.  I figure that any new tests for this code should
check that regex metacharacters are properly recognised.  However,
I could not find any documentation which documents the fact that the
syntax of global names in special case lists is based on regexes.
The extent to which regex syntax is supported in special case lists
should probably be decided on/documented before writing tests.

Differential Revision: http://llvm-reviews.chandlerc.com/D1150

llvm-svn: 187732
2013-08-05 17:48:04 +00:00
Alexey Samsonov f52b717db3 80-cols
llvm-svn: 187725
2013-08-05 13:19:49 +00:00
Nadav Rotem 5defea90e6 SLPVectorizer: Fix PR16777. PHInodes may use multiple extracted values that come from different blocks.
Thanks Alexey Samsonov.

llvm-svn: 187663
2013-08-02 18:40:24 +00:00
Alexey Samsonov 9096968de5 Fix dereferencing end iterator in SimplifyCFG. Patch by Ye Mei.
llvm-svn: 187646
2013-08-02 08:06:43 +00:00
Matt Arsenault 87dc60761f Teach getOrEnforceKnownAlignment about address spaces
llvm-svn: 187629
2013-08-01 22:42:18 +00:00
Nadav Rotem e4e6e9ed47 Move the optlevel check to the frontend.
llvm-svn: 187628
2013-08-01 22:41:58 +00:00
Nadav Rotem 9153b3871d Only enable SLP-vectorization on O3 builds.
llvm-svn: 187595
2013-08-01 18:28:15 +00:00
Nadav Rotem 25f15358d2 80-col
llvm-svn: 187535
2013-07-31 22:17:45 +00:00
Owen Anderson c7be519dc0 Preserve fast-math flags when folding (fsub x, (fneg y)) to (fadd x, y).
llvm-svn: 187462
2013-07-30 23:53:17 +00:00
Matt Arsenault cacbb2377a Change behavior of calling bitcasted alias functions.
It will now only convert the arguments / return value and call
the underlying function if the types are able to be bitcasted.
This avoids using fp<->int conversions that would occur before.

llvm-svn: 187444
2013-07-30 20:45:05 +00:00
Nadav Rotem d9c74cc6d3 SLPVectorier: update the debug location for the new instructions.
llvm-svn: 187363
2013-07-29 18:18:46 +00:00
Chandler Carruth cd7c8cdfa1 Teach the AllocaPromoter which is wrapped around the SSAUpdater
infrastructure to do promotion without a domtree the same smarts about
looking through GEPs, bitcasts, etc., that I just taught mem2reg about.
This way, if SROA chooses to promote an alloca which still has some
noisy instructions this code can cope with them.

I've not used as principled of an approach here for two reasons:
1) This code doesn't really need it as we were already set up to zip
   through the instructions used by the alloca.
2) I view the code here as more of a hack, and hopefully a temporary one.

The SSAUpdater path in SROA is a real sore point for me. It doesn't make
a lot of architectural sense for many reasons:
- We're likely to end up needing the domtree anyways in a subsequent
  pass, so why not compute it earlier and use it.
- In the future we'll likely end up needing the domtree for parts of the
  inliner itself.
- If we need to we could teach the inliner to preserve the domtree. Part
  of the re-work of the pass manager will allow this to be very powerful
  even in large SCCs with many functions.
- Ultimately, computing a domtree has gotten significantly faster since
  the original SSAUpdater-using code went into ScalarRepl. We no longer
  use domfrontiers, and much of domtree is lazily done based on queries
  rather than eagerly.
- At this point keeping the SSAUpdater-based promotion saves a total of
  0.7% on a build of the 'opt' tool for me. That's not a lot of
  performance given the complexity!

So I'm leaving this a bit ugly in the hope that eventually we just
remove all of this nonsense.

I can't even readily test this because this code isn't reachable except
through SROA. When I re-instate the patch that fast-tracks allocas
already suitable for promotion, I'll add a testcase there that failed
before this change. Before that, SROA will fix any test case I give it.

llvm-svn: 187347
2013-07-29 09:06:53 +00:00
Nadav Rotem 750e42cba3 Don't vectorize when the attribute NoImplicitFloat is used.
llvm-svn: 187340
2013-07-29 05:13:00 +00:00
Rafael Espindola caa776be91 Fix -Wdocumentation warnings.
llvm-svn: 187336
2013-07-28 23:43:28 +00:00
Chandler Carruth 6b55dbea86 Update comments for SSAUpdater to use the modern doxygen comment
standards for LLVM. Remove duplicated comments on the interface from the
implementation file (implementation comments are left there of course).
Also clean up, re-word, and fix a few typos and errors in the commenst
spotted along the way.

This is in preparation for changes to these files and to keep the
uninteresting tidying in a separate commit.

llvm-svn: 187335
2013-07-28 22:00:33 +00:00
Chandler Carruth d31370e060 Temporarily revert r187323 until I update SSAUpdater to match mem2reg.
I forgot that we had two totally independent things here. :: sigh ::

llvm-svn: 187327
2013-07-28 09:05:49 +00:00
Chandler Carruth 9d96100ff0 Now that mem2reg understands how to cope with a slightly wider set of
uses of an alloca, we can pre-compute promotability while analyzing an
alloca for splitting in SROA. That lets us short-circuit the common case
of a bunch of trivially promotable allocas. This cuts 20% to 30% off the
run time of SROA for typical frontend-generated IR sequneces I'm seeing.
It gets the new SROA to within 20% of ScalarRepl for such code. My
current benchmark for these numbers is PR15412, but it fits the general
pattern of IR emitted by Clang so it should be widely applicable.

llvm-svn: 187323
2013-07-28 08:27:12 +00:00
Chandler Carruth d5b806a27f Thread DataLayout through the callers and into mem2reg. This will be
useful in a subsequent patch, but causes an unfortunate amount of noise,
so I pulled it out into a separate patch.

llvm-svn: 187322
2013-07-28 06:43:11 +00:00
Nadav Rotem 3e50c68956 Update the comment
llvm-svn: 187316
2013-07-27 23:28:47 +00:00
Chandler Carruth 8e3c4dc50e Don't use all the #ifdefs to hide the stats counters and instead rely on
their being optimized out in debug mode. Realistically, this just isn't
going to be the slow part anyways. This also fixes unused variable
warnings that are breaking LLD build bots. =/ I didn't see these at
first, and kept losing track of the fact that they were broken.

llvm-svn: 187297
2013-07-27 10:17:49 +00:00
Chandler Carruth e8f5812a30 Merge the removal of dead instructions and lifetime markers with the
analysis of the alloca. We don't need to visit all the users twice for
this. We build up a kill list during the analysis and then just process
it afterward. This recovers the tiny bit of performance lost by moving
to the visitor based analysis system as it removes one entire use-list
walk from mem2reg. In some cases, this is now faster than mem2reg was
previously.

llvm-svn: 187296
2013-07-27 09:43:30 +00:00
Nick Lewycky 0b68245ec8 Reimplement isPotentiallyReachable to make nocapture deduction much stronger.
Adds unit tests for it too.

Split BasicBlockUtils into an analysis-half and a transforms-half, and put the
analysis bits into a new Analysis/CFG.{h,cpp}. Promote isPotentiallyReachable
into llvm::isPotentiallyReachable and move it into Analysis/CFG.

llvm-svn: 187283
2013-07-27 01:24:00 +00:00
Tom Stellard 8b1e021e85 SimplifyCFG: Use parallel-and and parallel-or mode to consolidate branch conditions
Merge consecutive if-regions if they contain identical statements.
Both transformations reduce number of branches.  The transformation
is guarded by a target-hook, and is currently enabled only for +R600,
but the correctness has been tested on X86 target using a variety of
CPU benchmarks.

Patch by: Mei Ye

llvm-svn: 187278
2013-07-27 00:01:07 +00:00
Nadav Rotem cfd40da9b1 SLP Vectorier: Don't vectorize really short chains because they are already handled by the SelectionDAG store-vectorizer, which does a better job in deciding when to vectorize.
llvm-svn: 187267
2013-07-26 23:07:55 +00:00
Nadav Rotem 9ce0f779bc SLP Vectorizer: Disable the vectorization of non power of two chains, such as <3 x float>, because we dont have a good cost model for these types.
llvm-svn: 187265
2013-07-26 22:53:11 +00:00
Owen Anderson d6d4da09f7 Fix variable name.
llvm-svn: 187253
2013-07-26 22:06:21 +00:00
Owen Anderson e37c2e4d11 When InstCombine tries to fold away (fsub x, (fneg y)) into (fadd x, y), it is
also worthwhile for it to look through FP extensions and truncations, whose
application commutes with fneg.

llvm-svn: 187249
2013-07-26 21:40:29 +00:00
Stephen Lin 4ef1387221 Correct case of m_UIToFp to m_UIToFP to match instruction name, add m_SIToFP for consistency.
llvm-svn: 187225
2013-07-26 17:55:00 +00:00
Chandler Carruth 9af38fc247 Re-implement the analysis of uses in mem2reg to be significantly more
robust. It now uses an InstVisitor and worklist to actually walk the
uses of the Alloca transitively and detect the pattern which we can
directly promote: loads & stores of the whole alloca and instructions we
can completely ignore.

Also, with this new implementation teach both the predicate for testing
whether we can promote and the promotion engine itself to use the same
code so we no longer have strange divergence between the two code paths.

I've added some silly test cases to demonstrate that we can handle
slightly more degenerate code patterns now. See the below for why this
is even interesting.

Performance impact: roughly 1% regression in the performance of SROA or
ScalarRepl on a large C++-ish test case where most of the allocas are
basically ready for promotion. The reason is because of silly redundant
work that I've left FIXMEs for and which I'll address in the next
commit. I wanted to separate this commit as it changes the behavior.
Once the redundant work in removing the dead uses of the alloca is
fixed, this code appears to be faster than the old version. =]

So why is this useful? Because the previous requirement for promotion
required a *specific* visit pattern of the uses of the alloca to verify:
we *had* to look for no more than 1 intervening use. The end goal is to
have SROA automatically detect when an alloca is already promotable and
directly hand it to the mem2reg machinery rather than trying to
partition and rewrite it. This is a 25% or more performance improvement
for SROA, and a significant chunk of the delta between it and
ScalarRepl. To get there, we need to make mem2reg actually capable of
promoting allocas which *look* promotable to SROA without have SROA do
tons of work to massage the code into just the right form.

This is actually the tip of the iceberg. There are tremendous potential
savings we can realize here by de-duplicating work between mem2reg and
SROA.

llvm-svn: 187191
2013-07-26 08:20:39 +00:00
Bill Schmidt 0a9170d931 [PowerPC] Support powerpc64le as a syntax-checking target.
This patch provides basic support for powerpc64le as an LLVM target.
However, use of this target will not actually generate little-endian
code.  Instead, use of the target will cause the correct little-endian
built-in defines to be generated, so that code that tests for
__LITTLE_ENDIAN__, for example, will be correctly parsed for
syntax-only testing.  Code generation will otherwise be the same as
powerpc64 (big-endian), for now.

The patch leaves open the possibility of creating a little-endian
PowerPC64 back end, but there is no immediate intent to create such a
thing.

The LLVM portions of this patch simply add ppc64le coverage everywhere
that ppc64 coverage currently exists.  There is nothing of any import
worth testing until such time as little-endian code generation is
implemented.  In the corresponding Clang patch, there is a new test
case variant to ensure that correct built-in defines for little-endian
code are generated.

llvm-svn: 187179
2013-07-26 01:35:43 +00:00
Rafael Espindola 17600e29fa Respect llvm.used in Internalize.
The language reference says that:

"If a symbol appears in the @llvm.used list, then the compiler,
assembler, and linker are required to treat the symbol as if there is
a reference to the symbol that it cannot see"

Since even the linker cannot see the reference, we must assume that
the reference can be using the symbol table. For example, a user can add
__attribute__((used)) to a debug helper function like dump and use it from
a debugger.

llvm-svn: 187103
2013-07-25 03:23:25 +00:00
Nick Lewycky 5b15037fc9 Check that TD isn't NULL before dereferencing it down this path.
llvm-svn: 187099
2013-07-25 02:55:14 +00:00
Rafael Espindola ec2375fb51 Make these methods const correct.
Thanks to Nick Lewycky for noticing it.

llvm-svn: 187098
2013-07-25 02:50:08 +00:00
Benjamin Kramer 328da33d19 TRE: Move class into anonymous namespace.
While there shrink a dangerously large SmallPtrSet.

llvm-svn: 187050
2013-07-24 16:12:08 +00:00
Chandler Carruth 58e25d3905 Fix a problem I introduced in r187029 where we would over-eagerly
schedule an alloca for another iteration in SROA. This only showed up
with a mixture of promotable and unpromotable selects and phis. Added
a test case for this.

llvm-svn: 187031
2013-07-24 12:12:17 +00:00
Chandler Carruth 83ea195d40 Fix PR16687 where we were incorrectly promoting an alloca that had
pending speculation for a phi node. The problem here is that we were
using growth of the specluation set as an indicator of whether
speculation would occur, and if the phi node is already in the set we
don't see it grow. This is a symptom of the fact that this signal is
a total hack.

Unfortunately, I couldn't really come up with a non-hacky way of
signaling that promotion remains valid *after* speculation occurs, such
that we only speculate when all else looks good for promotion. In the
end, I went with at least a much more explicit approach of doing the
work of queuing inside the phi and select processing and setting
a preposterously named flag to convey that we're in the special state of
requiring speculating before promotion.

Thanks to Richard Trieu and Nick Lewycky for the excellent work reducing
a testcase for this from a pretty giant, nasty assert in a big
application. =] The testcase was excellent.

llvm-svn: 187029
2013-07-24 09:47:28 +00:00
Matt Arsenault f64212b281 Fix spelling
llvm-svn: 186997
2013-07-23 22:20:57 +00:00
Nick Lewycky 6ab9d936d5 Remove extraneous null statement. No functionality change!
llvm-svn: 186893
2013-07-22 23:38:27 +00:00
Jakub Staszak d4d94065e3 Use switch instead of if. No functionality change.
llvm-svn: 186892
2013-07-22 23:38:16 +00:00
Jakub Staszak 8e1a6e7d53 Remove trailing spaces.
llvm-svn: 186890
2013-07-22 23:16:36 +00:00
Nadav Rotem cf0dcdc71c When we vectorize across multiple basic blocks we may vectorize PHINodes that create a cycle. We already break the cycle on phi-nodes, but arithmetic operations are still uplicated. This patch adds code that checks if the operation that we are vectorizing was vectorized during the visit of the operands and uses this value if it can.
llvm-svn: 186883
2013-07-22 22:18:07 +00:00
Jakub Staszak cb132face0 OldPtr is llvm::Instruction. Remove unneeded cast<>.
llvm-svn: 186880
2013-07-22 22:10:43 +00:00
Jakub Staszak 6b36db08f3 Change tabs to spaces.
llvm-svn: 186877
2013-07-22 21:11:30 +00:00
Matt Arsenault fb18323885 Fix spelling and grammar
llvm-svn: 186858
2013-07-22 18:59:58 +00:00
Nadav Rotem 8c45d4b27f Fix an obvious typo in the loop vectorizer where the cost model uses the wrong variable. The variable BlockCost is ignored.
We don't have tests for the effect of if-conversion loops because it requires a big test (that includes if-converted loops) and it is difficult to find and balance a loop to do the right thing.

llvm-svn: 186845
2013-07-22 17:10:48 +00:00
Nadav Rotem d7ff88a8d9 Delete unused helper functions.
llvm-svn: 186808
2013-07-22 05:19:22 +00:00
Benjamin Kramer 2fdb758ca8 mem2reg: Minor STL usage cleanup. No functionality change.
llvm-svn: 186790
2013-07-21 11:03:40 +00:00
Chandler Carruth 7aa9ebb546 Make the mem2reg interface use an ArrayRef as it keeps a copy of these
to iterate over.

llvm-svn: 186788
2013-07-21 08:37:58 +00:00
Nadav Rotem f6bb6a464c Revert a part of r186420. Don't forbid multiple store chains that merge.
llvm-svn: 186786
2013-07-21 06:12:57 +00:00
Chandler Carruth b1ca98c4d0 Hoist the rest of the logic for promoting single-store allocas into the
helper function. This leaves both trivial cases handled entirely in
helper functions and merely manages the list of allocas to process in
the run method.

The next step will be to handle all of the trivial promotion work prior
to even creating the core class and the subsequent simplifications that
enables.

llvm-svn: 186784
2013-07-21 01:52:33 +00:00
Chandler Carruth f9e7e1dd87 Hoist the rest of the logic for fully promoting allocas with all uses in
a single block into the helper routine. This takes advantage of the fact
that we can directly replace uses prior to any store with undef to
simplify matters and unconditionally promote allocas only used within
one block.

I've removed the special handling for the case of no stores existing.
This has no semantic effect but might slow things down. I'll fix that in
a later patch when I refactor this entire thing to be easier to manage
the different cases.

llvm-svn: 186783
2013-07-21 01:44:07 +00:00
Chandler Carruth e99f931516 Remove a method made dead by the prior refactoring.
llvm-svn: 186782
2013-07-21 00:01:34 +00:00
Chandler Carruth 420fafef93 Hoist the two trivial promotion routines out of the big class that
handles the general cases.

The hope is to refactor this so that we don't end up building the entire
class for the trivial cases. I also want to lift a lot of the early
pre-processing in the initial segment of run() into a separate routine,
and really none of it needs to happen inside the primary promotion
class.

These routines in particular used none of the actual state in the
promotion class, so they don't really make sense as members.

llvm-svn: 186781
2013-07-20 23:59:51 +00:00
Chandler Carruth 48e11fd76d Hoist the AllocaInfo struct to the top of the file.
This struct is nicely independent of everything else, and we already
needed a foward declaration here. It's simpler to just define it
immediately.

llvm-svn: 186780
2013-07-20 23:39:26 +00:00
Chandler Carruth 4711793e8a Sink a typedef and comparator down to the function that actually uses them.
llvm-svn: 186779
2013-07-20 23:36:19 +00:00
Rafael Espindola c2bb73fc8d Don't crash when llvm.compiler.used becomes empty.
GlobalOpt simplifies llvm.compiler.used by removing any members that are also
in the more strict llvm.used. Handle the special case where llvm.compiler.used
becomes empty.

llvm-svn: 186778
2013-07-20 23:33:15 +00:00
Chandler Carruth f3878f46ce Don't allocate the DIBuilder on the heap and remove all the complexity
that ensued from that.

llvm-svn: 186777
2013-07-20 23:33:06 +00:00
Chandler Carruth e62f211b77 Rename constructor parameters to follow the common member-shadowing
pattern and conform to the naming conventions.

llvm-svn: 186776
2013-07-20 23:23:47 +00:00
Chandler Carruth b3e8e6f10b Reformat the implementation of mem2reg with clang-format so that my
subsequent changes don't introduce inconsistencies.

llvm-svn: 186775
2013-07-20 23:20:08 +00:00
Chandler Carruth 985eb0b550 Remove a DenseMapInfo specialization for std::pair -- we have one of
those baked into DenseMap now.

llvm-svn: 186773
2013-07-20 23:09:05 +00:00
Chandler Carruth 019516109d Update mem2reg's comments to conform to the new doxygen standards. No
functionality changed.

llvm-svn: 186772
2013-07-20 22:20:05 +00:00
Benjamin Kramer 08e5070bf5 SROA: Microoptimization: Remove dead entries first, then sort.
While there replace an explicit struct with std::mem_fun.

llvm-svn: 186761
2013-07-20 08:38:34 +00:00
Stephen Lin a9b57f6bea InstCombine: call FoldOpIntoSelect for all floating binops, not just fmul
llvm-svn: 186759
2013-07-20 07:13:13 +00:00
Nadav Rotem e210839f5b fix an 80-col line.
llvm-svn: 186733
2013-07-19 23:14:01 +00:00
Nadav Rotem c069c25518 Use LLVMs ADTs that improve the compile time of this pass.
llvm-svn: 186732
2013-07-19 23:12:19 +00:00
Nadav Rotem 5c9a193a65 SLPVectorizer: Improve the compile time of isConsecutive by reordering the conditions that check GEPs and eliminate two of the calls to accumulateConstantOffset.
llvm-svn: 186731
2013-07-19 23:11:15 +00:00
Rafael Espindola 9aadcc4c0e s/compiler_used/compiler.used/.
We were incorrectly using compiler_used instead of compiler.used. Unfortunately
the passes using the broken name had tests also using the broken name.

llvm-svn: 186705
2013-07-19 18:44:51 +00:00
Chandler Carruth 6c321c131b Cleanup the stats counters for the new implementation. These actually
count the right things and have the right names.

llvm-svn: 186667
2013-07-19 10:57:36 +00:00
Chandler Carruth 1ed848d55c Fix another assert failure very similar to PR16651's test case. This
test case came from Benjamin and found the parallel bug in the vector
promotion code.

llvm-svn: 186666
2013-07-19 10:57:32 +00:00
Chandler Carruth 9f21fe1d65 Try to move to a more reasonable set of naming conventions given the new
implementation of the SROA algorithm. We were using the term 'partition'
in many places that no longer ever represented an actual partition, but
rather just an arbitrary slice of an alloca.

No functionality change intended here. Mostly just renaming of types,
functions, variables, and rewording of comments. Several comments were
rewritten to make a lot more sense in the new structure of things.

The stats are still weird and not reflective of how this really works.
I'll fix those up in a separate patch as it is a touch more semantic of
a change...

llvm-svn: 186659
2013-07-19 09:13:58 +00:00
Chandler Carruth 90a735d606 A long overdue cleanup in SROA to use 'DL' instead of 'TD' for the
DataLayout variables.

llvm-svn: 186656
2013-07-19 07:21:28 +00:00
Chandler Carruth 5955c9e4da Fix PR16651, an assert introduced in my recent re-work of the innards of
SROA.

The crux of the issue is that now we track uses of a partition of the
alloca in two places: the iterators over the partitioning uses and the
previously collected split uses vector. We weren't accounting for the
fact that the split uses might invalidate integer widening in ways other
than due to their width (in this case due to being volatile).

Further reduced testcase added to the tests.

llvm-svn: 186655
2013-07-19 07:12:23 +00:00
Eric Christopher 03b3e1118f Remove DIBuilder cache of variable TheCU and change the few
uses that wanted it. Also change the interface for createCompileUnit
to compensate. Fix comments that refer to TheCU as well.

llvm-svn: 186637
2013-07-19 00:51:47 +00:00
Nick Lewycky 03f3d34ffb Clean up some of this code a tiny bit, no functionality change.
llvm-svn: 186622
2013-07-18 22:32:32 +00:00
Eric Christopher a4b6cf14f6 Revert "Remove DIBuilder cache of variable TheCU and change the few"
This reverts commit r186599 as I didn't want to commit this yet.

llvm-svn: 186601
2013-07-18 19:13:06 +00:00
Eric Christopher d0b2150f01 Remove DIBuilder cache of variable TheCU and change the few
uses that wanted it. Also change the interface for createCompileUnit
to compensate. Fix comments that refer to TheCU as well.

llvm-svn: 186599
2013-07-18 19:11:29 +00:00
Nadav Rotem bb3398f000 Handle constants without going through SCEV.
llvm-svn: 186593
2013-07-18 18:34:21 +00:00
Nadav Rotem de2815a5f7 SLPVectorizer: Speedup isConsecutive by manually checking GEPs with multiple indices.
This brings the compile time of the SLP-Vectorizer to about 2.5% of OPT for my testcase.

llvm-svn: 186592
2013-07-18 18:20:45 +00:00
Chandler Carruth f0546402af Reapply r186316 with a fix for one bug where the code could walk off the
end of a vector. This was found with ASan. I've had one other report of
a crasher, but thus far been unable to reproduce the crash. It may well
be fixed with this version, and if not I'd like to get more information
from the build bots about what is happening.

See r186316 for the full commit log for the new implementation of the
SROA algorithm.

llvm-svn: 186565
2013-07-18 07:15:00 +00:00
Nadav Rotem 7d7036b8c6 SLPVectorizer: Speedup isConsecutive (that checks if two addresses are consecutive in memory) by checking for additional patterns that don't need to go through SCEV.
llvm-svn: 186563
2013-07-18 04:33:20 +00:00
Eric Christopher 7ab2c3ecb2 Add comparison operators for DIDescriptors to fix c++98 fallout
of operator bool change.

Also convert a variable in DebugIR.

llvm-svn: 186544
2013-07-17 23:25:22 +00:00
Nadav Rotem 43639e8492 Fix a comment.
llvm-svn: 186541
2013-07-17 22:41:16 +00:00
Stephen Lin 03f9fbbcd7 Restore r181216, which was partially reverted in r182499.
llvm-svn: 186533
2013-07-17 20:06:03 +00:00
Nadav Rotem 3072baeb9c Add a micro optimization to catch cases where the PtrA equals PtrB.
llvm-svn: 186531
2013-07-17 19:52:25 +00:00
Hal Finkel ec7cd26968 Fix comparisons of alloca alignment in inliner merging
Duncan pointed out a mistake in my fix in r186425 when only one of the allocas
being compared had the target-default alignment. This is essentially his
suggested solution. Thanks!

llvm-svn: 186510
2013-07-17 14:32:41 +00:00
Craig Topper 24048c9440 Mark a method 'const' and another 'static'.
llvm-svn: 186485
2013-07-17 03:54:53 +00:00
Craig Topper 1c4d667ca5 Make a few more static string pointers constant.
llvm-svn: 186484
2013-07-17 03:43:10 +00:00
Nadav Rotem 2202317fce SLPVectorizer: Accelerate the isConsecutive check by replacing the subtraction of the two values with a simple SCEV expression that adds the offset to one of the pointers that we compare.
llvm-svn: 186479
2013-07-17 00:48:31 +00:00
Nadav Rotem d2e8c4cdea flip the scev minus direction to simplify the code.
llvm-svn: 186466
2013-07-16 22:57:06 +00:00
Nadav Rotem 8f924f3891 SLPVectorizer: Improve the compile time of isConsecutive by adding a simple constant-gep check before using SCEV.
This check does not always work because not all of the GEPs use a constant offset, but it happens often enough to reduce the number of times we use SCEV.

llvm-svn: 186465
2013-07-16 22:51:07 +00:00
Rafael Espindola 6d35481c94 Add a wrapper for open.
This centralizes the handling of O_BINARY and opens the way for hiding more
differences (like how open behaves with directories).

llvm-svn: 186447
2013-07-16 19:44:17 +00:00
Peter Collingbourne 8b77f18da0 Make SpecialCaseList match full strings, as documented, using anchors.
Differential Revision: http://llvm-reviews.chandlerc.com/D1149

llvm-svn: 186431
2013-07-16 17:56:07 +00:00
Hal Finkel 9caa8f7ba7 When the inliner merges allocas, it must keep the larger alignment
For safety, the inliner cannot decrease the allignment on an alloca when
merging it with another.

I've included two variants of the test case for this: one with DataLayout
available, and one without. When DataLayout is not available, if only one of
the allocas uses the default alignment (getAlignment() == 0), then they cannot
be safely merged.

llvm-svn: 186425
2013-07-16 17:10:55 +00:00
Nadav Rotem 26bf9a0c75 SLPVectorizer: Reduce the compile time of the consecutive store lookup.
Process groups of stores in chunks of 16.

llvm-svn: 186420
2013-07-16 15:25:17 +00:00
Craig Topper d3a34f81f8 Add 'const' qualifiers to static const char* variables.
llvm-svn: 186371
2013-07-16 01:17:10 +00:00
Nadav Rotem 1c1d6c1666 PR16628: Fix a bug in the code that merges compares.
Compares return i1 but they compare different types.

llvm-svn: 186359
2013-07-15 22:52:48 +00:00
Stephen Lin 837bba1c51 Remove trailing whitespace
llvm-svn: 186333
2013-07-15 17:55:02 +00:00
Chandler Carruth e3899f2c2c Revert r186316 while I track down an ASan failure and an assert from
a bot.

This reverts the commit which introduced a new implementation of the
fancy SROA pass designed to reduce its overhead. I'll skip the huge
commit log here, refer to r186316 if you're looking for how this all
works and why it works that way.

llvm-svn: 186332
2013-07-15 17:36:21 +00:00
Chandler Carruth e74ff4c643 Reimplement SROA yet again. Same fundamental principle, but a totally
different core implementation strategy.

Previously, SROA would build a relatively elaborate partitioning of an
alloca, associate uses with each partition, and then rewrite the uses of
each partition in an attempt to break apart the alloca into chunks that
could be promoted. This was very wasteful in terms of memory and compile
time because regardless of how complex the alloca or how much we're able
to do in breaking it up, all of the datastructure work to analyze the
partitioning was done up front.

The new implementation attempts to form partitions of the alloca lazily
and on the fly, rewriting the uses that make up that partition as it
goes. This has a few significant effects:
1) Much simpler data structures are used throughout.
2) No more double walk of the recursive use graph of the alloca, only
   walk it once.
3) No more complex algorithms for associating a particular use with
   a particular partition.
4) PHI and Select speculation is simplified and happens lazily.
5) More precise information is available about a specific use of the
   alloca, removing the need for some side datastructures.

Ultimately, I think this is a much better implementation. It removes
about 300 lines of code, but arguably removes more like 500 considering
that some code grew in the process of being factored apart and cleaned
up for this all to work.

I've re-used as much of the old implementation as possible, which
includes the lion's share of code in the form of the rewriting logic.
The interesting new logic centers around how the uses of a partition are
sorted, and split into actual partitions.

Each instruction using a pointer derived from the alloca gets
a 'Partition' entry. This name is totally wrong, but I'll do a rename in
a follow-up commit as there is already enough churn here. The entry
describes the offset range accessed and the nature of the access. Once
we have all of these entries we sort them in a very specific way:
increasing order of begin offset, followed by whether they are
splittable uses (memcpy, etc), followed by the end offset or whatever.
Sorting by splittability is important as it simplifies the collection of
uses into a partition.

Once we have these uses sorted, we walk from the beginning to the end
building up a range of uses that form a partition of the alloca.
Overlapping unsplittable uses are merged into a single partition while
splittable uses are broken apart and carried from one partition to the
next. A partition is also introduced to bridge splittable uses between
the unsplittable regions when necessary.

I've looked at the performance PRs fairly closely. PR15471 no longer
will even load (the module is invalid). Not sure what is up there.
PR15412 improves by between 5% and 10%, however it is nearly impossible
to know what is holding it up as SROA (the entire pass) takes less time
than reading the IR for that test case. The analysis takes the same time
as running mem2reg on the final allocas. I suspect (without much
evidence) that the new implementation will scale much better however,
and it is just the small nature of the test cases that makes the changes
small and noisy. Either way, it is still simpler and cleaner I think.

llvm-svn: 186316
2013-07-15 10:30:19 +00:00
Craig Topper 06b3b6651e Add 'const' qualifier to some arrays.
llvm-svn: 186312
2013-07-15 08:02:13 +00:00
Craig Topper 5871321e49 Use llvm::array_lengthof to replace sizeof(array)/sizeof(array[0]).
llvm-svn: 186301
2013-07-15 04:27:47 +00:00
Nadav Rotem d9f3f4548e SLPVectorizer: change the order in which we search for vectorization candidates. Do stores first and PHIs second.
llvm-svn: 186277
2013-07-14 06:15:46 +00:00
Craig Topper b94011fd28 Use SmallVectorImpl& instead of SmallVector to avoid repeating small vector size.
llvm-svn: 186274
2013-07-14 04:42:23 +00:00
Arnold Schwaighofer a92eeebde8 LoopVectorizer: Disallow reductions whose header phi is used outside the loop
If an outside loop user of the reduction value uses the header phi node we
cannot just reduce the vectorized phi value in the vector code epilog because
we would loose VF-1 reductions.

lp:
  p = phi (0, lv)
  lv = lv + 1
  ...
  brcond , lp, outside

outside:
  usr = add 0, p

(Say the loop iterates two times, the value of p coming out of the loop is one).

We cannot just transform this to:

vlp:
  p = phi (<0,0>, lv)
  lv = lv + <1,1>
  ..
  brcond , lp, outside

outside:
  p_reduced = p[0] + [1];
  usr = add 0, p_reduced

(Because the original loop iterated two times the vectorized loop would iterate
one time, but p_reduced ends up being zero instead of one).

We would have to execute VF-1 iterations in the scalar remainder loop in such
cases. For now, just disable vectorization.

PR16522

llvm-svn: 186256
2013-07-13 19:09:29 +00:00
Andrew Trick 0ae8c94f8f LoopVectorize fix: LoopInfo must be valid when invoking utils like SCEVExpander.
In general, one should always complete CFG modifications first, update
CFG-based analyses, like Dominatores and LoopInfo, then generate
instruction sequences.

LoopVectorizer was creating a new loop, calling SCEVExpander to
generate checks, then updating LoopInfo. I just changed the order.

llvm-svn: 186241
2013-07-13 06:20:06 +00:00
Nick Lewycky 7459be6dc7 Add a microoptimization for urem.
llvm-svn: 186235
2013-07-13 01:16:47 +00:00
Joey Gouly a3250f22c2 Fix a crash in EvaluateInDifferentElementOrder where it would generate an
undef vector of the wrong type.

LGTM'd by Nick Lewycky on IRC.

llvm-svn: 186224
2013-07-12 23:08:06 +00:00
Andrew Trick a1e4118a46 LFTR improvement to avoid truncation.
This is a reimplemntation of the patch originally in r186107.

llvm-svn: 186215
2013-07-12 22:08:48 +00:00
Andrew Trick 2b71848ffe Cleanup LFTR logic.
llvm-svn: 186214
2013-07-12 22:08:44 +00:00
Andrew Trick 466555e50d Cleanup: rename a variable to make the logic easier to follow.
llvm-svn: 186213
2013-07-12 22:08:41 +00:00
Arnold Schwaighofer 9da9a43af8 TargetTransformInfo: address calculation parameter for gather/scather
Address calculation for gather/scather in vectorized code can incur a
significant cost making vectorization unbeneficial. Add infrastructure to add
cost.
Tests and cost model for targets will be in follow-up commits.

radar://14351991

llvm-svn: 186187
2013-07-12 19:16:02 +00:00
Chandler Carruth cf3715cadd Revert "indvars: Improve LFTR by eliminating truncation when comparing
against a constant."

This reverts commit r186107. It didn't handle wrapping arithmetic in the
loop correctly and thus caused the following C program to count from
0 to UINT64_MAX instead of from 0 to 255 as intended:

  #include <stdio.h>
  int main() {
    unsigned char first = 0, last = 255;
    do { printf("%d\n", first); } while (first++ != last);
  }

Full test case and instructions to reproduce with just the -indvars pass
sent to the original review thread rather than to r186107's commit.

llvm-svn: 186152
2013-07-12 11:18:55 +00:00
Nadav Rotem 89c41bf06a SLPVectorizer: Sink and enable CSE for ExtractElements.
llvm-svn: 186145
2013-07-12 06:09:24 +00:00
Nadav Rotem fa3c2db211 SLPVectorize: Replace the code that checks for vectorization candidates in successor blocks with code that scans PHINodes.
Before we could vectorize PHINodes scanning successors was a good way of finding candidates. Now we can vectorize the phinodes which is simpler.

llvm-svn: 186139
2013-07-12 00:04:18 +00:00
Nadav Rotem db06b139fd Remove an argument that we dont use anymore.
llvm-svn: 186116
2013-07-11 20:56:13 +00:00
Andrew Trick 3095993d6f indvars: Improve LFTR by eliminating truncation when comparing against a constant.
Patch by Michele Scandale!

Adds a special handling of the case where, during the loop exit
condition rewriting, the exit value is a constant of bitwidth lower
than the type of the induction variable: instead of introducing a
trunc operation in order to match correctly the operand types, it
allows to convert the constant value to an equivalent constant,
depending on the initial value of the induction variable and the trip
count, in order have an equivalent comparison between the induction
variable and the new constant.

llvm-svn: 186107
2013-07-11 17:08:59 +00:00
Benjamin Kramer fc3ea6f4bc Don't use a potentially expensive shift if all we want is one set bit.
No functionality change.

llvm-svn: 186095
2013-07-11 16:05:50 +00:00
Arnold Schwaighofer e97c71b8fd LoopVectorize: Vectorize all accesses in address space zero with unit stride
We can vectorize them because in the case where we wrap in the address space the
unvectorized code would have had to access a pointer value of zero which is
undefined behavior in address space zero according to the LLVM IR semantics.
(Thank you Duncan, for pointing this out to me).

Fixes PR16592.

llvm-svn: 186088
2013-07-11 15:21:55 +00:00
Duncan Sands e773c08021 TryToSimplifyUncondBranchFromEmptyBlock was checking that any common
predecessors of the two blocks it is attempting to merge supply the
same incoming values to any phi in the successor block.  This change
allows merging in the case where there is one or more incoming values
that are undef.  The undef values are rewritten to match the non-undef
value that flows from the other edge.  Patch by Mark Lacey.

llvm-svn: 186069
2013-07-11 08:28:20 +00:00
Nadav Rotem 08efb262a9 Fix a warning.
llvm-svn: 186064
2013-07-11 05:39:02 +00:00
Nadav Rotem b8dd66f655 SLPVectorizer: refactor the code that places extracts. Place the code that decides where to put extracts in the build-tree phase. This allows us to take the cost of the extracts into account.
llvm-svn: 186058
2013-07-11 04:54:05 +00:00
Michael Gottesman b40db26eae Teach TailRecursionElimination to handle certain cases of nocapture escaping allocas.
Without the changes introduced into this patch, if TRE saw any allocas at all,
TRE would not perform TRE *or* mark callsites with the tail marker.

Because TRE runs after mem2reg, this inadequacy is not a death sentence. But
given a callsite A without escaping alloca argument, A may not be able to have
the tail marker placed on it due to a separate callsite B having a write-back
parameter passed in via an argument with the nocapture attribute.

Assume that B is the only other callsite besides A and B only has nocapture
escaping alloca arguments (*NOTE* B may have other arguments that are not passed
allocas). In this case not marking A with the tail marker is unnecessarily
conservative since:

  1. By assumption A has no escaping alloca arguments itself so it can not
     access the caller's stack via its arguments.

  2. Since all of B's escaping alloca arguments are passed as parameters with
     the nocapture attribute, we know that B does not stash said escaping
     allocas in a manner that outlives B itself and thus could be accessed
     indirectly by A.

With the changes introduced by this patch:

  1. If we see any escaping allocas passed as a capturing argument, we do
     nothing and bail early.

  2. If we do not see any escaping allocas passed as captured arguments but we
     do see escaping allocas passed as nocapture arguments:

       i. We do not perform TRE to avoid PR962 since the code generator produces
          significantly worse code for the dynamic allocas that would be created
          by the TRE algorithm.

       ii. If we do not return twice, mark call sites without escaping allocas
           with the tail marker. *NOTE* This excludes functions with escaping
           nocapture allocas.

  3. If we do not see any escaping allocas at all (whether captured or not):

       i. If we do not have usage of setjmp, mark all callsites with the tail
          marker.

       ii. If there are no dynamic/variable sized allocas in the function,
           attempt to perform TRE on all callsites in the function.

Based off of a patch by Nick Lewycky.

rdar://14324281.

llvm-svn: 186057
2013-07-11 04:40:01 +00:00
Michael Gottesman 6eb95dc2f7 [objc-arc] Changed 'mode: c++' => 'C++' at Nick Lewycky's suggestion. Also removed unnecessary mode: c++ lines from .cpp files.
llvm-svn: 186026
2013-07-10 18:49:00 +00:00
Peter Collingbourne 49062a97cf Implement categories for special case lists.
A special case list can now specify categories for specific globals,
which can be used to instruct an instrumentation pass to treat certain
functions or global variables in a specific way, such as by omitting
certain aspects of instrumentation while keeping others, or informing
the instrumentation pass that a specific uninstrumentable function
has certain semantics, thus allowing the pass to instrument callers
according to those semantics.

For example, AddressSanitizer now uses the "init" category instead of
global-init prefixes for globals whose initializers should not be
instrumented, but which in all other respects should be instrumented.

The motivating use case is DataFlowSanitizer, which will have a
number of different categories for uninstrumentable functions, such
as "functional" which specifies that a function has pure functional
semantics, or "discard" which indicates that a function's return
value should not be labelled.

Differential Revision: http://llvm-reviews.chandlerc.com/D1092

llvm-svn: 185978
2013-07-09 22:03:17 +00:00
Peter Collingbourne 2eb048d230 Introduce a SpecialCaseList ctor which takes a MemoryBuffer to make
it more unit testable, and fix memory leak in the other ctor.

Differential Revision: http://llvm-reviews.chandlerc.com/D1090

llvm-svn: 185976
2013-07-09 22:03:09 +00:00
Peter Collingbourne 015370e23a Rename BlackList class to SpecialCaseList and move it to Transforms/Utils.
Differential Revision: http://llvm-reviews.chandlerc.com/D1089

llvm-svn: 185975
2013-07-09 22:02:49 +00:00
Nadav Rotem d7b574e5b3 Fix PR16571, which is a bug in the code that checks that all of the types in the bundle are uniform.
llvm-svn: 185970
2013-07-09 21:38:08 +00:00
Nadav Rotem 861bef7dd0 Set the default insert point to the first instruction, and not to end()
llvm-svn: 185953
2013-07-09 17:55:36 +00:00
David Majnemer eeed73b981 InstCombine: Fix typo in comment for visitICmpInstWithInstAndIntCst
llvm-svn: 185916
2013-07-09 09:24:35 +00:00
David Majnemer 72d76275ac InstCombine: variations on 0xffffffff - x >= 4
The following transforms are valid if -C is a power of 2:
(icmp ugt (xor X, C), ~C) -> (icmp ult X, C)
(icmp ult (xor X, C), -C) -> (icmp uge X, C)

These are nice, they get rid of the xor.

llvm-svn: 185915
2013-07-09 09:20:58 +00:00
David Majnemer 414d4e58aa InstCombine: X & -C != -C -> X <= u ~C
Tests were added in r185910 somehow.

llvm-svn: 185912
2013-07-09 08:09:32 +00:00
David Majnemer bafa537eb7 Commit r185909 was a misapplied patch, fix it
llvm-svn: 185910
2013-07-09 07:58:32 +00:00
David Majnemer f2a9a513c7 InstCombine: add more transforms
C1-X <u C2 -> (X|(C2-1)) == C1
C1-X >u C2 -> (X|C2) == C1
X-C1 <u C2 -> (X & -C2) == C1
X-C1 >u C2 -> (X & ~C2) == C1

llvm-svn: 185909
2013-07-09 07:50:59 +00:00
Eli Bendersky 07b0e451ca Fix comment
llvm-svn: 185888
2013-07-08 23:57:07 +00:00
Nadav Rotem c9c57518ab This patch changes the saved IRBuilder insert point from BasicBlock::iterator to AssertingVH.
Commit 185883 fixes a bug in the IRBuilder that should fix the ASan bot. AssertingVH can help in exposing some RAUW problems.

Thanks Ben and Alexey!

llvm-svn: 185886
2013-07-08 23:31:13 +00:00
Michael Gottesman c1b648f6c0 [objc-arc] Fix assertion in EraseInstruction so that noop on null calls when passed null do not trigger the assert.
The specific case of interest is when objc_retainBlock is passed null.

llvm-svn: 185885
2013-07-08 23:30:23 +00:00
David Majnemer fa90a0b325 InstCombine: Fold X-C1 <u 2 -> (X & -2) == C1
Back in r179493 we determined that two transforms collided with each
other.  The fix back then was to reorder the transforms so that the
preferred transform would give it a try and then we would try the
secondary transform.  However, it was noted that the best approach would
canonicalize one transform into the other, removing the collision and
allowing us to optimize IR given to us in that form.

llvm-svn: 185808
2013-07-08 11:53:08 +00:00
Nadav Rotem 2ee35771a8 Clear the builder insert point between tree-vectorization phases.
llvm-svn: 185777
2013-07-07 14:57:18 +00:00
Nadav Rotem 2041b742d4 SLPVectorizer: Implement DCE as part of vectorization.
This is a complete re-write if the bottom-up vectorization class.
Before this commit we scanned the instruction tree 3 times. First in search of merge points for the trees. Second, for estimating the cost. And finally for vectorization.
There was a lot of code duplication and adding the DCE exposed bugs. The new design is simpler and DCE was a part of the design.
In this implementation we build the tree once. After that we estimate the cost by scanning the different entries in the constructed tree (in any order). The vectorization phase also works on the built tree.

llvm-svn: 185774
2013-07-07 06:57:07 +00:00
Michael Gottesman 618df456e2 [objc-arc] Remove the alias analysis part of r185764.
Upon further reflection, the alias analysis part of r185764 is not a safe
change.

llvm-svn: 185770
2013-07-07 04:18:03 +00:00
Michael Gottesman a72630d453 [objc-arc] Teach the ARC optimizer that objc_sync_enter/objc_sync_exit do not modify the ref count of an objc object and additionally are inert for modref purposes.
llvm-svn: 185769
2013-07-07 01:52:55 +00:00
Michael Gottesman e557da26db [objc-arc] When we initialize ARCRuntimeEntryPoints, make sure we reset all references to entrypoint declarations as well.
llvm-svn: 185764
2013-07-06 18:43:05 +00:00
Benjamin Kramer 3d90a8f4f9 Reassociate: Remove unnecessary default operator=.
llvm-svn: 185757
2013-07-06 15:10:13 +00:00
Michael Gottesman 4d9439c73f [objc-arc] Performed some small cleanups in ARCRuntimeEntryPoints and added an llvm_unreachable after the switch to quiet -Wreturn_type errors.
llvm-svn: 185746
2013-07-06 02:18:56 +00:00
Michael Gottesman 574d521c85 [objc-arc] Renamed Module => TheModule in ARCRuntimeEntryPoints. Also did some small cleanups.
This fixes an issue that came up due to -fpermissive on the bots.

llvm-svn: 185744
2013-07-06 01:57:32 +00:00
Michael Gottesman 01df45056e Removed trailing whitespace.
llvm-svn: 185743
2013-07-06 01:41:35 +00:00
Michael Gottesman 0b912b2673 [objc-arc] Updated ObjCARCContract to use ARCRuntimeEntryPoints.
llvm-svn: 185742
2013-07-06 01:39:26 +00:00
Michael Gottesman 14acfacc48 [objc-arc] Updated ObjCARCOpts to use ARCRuntimeEntryPoints.
llvm-svn: 185741
2013-07-06 01:39:23 +00:00
Michael Gottesman a94186a4c0 [objc-arc] Refactor runtime entrypoint declaration entrypoint creation.
This is the first patch in a series of 3 patches which clean up how we create
runtime function declarations in the ARC optimizer when they do not exist
already in the IR.

Currently we have a bunch of duplicated code in ObjCARCOpts, ObjCARCContract
that does this. This patch refactors that code into a separate class called
ARCRuntimeEntryPoints which lazily creates the declarations for said
entrypoints.

The next two patches will consist of the work of refactoring
ObjCARCContract/ObjCARCOpts to use this new code.

llvm-svn: 185740
2013-07-06 01:39:18 +00:00
Nick Lewycky cff2cf8e3a Fix annotation of unlink. Should fix builder.
llvm-svn: 185738
2013-07-06 00:59:28 +00:00
Nick Lewycky c2ec0725ce Extend 'readonly' and 'readnone' to work on function arguments as well as
functions. Make the function attributes pass add it to known library functions
and when it can deduce it.

llvm-svn: 185735
2013-07-06 00:29:58 +00:00
Rafael Espindola 155cf0f3a6 Use sys::fs::createTemporaryFile.
llvm-svn: 185719
2013-07-05 20:14:52 +00:00
Sylvestre Ledru 751447a3ac Remove a useless declarations (found by scan-build)
llvm-svn: 185709
2013-07-05 15:58:12 +00:00
David Majnemer c2a990bc00 InstCombine: (icmp eq B, 0) | (icmp ult A, B) -> (icmp ule A, B-1)
This transform allows us to turn IR that looks like:
  %1 = icmp eq i64 %b, 0
  %2 = icmp ult i64 %a, %b
  %3 = or i1 %1, %2
  ret i1 %3

into:
  %0 = add i64 %b, -1
  %1 = icmp uge i64 %0, %a
  ret i1 %1

which means we go from lowering:
        cmpq    %rsi, %rdi
        setb    %cl
        testq   %rsi, %rsi
        sete    %al
        orb     %cl, %al
        ret

to lowering:
        decq    %rsi
        cmpq    %rdi, %rsi
        setae   %al
        ret

llvm-svn: 185677
2013-07-05 00:31:17 +00:00
David Majnemer 37f8f445de InstCombine: Reimplementation of visitUDivOperand
This transform was originally added in r185257 but later removed in
r185415.  The original transform would create instructions speculatively
and then discard them if the speculation was proved incorrect.  This has
been replaced with a scheme that splits the transform into two parts:
preflight and fold.  While we preflight, we build up fold actions that
inform the folding stage on how to act.

llvm-svn: 185667
2013-07-04 21:17:49 +00:00
Benjamin Kramer 371722288c SimplifyCFG: Teach switch generation some patterns that instcombine forms.
This allows us to create switches even if instcombine has munged two of the
incombing compares into one and some bit twiddling. This was motivated by enum
compares that are common in clang.

llvm-svn: 185632
2013-07-04 14:22:02 +00:00
Nick Lewycky a833c667e2 Tabs to spaces. No functionality change.
llvm-svn: 185612
2013-07-04 03:51:53 +00:00
Craig Topper af0dea1347 Use SmallVectorImpl::iterator/const_iterator instead of SmallVector to avoid specifying the vector size.
llvm-svn: 185606
2013-07-04 01:31:24 +00:00
Craig Topper 31ee5866de Use SmallVectorImpl::iterator/const_iterator instead of SmallVector to avoid specifying the vector size.
llvm-svn: 185540
2013-07-03 15:07:05 +00:00
Evgeniy Stepanov dc6d7eb860 [msan] Unpoison stack allocations and undef values in blacklisted functions.
This changes behavior of -msan-poison-stack=0 flag from not poisoning stack
allocations to actively unpoisoning them.

llvm-svn: 185538
2013-07-03 14:39:14 +00:00
Michael Gottesman 2db11161a8 Added support in FunctionAttrs for adding relevant function/argument attributes for the posix call gettimeofday.
This implies annotating it as nounwind and its arguments as nocapture. To be
conservative, we do not annotate the arguments with noalias since some platforms
do not have restrict on the declaration for gettimeofday.

llvm-svn: 185502
2013-07-03 04:00:54 +00:00
Manman Ren d0e67aa1ce Debug Info: cleanup
llvm-svn: 185456
2013-07-02 18:37:35 +00:00
Hal Finkel fdbe161b1a Revert r185257 (InstCombine: Be more agressive optimizing 'udiv' instrs with 'select' denoms)
I'm reverting this commit because:

 1. As discussed during review, it needs to be rewritten (to avoid creating and
then deleting instructions).

 2. This is causing optimizer crashes. Specifically, I'm seeing things like
this:

    While deleting: i1 %
    Use still stuck around after Def is destroyed:  <badref> = select i1 <badref>, i32 0, i32 1
    opt: /src/llvm-trunk/lib/IR/Value.cpp:79: virtual llvm::Value::~Value(): Assertion `use_empty() && "Uses remain when a value is destroyed!"' failed.

   I'd guess that these will go away once we're no longer creating/deleting
instructions here, but just in case, I'm adding a regression test.

Because the code is bring rewritten, I've just XFAIL'd the original regression test. Original commit message:

	InstCombine: Be more agressive optimizing 'udiv' instrs with 'select' denoms

	Real world code sometimes has the denominator of a 'udiv' be a
	'select'.  LLVM can handle such cases but only when the 'select'
	operands are symmetric in structure (both select operands are a constant
	power of two or a left shift, etc.).  This falls apart if we are dealt a
	'udiv' where the code is not symetric or if the select operands lead us
	to more select instructions.

	Instead, we should treat the LHS and each select operand as a distinct
	divide operation and try to optimize them independently.  If we can
	to simplify each operation, then we can replace the 'udiv' with, say, a
	'lshr' that has a new select with a bunch of new operands for the
	select.

llvm-svn: 185415
2013-07-02 05:21:11 +00:00
Nick Lewycky 26fcc51f15 Add missing break statements. Noticed by inspection.
llvm-svn: 185414
2013-07-02 05:02:56 +00:00
Manman Ren 74c188f026 Debug Info: clean up usage of Verify.
No functionality change. It should suffice to check the type of a debug info
metadata, instead of calling Verify.

llvm-svn: 185383
2013-07-01 21:02:01 +00:00
Arnold Schwaighofer ef51cf202b LoopVectorize: Math functions only read rounding mode
Math functions are mark as readonly because they read the floating point
rounding mode. Because we don't vectorize loops that would contain function
calls that set the rounding mode it is safe to ignore this memory read.

llvm-svn: 185299
2013-07-01 00:54:44 +00:00
Stephen Lin 2e551adcd9 DeadArgumentElimination: keep return value on functions that have a live argument with the 'returned' attribute (rather than generate invalid IR); however, if both can be eliminated, both will be
llvm-svn: 185290
2013-06-30 20:26:21 +00:00
Benjamin Kramer 4093f29366 InstCombine: Also turn selects fed by an and into arithmetic when the types don't match.
Inserting a zext or trunc is sufficient. This pattern is somewhat common in
LLVM's pointer mangling code.

llvm-svn: 185270
2013-06-29 21:17:04 +00:00
Benjamin Kramer 4ab72f9b9a LoopVectorizer: Pack MemAccessInfo pairs.
llvm-svn: 185263
2013-06-29 17:52:08 +00:00
Benjamin Kramer 53545693d7 Move helper classes into anonymous namespaces.
llvm-svn: 185262
2013-06-29 17:02:06 +00:00
David Majnemer 5953d3712a InstCombine: FoldGEPICmp shouldn't change sign of base pointer comparison
Changing the sign when comparing the base pointer would introduce all
sorts of unexpected things like:
  %gep.i = getelementptr inbounds [1 x i8]* %a, i32 0, i32 0
  %gep2.i = getelementptr inbounds [1 x i8]* %b, i32 0, i32 0
  %cmp.i = icmp ult i8* %gep.i, %gep2.i
  %cmp.i1 = icmp ult [1 x i8]* %a, %b
  %cmp = icmp ne i1 %cmp.i, %cmp.i1
  ret i1 %cmp

into:
  %cmp.i = icmp slt [1 x i8]* %a, %b
  %cmp.i1 = icmp ult [1 x i8]* %a, %b
  %cmp = xor i1 %cmp.i, %cmp.i1
  ret i1 %cmp

By preserving the original sign, we now get:
  ret i1 false

This fixes PR16483.

llvm-svn: 185259
2013-06-29 10:28:04 +00:00
David Majnemer 92a8a7d45a InstCombine: Small whitespace cleanup in FoldGEPICmp
llvm-svn: 185258
2013-06-29 09:45:35 +00:00
David Majnemer 797227eea6 InstCombine: Be more agressive optimizing 'udiv' instrs with 'select' denoms
Real world code sometimes has the denominator of a 'udiv' be a
'select'.  LLVM can handle such cases but only when the 'select'
operands are symmetric in structure (both select operands are a constant
power of two or a left shift, etc.).  This falls apart if we are dealt a
'udiv' where the code is not symetric or if the select operands lead us
to more select instructions.

Instead, we should treat the LHS and each select operand as a distinct
divide operation and try to optimize them independently.  If we can
to simplify each operation, then we can replace the 'udiv' with, say, a
'lshr' that has a new select with a bunch of new operands for the
select.

llvm-svn: 185257
2013-06-29 08:40:07 +00:00
Nadav Rotem 0a25727f31 We preserve the CFG and some of the analysis passes.
llvm-svn: 185251
2013-06-29 05:38:15 +00:00
Nadav Rotem e00343446c Update docs.
llvm-svn: 185250
2013-06-29 05:37:19 +00:00
David Majnemer b889e405eb InstCombine: Optimize (1 << X) Pred CstP2 to X Pred Log2(CstP2)
We may, after other optimizations, find ourselves with IR that looks
like:

  %shl = shl i32 1, %y
  %cmp = icmp ult i32 %shl, 32

Instead, we should just compare the shift count:

  %cmp = icmp ult i32 %y, 5

llvm-svn: 185242
2013-06-28 23:42:03 +00:00
Nadav Rotem 060be733a5 SLP Vectorizer: Add support for trees with external users.
To support this we have to insert 'extractelement' instructions to pick the right lane.
We had this functionality before but I removed it when we moved to the multi-block design because it was too complicated.

llvm-svn: 185230
2013-06-28 22:07:09 +00:00
Nadav Rotem 9ce3fedcdd LoopVectorizer: Refactor the code that checks if it is safe to predicate blocks.
In this code we keep track of pointers that we are allowed to read from, if they are accessed by non-predicated blocks.
We use this list to allow vectorization of conditional loads in predicated blocks because we know that these addresses don't segfault.

llvm-svn: 185214
2013-06-28 20:46:27 +00:00
Daniel Malea b17b1cd6f5 Remove needless include (unistd.h) in DebugIR pass
- should unbreak Windows builds

llvm-svn: 185198
2013-06-28 19:19:44 +00:00
Daniel Malea 0673464a92 Add missing header for DebugIR
- missed svn add...

llvm-svn: 185194
2013-06-28 19:07:59 +00:00
Daniel Malea 31321fa53d Remove limitation on DebugIR that made it require existing debug metadata.
- Build debug metadata for 'bare' Modules using DIBuilder
- DebugIR can be constructed to generate an IR file (to be seen by a debugger)
  or not in cases where the user already has an IR file on disk.

llvm-svn: 185193
2013-06-28 19:05:23 +00:00
Arnold Schwaighofer ce2c766f61 LoopVectorize: Pull dyn_cast into setDebugLocFromInst
llvm-svn: 185168
2013-06-28 17:14:48 +00:00
Arnold Schwaighofer 3b27b992ca LoopVectorize: Use static function instead of DebugLocSetter class
I used the class to safely reset the state of the builder's debug location.  I
think I have caught all places where we need to set the debug location to a new
one. Therefore, we can replace the class by a function that just sets the debug
location.

llvm-svn: 185165
2013-06-28 16:26:54 +00:00
Manman Ren 983a16c08a Debug Info: clean up usage of Verify.
No functionality change.
It should suffice to check the type of a debug info metadata, instead of
calling Verify. For cases where we know the type of a DI metadata, use
assert.

Also update testing cases to make them conform to the format of DI classes.

llvm-svn: 185135
2013-06-28 05:43:10 +00:00
Arnold Schwaighofer 12ecb331af LoopVectorize: Preserve debug location info
radar://14169017

llvm-svn: 185122
2013-06-28 00:38:54 +00:00
Matt Arsenault 5d2e85f6d7 Fix using arg_end() - arg_begin() instead of arg_size()
llvm-svn: 185121
2013-06-28 00:25:40 +00:00
Michael Gottesman 79b0967548 Revert "Revert "[APFloat] Removed APFloat constructor which initialized to either zero/NaN but allowed you to arbitrarily set the category of the float.""
This reverts commit r185099.

Looks like both the ppc-64 and mips bots are still failing after I reverted this
change.

Since:

1. The mips bot always performs a clean build,
2. The ppc64-bot failed again after a clean build (I asked the ppc-64
maintainers to clean the bot which they did... Thanks Will!),

I think it is safe to assume that this change was not the cause of the failures
that said builders were seeing. Thus I am recomitting.

llvm-svn: 185111
2013-06-27 21:58:19 +00:00
Michael Gottesman ccaf3321f1 Revert "[APFloat] Removed APFloat constructor which initialized to either zero/NaN but allowed you to arbitrarily set the category of the float."
This reverts commit r185095. This is causing a FileCheck failure on
the 3dnow intrinsics on at least the mips/ppc bots but not on the x86
bots.

Reverting while I figure out what is going on.

llvm-svn: 185099
2013-06-27 20:40:11 +00:00
Arnold Schwaighofer 38de7cd464 LoopVectorize: Cache edge masks created during if-conversion
Otherwise, we end up with an exponential IR blowup.
Fixes PR16472.

llvm-svn: 185097
2013-06-27 20:31:06 +00:00
Michael Gottesman 03255a1675 [APFloat] Removed APFloat constructor which initialized to either zero/NaN but allowed you to arbitrarily set the category of the float.
The category which an APFloat belongs to should be dependent on the
actual value that the APFloat has, not be arbitrarily passed in by the
user. This will prevent inconsistency bugs where the category and the
actual value in APFloat differ.

I also fixed up all of the references to this constructor (which were
only in LLVM).

llvm-svn: 185095
2013-06-27 19:50:52 +00:00
Arnold Schwaighofer a2dd195fb3 LoopVectorize: Use vectorized loop invariant gep index anchored in loop
Use vectorized instruction instead of original instruction anchored in the
original loop.

Fixes PR16452 and t2075.c of PR16455.

llvm-svn: 185081
2013-06-27 15:11:55 +00:00
Arnold Schwaighofer ccd6c9929b LoopVectorize: Don't store a reversed value in the vectorized value map
When we store values for reversed induction stores we must not store the
reversed value in the vectorized value map. Another instruction might use this
value.

This fixes 3 test cases of PR16455.

llvm-svn: 185051
2013-06-27 00:45:41 +00:00
Michael Gottesman 41748d7c86 Added support for the Builtin attribute.
The Builtin attribute is an attribute that can be placed on function call site that signal that even though a function is declared as being a builtin,

rdar://problem/13727199

llvm-svn: 185049
2013-06-27 00:25:01 +00:00
Nadav Rotem 8edefb3665 No need to use a Set when a vector would do.
llvm-svn: 185047
2013-06-27 00:14:13 +00:00
Nadav Rotem 93f880fb77 SLP: When searching for vectorization opportunities scan the blocks in post-order because we grow chains upwards.
llvm-svn: 185041
2013-06-26 23:44:45 +00:00
Nadav Rotem 7f0d6d7975 SLP: Dont erase instructions during vectorization because it prevents the outerloops from iterating over the instructions.
llvm-svn: 185040
2013-06-26 23:43:23 +00:00
Michael Gottesman c2af8d6273 In InstCombine{AddSub,MulDivRem} convert APFloat.isFiniteNonZero() && !APFloat.isDenormal => APFloat.isNormal.
llvm-svn: 185037
2013-06-26 23:17:31 +00:00
Eric Christopher b8c608ea39 Revert "Debug Info: clean up usage of Verify." as it's breaking bots.
This reverts commit r185020

llvm-svn: 185032
2013-06-26 22:44:57 +00:00
Manman Ren aa00ce0e8f Debug Info: clean up usage of Verify.
No functionality change.
It should suffice to check the type of a debug info metadata, instead of
calling Verify.

llvm-svn: 185020
2013-06-26 21:26:10 +00:00
Nadav Rotem 4c5b2d1de6 Erase all of the instructions that we RAUWed
llvm-svn: 184969
2013-06-26 17:16:09 +00:00
Nadav Rotem f4ca3994b8 Do not add cse-ed instructions into the visited map because we dont want to consider them as a candidate for replacement of instructions to be visited.
llvm-svn: 184966
2013-06-26 16:54:53 +00:00
Kostya Serebryany 5e276f9dbc [asan] workaround for PR16277: don't instrument AllocaInstr with alignment more than the redzone size
llvm-svn: 184928
2013-06-26 09:49:52 +00:00
Kostya Serebryany 9f5213f20f [asan] add option -asan-keep-uninstrumented-functions
llvm-svn: 184927
2013-06-26 09:18:17 +00:00
Nick Lewycky 5cd9538b90 dbgs() << Instruction doesn't print a newline on the end any more. Update these
debug statements to add a missing newline. Also canonicalize to '\n' instead of
"\n"; the latter calls a function with a loop the former does not.

llvm-svn: 184897
2013-06-26 00:30:18 +00:00
Nadav Rotem 0794acc1da SLPVectorizer: support slp-vectorization of PHINodes between basic blocks
llvm-svn: 184888
2013-06-25 23:04:09 +00:00
Bob Wilson acfc01dedf Fix SROA to avoid unnecessary scalar conversions for 1-element vectors.
When a 1-element vector alloca is promoted, a store instruction can often be
rewritten without converting the value to a scalar and using an insertelement
instruction to stuff it into the new alloca.  This patch just adds a check
to skip that conversion when it is unnecessary.  This turns out to be really
important for some ARM Neon operations where <1 x i64> is used to get around
the fact that i64 is not a legal type.

llvm-svn: 184870
2013-06-25 19:09:50 +00:00
Nadav Rotem 3de032a3b6 Fix a typo in the code that collected the costs recursively.
llvm-svn: 184827
2013-06-25 05:30:56 +00:00
Nadav Rotem 9c7c997a7e Rename the variable to fix a warning. Thanks Andy Gibbs.
llvm-svn: 184749
2013-06-24 15:59:47 +00:00
Arnold Schwaighofer b252c11ccc Reapply 184685 after the SetVector iteration order fix.
This should hopefully have fixed the stage2/stage3 miscompare on the dragonegg
testers.

"LoopVectorize: Use the dependence test utility class

We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

  for (i = 8; i < 256; ++i) {
    a[i] = a[i+4] * a[i+8];
  }

  for (i = 8; i < 256; ++i) {
    a[i] = a[i-4] * a[i-8];
  }

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598"

llvm-svn: 184724
2013-06-24 12:09:15 +00:00
Arnold Schwaighofer 91472fa4fc LoopVectorize: Use SetVector for the access set
We are creating the runtime checks using this set so we need a deterministic
iteration order.

llvm-svn: 184723
2013-06-24 12:09:12 +00:00
Chandler Carruth 08e1b8742b Add a flag to defer vectorization into a phase after the inliner and its
CGSCC pass manager. This should insulate the inlining decisions from the
vectorization decisions, however it may have both compile time and code
size problems so it is just an experimental option right now.

Adding this based on a discussion with Arnold and it seems at least
worth having this flag for us to both run some experiments to see if
this strategy is workable. It may solve some of the regressions seen
with the loop vectorizer.

llvm-svn: 184698
2013-06-24 07:21:47 +00:00
Arnold Schwaighofer 58ca945f38 Revert "LoopVectorize: Use the dependence test utility class"
This reverts commit cbfa1ca993363ca5c4dbf6c913abc957c584cbac.

We are seeing a stage2 and stage3 miscompare on some dragonegg bots.

llvm-svn: 184690
2013-06-24 06:10:41 +00:00
Arnold Schwaighofer b914a7e2ef LoopVectorize: Use the dependence test utility class
We now no longer need alias analysis - the cases that alias analysis would
handle are now handled as accesses with a large dependence distance.

We can now vectorize loops with simple constant dependence distances.

  for (i = 8; i < 256; ++i) {
    a[i] = a[i+4] * a[i+8];
  }

  for (i = 8; i < 256; ++i) {
    a[i] = a[i-4] * a[i-8];
  }

We would be able to vectorize about 200 more loops (in many cases the cost model
instructs us no to) in the test suite now. Results on x86-64 are a wash.

I have seen one degradation in ammp. Interestingly, the function in which we
now vectorize a loop is never executed so we probably see some instruction
cache effects. There is a 2% improvement in h264ref. There is one or the other
TSCV loop kernel that speeds up.

radar://13681598

llvm-svn: 184685
2013-06-24 03:55:48 +00:00
Arnold Schwaighofer d517976758 LoopVectorize: Add utility class for checking dependency among accesses
This class checks dependences by subtracting two Scalar Evolution access
functions allowing us to catch very simple linear dependences.

The checker assumes source order in determining whether vectorization is safe.
We currently don't reorder accesses.
Positive true dependencies need to be a multiple of VF otherwise we impede
store-load forwarding.

llvm-svn: 184684
2013-06-24 03:55:45 +00:00
Arnold Schwaighofer d57419696d LoopVectorize: Add utility class for building sets of dependent accesses
Sets of dependent accesses are built by unioning sets based on underlying
objects. This class will be used by the upcoming dependence checker.

llvm-svn: 184683
2013-06-24 03:55:44 +00:00
Nadav Rotem 210e86d7c4 SLP Vectorizer: Add support for vectorizing parts of the tree.
Untill now we detected the vectorizable tree and evaluated the cost of the
entire tree.  With this patch we can decide to trim-out branches of the tree
that are not profitable to vectorizer.

Also, increase the max depth from 6 to 12. In the worse possible case where all
of the code is made of diamond-shaped graph this can bring the cost to 2**10,
but diamonds are not very common.

llvm-svn: 184681
2013-06-24 02:52:43 +00:00