Commit Graph

9803 Commits

Author SHA1 Message Date
Bill Wendling c79e42c5ce Change 'AttrVal' to 'AttrKind' to better reflect that it's a kind of attribute instead of the value of the attribute.
llvm-svn: 170972
2012-12-22 00:37:52 +00:00
Roman Divacky a229186a82 Remove duplicate includes.
llvm-svn: 170902
2012-12-21 17:06:44 +00:00
Evgeniy Stepanov 4fbc0d08bf [msan] Remove unreachable blocks before instrumenting a function.
llvm-svn: 170883
2012-12-21 11:18:49 +00:00
Nadav Rotem 3b850b70b3 Enable if-conversion.
llvm-svn: 170841
2012-12-21 04:47:54 +00:00
Evan Cheng 99cafb1db2 Every pass deserves a name, even codegenprep.
llvm-svn: 170831
2012-12-21 01:48:14 +00:00
Nadav Rotem a4b53f20a3 BB-Vectorizer: Check the cost of the store pointer type
and not the return type, which is void. A number of test
cases fail after adding the assertion in TTImpl.

llvm-svn: 170828
2012-12-21 01:24:36 +00:00
Nadav Rotem e7785686a5 Fix a bug in the code that checks if we can vectorize loops while using dynamic
memory bound checks.  Before the fix we were able to vectorize this loop from
the Livermore Loops benchmark:

for ( k=1 ; k<n ; k++ )
  x[k] = x[k-1] + y[k];

llvm-svn: 170811
2012-12-21 00:07:35 +00:00
Nadav Rotem 2ababf68d7 LoopVectorize: Fix a bug in the scalarization of instructions.
Before if-conversion we could check if a value is loop invariant
if it was declared inside the basic block. Now that loops have
multiple blocks this check is incorrect.

This fixes External/SPEC/CINT95/099_go/099_go

llvm-svn: 170756
2012-12-20 20:24:40 +00:00
Nadav Rotem 8b20c0a814 Loop Vectorizer: turn-off if-conversion.
llvm-svn: 170708
2012-12-20 17:42:53 +00:00
James Molloy 4f6fb953a7 Add a new attribute, 'noduplicate'. If a function contains a noduplicate call, the call cannot be duplicated - Jump threading, loop unrolling, loop unswitching, and loop rotation are inhibited if they would duplicate the call.
Similarly inlining of the function is inhibited, if that would duplicate the call (in particular inlining is still allowed when there is only one callsite and the function has internal linkage).

llvm-svn: 170704
2012-12-20 16:04:27 +00:00
Craig Topper ae48cb2e5a Formatting fixes. Remove some unnecessary 'else' after 'return'. No functional change.
llvm-svn: 170676
2012-12-20 07:15:54 +00:00
Craig Topper 9d4171afed Removing trailing whitespace
llvm-svn: 170675
2012-12-20 07:09:41 +00:00
Nadav Rotem 7bdc45b570 Loop Vectorizer: Enable if-conversion.
llvm-svn: 170632
2012-12-20 02:00:02 +00:00
Nadav Rotem 28408a20c9 whitespace
llvm-svn: 170626
2012-12-20 00:49:56 +00:00
Paul Redmond 5917f4c715 Transform (x&C)>V into (x&C)!=0 where possible
When the least bit of C is greater than V, (x&C) must be greater than V
if it is not zero, so the comparison can be simplified.

Although this was suggested in Target/X86/README.txt, it benefits any
architecture with a directly testable form of AND.

Patch by Kevin Schoedel

llvm-svn: 170576
2012-12-19 19:47:13 +00:00
Evgeniy Stepanov abeae5c7d5 [msan] Add track-origins argument to the pass constructor.
llvm-svn: 170544
2012-12-19 13:55:51 +00:00
Evgeniy Stepanov d7571cd4bc [msan] Heuristically instrument unknown intrinsics.
This changes adds shadow and origin propagation for unknown intrinsics
by examining the arguments and ModRef behaviour. For now, only 3 classes
of intrinsics are handled:
- those that look like simple SIMD store
- those that look like simple SIMD load
- those that don't have memory effects and look like arithmetic/logic/whatever
  operation on simple types.

llvm-svn: 170530
2012-12-19 11:22:04 +00:00
Benjamin Kramer e300004bd5 LoopVectorize: Make iteration over induction variables not depend on pointer values.
MapVector is a bit heavyweight, but I don't see a simpler way. Also the
InductionList is unlikely to be large. This should help 3-stage selfhost
compares (PR14647).

llvm-svn: 170528
2012-12-19 11:09:15 +00:00
Bill Wendling d97b75d816 Inline the 'hasIncompatibleWithVarArgsAttrs' method into its only uses. And some minor comment reformatting.
llvm-svn: 170516
2012-12-19 08:57:40 +00:00
Bill Wendling 3d7b0b8ac7 Rename the 'Attributes' class to 'Attribute'. It's going to represent a single attribute in the future.
llvm-svn: 170502
2012-12-19 07:18:57 +00:00
Shuxin Yang 5b841c4a64 Make sure the buffer, which containas an instance of APFloat, has proper alignment.
llvm-svn: 170486
2012-12-19 01:10:17 +00:00
Shuxin Yang 37a1efe1c6 rdar://12801297
InstCombine for unsafe floating-point add/sub.

llvm-svn: 170471
2012-12-18 23:10:12 +00:00
Nadav Rotem 9aee065e3c Enable the loop vectorizer in clang and not in the pass manager, so that we can disable it in clang.
llvm-svn: 170470
2012-12-18 23:09:44 +00:00
Benjamin Kramer f0e5d2f032 LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops instead of scalar operations.
For example on x86 with SSE4.2 a <8 x i8> add reduction becomes
	movdqa	%xmm0, %xmm1
	movhlps	%xmm1, %xmm1            ## xmm1 = xmm1[1,1]
	paddw	%xmm0, %xmm1
	pshufd	$1, %xmm1, %xmm0        ## xmm0 = xmm1[1,0,0,0]
	paddw	%xmm1, %xmm0
	phaddw	%xmm0, %xmm0
	pextrb	$0, %xmm0, %edx

instead of
	pextrb	$2, %xmm0, %esi
	pextrb	$0, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$4, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$6, %xmm0, %edx
	addb	%sil, %dl
	pextrb	$8, %xmm0, %esi
	addb	%dl, %sil
	pextrb	$10, %xmm0, %edi
	pextrb	$14, %xmm0, %edx
	addb	%sil, %dil
	pextrb	$12, %xmm0, %esi
	addb	%dil, %sil
	addb	%sil, %dl

llvm-svn: 170439
2012-12-18 18:40:20 +00:00
Nadav Rotem c0699854dd Enable the loop vectorizer.
llvm-svn: 170416
2012-12-18 06:37:12 +00:00
Nadav Rotem a5024fc3e1 SROA: Replace calls to getScalarSizeInBits to DataLayout's API because
getScalarSizeInBits could not handle vectors of pointers.

llvm-svn: 170412
2012-12-18 05:23:31 +00:00
Rafael Espindola 46b9c8a2cd Initialize NoRedZone and remove unused default values.
llvm-svn: 170404
2012-12-18 03:35:05 +00:00
Chandler Carruth e3f4119b06 Fix another SROA crasher, PR14601.
This was a silly oversight, we weren't pruning allocas which were used
by variable-length memory intrinsics from the set that could be widened
and promoted as integers. Fix that.

llvm-svn: 170353
2012-12-17 18:48:07 +00:00
Evgeniy Stepanov 88b8dceddf [msan] Fix lint warning.
llvm-svn: 170347
2012-12-17 16:30:05 +00:00
Chandler Carruth 21eb4e96c2 Teach the rewriting of memcpy calls to support subvector copies.
This also cleans up a bit of the memcpy call rewriting by sinking some
irrelevant code further down and making the call-emitting code a bit
more concrete.

Previously, memcpy of a subvector would actually miscompile (!!!) the
copy into a single vector element copy. I have no idea how this ever
worked. =/ This is the memcpy half of PR14478 which we probably weren't
noticing previously because it didn't actually assert.

The rewrite relies on the newly refactored insert- and extractVector
functions to do the heavy lifting, and those are the same as used for
loads and stores which makes the test coverage a bit more meaningful
here.

llvm-svn: 170338
2012-12-17 14:51:24 +00:00
Evgeniy Stepanov 95a80abead Optimize tree walking in markAliveBlocks.
Check whether a BB is known as reachable before adding it to the worklist.
This way BB's with multiple predecessors are added to the list no more than
once.

llvm-svn: 170335
2012-12-17 14:28:00 +00:00
Chandler Carruth cacda256a1 Fix a secondary bug I introduced while fixing the first part of PR14478.
The first half of fixing this bug was actually in r170328, but was
entirely coincidental. It did however get me to realize the nature of
the bug, and adapt the test case to test more interesting behavior. In
turn, that uncovered the rest of the bug which I've fixed here.

This should fix two new asserts that showed up in the vectorize nightly
tester.

llvm-svn: 170333
2012-12-17 14:03:01 +00:00
Chandler Carruth 95e1fb8a42 Hoist a convertValue call to the two paths where it is needed.
I noticed this while looking at r170328. We only ever do a vector
rewrite when the alloca *is* the vector type, so it's good to not paper
over bugs here by doing a convertValue that isn't needed.

llvm-svn: 170331
2012-12-17 13:51:03 +00:00
Chandler Carruth ce4562bdcb Hoist the insertVector helper to be a static helper.
This will allow its use inside of memcpy rewriting as well. This routine
is more complex than extractVector, and some of its uses are not 100%
where I want them to be so there is still some work to do here.

While this can technically change the output in some cases, it shouldn't
be a change that matters -- IE, it can leave some dead code lying around
that prior versions did not, etc.

Yet another step in the refactorings leading up to the solution to the
last component of PR14478.

llvm-svn: 170328
2012-12-17 13:41:21 +00:00
Chandler Carruth b6bc8749e8 Lift the extractVector helper all the way out to a static helper function.
The method helpers all implicitly act upon the alloca, and what we
really want is a fully generic helper. Doing memcpy rewrites is more
special than all other rewrites because we are at times rewriting
instructions which touch pointers *other* than the alloca. As
a consequence all of the helpers needed by memcpy rewriting of
sub-vector copies will need to be generalized fully.

Note that all of these helpers ({insert,extract}{Integer,Vector}) are
woefully uncommented. I'm going to go back through and document them
once I get the factoring correct.

No functionality changed.

llvm-svn: 170325
2012-12-17 13:07:30 +00:00
Chandler Carruth 769445ef03 Factor the vector load rewriting into a more generic form.
This makes it suitable for use in rewriting memcpy in the presence of
subvector memcpy intrinsics.

No functionality changed.

llvm-svn: 170324
2012-12-17 12:50:21 +00:00
Chandler Carruth ccca504f3a Fix the first part of PR14478: memset now works.
PR14478 highlights a serious problem in SROA that simply wasn't being
exercised due to a lack of vector input code mixed with C-library
function calls. Part of SROA was written carefully to handle subvector
accesses via memset and memcpy, but the rewriter never grew support for
this. Fixing it required refactoring the subvector access code in other
parts of SROA so it could be shared, and then fixing the splat formation
logic and using subvector insertion (this patch).

The PR isn't quite fixed yet, as memcpy is still broken in the same way.
I'm starting on that series of patches now.

Hopefully this will be enough to bring the bullet benchmark back to life
with the bb-vectorizer enabled, but that may require fixing memcpy as
well.

llvm-svn: 170301
2012-12-17 04:07:37 +00:00
Chandler Carruth eae65a5629 Extract the logic for inserting a subvector into a vector alloca.
No functionality changed. Another step of refactoring toward solving
PR14487.

llvm-svn: 170300
2012-12-17 04:07:35 +00:00
Chandler Carruth 514f34f9c4 Lift the integer splat computation into a helper function.
No functionality changed. Refactoring leading up to the fix for PR14478
which requires some significant changes to the memset and memcpy
rewriting.

llvm-svn: 170299
2012-12-17 04:07:30 +00:00
Chandler Carruth 067edd342f Relax an overly aggressive assert to fix PR14572.
The alloca width is based on the alloc size, not the type size.

llvm-svn: 170270
2012-12-15 09:26:06 +00:00
NAKAMURA Takumi 8f45b6c709 Revert r170246, "Enable the loop vectorizer by default."
llvm-svn: 170267
2012-12-15 06:11:13 +00:00
Michael Ilseman e2754dc887 Add back FoldOpIntoPhi optimizations with fix. Included test cases to help catch these errors and to test the presence of the optimization itself
llvm-svn: 170248
2012-12-14 22:08:26 +00:00
Nadav Rotem acde77481d Enable the loop vectorizer by default.
llvm-svn: 170246
2012-12-14 21:30:23 +00:00
Shuxin Yang f8e9a5a061 rdar://12753946
Implement rule : "x * (select cond 1.0, 0.0) -> select cond x, 0.0"

llvm-svn: 170226
2012-12-14 18:46:06 +00:00
Evgeniy Stepanov 9b72e991c6 Fix lint warnings in MemorySanitizer.cpp.
llvm-svn: 170203
2012-12-14 13:48:31 +00:00
Evgeniy Stepanov 49175b237d [msan] Origin stores and loads do not need explicit alignment.
Origin address is always 4 byte aligned, and the access type is always i32.

llvm-svn: 170199
2012-12-14 13:43:11 +00:00
Evgeniy Stepanov f18e3af11f [msan] Refactor default shadow propagation and origin tracking.
This change moves the code for default shadow propagaition (handleShadowOr)
and origin tracking (setOriginForNaryOp) into a new builder-like class. Also
gets rid of handleShadowOrBinary.

llvm-svn: 170192
2012-12-14 12:54:18 +00:00
Nadav Rotem d3a3c9fdd5 revert r170166 - disable the loop vectorizer.
llvm-svn: 170172
2012-12-14 01:57:00 +00:00
Nadav Rotem 3b606d6fd5 Enable the loop vectorizer.
llvm-svn: 170166
2012-12-14 00:30:34 +00:00
Nadav Rotem b4ea4b3751 Disable the loop vectorizer.
llvm-svn: 170162
2012-12-14 00:02:07 +00:00
Nadav Rotem e5e28b48c8 Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion by default. I plan to revert this patch later today.
llvm-svn: 170157
2012-12-13 23:11:54 +00:00
NAKAMURA Takumi 38d2b2442f Revert r170020, "Simplify negated bit test", for now.
This assumes (1 << n) is always not zero. Consider n is greater than word size.
Although I know it is undefined, this transforms undefined behavior hidden.

This led clang unexpected behavior with some failures. I will investigate to fix undefined shl in clang.

llvm-svn: 170128
2012-12-13 14:28:16 +00:00
Eric Christopher a1bbeeca72 Revert "Restore the PHI optimization I accidently removed" temporarily since
it seems to be breaking self-host for a few people and is PR14592.

This reverts commit r170024.

llvm-svn: 170106
2012-12-13 06:48:05 +00:00
Rafael Espindola a2c107e661 Missed these calls from the previous rename somehow.
llvm-svn: 170094
2012-12-13 03:42:31 +00:00
Rafael Espindola 319f74cd11 Rename isPowerOfTwo to isKnownToBeAPowerOfTwo.
In a previous thread it was pointed out that isPowerOfTwo is not a very precise
name since it can return false for powers of two if it is unable to show that
they are powers of two.

llvm-svn: 170093
2012-12-13 03:37:24 +00:00
Michael Ilseman 536cc32ba0 Pattern matching code for intrinsics.
Provides m_Argument that allows matching against a CallSite's specified argument. Provides m_Intrinsic pattern that can be templatized over the intrinsic id and bind/match arguments similarly to other pattern matchers. Implementations provided for 0 to 4 arguments, though it's very simple to extend for more. Also provides example template specialization for bswap (m_BSwap) and example of code cleanup for its use.

llvm-svn: 170091
2012-12-13 03:13:36 +00:00
Quentin Colombet c0dba2035a Take into account minimize size attribute in the inliner.
Better controls the inlining of functions when the caller function has MinSize attribute.
Basically, when the caller function has this attribute, we do not "force" the inlining
of callee functions carrying the InlineHint attribute (i.e., functions defined with
inline keyword)

llvm-svn: 170065
2012-12-13 01:05:25 +00:00
Nadav Rotem 36510f7194 Teach the cost model about the optimization in r169904: Truncation of induction variables costs the same as scalar trunc.
llvm-svn: 170051
2012-12-13 00:21:03 +00:00
Chad Rosier e28ae30a8e Typo.
llvm-svn: 170050
2012-12-13 00:18:46 +00:00
Michael Ilseman 3c814128cd Restore the PHI optimization I accidently removed
llvm-svn: 170024
2012-12-12 20:59:36 +00:00
Michael Ilseman 9fc0f258fa Remove trailing whitespace
llvm-svn: 170022
2012-12-12 20:57:53 +00:00
David Majnemer 5226aa94ce Simplify negated bit test
llvm-svn: 170020
2012-12-12 20:48:54 +00:00
Nadav Rotem 6027bdf898 Fix indentation.
llvm-svn: 170005
2012-12-12 19:39:36 +00:00
Nadav Rotem d0bb22bba3 LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to increase the function size.
llvm-svn: 170004
2012-12-12 19:29:45 +00:00
Rafael Espindola e40238069e The TargetData is not used for the isPowerOfTwo determination. It has never
been used in the first place.  It simply was passed to the function and to the
recursive invocations.  Simply drop the parameter and update the callers for the
new signature.

Patch by Saleem Abdulrasool!

llvm-svn: 169988
2012-12-12 16:52:40 +00:00
Alexey Samsonov 3d43b63a6e Improve debug info generated with enabled AddressSanitizer.
When ASan replaces <alloca instruction> with
<offset into a common large alloca>, it should also patch
llvm.dbg.declare calls and replace debug info descriptors to mark
that we've replaced alloca with a value that stores an address
of the user variable, not the user variable itself.

See PR11818 for more context.

llvm-svn: 169984
2012-12-12 14:31:53 +00:00
Nadav Rotem 6798a04b15 Fix the ascii drawing that was ruined when I split the H and CPP
llvm-svn: 169955
2012-12-12 01:33:47 +00:00
Nadav Rotem 4fa2e3d5af fix a typo.
llvm-svn: 169953
2012-12-12 01:31:10 +00:00
Nadav Rotem aeb17df802 LoopVectorizer: When -Os is used, vectorize only loops that dont require a tail loop. There is no testcase because I dont know of a way to initialize the loop vectorizer pass without adding an additional hidden flag.
llvm-svn: 169950
2012-12-12 01:11:46 +00:00
Shuxin Yang 81b3678564 - Fix a problematic way in creating all-the-1 APInt.
- Propagate "exact" bit of [l|a]shr instruction.

llvm-svn: 169942
2012-12-12 00:29:03 +00:00
Michael Ilseman d5787be5ba Remove redunant optimizations from InstCombine, instead call the appropriate functions from SimplifyInstruction
llvm-svn: 169941
2012-12-12 00:28:32 +00:00
Nadav Rotem f707bf4ca3 PR14574. Fix a bug in the code that calculates the mask the converted PHIs in if-conversion.
llvm-svn: 169916
2012-12-11 21:30:14 +00:00
Nadav Rotem e266efb70b Loop Vectorize: optimize the vectorization of trunc(induction_var). The truncation is now done on scalars.
llvm-svn: 169904
2012-12-11 18:58:10 +00:00
Rafael Espindola a92da5b34f Use an ArrayRef instead of a std::vector&.
llvm-svn: 169881
2012-12-11 16:36:02 +00:00
Evgeniy Stepanov d2bd319adc [msan] Use explicitely aligned stores and loads with function argument shadow.
Use explicitely aligned store and load instructions to deal with argument and
retval shadow. This matters when an argument's alignment is higher than
__msan_param_tls alignment (which is the case with __m128i).

llvm-svn: 169859
2012-12-11 12:34:09 +00:00
Patrik Hagglund e98b7a0389 Revert EVT->MVT changes, r169836-169851, due to buildbot failures.
llvm-svn: 169854
2012-12-11 11:14:33 +00:00
Patrik Hagglund cbc9d4d0f9 Change TargetLowering::getLoadExtAction to take an MVT, instead of EVT.
llvm-svn: 169840
2012-12-11 09:39:09 +00:00
Nadav Rotem dbb3328194 Fix PR14565. Don't if-convert loops that have switch statements in them.
llvm-svn: 169813
2012-12-11 04:55:10 +00:00
Nadav Rotem 36cdd82627 Enable the loop vectorizer only on O2 and above. (Still disabled by default)
llvm-svn: 169774
2012-12-10 21:45:01 +00:00
Nadav Rotem 07df5ac1a1 Split the LoopVectorizer into H and CPP.
llvm-svn: 169771
2012-12-10 21:39:02 +00:00
Bill Wendling 74f334e476 Don't use a red zone for code coverage if the user specified `-mno-red-zone'.
The `-mno-red-zone' flag wasn't being propagated to the functions that code
coverage generates. This allowed some of them to use the red zone when that
wasn't allowed.
<rdar://problem/12843084>

llvm-svn: 169754
2012-12-10 19:46:49 +00:00
Nadav Rotem 7b5b55c195 Add support for reverse induction variables. For example:
while (i--)
 sum+=A[i];

llvm-svn: 169752
2012-12-10 19:25:06 +00:00
Chandler Carruth e41e7b7901 Add a new visitor for walking the uses of a pointer value.
This visitor provides infrastructure for recursively traversing the
use-graph of a pointer-producing instruction like an alloca or a malloc.
It maintains a worklist of uses to visit, so it can handle very deep
recursions. It automatically looks through instructions which simply
translate one pointer to another (bitcasts and GEPs). It tracks the
offset relative to the original pointer as long as that offset remains
constant and exposes it during the visit as an APInt offset. Finally, it
performs conservative escape analysis.

However, currently it has some limitations that should be addressed
going forward:
1) It doesn't handle vectors of pointers.
2) It doesn't provide a cheaper visitor when the constant offset
   tracking isn't needed.
3) It doesn't support non-instruction pointer values.

The current functionality is exactly what is required to implement the
SROA pointer-use visitors in terms of this one, rather than in terms of
their own ad-hoc base visitor, which was always very poorly specified.
SROA has been converted to use this, and the code there deleted which
this utility now provides.

Technically speaking, using this new visitor allows SROA to handle a few
more cases than it previously did. It is now more aggressive in ignoring
chains of instructions which look like they would defeat SROA, but in
fact do not because they never result in a read or write of memory.
While this is "neat", it shouldn't be interesting for real programs as
any such chains should have been removed by others passes long before we
get to SROA. As a consequence, I've not added any tests for these
features -- it shouldn't be part of SROA's contract to perform such
heroics.

The goal is to extend the functionality of this visitor going forward,
and re-use it from passes like ASan that can benefit from doing
a detailed walk of the uses of a pointer.

Thanks to Ben Kramer for the code review rounds and lots of help
reviewing and debugging this patch.

llvm-svn: 169728
2012-12-10 08:28:39 +00:00
Chandler Carruth e45f4658a3 Fix PR14548: SROA was crashing on a mixture of i1 and i8 loads and stores.
When SROA was evaluating a mixture of i1 and i8 loads and stores, in
just a particular case, it would tickle a latent bug where we compared
bits to bytes rather than bits to bits. As a consequence of the latent
bug, we would allow integers through which were not byte-size multiples,
a situation the later rewriting code was never intended to handle.

In release builds this could trigger all manner of oddities, but the
reported issue in PR14548 was forming invalid bitcast instructions.

The only downside of this fix is that it makes it more clear that SROA
in its current form is not capable of handling mixed i1 and i8 loads and
stores. Sometimes with the previous code this would work by luck, but
usually it would crash, so I'm not terribly worried. I'll watch the LNT
numbers just to be sure.

llvm-svn: 169719
2012-12-10 00:54:45 +00:00
Paul Redmond 2adb13c100 LoopVectorize: support vectorizing intrinsic calls
- added function to VectorTargetTransformInfo to query cost of intrinsics
- vectorize trivially vectorizable intrinsic calls such as sin, cos, log, etc.

Reviewed by: Nadav

llvm-svn: 169711
2012-12-09 20:42:17 +00:00
Paul Redmond f7cd6b391a test commit.
llvm-svn: 169709
2012-12-09 19:46:31 +00:00
Jakub Staszak 8432185ec2 Use m_OneUse pattern instead of hasOneUse() method.
No functionality change.

llvm-svn: 169703
2012-12-09 16:06:44 +00:00
Jakub Staszak 538e3861e3 Remove trailing spaces.
llvm-svn: 169701
2012-12-09 15:37:46 +00:00
Chandler Carruth 93ff2447ec Switch SROA to pop Uses off the back of its visitors' queues.
This will more closely match the behavior of the new PtrUseVisitor that
I am adding. Hopefully this will not change the actual behavior in any
way, but by making the processing order more similar help in debugging.

llvm-svn: 169697
2012-12-09 11:56:01 +00:00
Shuxin Yang 95de7c37e2 - Re-enable population count loop idiom recognization
- fix a bug which cause sigfault.
- add two testing cases which was causing crash

llvm-svn: 169687
2012-12-09 03:12:46 +00:00
Chandler Carruth 91e47532fe Revert the patches adding a popcount loop idiom recognition pass.
There are still bugs in this pass, as well as other issues that are
being worked on, but the bugs are crashers that occur pretty easily in
the wild. Test cases have been sent to the original commit's review
thread.

This reverts the commits:
  r169671: Fix a logic error.
  r169604: Move the popcnt tests to an X86 subdirectory.
  r168931: Initial commit adding the pass.

llvm-svn: 169683
2012-12-08 22:18:29 +00:00
Shuxin Yang 9c5c97647f Fix an inadvertent typo error.
llvm-svn: 169671
2012-12-08 05:00:59 +00:00
Bill Wendling e94d843e43 s/AttrListPtr/AttributeSet/g to better label what this class is going to be in the near future.
llvm-svn: 169651
2012-12-07 23:16:57 +00:00
Evgeniy Stepanov 383b61e791 [msan] Remove readonly/readnone attributes from all called functions.
MSan uses a TLS slot to pass shadow for function arguments and return values.
This makes all instrumented functions not readonly, and at the same time
requires that all callees of an instrumented function that may be
MSan-instrumented do not have readonly attribute (otherwise some of the
instrumentation may be optimized out).

llvm-svn: 169591
2012-12-07 09:08:32 +00:00
Jakub Staszak 772b893e5d Remove unused field.
llvm-svn: 169551
2012-12-06 22:08:59 +00:00
Jakub Staszak 9525a77bf5 Remove trailing spaces.
llvm-svn: 169550
2012-12-06 21:57:16 +00:00
NAKAMURA Takumi e0b1b4645a MemorySanitizer.cpp: Suppress a warning. [-Wunused-variable]
llvm-svn: 169504
2012-12-06 13:38:00 +00:00
Evgeniy Stepanov 47ac9ba9cc [msan] Fix a typo in a comment.
llvm-svn: 169491
2012-12-06 11:58:59 +00:00
Evgeniy Stepanov 4f220d96c5 [msan] Do not store origin for clean values.
Instead of unconditionally storing origin with every application store,
only do this when the shadow of the stored value is != 0.

This change also delays instrumentation of stores until after the walk over
function's instructions, because adding new basic blocks confuses InstVisitor.

We only keep 1 origin value per 4 bytes of application memory. This change
fixes the bug when a store of a single clean byte wiped the origin for the
whole 4-byte area.

Since stores of uninitialized values are relatively uncommon, this change
improves performance of track-origins mode by 5% median and by up to 47% on
specs.

llvm-svn: 169490
2012-12-06 11:41:03 +00:00
Bill Wendling ab417b644c Set the 'MadeChange' variable if we are deleting blocks.
llvm-svn: 169455
2012-12-06 00:30:20 +00:00
Evgeniy Stepanov 8b51bab495 [msan] Instrument bswap intrinsic.
llvm-svn: 169383
2012-12-05 14:39:55 +00:00
Evgeniy Stepanov 94b257df3c [msan] Initialize callbacks in runOnFunction as opposed to doInitialization.
This mirrors the change in ASan & TSan done in r168864.

llvm-svn: 169378
2012-12-05 13:14:33 +00:00
Evgeniy Stepanov 474cb3b3b5 [msan] Change linkage type of __msan_track_origins.
LinkOnceODRLinkage globals may be removed in GlobalOpt if not used in the
current module.

llvm-svn: 169377
2012-12-05 12:49:41 +00:00
Nadav Rotem a8f026e2d4 LoopVectorizer: Increase the number of pointers that can be tested at runtime. If we cant prove statically that the pointers are disjoint then we add the runtime check.
llvm-svn: 169334
2012-12-04 23:25:24 +00:00
Nadav Rotem 87fc988c5d Enable if-conversion during vectorization.
llvm-svn: 169331
2012-12-04 22:59:52 +00:00
Nadav Rotem 93fa5ef957 Fix a bug in vectorization of if-converted reduction variables. If the
reduction variable is not used outside the loop then we ran into an
endless loop. This change checks if we found the original PHI.

llvm-svn: 169324
2012-12-04 22:40:22 +00:00
Shuxin Yang 73285933c9 For rdar://12329730, last piece.
This change attempts to simplify (X^Y) -> X or Y in the user's context if we know that
only bits from X or Y are demanded.

  A minimized case is provided bellow. This change will simplify "t>>16" into "var1 >>16".

  =============================================================
  unsigned foo (unsigned val1, unsigned val2) {
    unsigned t = val1 ^ 1234;
    return (t >> 16) | t; // NOTE: t is used more than once.
  }
  =============================================================

  Note that if the "t" were used only once, the expression would be finally optimized as well.
However, with with this change, the optimization will take place earlier.

  Reviewed by Nadav, Thanks a lot!

llvm-svn: 169317
2012-12-04 22:15:32 +00:00
Nadav Rotem a10b311aec Add support for reduction variables when IF-conversion is enabled.
llvm-svn: 169288
2012-12-04 18:17:33 +00:00
Chandler Carruth 802d755533 Sort includes for all of the .h files under the 'lib' tree. These were
missed in the first pass because the script didn't yet handle include
guards.

Note that the script is now able to handle all of these headers without
manual edits. =]

llvm-svn: 169224
2012-12-04 07:12:27 +00:00
Nadav Rotem 07674cb566 Give scalar if-converted blocks half the score because they are not always executed due to CF.
llvm-svn: 169223
2012-12-04 07:11:52 +00:00
Nadav Rotem 628c2dba60 Add the last part that is needed for vectorization of if-converted code.
Added the code that actually performs the if-conversion during vectorization.

We can now vectorize this code:

for (int i=0; i<n; ++i) {
  unsigned k = 0;

  if (a[i] > b[i])   <------ IF inside the loop.
    k = k * 5 + 3;

  a[i] = k;          <---- K is a phi node that becomes vector-select.
}

llvm-svn: 169217
2012-12-04 06:15:11 +00:00
Kostya Serebryany 9b65726d24 [asan] add experimental -asan-realign-stack option (true by default, which does not change the current behavior)
llvm-svn: 169216
2012-12-04 06:14:01 +00:00
Matt Beaumont-Gay abfc446063 Add 'using' declarations to suppress -Woverloaded-virtual warnings.
llvm-svn: 169214
2012-12-04 05:41:27 +00:00
Shuxin Yang 86c0e232b7 rdar://12329730 (2nd part, revised)
The type of shirt-right (logical or arithemetic) should remain unchanged 
when transforming  "X << C1 >> C2" into "X << (C1-C2)"

llvm-svn: 169209
2012-12-04 03:28:32 +00:00
Alexey Samsonov 261177a1e1 ASan: add initial support for handling llvm.lifetime intrinsics in ASan - emit calls into runtime library that poison memory for local variables when their lifetime is over and unpoison memory when their lifetime begins.
llvm-svn: 169200
2012-12-04 01:34:23 +00:00
NAKAMURA Takumi f99b535fdb LoopVectorize.cpp: Suppress a warning. [-Wunused-variable]
llvm-svn: 169195
2012-12-04 00:49:34 +00:00
NAKAMURA Takumi 8b07bc579b Fix whitespace.
llvm-svn: 169194
2012-12-04 00:49:28 +00:00
Shuxin Yang 63e999edbf rdar://12329730 (2nd part)
This change tries to simmplify E1 = " X >> C1 << C2" into :
  - E2 = "X << (C2 - C1)" if C2 > C1, or
  - E2 = "X >> (C1 - C2)" if C1 > C2, or
  - E2 = X if C1 == C2.

 Reviewed by Nadav. Thanks!

llvm-svn: 169182
2012-12-04 00:04:54 +00:00
Nadav Rotem d479a57f68 minor renaming, documentation and cleanups.
llvm-svn: 169175
2012-12-03 22:57:09 +00:00
Nadav Rotem fad16be973 IF-conversion: teach the cost-model how to grade if-converted loops.
llvm-svn: 169171
2012-12-03 22:46:31 +00:00
Nadav Rotem eee203d885 Now that we have a basic if-conversion infrastructure we can rename the
"single basic block loop vectorizer" to "innermost loop vectorizer".

llvm-svn: 169158
2012-12-03 21:33:08 +00:00
Nadav Rotem a30aba7a01 Add initial support for IF-conversion. This patch implements the first 1/3,
which is the legality of the if-conversion transformation. The next step is to
implement the cost-model for the if-converted code as well as the
vectorization itself.

llvm-svn: 169152
2012-12-03 21:06:35 +00:00
Alexey Samsonov ef51c3ff81 ASan: add blacklist file to ASan pass options. Clang patch for this will follow.
llvm-svn: 169143
2012-12-03 19:09:26 +00:00
Nadav Rotem 2349531def Teach the jump threading optimization to stop scanning the basic block when calculating the cost after passing the threshold.
llvm-svn: 169135
2012-12-03 17:34:44 +00:00
Chandler Carruth ed0881b2a6 Use the new script to sort the includes of every file under lib.
Sooooo many of these had incorrect or strange main module includes.
I have manually inspected all of these, and fixed the main module
include to be the nearest plausible thing I could find. If you own or
care about any of these source files, I encourage you to take some time
and check that these edits were sensible. I can't have broken anything
(I strictly added headers, and reordered them, never removed), but they
may not be the headers you'd really like to identify as containing the
API being implemented.

Many forward declarations and missing includes were added to a header
files to allow them to parse cleanly when included first. The main
module rule does in fact have its merits. =]

llvm-svn: 169131
2012-12-03 16:50:05 +00:00
Chandler Carruth f02b8bf11b Remove some buggy and apparantly unnecessary code from SROA.
The partitioning logic attempted to handle uses of an alloca with an
offset starting before the alloca so long as the use had some overlap
with the alloca itself. However, there was a bug where we tested
'(uint64_t)Offset >= AllocSize' without first checking whether 'Offset'
was positive. As a consequence, essentially every negative offset (that
is, starting *before* the alloca does) would be thrown out, even if it
was overlapping. The subsequent code to throw out negative offsets which
were actually non-overlapping was essentially dead. The code to *handle*
overlapping negative offsets was actually dead!

I've just removed all of this, and taught SROA to discard any uses which
start prior to the alloca from the beginning. It has the lovely property
of simplifying the code. =] All the tests still pass, and in fact no new
tests are needed as this is already covered by our testsuite. Fixing the
code so that negative offsets work the way the comments indicate they
were supposed to work causes regressions. That's how I found this.

Anyways, this is all progress in the correct direction -- tightening up
SROA to be maximally aggressive. Some day, I really hope to turn
out-of-bounds accesses to an alloca into 'unreachable'.

llvm-svn: 169120
2012-12-03 10:59:55 +00:00
Nuno Lopes 5eec2679df fix stats for added checks
llvm-svn: 169119
2012-12-03 10:15:03 +00:00
Benjamin Kramer 47534c7440 SROA: Avoid struct and array types early to avoid creating an overly large integer type.
Fixes PR14465.

Differential Revision: http://llvm-reviews.chandlerc.com/D148

llvm-svn: 169084
2012-12-01 11:53:32 +00:00
Zhou Sheng 8e6d64a71d Revert previous check in r168581, r169079 as they are still in code review status.
llvm-svn: 169083
2012-12-01 10:54:28 +00:00
Zhou Sheng 13fb1ca44a The patch is to improve the memory footprint of pass GlobalOpt.
Also check in a case to repeat the issue, on which 'opt -globalopt' consumes 1.6GB memory.
The big memory footprint cause is that current GlobalOpt one by one hoists and stores the leaf element constant into the global array, in each iteration, it recreates the global array initializer constant and leave the old initializer alone. This may result in many obsolete constants left.
For example:  we have global array @rom = global [16 x i32] zeroinitializer
After the first element value is hoisted and installed:   @rom = global [16 x i32] [ 1, 0, 0, ... ]
After the second element value is installed:  @rom = global [16 x 32] [ 1, 2, 0, 0, ... ]        // here the previous initializer is obsolete
...
When the transform is done, we have 15 obsolete initializers left useless.

llvm-svn: 169079
2012-12-01 04:38:53 +00:00
Pedro Artigas 00b83c9b8d reversed the logic of the log2 detection routine to reduce the number of nested ifs
llvm-svn: 169049
2012-11-30 22:47:15 +00:00
Nadav Rotem 3ae24ee08a minor cleanups
llvm-svn: 169048
2012-11-30 22:37:11 +00:00
Bill Wendling c786b31233 Replace r168930 with a more reasonable patch.
The original patch removed a bunch of code that the SjLjEHPrepare pass placed
into the entry block if all of the landing pads were removed during the
CodeGenPrepare class. The more natural way of doing things is to run the CGP
*before* we run the SjLjEHPrepare pass.

Make it so!

llvm-svn: 169044
2012-11-30 22:08:55 +00:00
Pedro Artigas 993acd0c54 Addresses many style issues with prior checkin (r169025)
llvm-svn: 169043
2012-11-30 22:07:05 +00:00
Pedro Artigas d8795040de Add fast math inst combine X*log2(Y*0.5)-->X*log2(Y)-X
reviewed by Michael Ilseman <milseman@apple.com>

llvm-svn: 169025
2012-11-30 19:09:41 +00:00
Nadav Rotem 6b494be886 Remove the use of LPPassManager. We can remove LPM because we dont need to run any additional loop passes on the new vector loop.
llvm-svn: 169016
2012-11-30 17:27:53 +00:00
Kostya Serebryany 817b60af38 [asan] simplify the code around doesNotReturn call. It now magically works.
llvm-svn: 168995
2012-11-30 11:08:59 +00:00
Chandler Carruth d9ef81e133 Fix non-determinism introduced in r168970 and pointed out by Duncan.
We're iterating over a non-deterministically ordered container looking
for two saturating flags. To do this correctly, we have to saturate
both, and only stop looping if both saturate to their final value.
Otherwise, which flag we see first changes the result.

This is also a micro-optimization of the previous version as now we
don't go into the (possibly expensive) test logic once the first
violation of either constraint is detected.

llvm-svn: 168989
2012-11-30 09:34:29 +00:00
Chandler Carruth 77d433dafe Rearrange the comments, control flow, and variable names; no
functionality changed.

Evan's commit r168970 moved the code that the primary comment in this
function referred to to the other end of the function without moving the
comment, and there has been a steady creep of "boolean" logic in it that
is simpler if handled via early exit. That way each special case can
have its own comments. I've also made the variable name a bit more
explanatory than "AllFit". This is in preparation to fix the
non-deterministic output of this function.

llvm-svn: 168988
2012-11-30 09:26:25 +00:00
Meador Inge e3f2b26bfa Move library call simplification statistic to instcombine
The simplify-libcalls pass maintained a statistic to count the number
of library calls that have been simplified.  Now that library call
simplification is being carried out in instcombine the statistic should
be moved to there.

llvm-svn: 168975
2012-11-30 04:05:06 +00:00
Chandler Carruth dbd6958183 Move the InstVisitor utility into VMCore where it belongs. It heavily
depends on the IR infrastructure, there is no sense in it being off in
Support land.

This is in preparation to start working to expand InstVisitor into more
special-purpose visitors that are still generic and can be re-used
across different passes. The expansion will go into the Analylis tree
though as nothing in VMCore needs it.

llvm-svn: 168972
2012-11-30 03:08:41 +00:00
Evan Cheng 65df808f62 Fix logic to determine whether to turn a switch into a lookup table. When
the tables cannot fit in registers (i.e. bitmap), do not emit the table
if it's using an illegal type.

rdar://12779436

llvm-svn: 168970
2012-11-30 02:02:42 +00:00
Shuxin Yang abcc370423 rdar://12100355 (part 1)
This revision attempts to recognize following population-count pattern:

 while(a) { c++; ... ; a &= a - 1; ... },
  where <c> and <a>could be used multiple times in the loop body.

 TODO: On X8664 and ARM, __buildin_ctpop() are not expanded to a efficent 
instruction sequence, which need to be improved in the following commits.

Reviewed by Nadav, really appreciate!

llvm-svn: 168931
2012-11-29 19:38:54 +00:00
Bill Wendling a4a77edf2e Handle the situation where CodeGenPrepare removes a reference to a BB that has
the last invoke instruction in the function. This also removes the last landing
pad in an function. This is fine, but with SjLj EH code, we've already placed a
bunch of code in the 'entry' block, which expects the landing pad to stick
around.

When we get to the situation where CGP has removed the last landing pad, go
ahead and nuke the SjLj instructions from the 'entry' block.
<rdar://problem/12721258>

llvm-svn: 168930
2012-11-29 19:38:06 +00:00
Nadav Rotem ec739205cc No need to run LICM after loop vectorization because we dont generate invariant code any more.
llvm-svn: 168928
2012-11-29 19:28:29 +00:00
Nadav Rotem 8dd6ee8df5 When broadcasting invariant scalars into vectors, place the broadcast code in the preheader.
llvm-svn: 168927
2012-11-29 19:25:41 +00:00
Meador Inge 75798bb7fe instcombine: Migrate puts optimizations
This patch migrates the puts optimizations from the simplify-libcalls
pass into the instcombine library call simplifier.

All the simplifiers from simplify-libcalls have now been migrated to
instcombine.  Yay!  Just a few other bits to migrate (prototype attribute
inference and a few statistics) and simplify-libcalls can finally be put
to rest.

llvm-svn: 168925
2012-11-29 19:15:17 +00:00
Alexey Samsonov 9a956e8cd2 [ASan] Simplify check added in r168861. Bail out from module pass early if the module is blacklisted.
llvm-svn: 168913
2012-11-29 18:27:01 +00:00
Matt Beaumont-Gay c76536f886 Apply Takumi's patch to suppress unused-variable warnings in -Asserts builds.
llvm-svn: 168911
2012-11-29 18:15:49 +00:00
Alexey Samsonov df6245233c Add options to AddressSanitizer passes to make them configurable by frontend.
llvm-svn: 168910
2012-11-29 18:14:24 +00:00