Commit Graph

12685 Commits

Author SHA1 Message Date
Sanjay Patel 0fdb437b25 remove function names from comments; NFC
llvm-svn: 231826
2015-03-10 19:42:57 +00:00
Michael Zolotukhin 267e12f714 Enable loop-rotate before loop-vectorize by default
llvm-svn: 231820
2015-03-10 19:07:41 +00:00
Adam Nemet 98c4c5dd78 [LAA-memchecks 2/3] Move number of memcheck threshold checking to LV
Now the analysis won't "fail" if the memchecks exceed the threshold.  It
is the transform pass' responsibility to perform the check.

This allows the transform pass to further analyze/eliminate the
memchecks.  E.g. in Loop distribution we only need to check pointers
that end up in different partitions.

Note that there is a slight change of functionality here.  The logic in
analyzeLoop is that if dependence checking fails due to non-constant
distance between the pointers, another attempt is made to prove safety
of the dependences purely using run-time checks.

Before this patch we could fail the loop due to exceeding the memcheck
threshold after the first step, now we only check the threshold in the
client after the full analysis.  There is no measurable compile-time
effect but I wanted to record this here.

llvm-svn: 231817
2015-03-10 18:54:23 +00:00
Sanjay Patel abf7023c63 remove names from comments; NFC
llvm-svn: 231813
2015-03-10 18:41:22 +00:00
Sanjay Patel 51bd9421ac fix typos; NFC
llvm-svn: 231812
2015-03-10 18:37:05 +00:00
Sanjay Patel f1b0db1545 remove function names from comments; NFC
llvm-svn: 231801
2015-03-10 16:42:24 +00:00
Owen Anderson 58364dc4da Fix a crash in InstCombine where we could try to truncate a switch comparison to zero width.
llvm-svn: 231761
2015-03-10 06:51:39 +00:00
Owen Anderson 51b75b8c34 Fix an infinite loop in InstCombine when an instruction with no users and side effects can be constant folded.
ReplaceInstUsesWith needs to return nullptr when the input has no users,
because in that case it does not mutate the program.  Otherwise, we can
get stuck in an infinite loop of repeatedly attempting to constant fold
and instruction with no users.

llvm-svn: 231755
2015-03-10 05:13:47 +00:00
Mehdi Amini a28d91d81b DataLayout is mandatory, update the API to reflect it with references.
Summary:
Now that the DataLayout is a mandatory part of the module, let's start
cleaning the codebase. This patch is a first attempt at doing that.

This patch is not exactly NFC as for instance some places were passing
a nullptr instead of the DataLayout, possibly just because there was a
default value on the DataLayout argument to many functions in the API.
Even though it is not purely NFC, there is no change in the
validation.

I turned as many pointer to DataLayout to references, this helped
figuring out all the places where a nullptr could come up.

I had initially a local version of this patch broken into over 30
independant, commits but some later commit were cleaning the API and
touching part of the code modified in the previous commits, so it
seemed cleaner without the intermediate state.

Test Plan:

Reviewers: echristo

Subscribers: llvm-commits

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 231740
2015-03-10 02:37:25 +00:00
Kostya Serebryany 48a4023f40 [sanitizer] fix instrumentation with -mllvm -sanitizer-coverage-block-threshold=0 to actually do something useful.
llvm-svn: 231736
2015-03-10 01:58:27 +00:00
Kostya Serebryany 8fb05ac998 [sanitizer] decrease sanitizer-coverage-block-threshold from 1000 to 500 as another horrible workaround for PR17409
llvm-svn: 231733
2015-03-10 01:11:53 +00:00
Benjamin Kramer 7bd1f7cb58 Remove the remaining uses of abs64 and nuke it.
std::abs works just fine and we're already using it in many places. NFC intended.

llvm-svn: 231696
2015-03-09 20:20:16 +00:00
Benjamin Kramer f044d3f93b Make helper functions static.
Found by -Wmissing-prototypes. NFC.

llvm-svn: 231664
2015-03-09 16:23:46 +00:00
Benjamin Kramer fd3bc74460 SymbolRewriter: Hide implementation details
NFC.

llvm-svn: 231660
2015-03-09 15:50:47 +00:00
Kevin Qin 65b07b8e1b Revert r231630 - Run LICM pass after loop unrolling pass.
As it broke llvm bootstrap.

llvm-svn: 231635
2015-03-09 07:26:37 +00:00
Kevin Qin 715b01e979 Introduce runtime unrolling disable matadata and use it to mark the scalar loop from vectorization.
Runtime unrolling is an expensive optimization which can bring benefit
only if the loop is hot and iteration number is relatively large enough.
For some loops, we know they are not worth to be runtime unrolled.
The scalar loop from vectorization is one of the cases.

llvm-svn: 231631
2015-03-09 06:14:18 +00:00
Kevin Qin a998735def Run LICM pass after loop unrolling pass.
Runtime unrollng will introduce a runtime check in loop prologue.
If the unrolled loop is a inner loop, then the proglogue will be inside
the outer loop. LICM pass can help to promote the runtime check out if
the checked value is loop invariant.

llvm-svn: 231630
2015-03-09 06:14:07 +00:00
David Blaikie dc3f01e9cf Simplify expressions involving boolean constants with clang-tidy
Patch by Richard (legalize at xmission dot com).

Differential Revision: http://reviews.llvm.org/D8154

llvm-svn: 231617
2015-03-09 01:57:13 +00:00
Olivier Sallenave 049d803ce0 Do not restrict interleaved unrolling to small loops, depending on the target.
llvm-svn: 231528
2015-03-06 23:12:04 +00:00
Benjamin Kramer e8a64a20f2 LoopInterchange: Remove empty method.
llvm-svn: 231503
2015-03-06 19:37:26 +00:00
Benjamin Kramer 79442920bf LoopInterchange: Rephrase instruction moving using ilist's splice and factor it into a function
+ Random cleanups. No functional change.

llvm-svn: 231501
2015-03-06 18:59:14 +00:00
Benjamin Kramer 298a3a0567 Fold init() helpers into constructors. NFC.
llvm-svn: 231486
2015-03-06 16:21:15 +00:00
Daniel Jasper 6adbd7aecf Change the way in which error case is being handled.
Specifically this:
* Prevents an "unused" warning in non-assert builds.
* In that error case return with out removing a child loop instead of
  looping forever.

llvm-svn: 231459
2015-03-06 10:39:14 +00:00
Karthik Bhat 88db86dd29 Add a new pass "Loop Interchange"
This pass interchanges loops to provide a more cache-friendly memory access.

For e.g. given a loop like -
  for(int i=0;i<N;i++)
    for(int j=0;j<N;j++)
      A[j][i] = A[j][i]+B[j][i];

is interchanged to -
  for(int j=0;j<N;j++)
    for(int i=0;i<N;i++)
      A[j][i] = A[j][i]+B[j][i];

This pass is currently disabled by default.

To give a brief introduction it consists of 3 stages-

LoopInterchangeLegality : Checks the legality of loop interchange based on Dependency matrix.
LoopInterchangeProfitability: A very basic heuristic has been added to check for profitibility. This will evolve over time.
LoopInterchangeTransform : Which does the actual transform.

LNT Performance tests shows improvement in Polybench/linear-algebra/kernels/mvt and Polybench/linear-algebra/kernels/gemver becnmarks.

TODO:
1) Add support for reductions and lcssa phi.
2) Improve profitability model.
3) Improve loop selection algorithm to select best loop for interchange. Currently the innermost loop is selected for interchange.
4) Improve compile time regression found in llvm lnt due to this pass.
5) Fix issues in Dependency Analysis module.

A special thanks to Hal for reviewing this code.
Review: http://reviews.llvm.org/D7499

llvm-svn: 231458
2015-03-06 10:11:25 +00:00
Yaron Keren 322bdad085 Silence C4715 'not all control paths return a value' warnings.
llvm-svn: 231455
2015-03-06 07:49:14 +00:00
Michael Gottesman 6ff10c959a [objc-arc] Sprinkle some more auto on some iterators.
llvm-svn: 231447
2015-03-06 02:10:03 +00:00
Michael Gottesman 16e6a2057f [objc-arc] Move the detection of potential uses or altering of a ref count onto PtrState.
llvm-svn: 231446
2015-03-06 02:07:12 +00:00
Michael Gottesman 6080596328 [objc-arc] Move the checking of whether or not we can match onto PtrStates and out of the main dataflow.
These refactored computations check whether or not we are at a stage
of the sequence where we can perform a match. This patch moves the
computation out of the main dataflow and into
{BottomUp,TopDown}PtrState.

llvm-svn: 231439
2015-03-06 00:34:42 +00:00
Michael Gottesman 4eae396ae9 [objc-arc] Refactor (Re-)initialization of PtrState from dataflow -> {TopDown,BottomUp}PtrState Class.
This initialization occurs when we see a new retain or release. Before
we performed the actual initialization inline in the dataflow. That is
just messy.

llvm-svn: 231438
2015-03-06 00:34:39 +00:00
Michael Gottesman feb138e211 [objc-arc] Create two subclasses of PtrState in preparation for moving per ptr state change behavior onto a PtrState class.
This will enable the main ObjCARCOpts dataflow to work with higher
level concepts such as "can this ptr state be modified by this ref
count" and not need to understand the nitty gritty details of how that
is determined. This makes the dataflow cleaner.

llvm-svn: 231437
2015-03-06 00:34:36 +00:00
Michael Gottesman 41c01005ed [objc-arc] Extract out MDNodes into a cache structure so the information can be passed around.
llvm-svn: 231436
2015-03-06 00:34:33 +00:00
Michael Gottesman f6bcb81000 [objc-arc] Remove annotations code.
It will always be in the history if it is needed again. Now it is just dead
code.

llvm-svn: 231435
2015-03-06 00:34:29 +00:00
Michael Gottesman d45907bd38 Fix build error.
llvm-svn: 231430
2015-03-05 23:57:07 +00:00
Michael Gottesman a9fc016281 [objc-arc] Change some casts and loop iterators to use auto.
llvm-svn: 231427
2015-03-05 23:29:06 +00:00
Michael Gottesman 68b91dbf84 [objc-arc] Extract out state specific to a ref count from the main objc arc sequence dataflow. This will allow me to separate the actual ARC queries from the meat of the dataflow algorithm.
llvm-svn: 231426
2015-03-05 23:29:03 +00:00
Michael Gottesman 0be6920e23 [objc-arc] Extract blot map vector into its own file. NFC.
llvm-svn: 231425
2015-03-05 23:28:58 +00:00
Michael Kuperstein bcb26d6880 [InstCombine] Fix an assertion when fmul has a ConstantExpr operand
isNormalFp and isFiniteNonZeroFp should not assume vector operands can not be constant expressions.

Patch by Pawel Jurek <pawel.jurek@intel.com>
Differential Revision: http://reviews.llvm.org/D8053

llvm-svn: 231359
2015-03-05 08:38:57 +00:00
Kostya Serebryany 83ce8779d5 [sanitizer] add nosanitize metadata to more coverage instrumentation instructions
llvm-svn: 231333
2015-03-05 01:20:05 +00:00
Sanjoy Das a5397c0198 [IndVarSimplify] use the "canonical" way to infer no-wrap.
Summary:
rL225282 introduced an ad-hoc way to promote some additions to nuw or
nsw.  Since then SCEV has become smarter in directly proving no-wrap;
and using the canonical "ext(A op B) == ext(A) op ext(B)" method of
proving no-wrap is just as powerful now.  Rip out the existing
complexity in favor of getting SCEV to do all the heaving lifting
internally.

This change does not add any unit tests because it is supposed to be a
non-functional change.  Tests added in rL225282 and rL226075 are valid
tests for this change.

Reviewers: atrick, majnemer

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D7981

llvm-svn: 231306
2015-03-04 22:24:23 +00:00
Reid Kleckner 4276945161 Try to satisfy sanitizer lint check
llvm-svn: 231284
2015-03-04 20:38:59 +00:00
Mehdi Amini 46a43556db Make DataLayout Non-Optional in the Module
Summary:
DataLayout keeps the string used for its creation.

As a side effect it is no longer needed in the Module.
This is "almost" NFC, the string is no longer
canonicalized, you can't rely on two "equals" DataLayout
having the same string returned by getStringRepresentation().

Get rid of DataLayoutPass: the DataLayout is in the Module

The DataLayout is "per-module", let's enforce this by not
duplicating it more than necessary.
One more step toward non-optionality of the DataLayout in the
module.

Make DataLayout Non-Optional in the Module

Module->getDataLayout() will never returns nullptr anymore.

Reviewers: echristo

Subscribers: resistor, llvm-commits, jholewinski

Differential Revision: http://reviews.llvm.org/D7992

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 231270
2015-03-04 18:43:29 +00:00
Dmitry Vyukov b37b95ed3e asan: do not instrument direct inbounds accesses to stack variables
Do not instrument direct accesses to stack variables that can be
proven to be inbounds, e.g. accesses to fields of structs on stack.

But it eliminates 33% of instrumentation on webrtc/modules_unittests
(number of memory accesses goes down from 290152 to 193998) and
reduces binary size by 15% (from 74M to 64M) and improved compilation time by 6-12%.

The optimization is guarded by asan-opt-stack flag that is off by default.

http://reviews.llvm.org/D7583

llvm-svn: 231241
2015-03-04 13:27:53 +00:00
Philip Reames 6da37857d1 [RewriteStatepointsForGC] Fix a relocation bug w.r.t values defined by invoke instructions
RewriteStatepointsForGC pass emits an alloca for each GC pointer which will be relocated. It then inserts stores after def and all relocations, and inserts loads before each use as well. In the end, mem2reg is used to update IR with relocations in SSA form.

However, there is a problem with inserting stores for values defined by invoke instructions. The code didn't expect a def was a terminator instruction, and inserting instructions after these terminators resulted in malformed IR.

This patch fixes this problem by handling invoke instructions as a special case. If the def is an invoke instruction, the store will be inserted at the beginning of the normal destination block. Since return value from invoke instruction does not dominate the unwind destination block, no action is needed there.

Patch by: Chen Li
Differential Revision: http://reviews.llvm.org/D7923

llvm-svn: 231183
2015-03-04 00:13:52 +00:00
Kostya Serebryany be5e0ed919 [sanitizer/coverage] Add AFL-style coverage counters (search heuristic for fuzzing).
Introduce -mllvm -sanitizer-coverage-8bit-counters=1
which adds imprecise thread-unfriendly 8-bit coverage counters.

The run-time library maps these 8-bit counters to 8-bit bitsets in the same way
AFL (http://lcamtuf.coredump.cx/afl/technical_details.txt) does:
counter values are divided into 8 ranges and based on the counter
value one of the bits in the bitset is set.
The AFL ranges are used here: 1, 2, 3, 4-7, 8-15, 16-31, 32-127, 128+.

These counters provide a search heuristic for single-threaded
coverage-guided fuzzers, we do not expect them to be useful for other purposes.

Depending on the value of -fsanitize-coverage=[123] flag,
these counters will be added to the function entry blocks (=1),
every basic block (=2), or every edge (=3).

Use these counters as an optional search heuristic in the Fuzzer library.
Add a test where this heuristic is critical.

llvm-svn: 231166
2015-03-03 23:27:02 +00:00
David Majnemer 1bacc0abc9 InstCombine: Ensure select condition types are identical before merging
Selection conditions may be vectors or scalars.  Make sure InstCombine
doesn't indiscriminately assume that a select which is value dependent
on another select have identical select condition types.

This fixes PR22773.

llvm-svn: 231156
2015-03-03 22:40:36 +00:00
David Blaikie 9469072367 RewriteStatepointsForGC::PhiState: Remove explicit copy ctor in favor of the Rule of Zero
The assertion was just checking a class invariant that's pretty easy to
verify by inspection (no mutating operations, and the two non-copy ctors
already ensure the state is maintained) so remove the explicit copy ctor
in favor of the default, thus allowing the use of the default copy
assignment operator without hitting the C++11 deprecation here.

llvm-svn: 231143
2015-03-03 21:49:07 +00:00
David Blaikie 7f1e0565b3 Revert "Remove the explicit SDNodeIterator::operator= in favor of the implicit default"
Accidentally committed a few more of these cleanup changes than
intended. Still breaking these out & tidying them up.

This reverts commit r231135.

llvm-svn: 231136
2015-03-03 21:18:16 +00:00
David Blaikie bb8da4c08f Remove the explicit SDNodeIterator::operator= in favor of the implicit default
There doesn't seem to be any need to assert that iterator assignment is
between iterators over the same node - if you want to reuse an iterator
variable to iterate another node, that's perfectly acceptable. Just
don't mix comparisons between iterators into disjoint sequences, as
usual.

llvm-svn: 231135
2015-03-03 21:17:08 +00:00
Peter Collingbourne da2dbf21a9 LowerBitSets: Use byte arrays instead of bit sets to represent in-memory bit sets.
By loading from indexed offsets into a byte array and applying a mask, a
program can test bits from the bit set with a relatively short instruction
sequence. For example, suppose we have 15 bit sets to lay out:

A (16 bits), B (15 bits), C (14 bits), D (13 bits), E (12 bits),
F (11 bits), G (10 bits), H (9 bits), I (7 bits), J (6 bits), K (5 bits),
L (4 bits), M (3 bits), N (2 bits), O (1 bit)

These bits can be laid out in a 16-byte array like this:

      Byte Offset
    0123456789ABCDEF
Bit
  7 HHHHHHHHHIIIIIII
  6 GGGGGGGGGGJJJJJJ
  5 FFFFFFFFFFFKKKKK
  4 EEEEEEEEEEEELLLL
  3 DDDDDDDDDDDDDMMM
  2 CCCCCCCCCCCCCCNN
  1 BBBBBBBBBBBBBBBO
  0 AAAAAAAAAAAAAAAA

For example, to test bit X of A, we evaluate ((bits[X] & 1) != 0), or to
test bit X of I, we evaluate ((bits[9 + X] & 0x80) != 0). This can be done
in 1-2 machine instructions on x86, or 4-6 instructions on ARM.

This uses the LPT multiprocessor scheduling algorithm to lay out the bits
efficiently.

Saves ~450KB of instructions in a recent build of Chromium.

Differential Revision: http://reviews.llvm.org/D7954

llvm-svn: 231043
2015-03-03 00:49:28 +00:00
Benjamin Kramer 838752d3f6 LoopIdiom: Give globals for memset_pattern16 private linkage.
There's really no reason to have them have entries in the symbol table
anymore. Old versions of ld64 had some bugs in this area but those have
been fixed long ago.

llvm-svn: 231041
2015-03-03 00:17:09 +00:00