Commit Graph

14942 Commits

Author SHA1 Message Date
Derek Bruening d862c178b0 [esan] EfficiencySanitizer instrumentation pass
Summary:
Adds an instrumentation pass for the new EfficiencySanitizer ("esan")
performance tuning family of tools.  Multiple tools will be supported
within the same framework.  Preliminary support for a cache fragmentation
tool is included here.

The shared instrumentation includes:
+ Turn mem{set,cpy,move} instrinsics into library calls.
+ Slowpath instrumentation of loads and stores via callouts to
  the runtime library.
+ Fastpath instrumentation will be per-tool.
+ Which memory accesses to ignore will be per-tool.

Reviewers: eugenis, vitalybuka, aizatsky, filcab

Subscribers: filcab, vkalintiris, pcc, silvas, llvm-commits, zhaoqin, kcc

Differential Revision: http://reviews.llvm.org/D19167

llvm-svn: 267058
2016-04-21 21:30:22 +00:00
JF Bastien c22d29982b NFC: fix copy / paste comment
llvm-svn: 267039
2016-04-21 19:53:39 +00:00
JF Bastien 3e2e69f607 NFC: fix nonsensical comment
llvm-svn: 267036
2016-04-21 19:41:48 +00:00
Sanjoy Das a085cfc150 Folding compares with unescaped allocations
Summary:
If we know that the pointer allocated within a function does not escape,
we can fold away comparisons that are done with global pointers

Patch by Anna Thomas!

Reviewers: reames, majnemer, sanjoy

Subscribers: mgrang, mcrosier, majnemer, llvm-commits

Differential Revision: http://reviews.llvm.org/D19276

llvm-svn: 267035
2016-04-21 19:26:45 +00:00
Philip Reames a98c7ead30 [instcombine][unordered] Extend load(select) transform to handle unordered loads
llvm-svn: 267023
2016-04-21 17:59:40 +00:00
Andrew Kaylor f0f279291c Initial implementation of optimization bisect support.
This patch implements a optimization bisect feature, which will allow optimizations to be selectively disabled at compile time in order to track down test failures that are caused by incorrect optimizations.

The bisection is enabled using a new command line option (-opt-bisect-limit).  Individual passes that may be skipped call the OptBisect object (via an LLVMContext) to see if they should be skipped based on the bisect limit.  A finer level of control (disabling individual transformations) can be managed through an addition OptBisect method, but this is not yet used.

The skip checking in this implementation is based on (and replaces) the skipOptnoneFunction check.  Where that check was being called, a new call has been inserted in its place which checks the bisect limit and the optnone attribute.  A new function call has been added for module and SCC passes that behaves in a similar way.

Differential Revision: http://reviews.llvm.org/D19172

llvm-svn: 267022
2016-04-21 17:58:54 +00:00
Philip Reames 3ac0718423 [unordered] unordered loads from null are still unreachable
llvm-svn: 267019
2016-04-21 17:45:05 +00:00
Adam Nemet 6dcf0788fc [LoopUtils] Fix typo in comment
llvm-svn: 267016
2016-04-21 17:33:22 +00:00
Adam Nemet 293be666eb [LoopUtils] Add asserts to findStringMetadataForLoop. NFC
These ensure that operand array has at least one element and it is the
self-reference.

llvm-svn: 267015
2016-04-21 17:33:20 +00:00
Adam Nemet 963341c872 [LoopUtils] Move def of findStringMetadataForLoop to LoopUtils.cpp. NFC
The decl is in LoopUtils.h.  I think that this was added to
LoopVersioningLICM.cpp by mistake.

llvm-svn: 267014
2016-04-21 17:33:17 +00:00
Adam Nemet f787826b46 [LoopUtils] Rename {check->find}StringMetadata{Into->For}Loop. NFC
"Into" was misleading.  I am also planning to use this helper to look
for loop metadata and return the argument, so find seems like a better
name.

llvm-svn: 267013
2016-04-21 17:33:12 +00:00
Philip Reames ac55090e96 [instcombine][unordered] Implement *-load forwarding for unordered atomics
This builds on 266999 which made FindAvailableValue do the right thing.  Tests included show the newly enabled transforms and those which disabled either due to conservatism or correctness requirements.

llvm-svn: 267006
2016-04-21 17:03:33 +00:00
Sanjoy Das 54a3a006ca [SimplifyCFG] Fold `llvm.guard(false)` to unreachable
Summary:
`llvm.guard(false)` always bails out of the current compilation unit, so
we can prune any control flow following it.

Reviewers: hfinkel, pcc, reames

Subscribers: majnemer, reames, mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D19245

llvm-svn: 266955
2016-04-21 05:09:12 +00:00
Duncan P. N. Exon Smith 0ab44dbf8f ValueMapper: Map uniqued nodes in post-order
The iteratitive algorithm from r265456 claimed but failed to create a
post-order traversal.  It had the same error that was fixed in the
ValueEnumerator in r266947: now, instead of pushing all operands on the
worklist at once, we pause whenever an operand gets pushed in order to
go depth-first (I know, it sounds obvious).

Sadly, I have no idea how to observe this from outside the algorithm and
so I haven't written a test.  The output should be the same; it should
just use fewer temporary nodes now.  I've added some comments that I
hope make the current logic clear enough it's unlikely to regress.

llvm-svn: 266949
2016-04-21 02:34:36 +00:00
Mehdi Amini bda3c97c16 ThinLTO/ModuleLinker: add a flag to not always pull-in linkonce when performing importing
Summary:
The function importer already decided what symbols need to be pulled
in. Also these magically added ones will not be in the export list
for the source module, which can confuse the internalizer for
instance.

Reviewers: tejohnson, rafael

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19096

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266948
2016-04-21 01:59:39 +00:00
Dehao Chen a8bae82373 Refine instruction weight annotation algorithm for sample profiler.
Summary:
This patch refined the instruction weight anootation algorithm:
1. Do not use dbg_value intrinsics for annotation.
2. Annotate cold calls if the call is inlined in profile, but not inlined before preparation. This indicates that the annotation preparation step found no sample for the inlined callsite, thus the call should be very cold.

Reviewers: dnovillo, davidxl

Subscribers: mgrang, llvm-commits

Differential Revision: http://reviews.llvm.org/D19286

llvm-svn: 266936
2016-04-20 23:36:23 +00:00
Kostya Serebryany a83bfeac9d Rename asan-check-lifetime into asan-stack-use-after-scope
Summary:
This is done for consistency with asan-use-after-return.
I see no other users than tests.

Reviewers: aizatsky, kcc

Differential Revision: http://reviews.llvm.org/D19306

llvm-svn: 266906
2016-04-20 20:02:58 +00:00
Chad Rosier b346dcbc25 Typo.
llvm-svn: 266905
2016-04-20 19:16:23 +00:00
Chad Rosier 41dd31f0b0 [ValueTracking] Make isImpliedCondition return an Optional<bool>. NFC.
Phabricator Revision: http://reviews.llvm.org/D19277

llvm-svn: 266904
2016-04-20 19:15:26 +00:00
Teresa Johnson b35cc691ea [ThinLTO] Prevent importing of "llvm.used" values
Summary:
This patch prevents importing from (and therefore exporting from) any
module with a "llvm.used" local value. Local values need to be promoted
and renamed when importing, and their presense on the llvm.used variable
indicates that there are opaque uses that won't see the rename. One such
example is a use in inline assembly.

See also the discussion at:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098047.html

As part of this, move collectUsedGlobalVariables out of Transforms/Utils
and into IR/Module so that it can be used more widely. There are several
other places in LLVM that used copies of this code that can be cleaned
up as a follow on NFC patch.

Reviewers: joker.eph

Subscribers: pcc, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18986

llvm-svn: 266877
2016-04-20 14:39:45 +00:00
Mehdi Amini bb3a1d92f3 ThinLTO: never promote as external weak
This linkage is *not* intended to express that a declaration refers
to a weak symbol, but that the symbol might not be present at link
time. I don't believe it was the intent.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266856
2016-04-20 04:18:11 +00:00
Mehdi Amini 2c719cc117 FunctionImport: make sure we always select the right callee in presence of alias
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266854
2016-04-20 04:17:36 +00:00
Mehdi Amini 6968ef773b ThinLTO: Move alias importing decision on the summary
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266845
2016-04-20 01:04:20 +00:00
Marcin Koscielnicki ef2e7b4819 [Mips] [MSan] VarArgMIPS64Helper: Use target's endian, not host's.
Ugh.

Differential Revision: http://reviews.llvm.org/D19292

llvm-svn: 266833
2016-04-19 23:46:59 +00:00
David Majnemer b4b27230bf [ValueTracking, VectorUtils] Refactor getIntrinsicIDForCall
The functionality contained within getIntrinsicIDForCall is two-fold: it
checks if a CallInst's callee is a vectorizable intrinsic.  If it isn't
an intrinsic, it attempts to map the call's target to a suitable
intrinsic.

Move the mapping functionality into getIntrinsicForCallSite and rename
getIntrinsicIDForCall to getVectorIntrinsicIDForCall while
reimplementing it in terms of getIntrinsicForCallSite.

llvm-svn: 266801
2016-04-19 19:10:21 +00:00
Chad Rosier b7dfbb40a3 [ValueTracking] Improve isImpliedCondition for conditions with matching operands.
This patch improves SimplifyCFG to catch cases like:

  if (a < b) {
    if (a > b) <- known to be false
      unreachable;
  }

Phabricator Revision: http://reviews.llvm.org/D18905

llvm-svn: 266767
2016-04-19 17:19:14 +00:00
Mehdi Amini aeb1e59b71 Minor improvement to debug output for Function Importer (NFC)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266723
2016-04-19 09:21:30 +00:00
Daniel Berlin 77fa84eadd Correct IDF calculator for ReverseIDF
Summary:
Need to use predecessors for reverse graph, successors for forward graph.
succ_iterator/pred_iterator are not compatible, this patch is all the work necessary to work around that (which is what everywhere else does).  Not sure if there is a better way, so cc'ing some random folks to take a gander :)

Reviewers: dblaikie, qcolombet, echristo

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D18796

llvm-svn: 266718
2016-04-19 06:13:28 +00:00
Michael Kuperstein de16b44f74 Port DemandedBits to the new pass manager.
Differential Revision: http://reviews.llvm.org/D18679

llvm-svn: 266699
2016-04-18 23:55:01 +00:00
Xinliang David Li e6b892940f Port InstrProfiling pass to the new pass manager
Differential Revision: http://reviews.llvm.org/D18126

llvm-svn: 266637
2016-04-18 17:47:38 +00:00
Mehdi Amini b550cb1750 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266595
2016-04-18 09:17:29 +00:00
Duncan P. N. Exon Smith 724c503499 Transforms: Try harder to fix bootstrap after r266565
This catches two nullptr insertions into the ValueMap I missed in
r266567.  I missed CloneFunction becuase it never calls RemapInstruction
directly.  Here's one of the still-failing bots:
  http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/11496

llvm-svn: 266570
2016-04-17 20:11:09 +00:00
Duncan P. N. Exon Smith 0fdaf8c9c2 Linker: Don't double-schedule appending variables
Add an assertion to ValueMapper that prevents double-scheduling of
GlobalValues to remap, and fix the one place it happened.  There are
tons of tests that fail with this assertion in place and without the
code change, so I'm not adding another.

Although it looks related, r266563 was, indeed, removing dead code.
AFAICT, this cross-file double-scheduling started in r266510 when the
cross-file recursion was removed.

llvm-svn: 266569
2016-04-17 19:40:20 +00:00
Duncan P. N. Exon Smith a71301befa Transforms: Fix bootstrap after r266565
Apparently there isn't test coverage for all of these.  I'd appreciate
if someone with could reproduce and send me something to reduce, but for
now I've just looked for users of RemapInstruction and MapValue and
ensured they don't accidentally insert nullptr.  Here is one of the
bootstraps that caught:

  http://lab.llvm.org:8011/builders/clang-x64-ninja-win7/builds/11494

llvm-svn: 266567
2016-04-17 19:26:49 +00:00
Duncan P. N. Exon Smith 3d555ac96d ValueMapper: Don't allow explicit null mappings of Values, NFC
As a follow-up to r123058, assert that there are no null mappings in the
ValueMap instead of just ignoring them when they are there.  There were
a couple of accidental insertions in CloneFunction so I cleaned those up
(caught by testcases).

llvm-svn: 266565
2016-04-17 18:53:24 +00:00
Sanjoy Das 99042473d0 Fix a typo in rL265762
I accidentally replaced `mayBeOverridden` with `!isInterposable`.
Remove the negation and add a test case that would've caught this.

Many thanks to Håkan Hjort for spotting this!

llvm-svn: 266551
2016-04-17 04:30:43 +00:00
Duncan P. N. Exon Smith 5ab2be094e IR: Use an explicit map for debug info type uniquing
Rather than relying on the structural equivalence of DICompositeType to
merge type definitions, use an explicit map on the LLVMContext that
LLParser and BitcodeReader consult when constructing new nodes.
Each non-forward-declaration DICompositeType with a non-empty
'identifier:' field is stored/loaded from the type map, and the first
definiton will "win".

This map is opt-in: clients that expect ODR types from different modules
to be merged must call LLVMContext::ensureDITypeMap.

  - Clients that just happen to load more than one Module in the same
    LLVMContext won't magically merge types.

  - Clients (like LTO) that want to continue to merge types based on ODR
    identifiers should opt-in immediately.

I have updated LTOCodeGenerator.cpp, the two "linking" spots in
gold-plugin.cpp, and llvm-link (unless -disable-debug-info-type-map) to
set this.

With this in place, it will be straightforward to remove the DITypeRef
concept (i.e., referencing types by their 'identifier:' string rather
than pointing at them directly).

llvm-svn: 266549
2016-04-17 03:58:21 +00:00
Duncan P. N. Exon Smith 694ab4e966 ValueMapper: Separate mapping of distinct and uniqued nodes (again)
Since the result of a mapped distinct node is known up front, it's more
efficient to map them separately from uniqued nodes.  This commit pulls
them out of the post-order traversal and stores them in a worklist to be
remapped at the top-level.

This is essentially reapplying r244181 ("ValueMapper: Rotate distinct
node remapping algorithm") to the new iterative algorithm from r265456
("ValueMapper: Rewrite Mapper::mapMetadata without recursion").

Now that the traversal logic only handles uniqued MDNodes, it's much
simpler to inline it all into MDNodeMapper::createPOT (I've killed the
MDNodeMapper::push and MDNodeMapper::tryToPop helpers and localized the
traversal worklist).

The resulting high-level algorithm for MDNodeMapper::map now looks like
this:

  - Distinct nodes are immediately mapped and added to
    MDNodeMapper::DistinctWorklist using MDNodeMapper::mapDistinctNode.

  - Uniqued nodes are mapped via MDNodeMapper::mapTopLevelUniquedNode,
    which traverses the transitive uniqued subgraph of a node to
    calculate uniqued node mappings in bulk.

      - This is a simplified version of MDNodeMapper::map from before
        this commit (originally r265456) that doesn't traverse through
        any distinct nodes.

      - Distinct nodes are added to MDNodeMapper::DistinctWorklist via
        MDNodeMapper::mapDistinctNode.

      - This uses MDNodeMapper::createPOT to fill a
        MDNodeMapper::UniquedGraph (a post-order traversal and side
        table), UniquedGraph::propagateChanges to track which uniqued
        nodes need to change, and MDNodeMapper::mapNodesInPOT to create
        the uniqued nodes.

      - Placeholders for forward references are now only needed when
        there's a uniquing cycle (a cycle of uniqued nodes unbroken by
        distinct nodes).  This is the key functionality change that
        we're reintroducing (from r244181).  As of r265456, a temporary
        forward reference might be needed for any cycle that involved
        uniqued nodes.

  - After mapping the first node appropriately, MDNodeMapper::map works
    through MDNodeMapper::DistinctWorklist.  For each distinct node, its
    operands are remapped with MDNodeMapper::mapDistinctNode and
    MDNodeMapper::mapTopLevelUniquedNode until all nodes have been
    mapped.

Sadly there's nothing observable I can test here; no real functionality
change, just a compile-time speedup from reduced malloc traffic.

llvm-svn: 266537
2016-04-16 21:44:08 +00:00
Duncan P. N. Exon Smith 0cb5c344b4 ValueMapper: Only put cyclic nodes into CyclicNodes, NFCI
As a minor fixup to r266258, only track nodes that needed a placeholder
in CyclicNodes in MDNodeMapper::mapUniquedNodes.  There should be no
observable functionality change, just some local memory savings because
CyclicNodes only needs to grow to accommodate nodes that are actually
involved in cycles.  (This was the original intent of r266258, or else
the vector would have been called "ChangedNodes".)

llvm-svn: 266536
2016-04-16 21:09:53 +00:00
Simon Atanasyan e12bef7ea7 ValueMapper: Fix unused var warning. NFC
llvm-svn: 266529
2016-04-16 11:49:40 +00:00
Mehdi Amini 1aafabf752 ThinLTO: Move the ODR resolution to be based purely on the summary.
This is a requirement for the cache handling in D18494

Differential Revision: http://reviews.llvm.org/D18908

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266519
2016-04-16 07:02:16 +00:00
Mehdi Amini 2d28f7aa07 ThinLTO: Make aliases explicit in the summary
To be able to work accurately on the reference graph when taking
decision about internalizing, promoting, renaming, etc. We need
to have the alias information explicit.

Differential Revision: http://reviews.llvm.org/D18836

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266517
2016-04-16 06:56:44 +00:00
Duncan P. N. Exon Smith a77d073305 ValueMapper: Stop memoizing ConstantAsMetadata
Stop memoizing ConstantAsMetadata in ValueMapper::mapMetadata.  Now we
have to recompute it, but these metadata aren't particularly common, and
it restricts the lifetime of the Metadata map unnecessarily.

(The motivation is that I have a patch which uses a single Metadata map
for the lifetime of IRMover.  Mehdi profiled r266446 with the patch
applied and we saw a pretty big speedup in lib/Linker.)

llvm-svn: 266513
2016-04-16 03:39:44 +00:00
Duncan P. N. Exon Smith 39423b0294 Reapply "ValueMapper: Eliminate cross-file co-recursion, NFC"
This reverts commit r266507, reapplying r266503 (and r266505
"ValueMapper: Use API from r266503 in unit tests, NFC") completely
unchanged.

I reverted because of a bot failure here:
  http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/16810/

However, looking more closely, the failure was from a host-compiler
crash (clang 3.7.1) when building:
  lib/CodeGen/AsmPrinter/CMakeFiles/LLVMAsmPrinter.dir/DwarfAccelTable.cpp.o

I didn't modify that file, or anything it includes, with that commit.

The next build (which hadn't picked up my revert) got past it:
  http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/16811/

I think this was just unfortunate timing.  I suppose the bot must be
flakey.

llvm-svn: 266510
2016-04-16 02:29:55 +00:00
Duncan P. N. Exon Smith 6fe1ff260b Revert "ValueMapper: Eliminate cross-file co-recursion, NFC"
This reverts commit r266503, in case it's the root cause of this bot
failure:

  http://lab.llvm.org:8011/builders/lld-x86_64-freebsd/builds/16810

I'm also reverting r266505 -- "ValueMapper: Use API from r266503 in unit
tests, NFC" -- since it's in the way.

llvm-svn: 266507
2016-04-16 02:05:33 +00:00
Duncan P. N. Exon Smith f0d73f95c1 ValueMapper: Eliminate cross-file co-recursion, NFC
Eliminate co-recursion of Mapper::mapValue through
ValueMaterializer::materializeInitFor, through a major redesign of the
ValueMapper.cpp interface.

  - Expose a ValueMapper class that controls the entry points to the
    mapping algorithms.
  - Change IRLinker to use ValueMapper directly, rather than
    llvm::RemapInstruction, llvm::MapValue, etc.
  - Use (e.g.) ValueMapper::scheduleMapGlobalInit to add mapping work to
    a worklist in ValueMapper instead of recursing.

There were two fairly major complications.

Firstly, IRLinker::linkAppendingVarProto incorporates an on-the-fly IR
ugprade that I had to split apart.  Long-term, this upgrade should be
done in the bitcode reader (and we should only accept the "new" form),
but for now I've just made it work and added a FIXME.  The hold-op is
that we need to deprecate C API that relies on this.

Secondly, IRLinker has special logic to correctly implement aliases with
comdats, and uses two ValueToValueMapTy instances and two
ValueMaterializers.  I supported this by allowing clients to register an
alternate mapping context, whose MCID can be passed in when scheduling
new work.

While out of scope for this commit, it should now be straightforward to
remove recursion from Mapper::mapValue.

llvm-svn: 266503
2016-04-16 01:29:08 +00:00
Duncan P. N. Exon Smith db6861e7dd ValueMapper: Hide Mapper::VM behind an accessor, NFC
Change Mapper::VM to a pointer and add a `getVM()` accessor for it.
While this has no functionality change, it minimizes the diff on an
upcoming patch that allows switching between instances of
ValueToValueMapTy on a single Mapper instance.

llvm-svn: 266490
2016-04-15 23:18:43 +00:00
Evgeniy Stepanov 40cd1514cf [cfi] Support explicit sections for functions in cfi-icall.
Allow explicit section for indirectly called functions in cfi-icall.
Jumptables for functions in the same type class must be contiguous, so they
always go to the default text section.

Fixes PR25079.

llvm-svn: 266486
2016-04-15 22:55:38 +00:00
David Majnemer 2e02ba78d5 [InstCombine] Don't transform compares of calls to functions named fabs{f,l,}
InstCombine wants to optimize compares of calls to fabs with zero.
However, we didn't have the necessary legality checking to verify that
the function call had the same behavior as fabs.

llvm-svn: 266452
2016-04-15 17:21:03 +00:00
Adrian Prantl 75819aedf6 [PR27284] Reverse the ownership between DICompileUnit and DISubprogram.
Currently each Function points to a DISubprogram and DISubprogram has a
scope field. For member functions the scope is a DICompositeType. DIScopes
point to the DICompileUnit to facilitate type uniquing.

Distinct DISubprograms (with isDefinition: true) are not part of the type
hierarchy and cannot be uniqued. This change removes the subprograms
list from DICompileUnit and instead adds a pointer to the owning compile
unit to distinct DISubprograms. This would make it easy for ThinLTO to
strip unneeded DISubprograms and their transitively referenced debug info.

Motivation
----------

Materializing DISubprograms is currently the most expensive operation when
doing a ThinLTO build of clang.

We want the DISubprogram to be stored in a separate Bitcode block (or the
same block as the function body) so we can avoid having to expensively
deserialize all DISubprograms together with the global metadata. If a
function has been inlined into another subprogram we need to store a
reference the block containing the inlined subprogram.

Attached to https://llvm.org/bugs/show_bug.cgi?id=27284 is a python script
that updates LLVM IR testcases to the new format.

http://reviews.llvm.org/D19034
<rdar://problem/25256815>

llvm-svn: 266446
2016-04-15 15:57:41 +00:00
Sanjay Patel f11ab05bdb [SimplifyCFG] propagate branch metadata when creating select (PR27344)
This is almost identical to:
http://reviews.llvm.org/rL264527

This doesn't solve PR27344; it just allows the profile weights to survive. 
To solve the bug, we need to use the profile weights in the backend.

llvm-svn: 266442
2016-04-15 15:32:12 +00:00
Justin Lebar cf63b64fc6 [PM] Add a SpeculativeExecution pass for targets with divergent branches.
Summary:
This IR pass is helpful for GPUs, and other targets with divergent
branches.  It's a nop on targets without divergent branches.

Reviewers: chandlerc

Subscribers: llvm-commits, jingyue, rnk, joker.eph, tra

Differential Revision: http://reviews.llvm.org/D18626

llvm-svn: 266399
2016-04-15 00:32:12 +00:00
Justin Lebar cad81cf6b3 [Speculation] Add a SpeculativeExecution mode where the pass does nothing unless TTI::hasBranchDivergence() is true.
Summary:
This lets us add this pass to the IR pass manager unconditionally; it
will simply not do anything on targets without branch divergence.

Reviewers: tra

Subscribers: llvm-commits, jingyue, rnk, chandlerc

Differential Revision: http://reviews.llvm.org/D18625

llvm-svn: 266398
2016-04-15 00:32:09 +00:00
Renato Golin 5cb666add7 [ARM] Adding IEEE-754 SIMD detection to loop vectorizer
Some SIMD implementations are not IEEE-754 compliant, for example ARM's NEON.

This patch teaches the loop vectorizer to only allow transformations of loops
that either contain no floating-point operations or have enough allowance
flags supporting lack of precision (ex. -ffast-math, Darwin).

For that, the target description now has a method which tells us if the
vectorizer is allowed to handle FP math without falling into unsafe
representations, plus a check on every FP instruction in the candidate loop
to check for the safety flags.

This commit makes LLVM behave like GCC with respect to ARM NEON support, but
it stops short of fixing the underlying problem: sub-normals. Neither GCC
nor LLVM have a flag for allowing sub-normal operations. Before this patch,
GCC only allows it using unsafe-math flags and LLVM allows it by default with
no way to turn it off (short of not using NEON at all).

As a first step, we push this change to make it safe and in sync with GCC.
The second step is to discuss a new sub-normal's flag on both communitues
and come up with a common solution. The third step is to improve the FastMath
flags in LLVM to encode sub-normals and use those flags to restrict NEON FP.

Fixes PR16275.

llvm-svn: 266363
2016-04-14 20:42:18 +00:00
Sanjay Patel e998b91d86 [InstCombine] remove constant by inverting compare + logic (PR27105)
https://llvm.org/bugs/show_bug.cgi?id=27105

We can check if all bits outside of a constant mask are set with a 
single constant.

As noted in the bug report, although this form should be considered the
canonical IR, backends may want to transform this into an 'andn' / 'andc' 
comparison against zero because that could be a single machine instruction.

Differential Revision: http://reviews.llvm.org/D18842

llvm-svn: 266362
2016-04-14 20:17:40 +00:00
Dehao Chen 34cc676732 Fix null pointer access for discriminator assignment.
Summary: This fixes the buildbot failure.

Reviewers: dnovillo, davidxl

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19129

llvm-svn: 266360
2016-04-14 19:46:38 +00:00
Dehao Chen 46f8fbbb1b Update discriminator assignment algorithm to handle nested call correctly.
Summary: Add discriminator for nested call correctly.

Reviewers: davidxl, dnovillo

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19127

llvm-svn: 266354
2016-04-14 18:37:18 +00:00
Davide Italiano 96d2a1c603 [ValueMapper] Range-loopify to improve readability. NFC.
llvm-svn: 266350
2016-04-14 18:07:32 +00:00
Nicolai Haehnle 05b127da06 [StructurizeCFG] Annotate branches that were treated as uniform
Summary:
This fully solves the problem where the StructurizeCFG pass does not
consider the same branches as uniform as the SIAnnotateControlFlow pass.
The patch in D19013 helps with this problem, but is not sufficient
(and, interestingly, causes a "regression" with one of the existing
test cases).

No tests included here, because tests in D19013 already cover this.

Reviewers: arsenm, tstellarAMD

Subscribers: arsenm, llvm-commits

Differential Revision: http://reviews.llvm.org/D19018

llvm-svn: 266346
2016-04-14 17:42:35 +00:00
David Majnemer 0f26b0aeb4 [CodeGen] Teach LLVM how to lower @llvm.{min,max}num to {MIN,MAX}NAN
The behavior of {MIN,MAX}NAN differs from that of {MIN,MAX}NUM when only
one of the inputs is NaN: -NUM will return the non-NaN argument while
-NAN would return NaN.

It is desirable to lower to @llvm.{min,max}num to -NAN if they don't
have a native instruction for -NUM.  Notably, ARMv7 NEON's vmin has the
-NAN semantics.

N.B.  Of course, it is only safe to do this if the intrinsic call is
marked nnan.

llvm-svn: 266279
2016-04-14 07:13:24 +00:00
Tim Northover 5c02f9ad28 ARM: override cost function to re-enable ConstantHoisting (& fix it).
At some point, ARM stopped getting any benefit from ConstantHoisting because
the pass called a different variant of getIntImmCost. Reimplementing the
correct variant revealed some problems, however:

  + ConstantHoisting was modifying switch statements. This is simply invalid,
    the cases must remain integer constants no matter the notional cost.
  + ConstantHoisting was mangling alloca instructions in the entry block. These
    should be handled by FrameLowering, so constants actually have a cost of 0.
    Worse, the resulting bitcasts meant they became dynamic allocas.

rdar://25707382

llvm-svn: 266260
2016-04-13 23:08:27 +00:00
Duncan P. N. Exon Smith 11f60fd65a ValueMapper: Resolve cycles on the new nodes
Fix a major bug from r265456.  Although it's now much rarer, ValueMapper
sometimes has to duplicate cycles.  The
might-transitively-reference-a-temporary counts don't decrement on their
own when there are cycles, and you need to call MDNode::resolveCycles to
fix it.

r265456 was checking the input nodes to see if they were unresolved.
This is useless; they should never be unresolved.  Instead we should
check the output nodes and resolve cycles on them.

llvm-svn: 266258
2016-04-13 22:54:01 +00:00
JF Bastien 8331458deb NFC mergefunc: const correctness
Some of the comparators were const others weren't making it annoying to add new comparators which call existing ones.

llvm-svn: 266247
2016-04-13 21:12:21 +00:00
Betul Buyukkurt bf8554c279 [PGO] Remove redundant VP instrumentation
LLVM optimization passes may reduce a profiled target expression
to a constant. Removing runtime calls at such instrumentation points
would help speedup the runtime of the instrumented program.

llvm-svn: 266229
2016-04-13 18:52:19 +00:00
Mehdi Amini b5b289339b Revert "Make aliases explicit in the summary"
Inadvertently commited...

This reverts commit e618ec93786d99df2ddf280ad2d5e02f5516cecf.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266215
2016-04-13 17:20:07 +00:00
Mehdi Amini ce744a95fd Make aliases explicit in the summary
Summary:
To be able to work accurately on the reference graph when taking decision
about internalizing, promoting, renaming, etc. We need to have the alias
information explicit.

Reviewers: tejohnson

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D18836

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266214
2016-04-13 17:18:42 +00:00
David L Kreitzer 752c1448fe Simplify strlen to a subtraction for certain cases.
Patch by Li Huang (li1.huang@intel.com)

Differential Revision: http://reviews.llvm.org/D18230

llvm-svn: 266200
2016-04-13 14:31:06 +00:00
David Majnemer 3ee5f34469 [InstCombine] We folded an fcmp to an i1 instead of a vector of i1
Remove an ad-hoc transform in InstCombine and replace it with more
general machinery (ValueTracking, InstructionSimplify and VectorUtils).

This fixes PR27332.

llvm-svn: 266175
2016-04-13 06:55:52 +00:00
Mehdi Amini 105938302a Minor cleanup in Internalize, hide helper class using anonymous namespace (NFC)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266173
2016-04-13 06:32:29 +00:00
Mehdi Amini 59269a874f Really return whether Internalize did change the Module or not.
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266169
2016-04-13 05:25:16 +00:00
Mehdi Amini 3949b9e6dd Modernize Internalizer with for-range loop (NFC)
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266168
2016-04-13 05:25:12 +00:00
Mehdi Amini 24d3414f06 Refactor the InternalizePass into a helper class, and expose it through a public free function (NFC)
There is really no reason to require to instanciate a pass manager to
internalize.

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266167
2016-04-13 05:25:08 +00:00
Mehdi Amini 4078709957 Refactor Internalization pass to use as a callback instead of a StringSet (NFC)
This will save a bunch of copies / initialization of intermediate
datastructure, and (hopefully) simplify the code.

This also abstract the symbol preservation mechanism outside of the
Internalization pass into the client code, which is not forced
to keep a map of strings for instance (ThinLTO will prefere hashes).

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266163
2016-04-13 04:20:32 +00:00
Mehdi Amini ef7555fbb2 Fix FunctionImport export list computation: need to take a reference to a map entry to actually modify it
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266159
2016-04-13 01:52:32 +00:00
Mehdi Amini 818f67add5 Fix mismatch on returned type between header and implementation for createNameAnonFunctionPass()
From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266151
2016-04-12 23:25:11 +00:00
Sanjay Patel 5e5056d939 [x86, InstCombine] fix masked load pass-through operand to be a zero vector
This bug was introduced with:
http://reviews.llvm.org/rL262269

AVX masked loads are specified to set vector lanes to zero when the high bit of the mask
element for that lane is zero:
"If the mask is 0, the corresponding data element is set to zero in the load form of these
instructions, and unmodified in the store form." --Intel manual

Differential Revision: http://reviews.llvm.org/D19017

llvm-svn: 266148
2016-04-12 23:16:23 +00:00
Mehdi Amini d5faa267c4 Add a pass to name anonymous/nameless function
Summary:
For correct handling of alias to nameless
function, we need to be able to refer them through a GUID in the summary.
Here we name them using a hash of the non-private global names in the module.

Reviewers: tejohnson

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D18883

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266132
2016-04-12 21:35:28 +00:00
JF Bastien f90029bb14 NFC: MergeFunctions return early
Same effect, easier to read.

llvm-svn: 266128
2016-04-12 21:23:05 +00:00
Teresa Johnson c86af3345c [ThinLTO] Only compute imports for current module in FunctionImport pass
Summary:
The function import pass was computing all the imports for all the
modules in the index, and only using the imports for the current module.
Change this to instead compute only for the given module. This means
that the exports list can't be populated, but they weren't being used
anyway.

Longer term, the linker can collect all the imports and export lists
and serialize them out for consumption by the distributed backend
processes which use this pass.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18945

llvm-svn: 266125
2016-04-12 21:13:11 +00:00
JF Bastien 1bb32ac480 NFC: MergeFunctions update more comments
They are wordy. Some words were wrong.

llvm-svn: 266124
2016-04-12 21:13:01 +00:00
George Burgess IV 278199f615 Add the allocsize attribute to LLVM.
`allocsize` is a function attribute that allows users to request that
LLVM treat arbitrary functions as allocation functions.

This patch makes LLVM accept the `allocsize` attribute, and makes
`@llvm.objectsize` recognize said attribute.

The review for this was split into two patches for ease of reviewing:
D18974 and D14933. As promised on the revisions, I'm landing both
patches as a single commit.

Differential Revision: http://reviews.llvm.org/D14933

llvm-svn: 266032
2016-04-12 01:05:35 +00:00
JF Bastien 4f43cfd2c2 MergeFunctions: test alloca better
r237193 fix handling of alloca size / align in MergeFunctions, but only tested one and didn't follow FunctionComparator::cmpOperations's usual comparison pattern. It also didn't update Instruction.cpp:haveSameSpecialState which I'll do separately.

llvm-svn: 266022
2016-04-12 00:03:26 +00:00
Mehdi Amini ae280e54a9 ThinLTO renaming: use module hash instead of position in the summary
This is more robust to changes in the link ordering.

Differential Revision: http://reviews.llvm.org/D18946

From: Mehdi Amini <mehdi.amini@apple.com>
llvm-svn: 266018
2016-04-11 23:26:46 +00:00
Hans Wennborg e9134897f4 Fix a couple of redundant conditional expressions (PR27283, PR28282)
llvm-svn: 265987
2016-04-11 20:35:01 +00:00
Matthew Simpson 53207a99f9 [LoopUtils, LV] Fix PR27246 (first-order recurrences)
This patch ensures that when we detect first-order recurrences, we reject a phi
node if its previous value is also a phi node. During vectorization the initial
and previous values of the recurrence are shuffled together to create the value
for the current iteration. However, phi nodes are not widened like other
instructions. This fixes PR27246.

Differential Revision: http://reviews.llvm.org/D18971

llvm-svn: 265983
2016-04-11 19:48:18 +00:00
Sanjay Patel b91bcd704a add FIXME comment; NFC
llvm-svn: 265970
2016-04-11 17:35:57 +00:00
Sanjay Patel 3a48e9823e add an assert for safety; NFC
llvm-svn: 265969
2016-04-11 17:27:44 +00:00
Sanjay Patel 4b9c682acf variable names start with a capital letter; NFC
llvm-svn: 265968
2016-04-11 17:25:23 +00:00
Sanjay Patel 371290790f [InstCombine] use canEvaluateShiftedShift() to handle the lshr case (NFCI)
We need just a couple of logic tweaks to consolidate the shl and lshr cases.

This is step 5 of refactoring to solve PR26760:
https://llvm.org/bugs/show_bug.cgi?id=26760

llvm-svn: 265965
2016-04-11 17:11:55 +00:00
Sanjay Patel 816ec8882a [InstCombine] don't try to shift an illegal amount (PR26760)
This is the straightforward fix for PR26760:
https://llvm.org/bugs/show_bug.cgi?id=26760

But we still need to make some changes to generalize this helper function
and then send the lshr case into here.

llvm-svn: 265960
2016-04-11 16:50:32 +00:00
Sanjay Patel bd8b779d16 [InstCombine] rename variables in shifted-shift helper function (NFCI)
This is step 3 of refactoring to solve PR26760:
https://llvm.org/bugs/show_bug.cgi?id=26760

llvm-svn: 265954
2016-04-11 16:11:07 +00:00
Sanjay Patel 6eaff5cec6 [InstCombine] add helper function for shift-shift optimization (NFCI)
This is step 2 of refactoring to solve PR26760:
https://llvm.org/bugs/show_bug.cgi?id=26760

llvm-svn: 265951
2016-04-11 15:43:41 +00:00
Sanjoy Das f9d88e650b This reverts commit r265913 and r265912
See PR27315

r265913: "[IndVars] Eliminate op.with.overflow when possible"

r265912: "[SCEV] See through op.with.overflow intrinsics"
llvm-svn: 265950
2016-04-11 15:26:18 +00:00
Teresa Johnson 2d5487cf44 [ThinLTO] Move summary computation from BitcodeWriter to new pass
Summary:
This is the first step in also serializing the index out to LLVM
assembly.

The per-module summary written to bitcode is moved out of the bitcode
writer and to a new analysis pass (ModuleSummaryIndexWrapperPass).
The pass itself uses a new builder class to compute index, and the
builder class is used directly in places where we don't have a pass
manager (e.g. llvm-as).

Because we are computing summaries outside of the bitcode writer, we no
longer can use value ids created by the bitcode writer's
ValueEnumerator. This required changing the reference graph edge type
to use a new ValueInfo class holding a union between a GUID (combined
index) and Value* (permodule index). The Value* are converted to the
appropriate value ID during bitcode writing.

Also, this enables removal of the BitWriter library's dependence on the
Analysis library that was previously required for the summary computation.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D18763

llvm-svn: 265941
2016-04-11 13:58:45 +00:00
Sanjoy Das a07ad647ee [IndVars] Eliminate op.with.overflow when possible
Summary:
If we can prove that an op.with.overflow intrinsic does not overflow, we
can get rid of the intrinsic, and replace it with non-wrapping
arithmetic.

Reviewers: atrick, regehr

Subscribers: sanjoy, mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D18685

llvm-svn: 265913
2016-04-10 22:50:31 +00:00
Elena Demikhovsky 751ed0a06a Loop vectorization with uniform load
Vectorization cost of uniform load wasn't correctly calculated.
As a result, a simple loop that loads a uniform value wasn't vectorized.

Differential Revision: http://reviews.llvm.org/D18940

llvm-svn: 265901
2016-04-10 16:53:19 +00:00
Teresa Johnson 3255eec16c [ThinLTO] Remove unused parameter (NFC)
llvm-svn: 265900
2016-04-10 15:17:26 +00:00
Sanjoy Das dd77e1e6a5 Maintain calling convention when inling calls to llvm.deoptimize
The behavior here was buggy -- we'd forget the calling convention after
inlining a callsite calling llvm.deoptimize.

llvm-svn: 265867
2016-04-09 00:22:59 +00:00
Evgeny Stupachenko 8788048403 test commit
llvm-svn: 265840
2016-04-08 20:20:38 +00:00
Duncan P. N. Exon Smith bb2c3e199e ValueMapper: Extract llvm::RemapFunction from IRMover.cpp, NFC
Strip out the remapping parts of IRLinker::linkFunctionBody and put them
in ValueMapper.cpp under the name Mapper::remapFunction (with a
top-level entry-point llvm::RemapFunction).

This is a nice cleanup on its own since it puts the remapping code
together and shares a single Mapper context for the entire
IRLinker::linkFunctionBody Call.  Besides that, this will make it easier
to break the co-recursion between IRMover.cpp and ValueMapper.cpp in
follow ups.

llvm-svn: 265835
2016-04-08 19:26:32 +00:00
Duncan P. N. Exon Smith adcebdf2d1 ValueMapper: Always use Mapper::mapValue from remapInstruction, NFCI
Use Mapper::mapValue instead of llvm::MapValue from
Mapper::remapInstruction when mapping an incoming block for a PHINode
(follow-up to r265832).  This will implicitly pass along the
Materializer argument, but when this code was added in r133513 there was
no Materializer argument.  I suspect this call to MapValue was just
missed in r182776 since it's not observable (basic blocks can't be
materialized, and they don't reference other values).

llvm-svn: 265833
2016-04-08 19:17:13 +00:00
Duncan P. N. Exon Smith a574e7a7a4 ValueMapper: Roll RemapInstruction into Mapper, NFC
Add Mapper::remapInstruction, move the guts of llvm::RemapInstruction
into it, and use the same Mapper for most of the calls to MapValue and
MapMetadata.  There should be no functionality change here.

I left off the call to MapValue that wasn't passing in a Materializer
argument (for basic blocks of PHINodes).  It shouldn't change
functionality either, but I'm suspicious enough to commit separately.

llvm-svn: 265832
2016-04-08 19:09:34 +00:00
Duncan P. N. Exon Smith 69341e6abc ValueMapper: Don't memoize metadata when RF_NoModuleLevelChanges
Prevent the Metadata side-table in ValueMap from growing unnecessarily
when RF_NoModuleLevelChanges.  As a drive-by, make ValueMap::hasMD,
which apparently had no users until I used it here for testing, actually
compile.

llvm-svn: 265828
2016-04-08 18:49:36 +00:00
Duncan P. N. Exon Smith e05ff7c1a7 ValueMapper: Stop memoizing MDStrings
Stop adding MDString to the Metadata section of the ValueMap in
MapMetadata.  It blows up the size of the map for no benefit, since we
can always return quickly anyway.

There is a potential follow-up that I don't think I'll push on right
away, but maybe someone else is interested:  stop checking for a
pre-mapped MDString, and move the `isa<MDString>()` checks in
Mapper::mapSimpleMetadata and MDNodeMapper::getMappedOp in front of the
`VM.getMappedMD()` calls.  While this would preclude explicitly
remapping MDStrings it would probably be a little faster.

llvm-svn: 265827
2016-04-08 18:47:02 +00:00
David Majnemer 56737722e4 [InstCombine] Fix miscompile in FoldSPFofSPF
We had a select of a cast of a select but attempted to replace the outer
select with the inner select dispite their incompatible types.

Patch by Anton Korobeynikov!

This fixes PR27236.

llvm-svn: 265805
2016-04-08 16:51:49 +00:00
David Majnemer fcc5811797 [InstCombine] Add a peephole for redundant assumes
Two or more identical assumes are occasionally next to each other in a
basic block.
While our generic machinery will turn a redundant assume into a no-op,
it is not super cheap.
We can perform a simpler check to achieve the same result for this case.

llvm-svn: 265801
2016-04-08 16:37:12 +00:00
David Majnemer 60c6abc3cc [LoopVectorize] Register cloned assumptions
InstCombine cannot effectively remove redundant assumptions without them
registered in the assumption cache.  The vectorizer can create identical
assumptions but doesn't register them with the cache, resulting in
slower compile times because InstCombine tries to reason about a lot
more assumptions.

Fix this by registering the cloned assumptions.

llvm-svn: 265800
2016-04-08 16:37:10 +00:00
Silviu Baranga 6f444dfd55 Re-commit [SCEV] Introduce a guarded backedge taken count and use it in LAA and LV
This re-commits r265535 which was reverted in r265541 because it
broke the windows bots. The problem was that we had a PointerIntPair
which took a pointer to a struct allocated with new. The problem
was that new doesn't provide sufficient alignment guarantees.
This pattern was already present before r265535 and it just happened
to work. To fix this, we now separate the PointerToIntPair from the
ExitNotTakenInfo struct into a pointer and a bool.

Original commit message:

Summary:
When the backedge taken codition is computed from an icmp, SCEV can
deduce the backedge taken count only if one of the sides of the icmp
is an AddRecExpr. However, due to sign/zero extensions, we sometimes
end up with something that is not an AddRecExpr.

However, we can use SCEV predicates to produce a 'guarded' expression.
This change adds a method to SCEV to get this expression, and the
SCEV predicate associated with it.

In HowManyGreaterThans and HowManyLessThans we will now add a SCEV
predicate associated with the guarded backedge taken count when the
analyzed SCEV expression is not an AddRecExpr. Note that we only do
this as an alternative to returning a 'CouldNotCompute'.

We use new feature in Loop Access Analysis and LoopVectorize to analyze
and transform more loops.

Reviewers: anemet, mzolotukhin, hfinkel, sanjoy

Subscribers: flyingforyou, mcrosier, atrick, mssimpso, sanjoy, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D17201

llvm-svn: 265786
2016-04-08 14:29:09 +00:00
Duncan P. N. Exon Smith 4ec55f8ab6 Reapply "ValueMapper: Treat LocalAsMetadata more like function-local Values"
This reverts commit r265765, reapplying r265759 after changing a call from
LocalAsMetadata::get to ValueAsMetadata::get (and adding a unit test).  When a
local value is mapped to a constant (like "i32 %a" => "i32 7"), the new debug
intrinsic operand may no longer be pointing at a local.

    http://lab.llvm.org:8080/green/job/clang-stage1-configure-RA_build/19020/

The previous coommit message follows:

--

This is a partial re-commit -- maybe more of a re-implementation -- of
r265631 (reverted in r265637).

This makes RF_IgnoreMissingLocals behave (almost) consistently between
the Value and the Metadata hierarchy.  In particular:

  - MapValue returns nullptr or "metadata !{}" for missing locals in
    MetadataAsValue/LocalAsMetadata bridging paris, depending on
    the RF_IgnoreMissingLocals flag.

  - MapValue doesn't memoize LocalAsMetadata-related results.

  - MapMetadata no longer deals with LocalAsMetadata or
    RF_IgnoreMissingLocals at all.  (This wasn't in r265631 at all, but
    I realized during testing it would make the patch simpler with no
    loss of generality.)

r265631 went too far, making both functions universally ignore
RF_IgnoreMissingLocals.  This broke building (e.g.) compiler-rt.
Reassociate (and possibly other passes) don't currently maintain
dominates-use invariants for metadata operands, resulting in IR like
this:

    define void @foo(i32 %arg) {
      call void @llvm.some.intrinsic(metadata i32 %x)
      %x = add i32 1, i32 %arg
    }

If the inliner chooses to inline @foo into another function, then
RemapInstruction will call `MapValue(metadata i32 %x)` and assert that
the return is not nullptr.

I've filed PR27273 to add a Verifier check and fix the underlying
problem in the optimization passes.

As a workaround, return `!{}` instead of nullptr for unmapped
LocalAsMetadata when RF_IgnoreMissingLocals is unset.  Otherwise, match
the behaviour of r265631.

Original commit message:

    ValueMapper: Make LocalAsMetadata match function-local Values

    Start treating LocalAsMetadata similarly to function-local members of
    the Value hierarchy in MapValue and MapMetadata.

      - Don't memoize them.
      - Return nullptr if they are missing.

    This also cleans up ConstantAsMetadata to stop listening to the
    RF_IgnoreMissingLocals flag.

llvm-svn: 265768
2016-04-08 03:13:22 +00:00
Duncan P. N. Exon Smith 805873148a Revert "ValueMapper: Treat LocalAsMetadata more like function-local Values"
This reverts commit r265759, since even this limited version breaks some
bots:
  http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/3311
  http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-autoconf/builds/17696

This also reverts r265761 "ValueMapper: Unduplicate
RF_NoModuleLevelChanges check, NFC", since I had trouble separating it
from r265759.

llvm-svn: 265765
2016-04-08 00:56:21 +00:00
Sanjoy Das 5ce3272833 Don't IPO over functions that can be de-refined
Summary:
Fixes PR26774.

If you're aware of the issue, feel free to skip the "Motivation"
section and jump directly to "This patch".

Motivation:

I define "refinement" as discarding behaviors from a program that the
optimizer has license to discard.  So transforming:

```
void f(unsigned x) {
  unsigned t = 5 / x;
  (void)t;
}
```

to

```
void f(unsigned x) { }
```

is refinement, since the behavior went from "if x == 0 then undefined
else nothing" to "nothing" (the optimizer has license to discard
undefined behavior).

Refinement is a fundamental aspect of many mid-level optimizations done
by LLVM.  For instance, transforming `x == (x + 1)` to `false` also
involves refinement since the expression's value went from "if x is
`undef` then { `true` or `false` } else { `false` }" to "`false`" (by
definition, the optimizer has license to fold `undef` to any non-`undef`
value).

Unfortunately, refinement implies that the optimizer cannot assume
that the implementation of a function it can see has all of the
behavior an unoptimized or a differently optimized version of the same
function can have.  This is a problem for functions with comdat
linkage, where a function can be replaced by an unoptimized or a
differently optimized version of the same source level function.

For instance, FunctionAttrs cannot assume a comdat function is
actually `readnone` even if it does not have any loads or stores in
it; since there may have been loads and stores in the "original
function" that were refined out in the currently visible variant, and
at the link step the linker may in fact choose an implementation with
a load or a store.  As an example, consider a function that does two
atomic loads from the same memory location, and writes to memory only
if the two values are not equal.  The optimizer is allowed to refine
this function by first CSE'ing the two loads, and the folding the
comparision to always report that the two values are equal.  Such a
refined variant will look like it is `readonly`.  However, the
unoptimized version of the function can still write to memory (since
the two loads //can// result in different values), and selecting the
unoptimized version at link time will retroactively invalidate
transforms we may have done under the assumption that the function
does not write to memory.

Note: this is not just a problem with atomics or with linking
differently optimized object files.  See PR26774 for more realistic
examples that involved neither.

This patch:

This change introduces a new set of linkage types, predicated as
`GlobalValue::mayBeDerefined` that returns true if the linkage type
allows a function to be replaced by a differently optimized variant at
link time.  It then changes a set of IPO passes to bail out if they see
such a function.

Reviewers: chandlerc, hfinkel, dexonsmith, joker.eph, rnk

Subscribers: mcrosier, llvm-commits

Differential Revision: http://reviews.llvm.org/D18634

llvm-svn: 265762
2016-04-08 00:48:30 +00:00
Duncan P. N. Exon Smith 0fa8aca0e6 ValueMapper: Unduplicate RF_NoModuleLevelChanges check, NFC
llvm-svn: 265761
2016-04-08 00:41:10 +00:00
Duncan P. N. Exon Smith 267185ec92 ValueMapper: Treat LocalAsMetadata more like function-local Values
This is a partial re-commit -- maybe more of a re-implementation -- of
r265631 (reverted in r265637).

This makes RF_IgnoreMissingLocals behave (almost) consistently between
the Value and the Metadata hierarchy.  In particular:

  - MapValue returns nullptr or "metadata !{}" for missing locals in
    MetadataAsValue/LocalAsMetadata bridging paris, depending on
    the RF_IgnoreMissingLocals flag.

  - MapValue doesn't memoize LocalAsMetadata-related results.

  - MapMetadata no longer deals with LocalAsMetadata or
    RF_IgnoreMissingLocals at all.  (This wasn't in r265631 at all, but
    I realized during testing it would make the patch simpler with no
    loss of generality.)

r265631 went too far, making both functions universally ignore
RF_IgnoreMissingLocals.  This broke building (e.g.) compiler-rt.
Reassociate (and possibly other passes) don't currently maintain
dominates-use invariants for metadata operands, resulting in IR like
this:

    define void @foo(i32 %arg) {
      call void @llvm.some.intrinsic(metadata i32 %x)
      %x = add i32 1, i32 %arg
    }

If the inliner chooses to inline @foo into another function, then
RemapInstruction will call `MapValue(metadata i32 %x)` and assert that
the return is not nullptr.

I've filed PR27273 to add a Verifier check and fix the underlying
problem in the optimization passes.

As a workaround, return `!{}` instead of nullptr for unmapped
LocalAsMetadata when RF_IgnoreMissingLocals is unset.  Otherwise, match
the behaviour of r265631.

Original commit message:

    ValueMapper: Make LocalAsMetadata match function-local Values

    Start treating LocalAsMetadata similarly to function-local members of
    the Value hierarchy in MapValue and MapMetadata.

      - Don't memoize them.
      - Return nullptr if they are missing.

    This also cleans up ConstantAsMetadata to stop listening to the
    RF_IgnoreMissingLocals flag.

llvm-svn: 265759
2016-04-08 00:33:44 +00:00
Ulrich Weigand fc23907673 [GVN] Address review comments for D18662
As suggested by Chandler in his review comments for D18662, this
follow-on patch renames some variables in GetLoadValueForLoad and
CoerceAvailableValueToLoadType to hopefully make it more obvious
which variables hold value sizes and which hold load/store sizes.

No functional change intended.

llvm-svn: 265687
2016-04-07 15:55:11 +00:00
Ulrich Weigand 6e6966460a [GVN] Fix handling of sub-byte types in big-endian mode
When GVN wants to re-interpret an already available value in a smaller
type, it needs to right-shift the value on big-endian systems to ensure
the correct bytes are accessed.  The shift value is the difference of
the sizes of the two types.

This is correct as long as both types occupy multiples of full bytes.
However, when one of them is a sub-byte type like i1, this no longer
holds true: we still need to shift, but only to access the correct
*byte*.  Accessing bits within the byte requires no shift in either
endianness; e.g. an i1 resides in the least-significant bit of its
containing byte on both big- and little-endian systems.

Therefore, the appropriate shift value to be used is the difference of
the *storage* sizes of the two types.  This is already handled correctly
in one place where such a shift takes place (GetStoreValueForLoad), but
is incorrect in two other places: GetLoadValueForLoad and
CoerceAvailableValueToLoadType.

This patch changes both places to use the storage size as well.

Differential Revision: http://reviews.llvm.org/D18662

llvm-svn: 265684
2016-04-07 15:45:02 +00:00
Benjamin Kramer 4fb78518f1 Make helper functions static. NFC.
llvm-svn: 265653
2016-04-07 10:10:09 +00:00
Duncan P. N. Exon Smith 45601e867d Revert "ValueMapper: Make LocalAsMetadata match function-local Values"
This reverts commit r265631, since it caused bot failures:
http://lab.llvm.org:8011/builders/clang-ppc64be-linux/builds/3256
http://lab.llvm.org:8011/builders/clang-cmake-aarch64-42vma/builds/7272

Looks like something is depending on the old behaviour.  I'll try to
track it down and recommit.

llvm-svn: 265637
2016-04-07 02:10:50 +00:00
Duncan P. N. Exon Smith fdccad925c ValueMapper: Allow RF_IgnoreMissingLocals and RF_NullMapMissingGlobalValues
Remove the assertion that disallowed the combination, since
RF_IgnoreMissingLocals should have no effect on globals.  As it happens,
RF_NullMapMissingGlobalValues asserted in MapValue(Constant*,...), so I
also changed a cast to a cast_or_null to get my test passing.

llvm-svn: 265633
2016-04-07 01:22:45 +00:00
Duncan P. N. Exon Smith c1e4070708 ValueMapper: Make LocalAsMetadata match function-local Values
Start treating LocalAsMetadata similarly to function-local members of
the Value hierarchy in MapValue and MapMetadata.

  - Don't memoize them.
  - Return nullptr if they are missing.

This also cleans up ConstantAsMetadata to stop listening to the
RF_IgnoreMissingLocals flag.

llvm-svn: 265631
2016-04-07 01:08:39 +00:00
Duncan P. N. Exon Smith da68cbc4ad IR: RF_IgnoreMissingValues => RF_IgnoreMissingLocals, NFC
Clarify what this RemapFlag actually means.

  - Change the flag name to match its intended behaviour.
  - Clearly document that it's not supposed to affect globals.
  - Add a host of FIXMEs to indicate how to fix the behaviour to match
    the intent of the flag.

RF_IgnoreMissingLocals should only affect the behaviour of
RemapInstruction for function-local operands; namely, for operands of
type Argument, Instruction, and BasicBlock.  Currently, it is *only*
passed into RemapInstruction calls (and the transitive MapValue calls
that it makes).

When I split Metadata from Value I didn't understand the flag, and I
used it in a bunch of places for "global" metadata.

This commit doesn't have any functionality change, but prepares to
cleanup MapMetadata and MapValue.

llvm-svn: 265628
2016-04-07 00:26:43 +00:00
Michael Zolotukhin 56ad4048ae Follow-up for r265605: don't mutate vector we're iterating.
llvm-svn: 265625
2016-04-07 00:09:42 +00:00
Mike Aizatsky 70ea45306a [sancov] enabling coverage edge pruning by default.
Differential Revision: http://reviews.llvm.org/D18844

llvm-svn: 265615
2016-04-06 23:24:37 +00:00
Michael Zolotukhin 97567e141e [LoopUnroll] Fix the way we update DT after complete unrolling.
Updating dominators for exit-blocks of the unrolled loops is not enough,
as shown in PR27157. The proper way is to update dominators for all
dominance-children of original loop blocks.

llvm-svn: 265605
2016-04-06 21:47:12 +00:00
JF Bastien 800f87a871 NFC: make AtomicOrdering an enum class
Summary:
In the context of http://wg21.link/lwg2445 C++ uses the concept of
'stronger' ordering but doesn't define it properly. This should be fixed
in C++17 barring a small question that's still open.

The code currently plays fast and loose with the AtomicOrdering
enum. Using an enum class is one step towards tightening things. I later
also want to tighten related enums, such as clang's
AtomicOrderingKind (which should be shared with LLVM as a 'C++ ABI'
enum).

This change touches a few lines of code which can be improved later, I'd
like to keep it as NFC for now as it's already quite complex. I have
related changes for clang.

As a follow-up I'll add:
  bool operator<(AtomicOrdering, AtomicOrdering) = delete;
  bool operator>(AtomicOrdering, AtomicOrdering) = delete;
  bool operator<=(AtomicOrdering, AtomicOrdering) = delete;
  bool operator>=(AtomicOrdering, AtomicOrdering) = delete;
This is separate so that clang and LLVM changes don't need to be in sync.

Reviewers: jyknight, reames

Subscribers: jyknight, llvm-commits

Differential Revision: http://reviews.llvm.org/D18775

llvm-svn: 265602
2016-04-06 21:19:33 +00:00
Fiona Glaser 045afc4f66 Loop Unroll: add options and tweak to make Partial unrolling more useful
1. Add FullUnrollMaxCount option that works like MaxCount, but also limits
   the unroll count for fully unrolled loops. So if a loop has an iteration
   count over this, it won't fully unroll.
2. Add CLI options for MaxCount and the new option, so they can be tested
   (plus a test).
3. Make partial unrolling obey MaxCount.

An example use-case (the out of tree one this is originally designed for) is
a target’s TTI can analyze a loop and decide on a max unroll count separate
from the size threshold, e.g. based on register pressure, then constrain
LoopUnroll to not exceed that, regardless of the size of the unrolled loop.

llvm-svn: 265562
2016-04-06 16:57:25 +00:00
Fiona Glaser 16332ba861 LoopUnroll: only allow non-modulo Partial unrolling when Runtime=true
Patch by Evgeny Stupachenko <evstupac@gmail.com>.

llvm-svn: 265558
2016-04-06 16:43:45 +00:00
Silviu Baranga a393baf1fd Revert r265535 until we know how we can fix the bots
llvm-svn: 265541
2016-04-06 14:06:32 +00:00
Chad Rosier 074ce836f0 Simplify logic. NFC.
llvm-svn: 265537
2016-04-06 13:27:13 +00:00
Silviu Baranga 72b4a4a330 [SCEV] Introduce a guarded backedge taken count and use it in LAA and LV
Summary:
When the backedge taken codition is computed from an icmp, SCEV can
deduce the backedge taken count only if one of the sides of the icmp
is an AddRecExpr. However, due to sign/zero extensions, we sometimes
end up with something that is not an AddRecExpr.

However, we can use SCEV predicates to produce a 'guarded' expression.
This change adds a method to SCEV to get this expression, and the
SCEV predicate associated with it.

In HowManyGreaterThans and HowManyLessThans we will now add a SCEV
predicate associated with the guarded backedge taken count when the
analyzed SCEV expression is not an AddRecExpr. Note that we only do
this as an alternative to returning a 'CouldNotCompute'.

We use new feature in Loop Access Analysis and LoopVectorize to analyze
and transform more loops.

Reviewers: anemet, mzolotukhin, hfinkel, sanjoy

Subscribers: flyingforyou, mcrosier, atrick, mssimpso, sanjoy, mzolotukhin, llvm-commits

Differential Revision: http://reviews.llvm.org/D17201

llvm-svn: 265535
2016-04-06 13:18:26 +00:00
Richard Trieu f35d4b0928 Add parentheses to silence warning.
llvm-svn: 265516
2016-04-06 04:22:00 +00:00
Duncan P. N. Exon Smith 6f2e37429a ValueMapper: Fix delayed blockaddress handling after r265273
r265273 added Mapper::mapBlockAddress, which delays mapping a
blockaddress value until the function has a body.  The condition was
backwards, and should be checking Function::empty instead of
GlobalValue::isDeclaration.

llvm-svn: 265508
2016-04-06 02:25:12 +00:00
Sanjoy Das 99abb2728b [RS4GC] Add a comment
llvm-svn: 265503
2016-04-06 01:33:54 +00:00
Sanjoy Das 8d89a2b296 [RS4GC] NFC cleanup of the DeferredReplacement class
Instead of constructors use clearly named factory methods.

llvm-svn: 265486
2016-04-05 23:18:53 +00:00
Sanjoy Das 49e974b33b [RS4GC] Better codegen for deoptimize calls
Don't emit a gc.result for a statepoint lowered from
@llvm.experimental.deoptimize since the call into __llvm_deoptimize is
effectively noreturn.  Instead follow the corresponding gc.statepoint
with an "unreachable".

llvm-svn: 265485
2016-04-05 23:18:35 +00:00
Duncan P. N. Exon Smith 818e5f38d2 Try harder to appease MSVC after r265456
r265465 wasn't good enough.  I need to spell out all the moves.

llvm-svn: 265470
2016-04-05 21:25:33 +00:00
Duncan P. N. Exon Smith 1de3c7e790 IR: Introduce ConstantAggregate, NFC
Add a common parent class for ConstantArray, ConstantVector, and
ConstantStruct called ConstantAggregate.  These are the aggregate
subclasses of Constant that take operands.

This is mainly a cleanup, adding common `isa` target and removing
duplicated code.  However, it also simplifies caching which constants
point transitively at `GlobalValue` (a possible future direction).

llvm-svn: 265466
2016-04-05 21:10:45 +00:00
Duncan P. N. Exon Smith f880d35b80 Try to appease MSVC after r265456
I can't remember if adding `= default` will make MSVC happy, or if I
have to spell this out.  Let's try the cleaner version first.

llvm-svn: 265465
2016-04-05 21:07:01 +00:00
Duncan P. N. Exon Smith ea7df770ae ValueMapper: Rewrite Mapper::mapMetadata without recursion
This commit completely rewrites Mapper::mapMetadata (the implementation
of llvm::MapMetadata) using an iterative algorithm.  The guts of the new
algorithm are in MDNodeMapper::map, the entry function in a new class.

Previously, Mapper::mapMetadata performed a recursive exploration of the
graph with eager "just in case there's a reason" malloc traffic.

The new algorithm has these benefits:

  - New nodes and temporaries are not created eagerly.
  - Uniquing cycles are not duplicated (see new unit test).
  - No recursion.

Given a node to map, it does this:

 1. Use a worklist to perform a post-order traversal of the transitively
    referenced unmapped nodes.

 2. Track which nodes will change operands, and which will have new
    addresses in the mapped scheme.  Propagate the changes through the
    POT until fixed point, to pick up uniquing cycles that need to
    change.

 3. Map all the distinct nodes without touching their operands.  If
    RF_MoveDistinctMetadata, they get mapped to themselves; otherwise,
    they get mapped to clones.

 4. Map the uniqued nodes (bottom-up), lazily creating temporaries for
    forward references as needed.

 5. Remap the operands of the distinct nodes.

Mehdi helped me out by profiling this with -flto=thin.  On his workload
(importing/etc. for opt.cpp), MapMetadata sped up by 15%, contributed
about 50% less to persistent memory, and made about 100x fewer calls to
malloc.  The speedup is less than I'd hoped.  The profile mainly blames
DenseMap lookups; perhaps there's a way to reduce them (e.g., by
disallowing remapping of MDString).

It would be nice to break the strange remaining recursion on the Value
side: MapValue => materializeInitFor => RemapInstruction => MapValue.  I
think we could do this by having materializeInitFor return a worklist of
things to be remapped.

llvm-svn: 265456
2016-04-05 20:23:21 +00:00
David L Kreitzer 188de5ae69 Adds the ability to use an epilog remainder loop during loop unrolling and makes
this the default behavior.

Patch by Evgeny Stupachenko (evstupac@gmail.com).

Differential Revision: http://reviews.llvm.org/D18158

llvm-svn: 265388
2016-04-05 12:19:35 +00:00
Dmitry Polukhin a3d5b0b218 [IFUNC] Use GlobalIndirectSymbol when aliases and ifuncs have something similar
Second part extracted from http://reviews.llvm.org/D15525

Use GlobalIndirectSymbol in all cases when aliases and ifuncs have
something in common.

Differential Revision: http://reviews.llvm.org/D18754

llvm-svn: 265382
2016-04-05 08:47:51 +00:00
Sanjay Patel e77c7de459 use range loop; NFCI
llvm-svn: 265360
2016-04-04 23:05:06 +00:00
Zia Ansari a82a58a4e5 Enable unroll for constant bound loops when TripCount is not modulo of unroll factor, reducing it to maximum power-of-2 that satisfies threshold limit.
Commit for Evgeny Stupachenko (evstupac@gmail.com)

Differential Revision: http://reviews.llvm.org/D18290

llvm-svn: 265337
2016-04-04 19:24:46 +00:00
Betul Buyukkurt 18131c4216 [PGO] Avoid instrumenting direct callee's at value sites.
Direct callees' that are cast to other function prototypes,
show up in the Call/Invoke instructions as ConstantExpr's.
Currently llvm::CallSite's getCalledFunction() fails
to return the callees in such expressions as direct calls.
Value profiling should avoid instrumenting such cases. Mostly NFC.

llvm-svn: 265330
2016-04-04 18:56:36 +00:00
Teresa Johnson 0beb858e97 [ThinLTO] Augment FunctionImport dump with value name to GUID map
Summary:
To aid in debugging, dump out the correlation between value names and
GUID for each source module when it is materialized. This will make it
easier to comprehend the earlier summary-based function importing debug
trace which only has access to and prints the GUIDs.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18556

llvm-svn: 265326
2016-04-04 18:52:23 +00:00
Duncan P. N. Exon Smith 8e65f8ddfd ValueMapper: Remove old FIXMEs; almost NFC
Remove a few old FIXMEs from the original commit of the Metadata/Value
split in r223802.  These are commented out assertions to the effect that
calls between mapValue and mapMetadata never return nullptr.

(The only behaviour change is that Mapper::mapSimpleMetadata memoizes
the nullptr return.)

When I originally rewrote the mapping code, I thought we could be
stricter in the new metadata hierarchy and never return nullptr when
RF_NullMapMissingGlobalValues was off.  It's still not entirely clear to
me why these assertions failed (a few months ago, I had a theory that I
forgot to write down, but that's helping no one).

Understood or not, I no longer see how these commented-out assertions
would be useful.  I'm relegating them to the annals of source control
before making significant changes to ValueMapper.cpp.

llvm-svn: 265282
2016-04-04 04:59:56 +00:00
Duncan P. N. Exon Smith 756e1c3db4 ValueMapper: Disallow metadata mapping recursion through mapValue
This adds an assertion to maintain the property from r265273.  When
Mapper::mapSimpleMetadata calls Mapper::mapValue, it should not find its
way back to mapMetadataImpl.  This guarantees that mapSimpleMetadata is
not involved in any recursion.

Since Mapper::mapValue calls out to arbitrary materializers, we need to
save a bit on the ValueMap to make this assertion effective.

There should be no functionality change here.  This co-recursion should
already have been impossible.

llvm-svn: 265276
2016-04-03 20:54:51 +00:00
Duncan P. N. Exon Smith a997856b3d Work around MSVC failure from r265273
http://lab.llvm.org:8011/builders/sanitizer-windows/builds/19726

llvm-svn: 265275
2016-04-03 20:42:21 +00:00
Duncan P. N. Exon Smith c6065e3a25 ValueMapper: Avoid recursion in mapSimplifiedMetadata, NFC
The main change is to delay materializing GlobalValue initializers from
Mapper::mapValue until Mapper::~Mapper.  This effectively removes all
recursion from mapSimplifiedMetadata, as promised in r265270.
mapSimplifiedMetadata calls mapValue for ConstantAsMetadata nodes to
find the mapped constant, and now it shouldn't be possible for mapValue
to indirectly re-invoke mapMetadata.  I'll add an assertion to that
effect in a follow-up (separated so that the assertion can easily be
reverted independently, if it comes to that).

This a step toward a broader goal: converting Mapper::mapMetadataImpl
from a recursive to an iterative algorithm.

When a BlockAddress points at a BasicBlock inside an unmaterialized
function body, we need to delay it until the function body is
materialized in Mapper::~Mapper.  This commit creates a temporary
BasicBlock and returns a new BlockAddress, then RAUWs the BasicBlock
once it is known.  This situation should be extremely rare since a
BlockAddress is usually used from within the function it's referencing
(and BlockAddress itself is rare).

There should be no observable functionality change.

llvm-svn: 265273
2016-04-03 20:17:45 +00:00
Duncan P. N. Exon Smith ae8bd4bd11 ValueMapper: Split out mapSimpleMetadata, NFC
Split out a helper for mapping metadata without operands.  This is any
metadata that is not an MDNode, and any MDNode where the answer is known
without looking at operands.

Through some weird twists, this function is co-recursive:

    mapSimpleMetadata
    => MapValue
    => materializeInitFor
    => linkFunctionBody
    => RemapInstructions
    => MapMetadata
    => mapSimpleMetadata

I plan to break the recursion in a follow-up.

llvm-svn: 265270
2016-04-03 19:31:01 +00:00
Duncan P. N. Exon Smith 829dc87a68 ValueMapper: Introduce Mapper helper class, NFC
Remove a bunch of boilerplate from ValueMapper.cpp by using a new
file-local class called Mapper.

llvm-svn: 265268
2016-04-03 19:06:24 +00:00