Commit Graph

15033 Commits

Author SHA1 Message Date
Craig Topper 3ad8fbd205 [Reassociate] Teach ConvertShiftToMul to preserve nsw flag if the shift amount is not bitwidth - 1.
Multiply and shl have different signed overflow behavior in
some cases. But it looks like we should be ok as long as the
shift amount is less than bitwidth - 1.

Alive2: http://volta.cs.utah.edu:8080/z/MM4WZP

Differential Revision: https://reviews.llvm.org/D81189
2020-06-04 14:51:34 -07:00
Sanjay Patel 192cb71836 [InstCombine] avoid crashing on select-shuffle detection
As mentioned in the post-commit comments of D81013 -
the mask check API has to assume the shuffle is
not length-changing, but we have not ruled that out
in this code. Use the ShuffleVectorInst call instead.
2020-06-04 17:27:14 -04:00
Huihui Zhang 42048ff972 [NFC] Move test vscale-factor-out-constant.ll to AArch64 sub-directory.
Vscale scalable vector is specific to AArch64 target.

Bring back 'uglygep' check.
2020-06-04 12:55:28 -07:00
Sanjay Patel 8a96c1f627 [InstCombine] move vector select ahead of select-shuffle
select Cond, (shuf_sel X, Y), X --> shuf_sel X, (select Cond, Y, X)

A select of a select-shuffle ("blend" in x86 lingo) can be reversed
so that the select is done first.
This is a more limited version of what I was trying in D80658,
but it enables existing demanded bits transforms to catch some of the
motivating cases. The tricky bit in that seems to be that by moving
the shuffle later, we can always guarantee that poison is correctly
inhibited by the shuffle mask in the final value.

Alive2 checks for the basic tests:
http://volta.cs.utah.edu:8080/z/Qqd3RK
http://volta.cs.utah.edu:8080/z/S4wchM
http://volta.cs.utah.edu:8080/z/wf9zPL
http://volta.cs.utah.edu:8080/z/wJeEGk

Differential Revision: https://reviews.llvm.org/D81013
2020-06-04 14:29:13 -04:00
Huihui Zhang f7f1abdb88 [NFC] Temporarily disable check for 'uglygep' while investigating some buildbot failure.
The purpose of vscale-factor-out-constant.ll is to check we are crashing
with blind cast 'Factor' in a MulExpr to SCEVConstant.
2020-06-04 10:54:02 -07:00
Layton Kifer 7381fcdf62 [TRE] Allow accumulator elimination when base case returns non-constant
Remove the requirement, that when performing accumulator elimination,
all other cases must return the same dynamic constant. We can do this by
initializing the accumulator with the identity value of the accumulation
operation, and inserting an additional operation before any return.

Differential Revision: https://reviews.llvm.org/D80844
2020-06-04 10:34:42 -07:00
Huihui Zhang bd43f78c76 [LSR][SCEVExpander] Avoid blind cast 'Factor' to SCEVConstant in FactorOutConstant.
Summary:
In SCEVExpander FactorOutConstant(), when GEP indexing into/over scalable vector,
it is legal for the 'Factor' in a MulExpr to be the size of a scalable vector
instead of a compile-time constant.

Current upstream crash with the test attached.

Reviewers: efriedma, sdesmalen, sanjoy.google, mkazantsev

Reviewed By: efriedma

Subscribers: hiraditya, javed.absar, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80973
2020-06-04 10:33:39 -07:00
Sanjay Patel ecbf34c0e4 [PhaseOrdering] add more tests for vector reductions; NFC
More coverage for D80867.
2020-06-04 08:19:40 -04:00
Max Kazantsev 18134511d9 [Test] Add test showing missing opportunity of folding ICmp(Phi(Consts...)) 2020-06-04 18:26:24 +07:00
Yevgeny Rouban dcfa78a4cc Extend InvokeInst !prof branch_weights metadata to unwind branches
Allow InvokeInst to have the second optional prof branch weight for
its unwind branch. InvokeInst is a terminator with two successors.
It might have its unwind branch taken many times. If so
the BranchProbabilityInfo unwind branch heuristic can be inaccurate.
This patch allows a higher accuracy calculated with both branch
weights set.

Changes:
 - A new section about InvokeInst is added to
   the BranchWeightMetadata page. It states the old information that
   missed in the doc and adds new about the second branch weight.
 - Verifier is changed to allow either 1 or 2 branch weights
   for InvokeInst.
 - A new test is written for BranchProbabilityInfo to demonstrate
   the main improvement of the simple fix in calcMetadataWeights().
 - Several new testcases are created for Inliner. Those check that
    both weights are accounted for invoke instruction weight
    calculation.
 - PGOUseFunc::setBranchWeights() is fixed to be applicable to
   InvokeInst.

Reviewers: davidxl, reames, xur, yamauchi
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D80618
2020-06-04 15:37:15 +07:00
Yevgeny Rouban 417bcb8827 [Instruction] Remove setProfWeight()
Remove the function Instruction::setProfWeight() and make
use of Instruction::copyMetadata(.., {LLVMContext::MD_prof}).
This is correct for all use cases of setProfWeight() as it
is applied to CallBase instructions only.
This change results in prof metadata copied intact even if
the source has "VP". The old pair of calls
extractProfTotalWeight() + setProfWeight() resulted in
setting branch_weights if the source had "VP" data.

Reviewers: yamauchi, davidxl
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D80987
2020-06-04 15:10:55 +07:00
Fangrui Song c1ea672ddb [gcov] Delete XFAIL: host-byteorder-big-endian
Add two to function-numbering.ll and exit-block.ll
2020-06-03 19:58:28 -07:00
Arnold Schwaighofer 2e4c5d1c48 CoroSplit: Fix coroutine splitting for retcon and retcon.once
Summary:
For retcon and retcon.once coroutines we assume that all uses of spills
can be sunk past coro.begin. This simplifies handling of instructions
that escape the address of an alloca.

The current implementation would have issues if the address of the
alloca is escaped before coro.begin. (It also has issues with casts before and
uses of those casts after the coro.begin instruction)

  %alloca_addr = alloca ...
  %escape  = ptrtoint %alloca_addr
  coro.begin
  store %escape to %alloca_addr

rdar://60272809

Subscribers: hiraditya, modocache, mgrang, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81023
2020-06-03 12:10:58 -07:00
Simon Pilgrim c2e27ac1ce [VectorCombine] Add multi-use shl test for D80885 2020-06-03 19:42:15 +01:00
Florian Hahn 195fa4bfae [NewGVN] Add test to ensure metadata is preserved for ssa_copy (NFC)
Add a new test that checks that metadata is preserved when ssa_copy
calls introduced by PredicateInfo are replaced by their original
values.
2020-06-03 19:30:03 +01:00
Simon Pilgrim 9f8ea2e6cf [VectorCombine] Add multi-use multiply test for D80885 2020-06-03 18:54:03 +01:00
Dorit Nuzman a9fe69c359 [InstSimplify] fix bug in matching or-with-not op (PR46083) 2020-06-03 13:44:29 -04:00
Sanjay Patel a26cd73d33 [InstSimplify] add/move tests for or with not op (PR46083); NFC 2020-06-03 08:13:36 -04:00
Wei Mi 7a6c89427c [SampleFDO] Add use-sample-profile function attribute.
When sampleFDO is enabled, people may expect they can use
-fno-profile-sample-use to opt-out using sample profile for a certain file.
That could be either for debugging purpose or for performance tuning purpose.
However, when thinlto is enabled, if a function in file A compiled with
-fno-profile-sample-use is imported to another file B compiled with
-fprofile-sample-use, the inlined copy of the function in file B may still
get its profile annotated.

The inconsistency may even introduce profile unused warning because if the
target is not compiled with explicit debug information flag, the function
in file A won't have its debug information enabled (debug information will
be enabled implicitly only when -fprofile-sample-use is used). After it is
imported into file B which is compiled with -fprofile-sample-use, profile
annotation for the outline copy of the function will fail because the
function has no debug information, and that will trigger  profile unused
warning.

We add a new attribute use-sample-profile to control whether a function
will use its sample profile no matter for its outline or inline copies.
That will make the behavior of -fno-profile-sample-use consistent.

Differential Revision: https://reviews.llvm.org/D79959
2020-06-02 17:23:17 -07:00
Sanjay Patel 5a82dc62d2 [InstCombine] add tests for select-of-select-shuffle; NFC 2020-06-02 13:26:21 -04:00
Sanjay Patel 5b8c79ce76 [InstCombine] regenerate complete test checks; NFC 2020-06-02 13:26:21 -04:00
Florian Hahn b446ec56a2 [LV] Make sure the MaxVF is a power-of-2 by rounding down.
LV currently only supports power of 2 vectorization factors, which has
been made explicit with the assertion added in
840450549c.

However, if the widest type is not a power-of-2 the computed MaxVF won't
be a power-of-2 either. This patch updates computeFeasibleMaxVF to
ensure the returned value is a power-of-2 by rounding down to the
nearest power-of-2.

Fixes PR46139.

Reviewers: Ayal, gilr, rengolin

Reviewed By: Ayal

Differential Revision: https://reviews.llvm.org/D80870
2020-06-02 10:40:49 +01:00
Denis Antrushin 3c626c714c [EarlyCSE] Common gc.relocate calls.
gc.relocate intrinsic is special in that its second and third operands
are not real values, but indices into relocate's parent statepoint list
of GC pointers.
To be CSE'd, they need special handling in `isEqual()` and `getHashCode()`.

Reviewed By: reames
Differential Revision: https://reviews.llvm.org/D80445
2020-06-02 12:25:43 +03:00
Simon Pilgrim 6ce6960b92 [VectorCombine][X86] Add loaded insert tests from D80885 2020-06-02 10:04:05 +01:00
Joseph Huber 1a4fb2edcb [OpenMP] Replace Clang's OpenMP RTL Definitions with OMPKinds.def
Summary: This changes Clang's generation of OpenMP runtime functions to use the types and functions defined in OpenMPKinds and OpenMPConstants. New OpenMP runtime function information should now be added to OMPKinds.def. This patch also changed the definitions of __kmpc_push_num_teams and __kmpc_copyprivate to match those found in the runtime.

Reviewers: jdoerfert

Reviewed By: jdoerfert

Subscribers: jfb, AndreyChurbanov, openmp-commits, fghanim, hiraditya, sstefan1, cfe-commits, llvm-commits

Tags: #openmp, #clang, #llvm

Differential Revision: https://reviews.llvm.org/D80222
2020-06-01 16:23:10 -04:00
Stanislav Mekhanoshin 4e963299ee Temporarily removed unstable test. NFC. 2020-06-01 10:18:54 -07:00
Stanislav Mekhanoshin e132a9c012 Update some names in test. NFC.
There seems to be some instability with IR nameing between
platforms. Attempted to fix it with replacing dot-numbered
names.
2020-06-01 09:11:18 -07:00
Sanjay Patel b874dc4dda [InstCombine] add test for select-of-shuffle; NFC
This is based on an example in D80658
2020-06-01 11:52:07 -04:00
Stanislav Mekhanoshin 745c6c8458 Process gep (phi ptr1, ptr2) in SROA
Differential Revision: https://reviews.llvm.org/D79218
2020-06-01 08:41:05 -07:00
Sanjay Patel dd54432a0f [InstNamer] use 'i' for Instructions, not 'tmp'
As discussed in https://bugs.llvm.org/show_bug.cgi?id=45951 and
D80584, the name 'tmp' is almost always a bad choice, but we have
a legacy of regression tests with that name because it was baked
into utils/update_test_checks.py.

This change makes -instnamer more consistent (already using "arg"
and "bb", the common LLVM shorthand). And it avoids the conflict
in telling users of the FileCheck script to run "-instnamer" to
create a better regression test and having that cause a warn/fail
in update_test_checks.py.
2020-06-01 11:11:14 -04:00
Ehud Katz 85c3088049 [StructurizeCFG] Fix region nodes ordering
This is a reimplementation of the `orderNodes` function, as the old
implementation didn't take into account all cases.
The new implementation uses SCCs instead of Loops to take account of
irreducible loops.

Fix PR41509

Differential Revision: https://reviews.llvm.org/D79037
2020-06-01 12:50:35 +03:00
Sanjay Patel 91b45fb527 [PhaseOrdering] add test for hoisting/CSE (PR46115); NFC 2020-05-31 10:34:18 -04:00
Sanjay Patel e31f2a894a [VectorCombine] add tests for scalarizing binop-with-constant; NFC
Goes with proposal in D80885.

This is adapted from the InstCombine tests that were added for
D50992

But these should be adjusted further to provide more interesting
scenarios for x86-specific codegen. Eg, vector types/sizes will
have different costs depending on ISA attributes.

We also need to add tests that include a load of the scalar
variable and add tests that include extra uses of the insert
to further exercise the cost model.
2020-05-31 09:11:30 -04:00
Sanjay Patel 129c501aa9 [PhaseOrdering] add scalarization test for PR42174; NFC
Motivating test for vector-combine enhancement in D80885.
Make sure that vectorization and canonicalization are
working together as expected.
2020-05-31 08:43:34 -04:00
Whitney Tsang 0fee91a187 [LoopUnroll] Add a test case for rG7873376bb36b.
rG7873376bb36b fixes a build failure for allyesconfig.

The problem happened when the single exiting block doesn't dominate the
loop latch, then the immediate dominator of the exit block should not be
the exiting block after unrolling. As the exiting block of
different unrolled iteration can branch to the exit block, and the ith
exiting block doesn't dominate (i+1)th exiting block, the immediate
dominator of the exit block should not the nearest common dominator of
the exiting block and the loop latch of the same iteration.

Differential Revision: https://reviews.llvm.org/D80477
2020-05-30 20:34:27 +00:00
zoecarver 065bf124fd [DSE] Remove noop stores in MSSA.
Adds a simple fast-path check for the pattern:
v = load ptr
store v to ptr

I took the tests from the bugzilla post, I can add more if needed (but I think these should be sufficent).

Refs: https://bugs.llvm.org/show_bug.cgi?id=45795

Differential Revision: https://reviews.llvm.org/D79391
2020-05-30 09:57:30 -07:00
Eric Christopher c554c5e159 Fix full unrolling with new pass manager.
Last we looked at this and couldn't come up with a reason to change
it, but with a pragma for full loop unrolling we bypass every other
loop unroll and then fail to fully unroll a loop when the pragma is set.

Move the OnlyWhenForced out of the check and into the initialization
of the full unroll pass in the new pass manager. This doesn't show up
with the old pass manager.

Add a new option to opt so that we can turn off loop unrolling
manually since this is a difference between clang and opt.

Tested with check-clang and check-llvm.
2020-05-29 20:08:21 -07:00
Valery N Dmitriev a45688a72c [SLP] Apply external to vectorizable tree users cost adjustment for
relevant aggregate build instructions only (UserCost).
Users are detected with findBuildAggregate routine and the trick is
that following SLP vectorization may end up vectorizing entire list
with smaller chunks. Cost adjustment then is applied for individual
chunks and these adjustments obviously have to be smaller than the
entire aggregate build cost.

Differential Revision: https://reviews.llvm.org/D80773
2020-05-29 15:37:41 -07:00
Stanislav Mekhanoshin af852d6f36 Revert "Process gep (phi ptr1, ptr2) in SROA"
This reverts commit f66a43c11a.
2020-05-29 13:51:03 -07:00
Stanislav Mekhanoshin f66a43c11a Process gep (phi ptr1, ptr2) in SROA
Differential Revision: https://reviews.llvm.org/D79218
2020-05-29 13:05:51 -07:00
Ehud Katz c710bb44a6 [Local] Prevent `invertCondition` from creating a redundant instruction
Prevent `invertCondition` from creating the inversion instruction, in
case the given value is an argument which has already been inverted.
Note that this approach has already been taken in case the given value
is an instruction (and not an argument).

Differential Revision: https://reviews.llvm.org/D80399
2020-05-29 21:08:22 +03:00
Sanjay Patel 61412b762d [SLP] auto-generate complete test checks; NFC 2020-05-29 13:45:25 -04:00
Sanjay Patel db653ff6b7 [LoopVectorize] auto-generate complete test checks; NFC 2020-05-29 13:14:08 -04:00
Sanjay Patel f78eecbb93 [LoopVectorize] regenerate test checks; NFC
Align attributes are now visible.
2020-05-29 13:02:45 -04:00
Sanjay Patel 5e94273227 [LoopVectorize] auto-generate complete checks; NFC 2020-05-29 13:01:35 -04:00
Sanjay Patel 9d1f95bf9f [LoopVectorize] regenerate test checks; NFC
Align attributes are now visible.
2020-05-29 13:01:35 -04:00
Sanjay Patel 0b21c6706a [LoopVectorize] auto-generate complete test checks; NFC 2020-05-29 13:01:35 -04:00
Paul Robinson 8c2d2d971b Preserve DbgLoc when DeadArgumentElimination rewrites a 'ret'.
Fixes PR46002.
2020-05-29 10:00:33 -07:00
Florian Hahn 01f999ae88 [SCCP] Switch to widen at PHIs, stores and call edges.
Currently SCCP does not widen PHIs, stores or along call edges
(arguments/return values), but on operations that directly extend ranges
(like binary operators).

This means PHIs, stores and call edges are not pessimized by widening
currently, while binary operators are. The main reason for widening
operators initially was that opting-out for certain operations was
more straight-forward in the initial implementation (and it did not
matter too much, as range support initially was only implemented for a
very limited set of operations.

During the discussion in D78391, it was suggested to consider flipping
widening to PHIs, stores and along call edges. After adding support for
tracking the number of range extensions in ValueLattice, limiting the
number of range extensions per value is straight forward.

This patch introduces a MaxWidenSteps option to the MergeOptions,
limiting the number of range extensions per value. For PHIs, it seems
natural allow an extension for each (active) incoming value plus 1. For
the other cases, a arbitrary limit of 10 has been chosen initially. It would
potentially make sense to set it depending on the users of a
function/global, but that still needs investigating. This potentially
leads to more state-changes and longer compile-times.

The results look quite promising (MultiSource, SPEC):

Same hash: 179 (filtered out)
Remaining: 58
Metric: sccp.IPNumInstRemoved

Program                                        base    widen-phi diff
 test-suite...ks/Prolangs-C/agrep/agrep.test    58.00   82.00    41.4%
 test-suite...marks/SciMark2-C/scimark2.test    32.00   43.00    34.4%
 test-suite...rks/FreeBench/mason/mason.test     6.00    8.00    33.3%
 test-suite...langs-C/football/football.test   104.00  128.00    23.1%
 test-suite...cations/hexxagon/hexxagon.test    36.00   42.00    16.7%
 test-suite...CFP2000/177.mesa/177.mesa.test   214.00  249.00    16.4%
 test-suite...ngs-C/assembler/assembler.test    14.00   16.00    14.3%
 test-suite...arks/VersaBench/dbms/dbms.test    10.00   11.00    10.0%
 test-suite...oxyApps-C++/miniFE/miniFE.test    43.00   47.00     9.3%
 test-suite...ications/JM/ldecod/ldecod.test   179.00  195.00     8.9%
 test-suite...CFP2006/433.milc/433.milc.test   249.00  265.00     6.4%
 test-suite.../CINT2000/175.vpr/175.vpr.test    98.00  104.00     6.1%
 test-suite...peg2/mpeg2dec/mpeg2decode.test    70.00   74.00     5.7%
 test-suite...CFP2000/188.ammp/188.ammp.test    71.00   75.00     5.6%
 test-suite...ce/Benchmarks/PAQ8p/paq8p.test   111.00  117.00     5.4%
 test-suite...ce/Applications/Burg/burg.test    41.00   43.00     4.9%
 test-suite...000/197.parser/197.parser.test    66.00   69.00     4.5%
 test-suite...tions/lambda-0.1.3/lambda.test    23.00   24.00     4.3%
 test-suite...urce/Applications/lua/lua.test   301.00  313.00     4.0%
 test-suite...TimberWolfMC/timberwolfmc.test    76.00   79.00     3.9%
 test-suite...lications/ClamAV/clamscan.test   991.00  1030.00    3.9%
 test-suite...plications/d/make_dparser.test    53.00   55.00     3.8%
 test-suite...fice-ispell/office-ispell.test    83.00   86.00     3.6%
 test-suite...lications/obsequi/Obsequi.test    28.00   29.00     3.6%
 test-suite.../Prolangs-C/bison/mybison.test    56.00   58.00     3.6%
 test-suite.../CINT2000/254.gap/254.gap.test   170.00  176.00     3.5%
 test-suite.../Applications/lemon/lemon.test    30.00   31.00     3.3%
 test-suite.../CINT2000/176.gcc/176.gcc.test   1202.00 1240.00    3.2%
 test-suite...pplications/treecc/treecc.test    79.00   81.00     2.5%
 test-suite...chmarks/MallocBench/gs/gs.test   357.00  366.00     2.5%
 test-suite...eeBench/analyzer/analyzer.test   103.00  105.00     1.9%
 test-suite...T2006/445.gobmk/445.gobmk.test   1697.00 1724.00    1.6%
 test-suite...006/453.povray/453.povray.test   1812.00 1839.00    1.5%
 test-suite.../Benchmarks/Bullet/bullet.test   337.00  342.00     1.5%
 test-suite.../CINT2000/252.eon/252.eon.test   426.00  432.00     1.4%
 test-suite...T2000/300.twolf/300.twolf.test   214.00  217.00     1.4%
 test-suite...pplications/oggenc/oggenc.test   244.00  247.00     1.2%
 test-suite.../CINT2006/403.gcc/403.gcc.test   4008.00 4055.00    1.2%
 test-suite...T2006/456.hmmer/456.hmmer.test   175.00  177.00     1.1%
 test-suite...nal/skidmarks10/skidmarks.test   430.00  434.00     0.9%
 test-suite.../Applications/sgefa/sgefa.test   115.00  116.00     0.9%
 test-suite...006/447.dealII/447.dealII.test   1082.00 1091.00    0.8%
 test-suite...6/482.sphinx3/482.sphinx3.test   141.00  142.00     0.7%
 test-suite...ocBench/espresso/espresso.test   152.00  153.00     0.7%
 test-suite...3.xalancbmk/483.xalancbmk.test   4003.00 4025.00    0.5%
 test-suite...lications/sqlite3/sqlite3.test   548.00  551.00     0.5%
 test-suite...marks/7zip/7zip-benchmark.test   5522.00 5551.00    0.5%
 test-suite...nsumer-lame/consumer-lame.test   208.00  209.00     0.5%
 test-suite...:: External/Povray/povray.test   1556.00 1563.00    0.4%
 test-suite...000/186.crafty/186.crafty.test   298.00  299.00     0.3%
 test-suite.../Applications/SPASS/SPASS.test   2019.00 2025.00    0.3%
 test-suite...ications/JM/lencod/lencod.test   8427.00 8449.00    0.3%
 test-suite...6/464.h264ref/464.h264ref.test   6797.00 6813.00    0.2%
 test-suite...6/471.omnetpp/471.omnetpp.test   431.00  430.00    -0.2%
 test-suite...006/450.soplex/450.soplex.test   446.00  447.00     0.2%
 test-suite...0.perlbench/400.perlbench.test   1729.00 1727.00   -0.1%
 test-suite...000/255.vortex/255.vortex.test   3815.00 3819.00    0.1%

Reviewers: efriedma, nikic, davide

Reviewed By: efriedma

Differential Revision: https://reviews.llvm.org/D79036
2020-05-29 11:59:17 +01:00
David Sherwood f254f1d94e [SVE] Remove getNumElements() warnings in InstCombiner::visitBitCast
Whilst trying to compile this test to assembly:

  CodeGen/aarch64-sve-intrinsics/acle_sve_reinterpret.c

I discovered some warnings were firing in InstCombiner::visitBitCast
due to calls to getNumElements() for scalable vector types. These
calls only really made sense for fixed width vectors so I have fixed
up the code appropriately.

Differential Revision: https://reviews.llvm.org/D80559
2020-05-29 08:00:08 +01:00