maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity

[LV][NFC] Factor out calculation of "best" estimated trip count. 

This is just small refactoring to minimize changes in upcoming patch.
In the next path I'm going to introduce changes into heuristic for vectorization of "tiny trip count" loops.

Patch by Evgeniy Brevnov <evgueni.brevnov@gmail.com>

Reviewers: hsaito, Ayal, fhahn, reames

Reviewed By: hsaito

Differential Revision: https://reviews.llvm.org/D67690

llvm-svn: 374338
This commit is contained in:
Florian Hahn 2019-10-10 13:07:01 +00:00
parent 3aa7e76677
commit a3ca7acb4f
1 changed files with 26 additions and 28 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
View File

@ -355,6 +355,29 @@ static Constant *getSignedIntOrFpConstant(Type *Ty, int64_t C) {
: ConstantFP::get(Ty, C);
}
/// Returns "best known" trip count for the specified loop \p L as defined by
/// the following procedure:
/// 1) Returns exact trip count if it is known.
/// 2) Returns expected trip count according to profile data if any.
/// 3) Returns upper bound estimate if it is known.
/// 4) Returns None if all of the above failed.
static Optional<unsigned> getSmallBestKnownTC(ScalarEvolution &SE, Loop *L) {
// Check if exact trip count is known.
if (unsigned ExpectedTC = SE.getSmallConstantTripCount(L))
return ExpectedTC;
// Check if there is an expected trip count available from profile data.
if (LoopVectorizeWithBlockFrequency)
if (auto EstimatedTC = getLoopEstimatedTripCount(L))
return EstimatedTC;
// Check if upper bound estimate is known.
if (unsigned ExpectedTC = SE.getSmallConstantMaxTripCount(L))
return ExpectedTC;
return None;
}
namespace llvm {
/// InnerLoopVectorizer vectorizes loops which contain only one basic
@ -7483,36 +7506,11 @@ bool LoopVectorizePass::processLoop(Loop *L) {
ORE, BFI, PSI, Hints);
assert(L->empty() && "Inner loop expected.");
// Check the loop for a trip count threshold: vectorize loops with a tiny trip
// count by optimizing for size, to minimize overheads.
// Prefer constant trip counts over profile data, over upper bound estimate.
unsigned ExpectedTC = 0;
bool HasExpectedTC = false;
if (const SCEVConstant *ConstExits =
dyn_cast<SCEVConstant>(SE->getBackedgeTakenCount(L))) {
const APInt &ExitsCount = ConstExits->getAPInt();
// We are interested in small values for ExpectedTC. Skip over those that
// can't fit an unsigned.
if (ExitsCount.ult(std::numeric_limits<unsigned>::max())) {
ExpectedTC = static_cast<unsigned>(ExitsCount.getZExtValue()) + 1;
HasExpectedTC = true;
}
}
// ExpectedTC may be large because it's bound by a variable. Check
// profiling information to validate we should vectorize.
if (!HasExpectedTC && LoopVectorizeWithBlockFrequency) {
auto EstimatedTC = getLoopEstimatedTripCount(L);
if (EstimatedTC) {
ExpectedTC = *EstimatedTC;
HasExpectedTC = true;
}
}
if (!HasExpectedTC) {
ExpectedTC = SE->getSmallConstantMaxTripCount(L);
HasExpectedTC = (ExpectedTC > 0);
}
if (HasExpectedTC && ExpectedTC < TinyTripCountVectorThreshold) {
auto ExpectedTC = getSmallBestKnownTC(*SE, L);
if (ExpectedTC && *ExpectedTC < TinyTripCountVectorThreshold) {
LLVM_DEBUG(dbgs() << "LV: Found a loop with a very small trip count. "
<< "This loop is worth vectorizing only if no scalar "
<< "iteration overheads are incurred.");