When we bail out early we make the partially build new code path
practically dead, though it was not unreachable. To remove dominance
problems we now make it not only dead but also prevent the control
flow to join with the original code path, thus allow to use original
values after the SCoP without any PHI nodes.
This fixes bug 25447.
llvm-svn: 252420
The bail out in r252412 left the code generation without verifying the (so
far) generated IR. This will change now and ensure we always run the
verifier.
Suggested-by: Tobias Grosser <tobias@grosser.es>
llvm-svn: 252419
While the program cannot cause a dependence cycle between invariant
loads, additional constraints (e.g., to ensure finite loops) can
introduce them. It is hard to detect them in the SCoP description,
thus we will only check for them at code generation time. If such a
recursion is detected we will bail out the code generation and place a
"false" runtime check to guarantee the original code is used.
This fixes bug 25443.
llvm-svn: 252412
After loop versioning, a dominance check of a non-affine subregion's
exit node causes the dominance check to always fail on any block in the
subregion if it shares the same exit block with the scop. The
subregion's exit block has become polly_merge_new_and_old, which also
receives the control flow of the generated code. This would cause that
any value for implicit stores is assumed to be not from the scop.
We check dominance with the generated exit node instead.
This fixes llvm.org/PR25438
llvm-svn: 252375
Remove all the implicit ilist iterator conversions from polly, in
preparation for making them illegal in ADT. There was one oddity I came
across: at line 95 of lib/CodeGen/LoopGenerators.cpp, there was a
post-increment `Builder.GetInsertPoint()++`.
Since it was a no-op, I removed it, but I admit I wonder if it might be
a bug (both before and after this change)? Perhaps it should be a
pre-increment?
llvm-svn: 252357
We were adding all generated values in non-affine subregions to be used
for the subregions generated exit block. The thought was that only
values that are dominating the original exit block can be used there.
But it is possible for synthesizable values to be expanded in any
block. If the same values is also used for implicit writes, it would
try to reuse already synthesized values even if not dominating the exit
block.
The fix is to only add values to the list of values usable in the exit
block only if it is dominating the exit block. This fixes
llvm.org/PR25412.
llvm-svn: 252301
Before this commit memory reference identifiers have only been unique per
basic block, but not per (non-affine) ScopStmt. This commit now uses the
MemoryAccess base pointer to uniquely identify each Memory access.
llvm-svn: 252200
For generating scalar writes of non-affine subregions, all except phi
writes are generated in the exit block. The phi writes are generated in
the incoming block for which we errornously used the same BBMap. This
can conflict if a value for one block is synthesized, and then reused
for another block which is not dominated by the first block. This is
fixed by using block-specific BBMaps for phi writes.
llvm-svn: 252172
An incoming value from a block the is not inside the scop is an
external use, even if the phi is inside the scop. A previous fix in
r251208 did not apply if the phi is inside a non-affine subregion. We
move the check for this phi case before the non-affine subregion check.
llvm-svn: 252157
To simplify and correct the preloading of a base pointer origin, e.g.,
the base pointer for the current indirect invariant load, we now just
check if there is an invariant access class that involves the base
pointer of the current class.
llvm-svn: 251962
We do not need to model read-only statements in the SCoP as they will
not cause any side effects that are visible to the outside anyway.
Removing them should safe us time and might even simplify the ASTs we
generate.
Differential Revision: http://reviews.llvm.org/D14272
llvm-svn: 251948
If a base pointer of a preloaded value has a base pointer origin, thus it is
an indirect invariant load, we have to make sure the base pointer origin is
preloaded first.
llvm-svn: 251946
ScalarEvolution doesn't allow the operands of an AddRec to be variant in the
loop of the AddRec. When we rewrite parameter SCEVs it might seem like the
new SCEV violates this property and ScalarEvolution will trigger an
assertion. To avoid this we move the start part out of an AddRec when we
rewrite it, thus avoid the operands to be possibly variant completely.
llvm-svn: 251945
If a base pointer load is preloaded, we have change the base pointer of
the derived SAI. However, as the derived SAI relationship is is
coarse grained, we need to check if we actually preloaded the base
pointer or a different element of the base pointer SAI array.
llvm-svn: 251881
In some cases different memory accesses access the very same array using a
different multi-dimensional array layout where the same dimensions have
different sizes. Instead of asserting when encountering this issue, we
gracefully bail out for this scop.
This fixes llvm.org/PR25252
llvm-svn: 251791
We remove -polly-detect-unprofitable and -polly-no-early-exit. Both have been
superseeded by -polly-process-unprofitable and were only kept as aliases for
our buildbots to continue to work. As all buildbots have been moved to the new
options, we can now remove the old ones for good.
llvm-svn: 251787
These maps are only needed during the construction of a single region statement.
Clearing them is important, as we otherwise get an assert in case some of the
referenced values are erased before the RegionGenerator is deleted.
llvm-svn: 251341
Volatile or atomic memory accesses are currently not supported. Neither did
we think about any special handling needed nor do we support the unknown
instructions the alias set tracker turns them into sometimes. Before this
patch, us not supporting unkown instructions in an alias set caused the
following assertion failures:
Assertion `AG.size() > 1 && "Alias groups should contain at least two accesses"'
failed
llvm-svn: 251234
When verifying if a scop is still valid we rerun all analysis, but did not
update DetectionContextMap. This change ensures that information, e.g. about
non-affine regions, is correctly updated
llvm-svn: 251227
the size expression.
We previously only checked if the size expression is 'undef', but allowed size
expressions of the form 'undef * undef' by accident. After this change we now
require size expressions to be affine which implies no 'undef' appears anywhere
in the expression.
llvm-svn: 251225
of the Region are external.
During code generation we split off the parts of the PHI nodes in the entry
block, which have incoming blocks that are not part of the region. As these
split-off PHI nodes then are external uses, we consequently also need to model
these uses in ScopInfo.
llvm-svn: 251208
Such PHI nodes can not only appear in the ExitBlock of the Scop, but indeed
any scalar PHI node above the scop and used in the scop is modeled as scalar
read access.
llvm-svn: 251198
This change adds code to directly code-generate multi-exit PHI nodes, instead
of trying to reuse the EscapeMap infrastructure for this. Using escape maps
adds a level of indirection that is hard to understand and - more importantly -
breaks in certain cases.
Specifically, the original code relied on simplifyRegion() to split the original
PHI node in two PHI nodes, one merging the values coming from within the scop
and a second that merges the first PHI node with the values that come from
outside the scop. To generate code the first PHI node is then just handled like
any other in-scop value that is used somewhere outside the scop. This fails for
the case where all values from inside the scop are identical, as the first PHI
node is in such cases automatically simplified and eliminated by LLVM right at
construction. As a result, there is no instruction that can be pass to the
EscapeMap handling, which means the references in the second PHI node are not
updated and may still reference values from within the original scop that do not
dominate it.
Our new code iterates directly over all modeled ScopArrayInfo objects that
represent multi-exit PHI nodes and generates code for them without relying on
the EscapeMap infrastructure. Hence, it works also for the case where the first
PHI node is eliminated.
llvm-svn: 251191
We isolate full tiles from partial tiles to be able to, for example, vectorize
loops with parametric lower and/or upper bounds.
If we use -polly-vectorizer=stripmine, we can see execution-time improvements:
correlation from 1m7361s to 0m5720s (-67.05 %), covariance from 1m5561s to
0m5680s (-63.50 %), ary3 from 2m3201s to 1m2361s (-46.72 %), CrystalMk from
8m5565s to 7m4285s (-13.18 %).
The current full/partial tile separation increases compile-time more than
necessary. As a result, we see in compile time regressions, for example, for 3mm
from 0m6320s to 0m9881s (56.34%). Some of this compile time increase is expected
as we generate more IR and consequently more time is spent in the LLVM backends.
However, a first investiagation has shown that a larger portion of compile time
is unnecessarily spent inside Polly's parallelism detection and could be
eliminated by propagating existing knowledge about vector loop parallelism.
Before enabling -polly-vectorizer=stripmine by default, it is necessary to
address this compile-time issue.
Contributed-by: Roman Gareev <gareevroman@gmail.com>
Reviewers: jdoerfert, grosser
Subscribers: grosser, #polly
Differential Revision: http://reviews.llvm.org/D13779
llvm-svn: 250809
New values were always synthesized in the block of the instruction
that needed them. This is incorrect for PHI node whose' value must be
defined in the respective incoming block. This patch temporarily moves
the builder's insert point to the incoming block while synthesizing phi
node arguments.
This fixes PR25241 (http://llvm.org/bugs/show_bug.cgi?id=25241)
llvm-svn: 250693
There are several different kinds of constants that could occur in a
branch condition, however we can only handle the most interesting one
namely constant integers. To this end we have to treat others as
non-affine.
This fixes bug 25244.
llvm-svn: 250669
We build the schedule based on a traversal of the region and accumulate
information for each loop in it. The total schedule is associated with the
loop surrounding the SCoP, though it can happen that there are blocks in the
SCoP which are part of loops that are only partially in the SCoP. Instead of
associating information with them (they are not part of the SCoP and
consequently are not modeled) we have to associate the schedule information
with the surrounding loop if any.
This fixes bug 25240.
llvm-svn: 250668
Accesses that have a relative offset (in bytes) that is not divisible
by the type size (in bytes) will be represented as empty in the SCoP
description. This is on its own not good but it also crashed the
invariant load hoisting. This patch will fix the latter problem while
the former should be addressed too.
This fixes bug 25236.
llvm-svn: 250664
If the base pointer of a load is invariant and defined in the SCoP but
not loaded we cannot hoist the load as we would not hoist the base
pointer definition.
This fixes bug 25237.
llvm-svn: 250663
Sorting is replaced by a demand driven code generation that will pre-load a
value when it is needed or, if it was not needed before, at some point
determined by the order of invariant accesses in the program. Only in very
little cases this demand driven pre-loading will kick in, though it will
prevent us from generating faulty code. An example where it is needed is
shown in:
test/ScopInfo/invariant_loads_complicated_dependences.ll
Invariant loads that appear in parameters but are not on the top-level (e.g.,
the parameter is not a SCEVUnknown) will now be treated correctly.
Differential Revision: http://reviews.llvm.org/D13831
llvm-svn: 250655
Polly can now be used as a analysis only tool as long as the code
generation is disabled. However, we do not have an alternative to the
independent blocks pass in place yet, though in the relevant cases
this does not seem to impact the performance much. Nevertheless, a
virtual alternative that allows the same transformations without
changing the input region will follow shortly.
llvm-svn: 250652
Expressing this in terms of BlockGenerator::getOrCreateAlloca(const
ScopArrayInfo *Array) does not work as the MemoryAccess BasePtr is in case of
invariant load hoisting different to the ScopArrayInfo BasePtr. Until this is
investigated and fixed, we move back to code that just uses the baseptr of
MemoryAccess.
llvm-svn: 250637
This allows the caller to get the alloca locations of an array without the
need to thank if Array is a PHI or a non-PHI Array. We directly make use of this
in BlockGenerator::getOrCreateAlloca(MemoryAccess &Access).
llvm-svn: 250628
Other places (e.g. hoistInvariantLoads) assume that an empty lookup
will return nullptr. The situation can currently not arise because
MemoryAccesses are not removed before hoistInvariantLoads.
llvm-svn: 250627
While clang-format takes care that the line-length is not surpassed, the
resulting comments sometimes look not optimal. We re-flow the text in the
comment to avoid these ugly single-word lines.
llvm-svn: 250626
Instead of generating implicit loads within basic blocks, put them
before the instructions of the statment itself, including non-affine
subregions. The region's entry node is dominating all blocks in the
region and therefore the loaded value will be available there.
Implicit writes in block-stmts were already stored back at the end of
the block. Now, also generate the stores of non-affine subregions when
leaving the statement, i.e. in the exiting block.
This change is required for array-mapped implicits ("De-LICM") to
ensure that there are no dependencies of demoted scalars within
statments. Statement load all required values, operator on copied in
registers, and then write back the changed value to the demoted memory.
Lifetimes analysis within statements becomes unecessary.
Differential Revision: http://reviews.llvm.org/D13487
llvm-svn: 250625
Accesses for exit node phis will be handled separately by
buildPHIAccesses if there is more than one exiting edge,
buildScalarDependences does not need to create additional SCALAR
accesses.
This is a corrected version of r250517, which was reverted in r250607.
Differential Revision: http://reviews.llvm.org/D13848
llvm-svn: 250622
Instead of checking at code generation time for each ScopStmt if a scalar has
external uses, we just iterate over the ScopArrayInfo descriptions we have and
check each of these for possible external uses.
Besides being somehow clearer, this approach has the benefit that we will always
create valid LLVM-IR even in case we disable the code generation of ScopStmt
bodies e.g. for testing purposes.
llvm-svn: 250608
In r250408 'CHECK-NEXT: br' lines were removed as they also matched a
'%polly.subregion.iv.inc' instruction and did consequently not check what they
were supposed to check. However, without these lines we can not test that the
.s2a instructions that are not any more generated since r250411 really are not
emitted. Hence, we add back the CHECK-NEXT lines to ensure there are really no
instructions generated between the store that we check for and the branch at the
end of the basic block. To ensure we do not match too early, we now check for
'br i1' or 'br label'.
llvm-svn: 250435
When pulling a llvm::Value to be written as a PHI write, the former
code did only check whether it is within the same basic block, but it
could also be the same non-affine subregion. In that case some
unecessary pair of MemoryAccesses would have been created.
Two unit test were explicitely checking for the unecessary writes,
including the comments that the writes are unecessary.
llvm-svn: 250411
They happen to match
%polly.subregion.iv.inc = add i32 %polly.subregion.iv, 1
^^ ^^
that is, are misleading in what they actually check.
llvm-svn: 250408
When sharing the same map from old to new value, CodeGeneration would
reuse the same new value for each basic block. However, the SCEV
expander might emit code in a basic block that does not dominate a use
of the SCEV in another basic block. This test checks whether both such
blocks have their own expanded new values.
llvm-svn: 250389
We harden one test case by ensuring no additional stores may possibly be
introduced between the stores we check for and the basic block terminator
statements.
We also add a test case for the situation where a value that is passed from
a non-affine region to a PHI node does not dominate the exit of the non-affine
region. This case has come up in patch reviews, so we make sure it is properly
handled today and in the future.
llvm-svn: 250217
This will allow us to optimize C++ template code with Polly. This support is
mostly for debugging purpose and individual experiments. The ultimate goal is
still to run Polly later in the pass manager when inlining already happened.
llvm-svn: 250092
instead of llvm::PassManagerBuilder::EP_EarlyAsPossible. This will allow us
to run actual module passes in Polly's canonicalization sequence, but should
otherwise have only little impact.
llvm-svn: 250091
We also allow such products for cases where 'Parameter' is loaded within the
scop, but where we can dynamically verify that the value of 'Parameter' remains
unchanged during the execution of the scop.
This change relies on Polly's new RequiredILS tracking infrastructure recently
contributed by Johannes.
llvm-svn: 250019
The domain generation can handle lazy && and || by default but eager
evaluated expressions were dismissed as non-affine. With this patch we
will allow arbitrary combinations of and/or bit-operations in the
conditions of branches.
Differential Revision: http://reviews.llvm.org/D13624
llvm-svn: 249971
Helper functions in the BlockGenerators.h/cpp introduce dependences
from the frontend to the backend of Polly. As they are used in
ScopDetection, ScopInfo, etc. we move them to the ScopHelper file.
llvm-svn: 249919
If a (assumed) invariant location is loaded multiple times we
generated a parameter for each location. However, this caused compile
time problems for several benchmarks (e.g., 445_gobmk in SPEC2006 and
BT in the NAS benchmarks). Additionally, the code we generate is
suboptimal as we preload the same location multiple times and perform
the same checks on all the parameters that refere to the same value.
With this patch we consolidate the invariant loads in three steps:
1) During SCoP initialization required invariant loads are put in
equivalence classes based on their pointer operand. One
representing load is used to generate a parameter for the whole
class, thus we never generate multiple parameters for the same
location.
2) During the SCoP simplification we remove invariant memory
accesses that are in the same equivalence class. While doing so
we build the union of all execution domains as it is only
important that the location is at least accessed once.
3) During code generation we only preload one element of each
equivalence class with the unified execution domain. All others
are mapped to that preloaded value.
Differential Revision: http://reviews.llvm.org/D13338
llvm-svn: 249853
This was left out from the original patch proposed in
http://reviews.llvm.org/D13195
even though it is needed to define an order invariant loads
are hoisted.
llvm-svn: 249680
Drop an unused flag polly-allow-non-scev-backedge-taken-count and also
its occurrences from the tests.
Contributed-by: Chris Jenneisch <chrisj@codeaurora.org>
Differential Revision: http://reviews.llvm.org/D13400
llvm-svn: 249675
ScopDetection users are interested in the detection context and access
these via different get-methods. However, not all information was
exposed though the number of maps to hold it was increasing steadily.
With this change only the detection contexts the rejection log and the
ValidRegions set are mapped. The former is needed, the second could be
integrated in the first and the ValidRegions set is only needed for the
deterministic order of the regions.
llvm-svn: 249614
This replaces the support for user defined error functions by a
heuristic that tries to determine if a call to a non-pure function
should be considered "an error". If so the block is assumed not to be
executed at runtime. While treating all non-pure function calls as
errors will allow a lot more regions to be analyzed, it will also
cause us to dismiss a lot again due to an infeasible runtime context.
This patch tries to limit that effect. A non-pure function call is
considered an error if it is executed only in conditionally with
regards to a cheap but simple heuristic.
llvm-svn: 249611
This patch allows invariant loads to be used in the SCoP description,
e.g., as loop bounds, conditions or in memory access functions.
First we collect "required invariant loads" during SCoP detection that
would otherwise make an expression we care about non-affine. To this
end a new level of abstraction was introduced before
SCEVValidator::isAffineExpr() namely ScopDetection::isAffine() and
ScopDetection::onlyValidRequiredInvariantLoads(). Here we can decide
if we want a load inside the region to be optimistically assumed
invariant or not. If we do, it will be marked as required and in the
SCoP generation we bail if it is actually not invariant. If we don't
it will be a non-affine expression as before. At the moment we
optimistically assume all "hoistable" (namely non-loop-carried) loads
to be invariant. This causes us to expand some SCoPs and dismiss them
later but it also allows us to detect a lot we would dismiss directly
if we would ask e.g., AliasAnalysis::canBasicBlockModify(). We also
allow potential aliases between optimistically assumed invariant loads
and other pointers as our runtime alias checks are sound in case the
loads are actually invariant. Together with the invariant checks this
combination allows to handle a lot more than LICM can.
The code generation of the invariant loads had to be extended as we
can now have dependences between parameters and invariant (hoisted)
loads as well as the other way around, e.g.,
test/Isl/CodeGen/invariant_load_parameters_cyclic_dependence.ll
First, it is important to note that we cannot have real cycles but
only dependences from a hoisted load to a parameter and from another
parameter to that hoisted load (and so on). To handle such cases we
materialize llvm::Values for parameters that are referred by a hoisted
load on demand and then materialize the remaining parameters. Second,
there are new kinds of dependences between hoisted loads caused by the
constraints on their execution. If a hoisted load is conditionally
executed it might depend on the value of another hoisted load. To deal
with such situations we sort them already in the ScopInfo such that
they can be generated in the order they are listed in the
Scop::InvariantAccesses list (see compareInvariantAccesses). The
dependences between hoisted loads caused by indirect accesses are
handled the same way as before.
llvm-svn: 249607
Value maps are created and used in many places and it is not always
possible to include CodeGen/Blockgenerators.h. To this end, ValueMapT
now lives in the ScopHelper.h which does not have any dependences itself.
This patch also replaces uses of different other value map types with
the ValueMapT.
llvm-svn: 249606
Do not use "Map[Key] == nullptr" to check if a Key is in the map, but use
"Map.find(Key) == Map.end()". Map[Key] always adds Key into the map, a
side-effect we do not want.
Found by inspection. This is hard to test outside of a targetted unit test,
which seems too much overhead for this individual issue.
llvm-svn: 249544
Clang's been taught to warn on more things here (unused values when
calling pure functions and ignoring their result, for example).
It might be better to figure out how to have cmake compile these tests
without -Werror/without any warnings enabled. But this'll do for now & I
don't know enough about cmake to fix it any other way, or to understand
the tradeoffs in that space.
llvm-svn: 249472
This single option replaces -polly-detect-unprofitable and -polly-no-early-exit
and is supposed to be the only option that disables compile-time heuristics that
aim to bail out early on scops that are believed to not benefit from Polly
optimizations.
Suggested-by: Johannes Doerfert
llvm-svn: 249426
These flags are now always passed to all tests and need to be disabled if
not needed. Disabling these flags, rather than passing them to almost all
tests, significantly simplfies our RUN: lines.
llvm-svn: 249422
Polly's profitability heuristic saves compile time by skipping trivial scops or
scops were we know no good optimization can be applied. For almost all our tests
this heuristic makes little sense as we aim for minimal test cases when testing
functionality. Hence, in almost all cases this heuristic is better be disabled.
In preparation of disabling Polly's compile time heuristic by default in the
test suite we first explicitly enable it in the couple of test cases that really
use it (or run with/without heuristic side-by-side).
llvm-svn: 249418
This test case was XFAILed under the assumption Polly is unable to detect the
scop. However, disabling Polly's profitability heuristics is sufficient to
detect this scop.
llvm-svn: 249414
A statement with an empty domain complicates the invariant load
hoisting and does not help any subsequent analysis or transformation.
In fact it might introduce parameter dimensions or increase the
schedule dimensionality. To this end, we remove statements with an
empty domain early in the SCoP simplification.
llvm-svn: 249276
Before isValidCFG() could hide the fact that a loop is non-affine by
over-approximation. This is problematic if a subregion of the loop contains
an exit/latch block and is over-approximated. Now we do not over-approximate
in the isValidCFG function if we check loop control. If such control is
non-affine the whole loop is over-approximated, not only a subregion.
llvm-svn: 249273
This patch cannot be tested on its own as the isValidCFG currently
hides the fact that control is actually non-affine with
over-approximation. This will be corrected in the next patch and a
test for non-affine latches will be added.
llvm-svn: 249272
When the ScopAnnotator was a class member variable some of the maps it contains
have not been properly cleared. As a result we had dangling pointers to
llvm::Value(s) which got detected by the AssertingVH we recently added.
No test case as this issue is hard to reproduce reliably as subsequent
optimizations need to delete some of the llvm::Values we still keep in our
lists.
llvm-svn: 249269
By using AssertingVH we will see assertions in case Values to which still
pointers in our maps exists are deleted. This is very useful as we previously
had some bugs that were caused by such stale Value pointers.
llvm-svn: 249267
The use of const qualified Value pointers prevents the use of AssertingVH. We
could probably think of adding const support to AssertingVH, but as const
correctness seems to currently provide limited benefit in Polly, we do not do
this yet.
llvm-svn: 249266
There have been various places where llvm::DenseMap<const llvm::Value *,
llvm::Value *> types have been defined, but all types have been expected to be
identical. We make this more clear by consolidating the different types and use
BlockGenerator::ValueMapT wherever there is a need for types to match
BlockGenerator::ValueMapT.
llvm-svn: 249264
By using asserting value handles, we will get assertions when we forget to clear
any of the Value maps instead of difficult to debug undefined behavior.
llvm-svn: 249238
By using asserting value handles, we will get assertions when we forget to clear
any of the Value maps instead of difficult to debug undefined behavior.
llvm-svn: 249237
We have to skip accesses in non-affine subregions during hoisting as
they might not be executed under the same condition as the entry of
the non-affine subregion.
llvm-svn: 249139
This moves the construction of ScopStmt to the beginning of the
ScopInfo pass. The late creation was a result of the earlier separation
of ScopInfo and TempScopInfo. This will avoid introducing more
ScopStmt-like maps in future commits. The AccFuncMap will also be
removed in some future commit. DomainMap might also be included into
ScopStmt.
The order in which ScopStmt are created changes and initially creates
empty statements that are removed in a simplification.
Differential Revision: http://reviews.llvm.org/D13341
llvm-svn: 249132
If a value is globally mapped (IslNodeBuilder::ValueMap) and
referenced in the code that will be put into a subfunction, we hand
down the new value to the subfunction.
This patch also removes code that handed down all invariant loads to
the subfunction. Instead, only needed invariant loads are given to the
subfunction. There are two possible reasons for an invariant load to
be handed down:
1) The invariant load is used in a block that is placed in the
subfunction but which is not the parent of the load. In this
case, the scalar access that will read the loaded value, will
cause its base pointer (the preloaded value) to be handed down to
the subfunction.
2) The invariant load is defined and used in a block that is placed
in the subfunction. With this patch we will hand down the
preloaded value to the subfunction as the invariant load is
globally mapped to that value.
llvm-svn: 249126
When error blocks are not terminated by an unreachable they have successors
that might only be reachable via error blocks. Additionally, branches in
error blocks are not checked during SCoP detection, thus we might not be able
to handle them. With this patch we do not try to model error block exit
conditions. Anything that is only reachable via error blocks is ignored too,
as it will not be executed in the optimized version of the SCoP anyway.
llvm-svn: 249099
The user can provide function names with
-polly-error-functions=name1,name2,name3
that will be treated as error functions. Any call to them is assumed
not to be executed.
This feature is mainly for developers to play around with the new
"error block" feature.
llvm-svn: 249098
With this patch we erase cached results for regions that are invalid
as early as possible. If we do not (as before), it is possible that
two expanded regions will have the same address and the tracked
results for both are mixed. Currently this would "only" cause
pessimism in later passes but that will change when we allow invariant
loads in the SCoP. Additionally, it triggers non-deterministic
results as we might dismiss a later region because of results cached
for an earlier one.
There is no test case as the problem occurs only non-deterministically.
llvm-svn: 249000
Because we handle more than SCEV does it is not possible to rewrite an
expression on the top-level using the SCEVParameterRewriter only. With
this patch we will do the rewriting on demand only and also
recursively, thus not only on the top-level.
llvm-svn: 248916
Instructions which we can synthesis from a SCEV expression are not
generated directly, but only when they are used as an operand of
another instruction. This avoids generating unnecessary instructions
and works more reliably than first inserting them and then deleting
them later on.
This commit was reverted in r248860 due to a remaining miscompile, where
we forgot to synthesis the operand values that were referenced from scalar
writes. test/Isl/CodeGen/scalar-store-from-same-bb.ll tests that we do this
now correctly.
llvm-svn: 248900
Before we unconditinoally forced all users outside the SCoP to use
the preloaded value. However, if the SCoP is not executed due to the
runtime checks, we need to use the original value because it might not
be invariant in the first place.
llvm-svn: 248881
This makes ScopInfo's scop member available earlier to other methods which will make some planned changes simpler.
No behavioral change intended
llvm-svn: 248879
As a first step in the direction of assumed invariant loads (loads
that are not written in some context) we now detect and hoist
definitively invariant loads. These invariant loads will be preloaded
in the code generation and used in the optimized version of the SCoP.
If the load is only conditionally executed the preloaded version will
also only be executed under the same condition, hence we will never
access memory that wouldn't have been accessed otherwise. This is also
the most distinguishing feature to licm.
As hoisting can make statements empty we will simplify the SCoP and
remove empty statements that would otherwise cause artifacts in the
code generation.
Differential Revision: http://reviews.llvm.org/D13194
llvm-svn: 248861
This reverts commit 07830c18d789ee72812d5b5b9b4f8ce72ebd4207.
The commit broke at least one test in lnt,
MultiSource/Benchmarks/Ptrdist/bc/number.c
was miss compiled and the test produced a wrong result.
One Polly test case that was added later was adjusted too.
llvm-svn: 248860
This hopefully helps to to address the following compile error on our buildbots
BlockGenerators.h:683:7: error: looser throw specifier for ‘virtual
polly::RegionGenerator::~RegionGenerator()’
BlockGenerators.h:164:11: error: overriding ‘virtual polly::BlockGenerator::~BlockGenerator() noexcept
(true)`
llvm-svn: 248812
Every once in a while we see code that accesses memory with different types,
e.g. to perform operations on a piece of memory using type 'float', but to copy
data to this memory using type 'int'. Modeled in C, such codes look like:
void foo(float A[], float B[]) {
for (long i = 0; i < 100; i++)
*(int *)(&A[i]) = *(int *)(&B[i]);
for (long i = 0; i < 100; i++)
A[i] += 10;
}
We already used the correct types during normal operations, but fall back to our
detected type as soon as we import changed memory access functions. For these
memory accesses we may generate invalid IR due to a mismatch between the element
type of the array we detect and the actual type used in the memory access. To
address this issue, we always cast the newly created address of a memory access
back to the type of the memory access where the address will be used.
llvm-svn: 248781
This is a bit of an awkward API and I'm not sure what the right solution
is. Having a publicly copy constructible base class makes it easy to
accidentally slice derived objects in a number of contexts.
llvm-svn: 248764
Instructions which we can synthesis from a SCEV expression are not generated
directly, but only when they are used as an operand of another instruction. This
avoids generating unnecessary instruction and works more reliably than first
inserting them and then deleting them later on.
Suggested-by: Johannes Doerfert <doerfert@cs.uni-saarland.de>
Differential Revision: http://reviews.llvm.org/D13208
llvm-svn: 248712
This patch allows switch instructions with affine conditions in the
SCoP. Also switch instructions in non-affine subregions are allowed.
Both did not require much changes to the code, though there was some
refactoring needed to integrate them without code duplication.
In the llvm-test suite the number of profitable SCoPs increased from
135 to 139 but more importantly we can handle more benchmarks and user
inputs without preprocessing.
Differential Revision: http://reviews.llvm.org/D13200
llvm-svn: 248701
This check was needed at some point but seems not useful anymore. Only
one adjustment in the domain generation was needed to cope with the
cases this check prevented from happening before.
llvm-svn: 248695
We now only delete trivially dead instructions in the BB we copy (copyBB), but
not in any other BB. Only for copyBB we know that there will _never_ be any
future uses of instructions that have no use after copyBB has been generated.
Other instructions in the AST that have been generated by IslNodeBuilder may
look dead at the moment, but may possibly still be referenced by GlobalMaps. If
we delete them now, later uses would break surprisingly.
We do not have a test case that breaks due to us deleting too many instructions.
This issue was found by inspection.
llvm-svn: 248688
The changes affect methods that are part of the Pass interface and
include:
- Comments that describe the methods purpose.
- A consistent use of the keywords override and virtual.
Additionally, the printScop method is now optional and removed from
SCoP passes that do not implement it.
llvm-svn: 248685
After having generated a new user statement a couple of inefficient or
trivially dead instructions may remain. This commit runs instruction
simplification over the newly generated blocks to ensure unneeded
instructions are removed right away.
This commit does adds simplification for non-affine subregions which was not
yet part of 248681.
llvm-svn: 248683
Otherwise, part of the computation will be just simplified away when we add
instruction simplification support to the RegionGenerator.
llvm-svn: 248682
After having generated a new user statement a couple of inefficient or trivially
dead instructions may remain. This commit runs instruction simplification over
the newly generated blocks to ensure unneeded instructions are removed right
away.
This commit does not yet add simplification for non-affine subregions.
llvm-svn: 248681
This commit basically reverts r246427 but still solves the issue
tackled by that commit. Instead of emitting initialization code in the
beginning of the start block we now generate parallel code in its own
block and thereby guarantee separation. This is necessary as we cannot
generate code for hoisted loads prior to the start block but it still
needs to be placed prior to everything else.
llvm-svn: 248674
The new domain construction algorithm now correctly models this test case (and
derives an empty run-time condition). Add this test case to ensure we do not
regress.
llvm-svn: 248669
When the whole SCoP is a non-affine region we need to use the
surrounding loop in the construction of the schedule as that is
the one that will be looked up after the schedule generation.
This fixes bug 24947
llvm-svn: 248667
When recovering multi-dimensional memory accesses, it may happen that different
accesses to the same base array are recovered with different dimensionality.
This patch ensures that the dimensionalities are unified by adding zero valued
dimensions to acesses with lower dimensionality. When starting to model
fixed-size arrays as multi-dimensional in 247906, this has not been taken
care of.
llvm-svn: 248662
There are three possible reasons to add a memory memory access: For explicit load and stores, for llvm::Value defs/uses, and to emulate PHI nodes (the latter two called implicit accesses). Previously MemoryAccess only stored IsPHI. Register accesses could be identified through the isScalar() method if it was no IsPHI. isScalar() determined the number of dimensions of the underlaying array, scalars represented by zero dimensions.
For the work on de-LICM, implicit accesses can have more than zero dimensions, making the distinction of isScalars() useless, hence now stored explicitly in the MemoryAccess. Instead, we replace it by isImplicit() and avoid the term scalar for zero-dimensional arrays as it might be confused with llvm::Value which are also often referred to as scalars (or alternatively, as registers).
No behavioral change intended, under the condition that it was impossible to create explicit accesses to zero-dimensional "arrays".
llvm-svn: 248616
This makes the intent of each created object clearer and allows to add more specific asserts. The bug fixed in r248535 has been discovered this way.
No functional change intended; everything should behave as before.
llvm-svn: 248603
This change addresses three issues:
- Read only scalars that enter a PHI node through an edge that comes from
outside the scop are not modeled any more, as such PHI nodes will always
be initialized to this initial value right before the SCoP is entered.
- For PHI nodes that depend on a scalar value that is defined outside the
scop, but where the scalar values is passed through an edge that itself
comes from a BB that is part of the region, we introduce in this basic
block a read of the out-of-scop value to ensure it's value is available
to write it into the PHI alloc location.
- Read only uses of scalars by PHI nodes are ignored in the general read only
handling code, as they are taken care of by the general PHI node modeling
code.
llvm-svn: 248535
After the merge of TempScopInfo into ScopInfo the analysis output
remained because of the existing unit tests. These remains are removed
and the units tests converted to match the equivalent output of
ScopInfo's analysis output. The unit tests are also moved into the
directory of ScopInfo tests.
Differential Revision: http://reviews.llvm.org/D13116
llvm-svn: 248485
Refactor all ISL-related cmake build instructions into its own
CMakeLists.txt and build as a separate library.
This is useful to apply ISL-related build flags to ISL only and not to
Polly's files. Also, it the separation of both projects becomes clearer.
Proposed name of the library is Polly_isl. It is not "isl" to avoid
mix-up with potentially installed libisl.{a|so}.
Tested configurations:
- Windows with cmake 3.2
- Ubuntu with cmake 3.0.2
- Ubuntu with cmake 3.0.2 BUILD_SHARED_LIBS on
- Ubuntu with cmake 2.8.12.2 (LLVM minimum version)
- Ubuntu out-of-LLVM-tree
Differential Revision: http://reviews.llvm.org/D12810
llvm-svn: 248484
A missing return statement that previously did not have a visibly negative
effect caused after some data-structure changes in r248024 multi-dimensional
accesses to be modeled both multi-dimensional as well as linearized. This
commit adds the missing return to avoid the incorrect double modeling as
well as the compile time increases it caused.
llvm-svn: 248171
If we encounter a <nsw> tagged AddRec for a loop we know the trip count of
that loop has to be bounded or the semantics is undefined anyway. Hence, we
only need to add unbounded assumptions if no such AddRec is known.
llvm-svn: 248128
So far we ignored the unbounded parts of the iteration domain, however
we need to assume they do not occure at all to remain sound if they do.
llvm-svn: 248126
We now add loop carried information during the second traversal of the
region instead of in a intermediate step in-between. This makes the
generation simpler, removes code and should even be faster.
llvm-svn: 248125
In order to allow multiple back edges we:
- compute the conditions under which each back edge is taken
- build the union over all these conditions, thus the condition that
any back edge is taken
- apply the same logic to the union we applied to a single back edge
llvm-svn: 248120
As we currently do not perform any optimizations that targets (or is
even aware) small trip counts we will skip them when we count the
loops in a region.
llvm-svn: 248119
All MemoryAccess objects will be owned by ScopInfo::AccFuncMap which
previously stored the IRAccess objects. Instead of creating new
MemoryAccess objects, the already created ones are reused, but their
order might be different now. Some fields of IRAccess and MemoryAccess
had the same meaning and are merged.
This is the last step of fusioning TempScopInfo.{h|cpp} and
ScopInfo.{h.cpp}. Some refactoring might still make sense.
Differential Revision: http://reviews.llvm.org/D12843
llvm-svn: 248024
In some cases instcombine introduces bitcasts that slightly obfuscate the
multi-dimensionality of an array. This patch teaches our fixed-size
delinearization how to look through bitcasts.
llvm-svn: 247928
This information is implicitly available through the multi-dimensionality of
memory accesses. This reduces compile time for 3mm from 430ms to 400ms and
should generally benefit compile time for cases where the assumed context
is complex.
llvm-svn: 247907
If the GEP instructions give us enough insights, model scalar accesses as
multi-dimensional (and generate the relevant run-time checks to ensure
correctness). This will allow us to simplify the dependence computation in
a subsequent commit.
llvm-svn: 247906
This will allow to generate non-wrap assumptions for integer expressions
that are part of the SCoP. We compare the common isl representation of
the expression with one computed with modulo semantic. For all parameter
combinations they are not equal we can have integer overflows.
The nsw flags are respected when the modulo representation is computed,
nuw and nw flags are ignored for now.
In order to not increase compile time to much, the non-wrap assumptions
are collected in a separate boundary context instead of the assumed
context. This helps compile time as the boundary context can become
complex and it is therefor not advised to use it in other operations
except runtime check generation. However, the assumed context is e.g.,
used to tighten dependences. While the boundary context might help to
tighten the assumed context it is doubtful that it will help in practice
(it does not effect lnt much) as the boundary (or no-wrap assumptions)
only restrict the very end of the possible value range of parameters.
PET uses a different approach to compute the no-wrap context, though lnt runs
have shown that this version performs slightly better for us.
llvm-svn: 247732
Due to the new domain generation, the SCoP keeps track of the domain
for all blocks, thus the SCEVAffinator can now work with blocks to avoid
duplication of the domains.
llvm-svn: 247731
Add polly-check-format as dependency of check-polly if clang-format is
available in the same build.
Differential Revision: http://reviews.llvm.org/D12850
llvm-svn: 247600
Summary:
Make clang-format run on each file independently using
add_custom_format (instead using a shell script in utils/). The targets
polly-{update|check}-format depend on these.
The primary motivation is to make them work on Windows, but also
improves them generally:
- Each file update/check can run in parallel (Although they do not take
long to run anyway)
- Implicit dependency on clang-format, so it recompiles if necessary
- polly-check-format shows the formatting difference if failing
Differential Revision: http://reviews.llvm.org/D12837
llvm-svn: 247581
At some point we build loop trip counts using this method. It was replaced by
a simpler trick that works only for affine (e.g., not modulo) constraints and
relies on the removal of unbounded parts. In order to allow modulo constrains
again we go back to the former, more accurate method.
llvm-svn: 247540
Summary:
TempScop is basically a holder for AccFuncMap, the dictionary from BasicBlocks to IRAccess lists. We move the list into polly::Scop and remove the polly::TempScop class.
There is one small change in behavior: If ScopInfo finds that its AssumedContext is impossible, it bails out by deleting the Scop object. The TempScop::print (invoked with opt -polly-scops -analyze) cannot print the AccFuncMap anymore as it would with a separate TempScop.
Differential Revision: http://reviews.llvm.org/D12803
llvm-svn: 247480
When compiling Polly without LLVM sources but with system-installed
LLVM, the build process would not honor the LLVM_ENABLE_ASSERTIONS
setting LLVM was compiled with, but effectively assume that it is
switched off when compiling. During unit-tests llvm-lit would still
query the LLVM_ENABLE_ASSERTIONS flag and enable tests which require
assertions. Even if enabled for LLVM, Polly does not output its debug
info and statistics in this this mode such that 7 tests fail.
To fix, we query llvm-config --assertion-mode and if on, enable
assertions as HandleLLVMOptions.cmake would do.
We cannot reliably use HandleLLVMOptions.cmake itself as the host's
LLVM build might have been built using automake and distributions
change file locations (e.g. Debian to
/usr/share/llvm-${VERSION}/cmake/HandleLLVMOptions.cmake).
llvm-svn: 247470
The function use_after_scop would iterate from 0 to 1024 and accessing element A[1024] where A has only valid indexes from 0 to 1023. Polly detects the situation of unconditionally undefined behavior and bail out in ScopInfo as non-feasible for optimization.
Other tests add impossible context assumptions as well, hance might show the same problem.
llvm-svn: 247412
Hoist runtime checks in the loop nest if they guard an "error" like event.
Such events are recognized as blocks with an unreachable terminator or a call
to the ubsan function that deals with out of bound accesses. Other "error"
events can be added easily.
We will ignore these blocks when we detect/model/optmize and code generate SCoPs
but we will make sure that they would not have been executed using the assumption
framework.
llvm-svn: 247310
As we do not rely on ScalarEvolution any more we do not need to get
the backedge taken count. Additionally, our domain generation handles
everything that is affine and has one latch and our ScopDetection will
over-approximate everything else.
This change will therefor allow loops with:
- one latch
- exiting conditions that are affine
Additionally, it will not check for structured control flow anymore.
Hence, loops and conditionals are not necessarily single entry single
exit regions any more.
Differential Version: http://reviews.llvm.org/D12758
llvm-svn: 247289
The TempScopInfo (-polly-analyze-ir) pass is removed and its work taken
over by ScopInfo (-polly-scops). Several tests depend on
-polly-analyze-ir and use -polly-scops instead which for the moment
prints the output of both passes. This again is not expected by some
other tests, especially those with negative searches, which have been
adapted.
Differential Version: http://reviews.llvm.org/D12694
llvm-svn: 247288
This patch replaces the last legacy part of the domain generation, namely the
ScalarEvolution part that was used to obtain loop bounds. We now iterate over
the loops in the region and propagate the back edge condition to the header
blocks. Afterwards we propagate the new information once through the whole
region. In this process we simply ignore unbounded parts of the domain and
thereby assume the absence of infinite loops.
+ This patch already identified a couple of broken unit tests we had for
years.
+ We allow more loops already and the step to multiple exit and multiple back
edges is minimal.
+ It allows to model the overflow checks properly as we actually visit
every block in the SCoP and know where which condition is evaluated.
- It is currently not compatible with modulo constraints in the
domain.
Differential Revision: http://reviews.llvm.org/D12499
llvm-svn: 247279
The support for modulo expressions is not comlete and makes the new
domain generation harder. As the currently broken domain generation
needs to be replaced, we will first swap in the new, fixed domain
generation and make it compatible with the modulo expressions later.
llvm-svn: 247278
This prepares for a series of patches that merges TempScopInfo into ScopInfo to
reduce Polly's code complexity. Only ScopInfo.{cpp|h} will be left thereafter.
Moving the code of TempScopInfo in one commit makes the mains diffs simpler to
understand.
In detail, merging the following classes is planned:
TempScopInfo into ScopInfo
TempScop into Scop
IRAccess into MemoryAccess
Only moving code, no functional changes intended.
Differential Version: http://reviews.llvm.org/D12693
llvm-svn: 247274
The support for pointer expressions is broken as it can only handle
some patterns in the IslExprBuilder. We should to treat pointers in
expressions the same as integers at some point and revert this patch.
llvm-svn: 247147
Add a custom makefile rule to generate lib/External/isl/gitversion.h
from GIT_HEAD_ID and trigger it using BULIT_SOURCES to ensure the file
exists before compilation starts.
The latest ISL creates gitversion.h from Makefile.am only, instead also
from configure.ac in previous version. While the Polly build invokes
configure, it does not invoke ISL's make such that the file was missing.
Invoking ISL's make would come with additional problems such as
triggering automake because of not preserved file time stamps.
Re-running automake might not be successful on other system
configurations for instance because it was preconfigured without
--with-clang option.
llvm-svn: 247142
Not all users of our IslNodeBuilder will attach scheduling information to the
AST in the same way IslAstInfo is doing it today. By going through a virtual
function when extracting the schedule of an AST node other users can provide
their own functions for extract scheduling information in case they attach
scheduling information in a different way to the AST nodes.
No functional change for Polly itself intended.
llvm-svn: 247126
While we do not need to model PHI nodes in the region exit (as it is not part
of the SCoP), we need to prepare for the case that the exit block is split in
code generation to create a single exiting block. If this will happen, hence
if the region did not have a single exiting block before, we will model the
operands of the PHI nodes as escaping scalars in the SCoP.
Differential Revision: http://reviews.llvm.org/D12051
llvm-svn: 247078
Instead of having two separate options
-polly-detect-scops-in-functions-without-loops and
-polly-detect-scops-in-regions-without-loops we now just use
-polly-detect-unprofitable to force the detection of scops ignoring any compile
time saving bailout heuristics.
llvm-svn: 247057
Certain backends, e.g. NVPTX, do not support '.' in function names. Hence,
we ensure all '.' are replaced by '_' when generating function names for
subfunctions. For the current OpenMP code generation, this is not strictly
necessary, but future uses cases (e.g. GPU offloading) need this issue to be
fixed.
llvm-svn: 246980
Our alias metadata is currently not emitted in a deterministic order. As it
is not needed in this test, we just drop it for now (but keep in mind to fix
this).
llvm-svn: 246942
When this option is enabled, Polly will emit printf calls for each scalar
load/and store which dump the scalar value loaded/stored at run time.
This patch also refactors the RuntimeDebugBuilder to use variadic templates
when generating CPU printfs. As result, it now becomes easier to print
strings that consist of a set of arguments. Also, as a single printf
call is emitted, it is more likely for such strings to be emitted atomically
if executed multi-threaded.
llvm-svn: 246941
Some of the structures are renamed, subfunction introduced to clarify the
individual steps and comments are added describing their functionality.
llvm-svn: 246929
In the common case, the access functions are not modified, hence there is no
need to obtain the IslAstBuild context at all. This should not only be minimally
faster, but this also allows the IslNodeBuilder to work on asts that are not
annotated with isl_ast_builds as long as the memory accesses are not modified.
llvm-svn: 246928
By inspection the update of the GlobalMaps in the RegionGenerator seems unneed,
and is removed as also no test cases fail when dropping this. Johannes Doerfert
confirmed that this is indeed save:
"I think that code was needed when we did not use the scalar codegen by default.
Now everything defined in a non-affine region should be communicated via memory
and reloaded in the user block. Hence, we should be good removing this code."
llvm-svn: 246926
When computing the index expressions for new, multi-dimensional memory accesses
these new index expressions may reference original llvm::Values that are not
transfered into the OpenMP subfunction. Using GlobalMap we now replace
references to such values with the rewritten values that have e.g. been passed
to the OpenMP subfunction.
llvm-svn: 246923
The GlobalMap variable used in BlockGenerator should always reference the same
list througout the entire code generation, hence we can make it a member
variable to avoid passing it around through every function call.
History: Before we switched to the SCEV based code generation the GlobalMap
also contained a mapping form old to new induction variables, hence it was
different for each ScopStmt, which is why we passed it as function argument
to copyStmt. The new SCEV based code generation now uses a separate mapping
called LTS -> LoopToSCEV that maps each original loop to a new loop iteration
variable provided as a SCEVExpr. The GlobalMap is currently mostly used for
OpenMP code generation, where references to parameters in the original function
need to be rewritten to the locations of these variables after they have been
passed to the subfunction.
Suggested-by: Johannes Doerfert <doerfert@cs.uni-saarland.de>
llvm-svn: 246920
Originally, we disallowed the import of multi-dimensional access functions due
to our code generation not supporting the generation of new address expressions
for multi-dimensional memory accesses. When building our run-time alias check
infrastructure we added code generation support for multi-dimensional address
calculations. Hence, we can now savely allow the import of new
multi-dimensional access functions.
llvm-svn: 246917
By just removing dimensions (and the constraints they are involved in) we
may loose information about the dimensions we do not remove. By instead
using project_out, we are sure all constraints on the outer dimensions are
preserved.
No test case, as this error condition is very unlikely to be triggered by
isl's current code. We still 'fix' this, as isl gives little guarantees
regarding the behavior of remove_divs.
llvm-svn: 246567
This case probably does not happen for LLVM generated code that is in loop
simplify form, but Polly does support such kind of loops. This commit ensures we
have test coverage as well.
llvm-svn: 246543
Code generation currently does not expect unbounded loops. When
using ISL to compute the loop trip count, if we find that the
iteration domain remains unbounded, we invalidate the Scop by
creating an infeasible context.
Contributed-by: Matthew Simpson <mssimpso@codeaurora.org>
This fixes PR24634.
Differential Revision: http://reviews.llvm.org/D12493
llvm-svn: 246477
Instead of building domains with MaxLoopDepth dimensions, we now build
the domains such that they have the right amount of dimensions all the
time.
llvm-svn: 246443
Before this commit we did this only for Arguments or Constants, but indeed
an instruction may define a value a lot higher up in the dominance tree, but
the actual write generally needs to happen right before branching to the
PHI node. Otherwise, the writes of different branches into PHI nodes may get
intermixed if they lay higher up in the dominance tree.
llvm-svn: 246441
Johannes Doerfert