We now generate runtime checks __after__ the SCoP code generation and
not before, though they are still inserted at the same position int
the code. This allows to modify the runtime check during SCoP code
generation.
llvm-svn: 271894
IntToPtr and PtrToInt instructions are basically no-ops that we can handle as
such. In order to generate them properly as parameters we had to improve the
ScopExpander, though the change is the first in the direction of a more
aggressive scalar synthetization.
llvm-svn: 271888
We now use the minimal necessary bit width for the generated code. If
operations might overflow (add/sub/mul) we will try to adjust the types in
order to ensure a non-wrapping computation. If the type adjustment is not
possible, thus the necessary type is bigger than the type value of
--polly-max-expr-bit-width, we will use assumptions to verify the computation
will not wrap. However, for run-time checks we cannot build assumptions but
instead utilize overflow tracking intrinsics.
llvm-svn: 271878
In case of modulo compared to zero, we need to do signed modulo
operation as unsigned can give different results based on whether the
dividend is negative or not.
This addresses llvm.org/PR27707
Contributed-by: Chris Jenneisch <chrisj@codeaurora.org>
Reviewers: _jdoerfert, grosser, Meinersbur
Differential Revision: http://reviews.llvm.org/D20145
llvm-svn: 271707
Operands of binary operations that might overflow will be temporarily
promoted to i64 again, though that is not a sound solution for the problem.
llvm-svn: 271538
We now have a simple function to adjust/unify the types of two (or three)
operands before an operation that requieres the same type for all operands.
Due to this change we will not promote parameters that are added to i64
anymore if that is not needed.
llvm-svn: 271513
multiplication
Fix small issues related to characters, operators and descriptions of tests.
Differential Revision: http://reviews.llvm.org/D20806
llvm-svn: 271264
Created a new pass ScopInfoRegionPass. As name suggests, it is a
region pass and it is there to preserve compatibility with our
existing Polly passes. ScopInfoRegionPass will return a SCoP object
for a valid region while the creation of the SCoP stays in the
ScopInfo class.
Contributed-by: Utpal Bora <cs14mtech11017@iith.ac.in>
Reviewed-by: Tobias Grosser <tobias@grosser.es>,
Johannes Doerfert <doerfert@cs.uni-saarland.de>
Differential Revision: http://reviews.llvm.org/D20770
llvm-svn: 271259
This header is required to make the ISO 646 alternative operator
spellings ("and", "or" instead of "&&", "||") work. Should these
operators be replaced by the standard ones as already suggested by
Johannes, also remove this #include again.
llvm-svn: 271206
Summary:
API-wise `apply` is a somewhat unidiomatic one-off function, and
removing the only(?) use in polly will let me remove it from SCEV's
exposed interface.
Reviewers: jdoerfert, Meinersbur, grosser
Subscribers: grosser, mcrosier, pollydev
Differential Revision: http://reviews.llvm.org/D20779
llvm-svn: 271177
Add determination of statements that contain, in particular,
matrix multiplications and can be optimized with [1] to try to
get close-to-peak performance. It can be enabled
via polly-pm-based-opts, which is false by default.
Refs:
[1] - http://www.cs.utexas.edu/users/flame/pubs/TOMS-BLIS-Analytical.pdf
Contributed-by: Roman Gareev <gareevroman@gmail.com>
Reviewed-by: Tobias Grosser <tobias@grosser.es>
Differential Revision: http://reviews.llvm.org/D20575
llvm-svn: 271128
Before this patch we bailed if a required invariant load was potentially
overwritten. However, now we will optimistically assume it is actually
invariant and, to this end, restrict the valid parameter space as well as the
execution context with regards to potential overwrites of the location.
llvm-svn: 270416
Since the base pointer of a possibly aliasing pointer might not alias
with any other pointer it (the base pointer) might not be tagged as
"required invariant". However, we need it do be in order to compare
the accessed addresses of the derived (possibly aliasing) pointer.
This patch also tries to clean up the load hoisting a little bit.
llvm-svn: 270412
So far we bailed if a required invariant load was potentially overwritten in
the SCoP. From now on we will optimistically assume it is actually invariant
and, to this end, restrict the valid parameter space.
llvm-svn: 270060
The SCoP now holds a reference to the ScopDetection::DetectionContext
which allows to simplify the type of various methods and remove code.
llvm-svn: 270053
Before this patch we only expanded valid __and__ profitable region. Therefor
we did not allow the expansion to create a profitable region from a
non-profitable one. With this patch we will remember and expand all valid
regions and check for profitability only at the end.
This patch increases the number of valid SCoPs in the LLVM-TS and SPEC
2000/2006 by 28% (from 303 to 390), including the hot loop in hmmer.
llvm-svn: 269343
This patch cleans up the rejection log handling during the
ScopDetection. It consists of two interconnected parts:
- We keep all detection contexts for a function in order to provide
more information to the user, e.g., about the rejection of
extended/intermediate regions.
- We remove the mutable "RejectLogs" member as the information is
available through the detection contexts.
llvm-svn: 269323
Truncate operations are basically modulo operations, thus we can model
them that way. However, for large types we assume the operand to fit
in the new type size instead of introducing a modulo with a very large
constant.
llvm-svn: 269300
We utilize assumptions on the input to model IR in polyhedral world.
To verify these assumptions we version the code and guard it with a
runtime-check (RTC). However, since the RTCs are themselves generated
from the polyhedral representation we generate them under the same
assumptions that they should verify. In other words, the guarantees
that we try to provide with the RTCs do not hold for the RTCs
themselves. To this end it is necessary to employ a different check
for the RTCs that will verify the assumptions did hold for them too.
Differential Revision: http://reviews.llvm.org/D20165
llvm-svn: 269299
If a profitable run is performed we will check if the SCoP seems to be
profitable after creation but before e.g., dependence are computed. This is
needed as SCoP detection only approximates the actual SCoP representation.
In the end this should allow us to be less conservative during the SCoP
detection while keeping the compile time in check.
llvm-svn: 269074
Regions with one affine loop can be profitable if the loop is
distributable. To this end we will allow them to be treated as
profitable if they contain at least two non-trivial basic blocks.
llvm-svn: 269064
The assumption attached to an llvm.assume in the SCoP needs to be
combined with the domain of the surrounding statement but can
nevertheless be used to refine the context.
This fixes the problems mentioned in PR27067.
llvm-svn: 269060
This patches makes the propagation of complexity problems during
domain generation consistent. Additionally, it makes it less likely to
encounter ill-formed domains later, e.g., during schedule generation.
llvm-svn: 269055
Before this patch we generated error-restrictions only for
error-blocks, thus blocks (or regions) containing a not represented
function call. However, the same reasoning is needed if the invalid
domain of a statement subsumes its actual domain. To this end we move
the generation of error-restrictions after the propagation of the
invalid domains. Consequently, error-statements are now defined more
general as statements that are assumed to be not executed.
Additionally, we do not record an empty domain for such statements but
a nullptr instead. This allows to distinguish between error-statements
and dead-statements.
llvm-svn: 269053
We now use context information to simplify the domains and access
functions of the SCoP instead of just aligning them with the parameter
space.
llvm-svn: 269048
Previously we checked the number of pieces to decide whether or not a
invariant load was to complex to be generated. However, there are
cases when e.g., divisions cause the complexity to spike regardless of
the number of pieces. To this end we now check the number of totally
involved dimensions which will increase with the number of pieces but
also the number of divisions.
llvm-svn: 269045
This exposes the functionality to interpret a SCEV, or better the
piece-wise function created from the SCEV, as an unsigned value
instead of a signed one.
llvm-svn: 269044
Min/max expressions are easier to read and can in some cases also result in
more concise IR that is generated as the min/max --- when lowered to a
cmp+select pattern -- commonly has a simpler condition then the ternary
condition isl would normally generate.
llvm-svn: 268855
This release includes sevaral improvments compared to the previous
version isl-0.16.1-145-g243bf7c (from the ISL 0.17 announcement):
- optionally combine SCCs incrementally in scheduler
- optionally maximize coincidence in scheduler
- optionally avoid loop coalescing in scheduler
- minor AST generator improvements
- improve support for expansions in schedule trees
llvm-svn: 268500
The check for complexity compares the number of polyhedra in a set,
which are combined by disjunctions (union, "OR"),
not conjunctions (intersection, "AND").
llvm-svn: 268223
Add a command line switch to set the
isl_options_set_schedule_outer_coincidence option. ISL then tries to
build schedules where the outer member of a band satisfies the
coincidence constraints.
In practice this allows loop skewing for more parallelism in inner
loops.
llvm-svn: 268222
After zero-extend operations and unsigned comparisons we now allow
unsigned divisions. The handling is basically the same as for signed
division, except the interpretation of the operands. As the divisor
has to be constant in both cases we can simply interpret it as an
unsigned value without additional complexity in the representation.
For the dividend we could choose from the different representation
schemes introduced for zero-extend operations but for now we will
simply use an assumption.
llvm-svn: 268032
For debugging it is often convenient to not abort at the very first memory
management error. This option allows to control this behavior at run-time.
llvm-svn: 268030
It does not suffice to take a global assumptions for unsigned comparisons but
we also need to adjust the invalid domain of the statements guarded by such
an assumption. To this end we allow to specialize the getPwAff call now in
order to indicate unsigned interpretation.
llvm-svn: 268025
When we materialize parameter SCEVs we did so without considering the
side effects they might have, e.g., both division and modulo are
undefined if the right hand side is zero. This is a problem because we
potentially extended the domain under which we evaluate parameters,
thus we might have introduced such undefined behaviour. To prevent
that from happening we will now guard divisions and modulo operations
in the parameters with a compare and select.
llvm-svn: 268023
Assumptions and restrictions can both be simplified with the domain of a
statement but not the same way. After this patch we will correctly
distinguish them.
llvm-svn: 267885
If the base pointer of an invariant load is is loaded conditionally, that
condition needs to hold for the invariant load too. The structure of the
program will imply this for domain constraints but not for imprecisions in
the modeling. To this end we will propagate the execution context of base
pointers during code generation and thus ensure the derived pointer does
not access an invalid base pointer.
llvm-svn: 267707
With this patch we will optimistically assume that the result of an unsigned
comparison is the same as the result of the same comparison interpreted as
signed.
llvm-svn: 267559
Additive expressions can have constant factors too that we can extract
and thereby simplify the internal representation. For now we do
compute the gcd of all constant factors but only extract the same
(possibly negated) factor if there is one.
llvm-svn: 267445
Before, we checked all GEPs in a statement in order to derive
out-of-bound assumptions. However, this can not only introduce new
parameters but it is also not clear what we can learn from GEPs that
are not immediately used in a memory accesses inside the SCoP. As this
case is very rare, no actual change in the behaviour is expected.
llvm-svn: 267442
Before, assumptions derived from llvm.assume could reference new
parameters that were not known to the SCoP before. These were neither
beneficial to the representation nor to the user that reads the
emitted remark. Now we project them out and keep only user assumptions
on known parameters. Nevertheless, the new parameters are still part
of the SCoPs parameter space as the SCEVAffinator currently adds them
on demand.
llvm-svn: 267441
The new handling is consistent with the remaining code, e.g., we do
not create a new parameter id for each lookup call but copy an
existing one. Additionally, we now use the implicit order defined by
the Parameters set instead of an explicit one defined in a map.
llvm-svn: 267423
A zero-extended value can be interpreted as a piecewise defined signed
value. If the value was non-negative it stays the same, otherwise it
is the sum of the original value and 2^n where n is the bit-width of
the original (or operand) type. Examples:
zext i8 127 to i32 -> { [127] }
zext i8 -1 to i32 -> { [256 + (-1)] } = { [255] }
zext i8 %v to i32 -> [v] -> { [v] | v >= 0; [256 + v] | v < 0 }
However, LLVM/Scalar Evolution uses zero-extend (potentially lead by a
truncate) to represent some forms of modulo computation. The left-hand side
of the condition in the code below would result in the SCEV
"zext i1 <false, +, true>for.body" which is just another description
of the C expression "i & 1 != 0" or, equivalently, "i % 2 != 0".
for (i = 0; i < N; i++)
if (i & 1 != 0 /* == i % 2 */)
/* do something */
If we do not make the modulo explicit but only use the mechanism described
above we will get the very restrictive assumption "N < 3", because for all
values of N >= 3 the SCEVAddRecExpr operand of the zero-extend would wrap.
Alternatively, we can make the modulo in the operand explicit in the
resulting piecewise function and thereby avoid the assumption on N. For the
example this would result in the following piecewise affine function:
{ [i0] -> [(1)] : 2*floor((-1 + i0)/2) = -1 + i0;
[i0] -> [(0)] : 2*floor((i0)/2) = i0 }
To this end we can first determine if the (immediate) operand of the
zero-extend can wrap and, in case it might, we will use explicit modulo
semantic to compute the result instead of emitting non-wrapping assumptions.
Note that operands with large bit-widths are less likely to be negative
because it would result in a very large access offset or loop bound after the
zero-extend. To this end one can optimistically assume the operand to be
positive and avoid the piecewise definition if the bit-width is bigger than
some threshold (here MaxZextSmallBitWidth).
We choose to go with a hybrid solution of all modeling techniques described
above. For small bit-widths (up to MaxZextSmallBitWidth) we will model the
wrapping explicitly and use a piecewise defined function. However, if the
bit-width is bigger than MaxZextSmallBitWidth we will employ overflow
assumptions and assume the "former negative" piece will not exist.
llvm-svn: 267408
Memory accesses can have non-precisely modeled access functions that
would cause us to build incorrect execution context for hoisted loads.
This is the same issue that occurred during the domain construction for
statements and it is dealt with the same way.
llvm-svn: 267289
The SCEVAffinator will now produce not only the isl representaiton of
a SCEV but also the domain under which it is invalid. This is used to
record possible overflows that can happen in the statement domains in
the statements invalid domain. The result is that invalid loads have
an accurate execution contexts with regards to the validity of their
statements domain. While the SCEVAffinator currently is only taking
"no-wrapping" assumptions, we can add more withouth worrying about the
execution context of loads that are optimistically hoisted.
llvm-svn: 267288
The invalid context is not enough to describe the parameter constraints under
which a statement is not modeled precisely. The reason is that during the
domain construction the bounds on the induction variables are not known but
needed to check if e.g., an overflow can actually happen. To this end we
replace the invalid context of a statement with an invalid domain. It is
initialized during domain construction and intersected with the domain once
it was completely build. Later this invalid domain allows to eliminate
falsely assumed wrapping cases and other falsely assumed mismatches in the
modeling.
llvm-svn: 267286
We used checks to minimize the number of remarks we present to a user
but these checks can become expensive, especially since all wrapping
assumptions are emitted separately. Because there is not benefit for a
"headless" run we put these checks under a command line flag. Thus, if
the flag is not given we will emit "non-effective" remarks, e.g.,
duplicates and revert to the old behaviour if it is given. As this
also changes the internal representation of some sets we set the flag
by default for our unit tests.
llvm-svn: 266087
Utilizing the record option for assumptions we can simplify the wrapping
assumption generation a lot. Additionally, we can now report locations
together with wrapping assumptions, though they might not be accurate yet.
llvm-svn: 266069
There are three reasons why we want to record assumptions first before we
add them to the assumed/invalid context:
1) If the SCoP is not profitable or otherwise invalid without the
assumed/invalid context we do not have to compute it.
2) Information about the context are gathered rather late in the SCoP
construction (basically after we know all parameters), thus the user
might see overly complicated assumptions to be taken while they would
have been simplified later on.
3) Currently we cannot take assumptions at any point but have to wait,
e.g., for the domain generation to finish. This makes wrapping
assumptions much more complicated as they need to be and it will
have a similar effect on "signed-unsigned" assumptions later.
llvm-svn: 266068
Collect the error domain contexts (formerly in the ErrorDomainCtxMap)
for each statement in the new InvalidContext member variable. While
this commit is basically a [NFC] it is a first step to make hoisting
sound by allowing a more fine grained record of invalid contexts,
e.g., here on statement level.
llvm-svn: 266053
Allow overflow of indices into the next higher dimension if it has
constant size. E.g.
float A[32][2];
((float*)A)[5];
is effectively the same as
A[2][1];
This can happen since r265379 as a side effect if ScopDetection
recognizes an access as affine, but ScopInfo rejects the GetElementPtr.
Differential Revision: http://reviews.llvm.org/D18878
llvm-svn: 265942
MSVC warns with:
warning C4239: nonstandard extension used: 'initializing': conversion from 'llvm::DebugLoc' to 'llvm::DebugLoc &'
note: A non-const reference may only be bound to an lvalue
Change the reference to a const reference.
llvm-svn: 265937
In r247147 we disabled pointer expressions because the IslExprBuilder did not
fully support them. This patch reintroduces them by simply treating them as
integers. The only special handling for pointers that is left detects the
comparison of two address_of operands and uses an unsigned compare.
llvm-svn: 265894
This reverts commit 2879c53e80e05497f408f21ce470d122e9f90f94.
Additionally, it adds SDiv and SRem instructions to the set of values
discovered by the findValues function even if we add the operands to
be able to recompute the SCEVs. In subfunctions we do not want to
recompute SDiv and SRem instructions but pass them instead as they
might have been created through the IslExprBuilder and are more
complicated than simple SDiv/SRem instructions in the code.
llvm-svn: 265873
We verify the optimized function now for a long time and it helped to track
down bugs early. This will now also happen for all parallel subfunctions we
generate.
llvm-svn: 265823
The way to get the elements size with getPrimitiveSizeInBits() is not
the same as used in other parts of Polly which should use
DataLayout::getTypeAllocSize(). Its use only queries the size of the
pointer and getPrimitiveSizeInBits returns 0 for types that require a
DataLayout object such as pointers.
Together with r265379, this should fix PR27195.
llvm-svn: 265795
If we build the domains for error blocks and later remove them we lose
the information that they are not executed. Thus, in the SCoP it looks
like the control will always reach the statement S:
for (i = 0 ... N)
if (*valid == 0)
doSth(&ptr);
S: A[i] = *ptr;
Consequently, we would have assumed "ptr" to be always accessed and
preloaded it unconditionally. However, only if "*valid != 0" we would
execute the optimized version of the SCoP. Nevertheless, we would have
hoisted and accessed "ptr"regardless of "*valid". This changes the
semantic of the program as the value of "*valid" can cause a change of
"ptr" and control if it is executed or not.
To fix this problem we adjust the execution context of hoisted loads
wrt. error domains. To this end we introduce an ErrorDomainCtxMap that
maps each basic block to the error context under which it might be
executed. Thus, to the context under which it is executed but an error
block would have been executed to. To fill this map one traversal of
the blocks in the SCoP suffices. During this traversal we do also
"remove" error statements and those that are only reachable via error
statements. This was previously done by the removeErrorBlockDomains
function which is therefor not needed anymore.
This fixes bug PR26683 and thereby several SPEC miscompiles.
Differential Revision: http://reviews.llvm.org/D18822
llvm-svn: 265778
If ScalarEvolution cannot look through some expression but we do, it
might happen that a multiplication will arrive at the
SCEVAffinator::visitMulExpr. While we could always try to improve the
extractConstantFactor function we might still miss something, thus we
reintroduce the code to generate multiplicative piecewise-affine
functions as a fall-back.
llvm-svn: 265777
The findValues() function did not look through div & srem instructions
that were part of the argument SCEV. However, in different other
places we already look through it. This mismatch caused us to preload
values in the wrong order.
llvm-svn: 265775
If all exiting blocks of a SCoP are error blocks and therefor not
represented we will not generate accesses and consequently no SAI
objects for exit PHIs. However, they are needed in the code generation
to generate the merge PHIs between the original and optimized region.
With this patch we enusre that the SAI objects for exit PHIs exist
even if all exiting blocks turn out to be eror blocks.
This fixes the crash reported in PR27207.
llvm-svn: 265393
We currently only consider the first GEP when delinearizing access functions,
which makes us loose information about additional index expression offsets,
which results in our SCoP model to be incorrect. With this patch we now
compare the base pointers used to ensure we do not miss any additional offsets.
This fixes llvm.org/PR27195.
We may consider supporting nested GEP in our delinearization heuristics in
the future.
llvm-svn: 265379
Even before we build the domain the branch condition can become very
complex, especially if we have to build the complement of a lot of
equality constraints. With this patch we bail if the branch condition
has a lot of basic sets and parameters.
After this patch we now successfully compile
External/SPEC/CINT2000/186_crafty/186_crafty
with "-polly-process-unprofitable -polly-position=before-vectorizer".
llvm-svn: 265286
As a CFG is often structured we can simplify the steps performed during
domain generation. When we push domain information we can utilize the
information from a block A to build the domain of a block B, if A dominates B
and there is no loop backede on a path from A to B. When we pull domain
information we can use information from a block A to build the domain of a
block B if B post-dominates A. This patch implements both ideas and thereby
simplifies domains that were not simplified by isl. For the FINAL basic block
in test/ScopInfo/complex-successor-structure-3.ll we used to build a universe
set with 81 basic sets. Now it actually is represented as universe set.
While the initial idea to utilize the graph structure depended on the
dominator and post-dominator tree we can use the available region
information as a coarse grained replacement. To this end we push the
region entry domain to the region exit and pull it from the region
entry for the region exit if applicable.
With this patch we now successfully compile
External/SPEC/CINT2006/400_perlbench/400_perlbench
and
SingleSource/Benchmarks/Adobe-C++/loop_unroll.
Differential Revision: http://reviews.llvm.org/D18450
llvm-svn: 265285
If a loop has no exiting blocks the region covering we use during
schedule genertion might not cover that loop properly. For now we bail
out as we would not optimize these loops anyway.
llvm-svn: 265280
If an exit PHI is written and also read in the SCoP we should not create two
SAI objects but only one. As the read is only modeled to ensure OpenMP code
generation knows about it we can simply use the EXIT_PHI MemoryKind for both
accesses.
llvm-svn: 265261
If a loop has no exiting blocks the region covering we use during
schedule genertion might not cover that loop properly. For now we bail
out as we would not optimize these loops anyway.
llvm-svn: 265260
If a non-affine region PHI is generated we should not move the insert
point prior to the synthezised value in the same block as we might
split that block at the insert point later on. Only if the incoming
value should be placed in a different block we should change the
insertion point.
llvm-svn: 265132
... instead of hardcoding something that has been free at some point. This fixes
a crash triggered by r265084, where the diagnostic IDs have been shifted in a
way that resulted our hardcode ID to not be assigned any implementation. Our ID
was likely already wrong earlier on, but this time we really crashed nicely.
llvm-svn: 265114
These caused LNT failures due to new assertions when running with
-polly-position=before-vectorizer -polly-process-unprofitable for:
FAIL: clamscan.compile_time
FAIL: cjpeg.compile_time
FAIL: consumer-jpeg.compile_time
FAIL: shapes.compile_time
FAIL: clamscan.execution_time
FAIL: cjpeg.execution_time
FAIL: consumer-jpeg.execution_time
FAIL: shapes.execution_time
The failures have been introduced by r264782, but r264789 had to be reverted
as it depended on the earlier patch.
llvm-svn: 264885
As a CFG is often structured we can simplify the steps performed
during domain generation. When we push domain information we can
utilize the information from a block A to build the domain of a
block B, if A dominates B. When we pull domain information we can
use information from a block A to build the domain of a block B
if B post-dominates A. This patch implements both ideas and thereby
simplifies domains that were not simplified by isl. For the FINAL
basic block in
test/ScopInfo/complex-successor-structure-3.ll .
we used to build a universe set with 81 basic sets. Now it actually is
represented as universe set.
While the initial idea to utilize the graph structure depended on the
dominator and post-dominator tree we can use the available region
information as a coarse grained replacement. To this end we push the
region entry domain to the region exit and pull it from the region
entry for the region exit.
Differential Revision: http://reviews.llvm.org/D18450
llvm-svn: 264789
Instead of waiting for the domain construction to finish we will now
bail as early as possible in case a complexity problem is encountered.
This might save compile time but more importantly it makes the "abort"
explicit. While we can always check if we invalidated the assumed
context we can simply propagate the result of the construction back.
This also removes the HasComplexCFG flag that was used for the very
same reason.
Differential Revision: http://reviews.llvm.org/D18504
llvm-svn: 264775
Johannes Doerfert