Adapt for 05da2fe521 "Sink all InitializePasses.h includes" which
forgot the GPGPU files (presumably because POLLY_ENABLE_GPGPU_CODEGEN
is OFF by default).
Avoids the need to include TargetMachine.h from various places just for
an enum. Various other enums live here, such as the optimization level,
TLS model, etc. Data suggests that this change probably doesn't matter,
but it seems nice to have anyway.
This file lists every pass in LLVM, and is included by Pass.h, which is
very popular. Every time we add, remove, or rename a pass in LLVM, it
caused lots of recompilation.
I found this fact by looking at this table, which is sorted by the
number of times a file was changed over the last 100,000 git commits
multiplied by the number of object files that depend on it in the
current checkout:
recompiles touches affected_files header
342380 95 3604 llvm/include/llvm/ADT/STLExtras.h
314730 234 1345 llvm/include/llvm/InitializePasses.h
307036 118 2602 llvm/include/llvm/ADT/APInt.h
213049 59 3611 llvm/include/llvm/Support/MathExtras.h
170422 47 3626 llvm/include/llvm/Support/Compiler.h
162225 45 3605 llvm/include/llvm/ADT/Optional.h
158319 63 2513 llvm/include/llvm/ADT/Triple.h
140322 39 3598 llvm/include/llvm/ADT/StringRef.h
137647 59 2333 llvm/include/llvm/Support/Error.h
131619 73 1803 llvm/include/llvm/Support/FileSystem.h
Before this change, touching InitializePasses.h would cause 1345 files
to recompile. After this change, touching it only causes 550 compiles in
an incremental rebuild.
Reviewers: bkramer, asbirlea, bollu, jdoerfert
Differential Revision: https://reviews.llvm.org/D70211
Commit 395124 "NVPTX: Don't insert an extra empty line at the end of the last section"
changed the length of the kernel payload. Update the regression test to the new binary size.
Root cause is VectorBlockGenerator::copyStmt iterates all instructions
in basic block, however some load instructions may be not unnecessary
thus removed by simplification. As a result, these load instructions
don't have a corresponding array.
Looking at BlockGenerator::copyBB, it only iterates instructions list
of ScopStmt. Given it must be a block type scop in case of
vectorization, I think we should do the same in
VectorBlockGenerator::copyStmt.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D70076
ScopBuilder::buildEqivClassBlockStmts creates ScopStmts for instruction
groups in basic block and inserts these ScopStmts into Scop::StmtMap,
however, as described in llvm.org/PR38358, comment #5, StmtScops are
inserted into vector ScopStmt[BB] in wrong order. As a result,
ScopBuilder::buildSchedule creates wrong order sequence node.
Looking closer to code, it's clear there is no equivalent classes with
interleaving isOrderedInstruction(memory access) instructions after
joinOrderedInstructions. Afterwards, ScopStmts need to be created and
inserted in the original order of memory access instructions, however,
at the moment ScopStmts are inserted in the order of leader instructions
which are probably not memory access instructions.
The fix is simple with a standalone loop scanning
isOrderedInstruction(memory access) instructions in basic block and
inserting elements into LeaderToInstList one by one. The patch also
removes double reversing operations which are now unnecessary.
New test preserve-equiv-class-order-in-basic_block.ll is also added.
Differential Revision: https://reviews.llvm.org/D68941
llvm-svn: 375192
Since the removal of extensions nodes from schedule trees in r362257 it
is possible to emit parallel code for SCoPs containing
matrix-multiplications. However, the code looking for references used in
outlined statement was not prepared to handle CopyStmts introduced by
the matrix-matrix multiplication detection.
In this case, CopyStmts do not introduce references in addition to the
ones captured by MemoryAccesses, i.e. we change the assertion to accept
CopyStmts and add a regression test for this case.
This fixes llvm.org/PR43164
llvm-svn: 372188
lib/Transform/ScheduleOptimizer.cpp fails to compile on Solaris, both on the 9.x
branch (first noticed when running test-release.sh without -no-polly) and on trunk:
/var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/lib/Transform/ScheduleOptimizer.cpp: In function ‘MicroKernelParamsTy getMicroKernelParams(const llvm::TargetTransformInfo*, polly::MatMulInfoTy)’:
/var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/lib/Transform/ScheduleOptimizer.cpp:914:62: error: call of overloaded ‘sqrt(long unsigned int)’ is ambiguous
914 | ceil(sqrt(Nvec * LatencyVectorFma * ThroughputVectorFma) / Nvec) * Nvec;
| ^
In file included from /usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/math.h:24,
from /usr/gcc/9/include/c++/9.1.0/cmath:45,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm-c/DataTypes.h:28,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm/Support/DataTypes.h:16,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm/ADT/Hashing.h:47,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm/ADT/ArrayRef.h:12,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/include/polly/ScheduleOptimizer.h:12,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/lib/Transform/ScheduleOptimizer.cpp:48:
/usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/iso/math_iso.h:220:21: note: candidate: ‘long double std::sqrt(long double)’
220 | inline long double sqrt(long double __X) { return __sqrtl(__X); }
| ^~~~
/usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/iso/math_iso.h:186:15:
note: candidate: ‘float std::sqrt(float)’
186 | inline float sqrt(float __X) { return __sqrtf(__X); }
| ^~~~
/usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/iso/math_iso.h:74:15:
note: candidate: ‘double std::sqrt(double)’
74 | extern double sqrt __P((double));
| ^~~~
/var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/lib/Transform/ScheduleOptimizer.cpp:915:67:
error: call of overloaded ‘ceil(long unsigned int)’ is ambiguous
915 | int Mr = ceil(Nvec * LatencyVectorFma * ThroughputVectorFma / Nr);
| ^
In file included from /usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/math.h:24,
from /usr/gcc/9/include/c++/9.1.0/cmath:45,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm-c/DataTypes.h:28,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm/Support/DataTypes.h:16,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm/ADT/Hashing.h:47,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/include/llvm/ADT/ArrayRef.h:12,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/include/polly/ScheduleOptimizer.h:12,
from /var/llvm/llvm-9.0.0-rc4/rc4/llvm.src/tools/polly/lib/Transform/ScheduleOptimizer.cpp:48:
/usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/iso/math_iso.h:196:21: note: candidate: ‘long double std::ceil(long double)’
196 | inline long double ceil(long double __X) { return __ceill(__X); }
| ^~~~
/usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/iso/math_iso.h:160:15:
note: candidate: ‘float std::ceil(float)’
160 | inline float ceil(float __X) { return __ceilf(__X); }
| ^~~~
/usr/gcc/9/lib/gcc/x86_64-pc-solaris2.11/9.1.0/include-fixed/iso/math_iso.h:76:15:
note: candidate: ‘double std::ceil(double)’
76 | extern double ceil __P((double));
| ^~~~
Fixed by adding casts to disambiguate, checked that it now compiles on both
amd64-pc-solaris2.11 and sparcv9-sun-solaris2.11 and on x86_64-pc-linux-gnu.
Differential Revision: https://reviews.llvm.org/D67442
llvm-svn: 371825
Function joinOrderedInstructions merges instructions when a leader is encountered twice.
It also notices that leaders in SeenLeaders may lose their leadership in previous merging,
and tries to handle the case using following code:
Instruction *PrevLeader = UnionFind.getLeaderValue(SeenLeaders.back());
However, this is wrong because it always gets leader for the last element of SeenLeaders,
and I believe it's wrong even we get leader for Prev here. As a result, Statements in cases
like the one in patch aren't merged as expected. After investigation, I believe it's
unnecessary to get leader instruction at all. This is based on fact: Although leaders in
SeenLeaders could lose leadership, they only lose to others in SeenLeaders, in other words,
one existing leader will be chosen as new leader of merged equivalent statements. We can
take advantage of this and simply check if current leader equals to Prev and break merging
if it does.
The patch also adds a new test.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D67007
llvm-svn: 371801
When reading code of Dependences::calculateDependences, I noticed that
WAR is computed specifically by buildWAR. Given ISL now
supports "kills" in approximate dataflow analysis, this patch takes
advantage of it.
This patch also cleans up a couple lines redundant codes.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D66741
llvm-svn: 370396
The while loop iterating parent loop in ScopBuilder::buildDomains is
unnecessary because either L or LD are later unused, this is a simple
patch removing it.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D66698
llvm-svn: 370368
When reading code in ScopBuilder::buildEqivClassBlockStmts, I think the
main statement flag computation can be simplified, here is the patch.
It's based on two simple facts that:
1. Instruction won't be removed once it's inserted into UnionFind.
2. Main statement must be set if there is non-trivial statement besides the last one.
The patch also saves std::find call.
Patch by bin.narwal <bin.narwal@gmail.com>
Differential Revision: https://reviews.llvm.org/D66477
llvm-svn: 369972
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368935
Scope of changes:
1) Moved buildDomains function to ScopBuilder class.
2) Moved buildDomainsWithBranchConstraints function to ScopBuilder class.
3) Moved propagateDomainConstraints to ScopBuilder class.
4) Moved propagateDomainConstraintsToRegionExit to ScopBuilder class.
5) Moved propagateInvalidStmtDomains to ScopBuilder class.
6) Moved getPredecessorDomainConstraints function to ScopBuilder class.
7) Moved addLoopBoundsToHeaderDomain function to ScopBuilder class.
8) Moved getPwAff function to ScopBuilder class.
9) Moved buildConditionSets functions to ScopBuilder class.
10) Added updateMaxLoopDepth, notifyErrorBlock, getOrInitEmptyDomain, isDomainDefined, setDomain functions to Scop class. They are used by ScopBuilder.
11) Moved helper functions: getRegionNodeBasicBlock, getRegionNodeSuccessor, containsErrorBlock, createNextIterationMap, collectBoundedParts, partitionSetParts, buildConditionSet to ScopBuilder.cpp file.
Differential Revision: https://reviews.llvm.org/D65729
llvm-svn: 368100
Scope of changes:
1) Moved addUserAssumptions function to ScopBuilder class.
2) Moved buildConditionSets functions to polly namespace.
3) Moved getRepresentingInvariantLoadSCEV to public section of the Scop class
Differential Revision: https://reviews.llvm.org/D65241
llvm-svn: 368089
Scope of changes:
1. Moved buildSchedule functions to ScopBuilder.
2. Moved combineInSequence function to ScopBuilder.
3. Moved mapToDimension function to ScopBuilder.
4. Moved LoopStackTy to ScopBuilder.
5. Moved getLoopSurroundingScop to ScopHelper.
6. Moved getNumBlocksInLoop to ScopHelper.
7. Moved getNumBlocksInRegionNode to ScopHelper.
8. Moved getRegionNodeLoop to ScopHelper.
Differential Revision: https://reviews.llvm.org/D64223
llvm-svn: 366377
Scope of changes:
1) Moved finalizeAccesses to ScopBuilder
2) Moved updateAccessDimensionality to ScopBuilder
3) Moved foldSizeConstantsToRight to ScopBuilder
4) Moved foldSizeConstantsToRight to ScopBuilder
5) Moved assumeNoOutOfBounds to ScopBuilder
6) Moved markFortranArrays to ScopBuilder
7) Added iterator range for AccessFunctions vector.
Differential Revision: https://reviews.llvm.org/D63794
llvm-svn: 366374
Scope of changes:
1) Moved addUserContext to ScopBuilder.
2) Moved command line option UserContextStr to ScopBuilder.
Differential Revision: https://reviews.llvm.org/D63740
llvm-svn: 366266
Scope of changes:
1) Moved buildAliasChecks to ScopBuilder.
2) Moved buildAliasGroup to ScopBuilder.
3) Moved buildAliasGroups to ScopBuilder.
4) Moved buildAliasGroupsForAccesses to ScopBuilder.
5) Moved splitAliasGroupsByDomain to ScopBuilder.
6) Moved addNonEmptyDomainConstraints to ScopBuilder.
7) Moved buildMinMaxAccess to ScopBuilder.
8) Moved calculateMinMaxAccess to ScopBuilder.
9) Moved getAccessDomain to ScopBuilder.
10) Moved command line options used only by buildAliasChecks functions to ScopBuilder.
11) Refactored buildAliasGroup function. Added addAliasGroup function to Scop class for pushing back calculated min/max accesses.
12) Added function incrementNumberOfAliasingAssumptions which increments number of statistic variable AssumptionsAliasing. AssumptionsAliasing variable is defined by STATISTIC macro inside ScopInfo.cpp and it is also used by function trackAssumption from Scop class.
13) Added reference to OptimizationRemarkEmitter to ScopBuilder class.
14) Moved calculateMinMaxAccess function to ScopBuilder class.
Differential Revision: https://reviews.llvm.org/D63693
llvm-svn: 366262
Scope of changes:
1) Moved addRecordedAssumptions to ScopBuilder.
2) Moved Assumption struct outside Scop class.
3) Refactored addRecordedAssumptions function. Replaced while loop by
for range loop.
4) Added function to clear processed Assumptions.
Differential Revision: https://reviews.llvm.org/D63572
llvm-svn: 366260
The Loads.h API changed so that a Type parameter is now mandatory in
preparation for pointer types being opaque. Unfortunately I don't build
polly routinely and it still had some uses. This just provides the
(obvious) load type in each case.
llvm-svn: 365470
https://reviews.llvm.org/D61934, committed as r362687, r363540, r363364
and r363147, made some emitted instruction nus/nsw. Add these falgs to
Polly's regression tests.
This should fix
Polly :: Isl/CodeGen/partial_write_in_region_with_loop.ll
Polly :: Isl/CodeGen/scev_expansion_in_nonaffine.ll
llvm-svn: 363599
FunctionAnalysisManagerModuleProxy started to be used by the
AlwaysInlinerPass in r363287 and therefore had to be registered in the
New PassManager.
Should fix the regression tests
Polly :: ScopInliner/invariant-load-func.ll
Polly :: ScopInliner/simple-inline-loop.ll
llvm-svn: 363572
Moved addInvariantLoads and functions listed below to ScopBuilder:
isAParameter
canAlwaysBeHoisted
These functions were referenced only by getNonHoistableCtx.
Moved CLI parameter PollyAllowDereferenceOfAllFunctionParams to
ScopBuilder.
Added iterator range through InvariantEquivClasses.
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D63172
llvm-svn: 363216
This review is based on review: https://reviews.llvm.org/D62925 . It is
part of moving hoistInvariantLoads function and all functions referenced
only by hoistInvariantLoads to ScopBuilder.
Moved getNonHoistableCtx and functions listed below to ScopBuilder:
isRequiredInvariantLoad
hasNonHoistableBasePtrInScop
isAccessRangeTooComplex
These functions were referenced only by getNonHoistableCtx.
MaxDimensionsInAccessRange and MaxDisjunctsInDomain constant is marked
as extern and it is added to polly namespace. It is used by Scop and
ScopBuilder classes.
MaxDimensionsInAccessRange constant moved to ScopBuilder. It is not used
outside ScopBuilder.
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D63066
llvm-svn: 363214
Refactor Scop and ScopBuilder class:
1. Move hoistInvariantLoads function from Scop to ScopBuilder class.
2. Private functions (addInvariantLoads, getNonHoistableCtx) are moved
to public section of Scop class. hoistInvariantLoads function
references these functions. These functions will be moved to
ScopBuilder as well in the next steps.
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D62925
llvm-svn: 363121
Refactor Scop and ScopBuilder class. Move canonicalizeDynamicsBasePtrs
and corresponding static functions from ScopInfo.cpp to ScopBuilder.cpp
Patch by Dominik Adamski <adamski.dominik@gmail.com>
Differential Revision: https://reviews.llvm.org/D62781
llvm-svn: 362554
Heejin Ahn