maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/lib/CodeGen/CGLoopInfo.cpp

402 lines
14 KiB
C++
Raw Normal View History

This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
//===---- CGLoopInfo.cpp - LLVM CodeGen for loop metadata -*- C++ -*-------===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 16:50:56 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
//
//===----------------------------------------------------------------------===//
#include "CGLoopInfo.h"
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
#include "clang/AST/ASTContext.h"
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
#include "clang/AST/Attr.h"
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
#include "llvm/IR/BasicBlock.h"
[TI removal] Test predicate rather than casting to detect a terminator and use the range based successor API. llvm-svn: 344730
2018-10-18 16:16:20 +08:00
#include "llvm/IR/CFG.h"
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
#include "llvm/IR/Constants.h"
#include "llvm/IR/InstrTypes.h"
#include "llvm/IR/Instructions.h"
#include "llvm/IR/Metadata.h"
Eliminate unnecessary namespace to prevent conflicts. llvm-svn: 239365
2015-06-09 07:27:35 +08:00
using namespace clang::CodeGen;
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
using namespace llvm;
Add a loop's debug location to its llvm.loop metadata Getting accurate locations for loops is important, because those locations are used by the frontend to generate optimization remarks. Currently, optimization remarks for loops often appear on the wrong line, often the first line of the loop body instead of the loop itself. This is confusing because that line might itself be another loop, or might be somewhere else completely if the body was an inlined function call. This happens because of the way we find the loop's starting location. First, we look for a preheader, and if we find one, and its terminator has a debug location, then we use that. Otherwise, we look for a location on an instruction in the loop header. The fallback heuristic is not bad, but will almost always find the beginning of the body, and not the loop statement itself. The preheader location search often fails because there's often not a preheader, and even when there is a preheader, depending on how it was formed, it sometimes carries the location of some preceeding code. I don't see any good theoretical way to fix this problem. On the other hand, this seems like a straightforward solution: Put the debug location in the loop's llvm.loop metadata. When emitting debug information, this commit causes us to add the debug location as an operand to each loop's llvm.loop metadata. Thus, we now generate this metadata for all loops (not just loops with optimization hints) when we're otherwise generating debug information. The remark test case changes depend on the companion LLVM commit r270771. llvm-svn: 270772
2016-05-26 05:53:24 +08:00
static MDNode *createMetadata(LLVMContext &Ctx, const LoopAttributes &Attrs,
[CodeGen] Pass objects that are expensive to copy by const ref. No functionality change. Found by clang-tidy's performance-unnecessary-value-param. llvm-svn: 287894
2016-11-25 00:01:20 +08:00
const llvm::DebugLoc &StartLoc,
[CodeGen] Generate llvm.loop.parallel_accesses instead of llvm.mem.parallel_loop_access metadata. Instead of generating llvm.mem.parallel_loop_access metadata, generate llvm.access.group on instructions and llvm.loop.parallel_accesses on loops. There is one access group per generated loop. This is clang part of D52116/r349725. Differential Revision: https://reviews.llvm.org/D52117 llvm-svn: 349823
2018-12-21 05:24:54 +08:00
const llvm::DebugLoc &EndLoc, MDNode *&AccGroup) {
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
if (!Attrs.IsParallel && Attrs.VectorizeWidth == 0 &&
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
Attrs.InterleaveCount == 0 && Attrs.UnrollCount == 0 &&
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
Attrs.UnrollAndJamCount == 0 && !Attrs.PipelineDisabled &&
Attrs.PipelineInitiationInterval == 0 &&
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
Attrs.VectorizeEnable == LoopAttributes::Unspecified &&
Add a loop's debug location to its llvm.loop metadata Getting accurate locations for loops is important, because those locations are used by the frontend to generate optimization remarks. Currently, optimization remarks for loops often appear on the wrong line, often the first line of the loop body instead of the loop itself. This is confusing because that line might itself be another loop, or might be somewhere else completely if the body was an inlined function call. This happens because of the way we find the loop's starting location. First, we look for a preheader, and if we find one, and its terminator has a debug location, then we use that. Otherwise, we look for a location on an instruction in the loop header. The fallback heuristic is not bad, but will almost always find the beginning of the body, and not the loop statement itself. The preheader location search often fails because there's often not a preheader, and even when there is a preheader, depending on how it was formed, it sometimes carries the location of some preceeding code. I don't see any good theoretical way to fix this problem. On the other hand, this seems like a straightforward solution: Put the debug location in the loop's llvm.loop metadata. When emitting debug information, this commit causes us to add the debug location as an operand to each loop's llvm.loop metadata. Thus, we now generate this metadata for all loops (not just loops with optimization hints) when we're otherwise generating debug information. The remark test case changes depend on the companion LLVM commit r270771. llvm-svn: 270772
2016-05-26 05:53:24 +08:00
Attrs.UnrollEnable == LoopAttributes::Unspecified &&
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
Attrs.UnrollAndJamEnable == LoopAttributes::Unspecified &&
Attrs.DistributeEnable == LoopAttributes::Unspecified && !StartLoc &&
!EndLoc)
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
return nullptr;
IR: Update clang for Metadata/Value split in r223802 Match LLVM API changes from r223802. llvm-svn: 223803
2014-12-10 02:39:32 +08:00
SmallVector<Metadata *, 4> Args;
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
// Reserve operand 0 for loop id self reference.
CodeGen: Update LoopAttributes for LLVM API change `MDNode::getTemporary()` returns a `unique_ptr<>` as of r226504. llvm-svn: 226505
2015-01-20 05:30:48 +08:00
auto TempNode = MDNode::getTemporary(Ctx, None);
Args.push_back(TempNode.get());
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
Add the loop end location to the loop metadata. This additional information can be used to improve the locations when generating remarks for loops. Depends on the companion LLVM change r286227. Patch by Florian Hahn. Differential Revision: https://reviews.llvm.org/D25764 llvm-svn: 286456
2016-11-10 22:44:30 +08:00
// If we have a valid start debug location for the loop, add it.
if (StartLoc) {
Args.push_back(StartLoc.getAsMDNode());
// If we also have a valid end debug location for the loop, add it.
if (EndLoc)
Args.push_back(EndLoc.getAsMDNode());
}
Add a loop's debug location to its llvm.loop metadata Getting accurate locations for loops is important, because those locations are used by the frontend to generate optimization remarks. Currently, optimization remarks for loops often appear on the wrong line, often the first line of the loop body instead of the loop itself. This is confusing because that line might itself be another loop, or might be somewhere else completely if the body was an inlined function call. This happens because of the way we find the loop's starting location. First, we look for a preheader, and if we find one, and its terminator has a debug location, then we use that. Otherwise, we look for a location on an instruction in the loop header. The fallback heuristic is not bad, but will almost always find the beginning of the body, and not the loop statement itself. The preheader location search often fails because there's often not a preheader, and even when there is a preheader, depending on how it was formed, it sometimes carries the location of some preceeding code. I don't see any good theoretical way to fix this problem. On the other hand, this seems like a straightforward solution: Put the debug location in the loop's llvm.loop metadata. When emitting debug information, this commit causes us to add the debug location as an operand to each loop's llvm.loop metadata. Thus, we now generate this metadata for all loops (not just loops with optimization hints) when we're otherwise generating debug information. The remark test case changes depend on the companion LLVM commit r270771. llvm-svn: 270772
2016-05-26 05:53:24 +08:00
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
// Setting vectorize.width
if (Attrs.VectorizeWidth > 0) {
IR: Update clang for Metadata/Value split in r223802 Match LLVM API changes from r223802. llvm-svn: 223803
2014-12-10 02:39:32 +08:00
Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.vectorize.width"),
ConstantAsMetadata::get(ConstantInt::get(
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
Type::getInt32Ty(Ctx), Attrs.VectorizeWidth))};
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
Args.push_back(MDNode::get(Ctx, Vals));
}
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
// Setting interleave.count
if (Attrs.InterleaveCount > 0) {
IR: Update clang for Metadata/Value split in r223802 Match LLVM API changes from r223802. llvm-svn: 223803
2014-12-10 02:39:32 +08:00
Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.interleave.count"),
ConstantAsMetadata::get(ConstantInt::get(
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
Type::getInt32Ty(Ctx), Attrs.InterleaveCount))};
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
Args.push_back(MDNode::get(Ctx, Vals));
}
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
// Setting unroll.count
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
if (Attrs.UnrollCount > 0) {
Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.unroll.count"),
ConstantAsMetadata::get(ConstantInt::get(
Type::getInt32Ty(Ctx), Attrs.UnrollCount))};
Args.push_back(MDNode::get(Ctx, Vals));
}
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
// Setting unroll_and_jam.count
if (Attrs.UnrollAndJamCount > 0) {
Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.unroll_and_jam.count"),
ConstantAsMetadata::get(ConstantInt::get(
Type::getInt32Ty(Ctx), Attrs.UnrollAndJamCount))};
Args.push_back(MDNode::get(Ctx, Vals));
}
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
// Setting vectorize.enable
if (Attrs.VectorizeEnable != LoopAttributes::Unspecified) {
Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.vectorize.enable"),
ConstantAsMetadata::get(ConstantInt::get(
Type::getInt1Ty(Ctx), (Attrs.VectorizeEnable ==
LoopAttributes::Enable)))};
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
Args.push_back(MDNode::get(Ctx, Vals));
}
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
// Setting unroll.full or unroll.disable
if (Attrs.UnrollEnable != LoopAttributes::Unspecified) {
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
std::string Name;
if (Attrs.UnrollEnable == LoopAttributes::Enable)
Name = "llvm.loop.unroll.enable";
else if (Attrs.UnrollEnable == LoopAttributes::Full)
Name = "llvm.loop.unroll.full";
else
Name = "llvm.loop.unroll.disable";
Metadata *Vals[] = {MDString::get(Ctx, Name)};
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
Args.push_back(MDNode::get(Ctx, Vals));
}
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
// Setting unroll_and_jam.full or unroll_and_jam.disable
if (Attrs.UnrollAndJamEnable != LoopAttributes::Unspecified) {
std::string Name;
if (Attrs.UnrollAndJamEnable == LoopAttributes::Enable)
Name = "llvm.loop.unroll_and_jam.enable";
else if (Attrs.UnrollAndJamEnable == LoopAttributes::Full)
Name = "llvm.loop.unroll_and_jam.full";
else
Name = "llvm.loop.unroll_and_jam.disable";
Metadata *Vals[] = {MDString::get(Ctx, Name)};
Args.push_back(MDNode::get(Ctx, Vals));
}
Add loop pragma for Loop Distribution Summary: This is similar to other loop pragmas like 'vectorize'. Currently it only has state values: distribute(enable) and distribute(disable). When one of these is specified the corresponding loop metadata is generated: !{!"llvm.loop.distribute.enable", i1 true/false} As a result, loop distribution will be attempted on the loop even if Loop Distribution in not enabled globally. Analogously, with 'disable' distribution can be turned off for an individual loop even when the pass is otherwise enabled. There are some slight differences compared to the existing loop pragmas. 1. There is no 'assume_safety' variant which makes its handling slightly different from 'vectorize'/'interleave'. 2. Unlike the existing loop pragmas, it does not have a corresponding numeric pragma like 'vectorize' -> 'vectorize_width'. So for the consistency checks in CheckForIncompatibleAttributes we don't need to check it against other pragmas. We just need to check for duplicates of the same pragma. Reviewers: rsmith, dexonsmith, aaron.ballman Subscribers: bob.wilson, cfe-commits, hfinkel Differential Revision: http://reviews.llvm.org/D19403 llvm-svn: 272656
2016-06-14 20:04:26 +08:00
if (Attrs.DistributeEnable != LoopAttributes::Unspecified) {
Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.distribute.enable"),
ConstantAsMetadata::get(ConstantInt::get(
Type::getInt1Ty(Ctx), (Attrs.DistributeEnable ==
LoopAttributes::Enable)))};
Args.push_back(MDNode::get(Ctx, Vals));
}
[CodeGen] Generate llvm.loop.parallel_accesses instead of llvm.mem.parallel_loop_access metadata. Instead of generating llvm.mem.parallel_loop_access metadata, generate llvm.access.group on instructions and llvm.loop.parallel_accesses on loops. There is one access group per generated loop. This is clang part of D52116/r349725. Differential Revision: https://reviews.llvm.org/D52117 llvm-svn: 349823
2018-12-21 05:24:54 +08:00
if (Attrs.IsParallel) {
AccGroup = MDNode::getDistinct(Ctx, {});
Args.push_back(MDNode::get(
Ctx, {MDString::get(Ctx, "llvm.loop.parallel_accesses"), AccGroup}));
}
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
if (Attrs.PipelineDisabled) {
Metadata *Vals[] = {
MDString::get(Ctx, "llvm.loop.pipeline.disable"),
ConstantAsMetadata::get(ConstantInt::get(
Type::getInt1Ty(Ctx), (Attrs.PipelineDisabled == true)))};
Args.push_back(MDNode::get(Ctx, Vals));
}
if (Attrs.PipelineInitiationInterval > 0) {
Metadata *Vals[] = {
MDString::get(Ctx, "llvm.loop.pipeline.initiationinterval"),
ConstantAsMetadata::get(ConstantInt::get(
Type::getInt32Ty(Ctx), Attrs.PipelineInitiationInterval))};
Args.push_back(MDNode::get(Ctx, Vals));
}
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
// Set the first operand to itself.
IR: Update clang for Metadata/Value split in r223802 Match LLVM API changes from r223802. llvm-svn: 223803
2014-12-10 02:39:32 +08:00
MDNode *LoopID = MDNode::get(Ctx, Args);
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
LoopID->replaceOperandWith(0, LoopID);
return LoopID;
}
LoopAttributes::LoopAttributes(bool IsParallel)
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
: IsParallel(IsParallel), VectorizeEnable(LoopAttributes::Unspecified),
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
UnrollEnable(LoopAttributes::Unspecified),
UnrollAndJamEnable(LoopAttributes::Unspecified), VectorizeWidth(0),
InterleaveCount(0), UnrollCount(0), UnrollAndJamCount(0),
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
DistributeEnable(LoopAttributes::Unspecified), PipelineDisabled(false),
PipelineInitiationInterval(0) {}
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
void LoopAttributes::clear() {
IsParallel = false;
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
VectorizeWidth = 0;
InterleaveCount = 0;
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
UnrollCount = 0;
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
UnrollAndJamCount = 0;
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
VectorizeEnable = LoopAttributes::Unspecified;
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
UnrollEnable = LoopAttributes::Unspecified;
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
UnrollAndJamEnable = LoopAttributes::Unspecified;
[Pragma] Clear loop distribution attribute between loops llvm-svn: 279608
2016-08-24 12:31:56 +08:00
DistributeEnable = LoopAttributes::Unspecified;
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
PipelineDisabled = false;
PipelineInitiationInterval = 0;
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
}
Add a loop's debug location to its llvm.loop metadata Getting accurate locations for loops is important, because those locations are used by the frontend to generate optimization remarks. Currently, optimization remarks for loops often appear on the wrong line, often the first line of the loop body instead of the loop itself. This is confusing because that line might itself be another loop, or might be somewhere else completely if the body was an inlined function call. This happens because of the way we find the loop's starting location. First, we look for a preheader, and if we find one, and its terminator has a debug location, then we use that. Otherwise, we look for a location on an instruction in the loop header. The fallback heuristic is not bad, but will almost always find the beginning of the body, and not the loop statement itself. The preheader location search often fails because there's often not a preheader, and even when there is a preheader, depending on how it was formed, it sometimes carries the location of some preceeding code. I don't see any good theoretical way to fix this problem. On the other hand, this seems like a straightforward solution: Put the debug location in the loop's llvm.loop metadata. When emitting debug information, this commit causes us to add the debug location as an operand to each loop's llvm.loop metadata. Thus, we now generate this metadata for all loops (not just loops with optimization hints) when we're otherwise generating debug information. The remark test case changes depend on the companion LLVM commit r270771. llvm-svn: 270772
2016-05-26 05:53:24 +08:00
LoopInfo::LoopInfo(BasicBlock *Header, const LoopAttributes &Attrs,
[CodeGen] Pass objects that are expensive to copy by const ref. No functionality change. Found by clang-tidy's performance-unnecessary-value-param. llvm-svn: 287894
2016-11-25 00:01:20 +08:00
const llvm::DebugLoc &StartLoc, const llvm::DebugLoc &EndLoc)
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
: LoopID(nullptr), Header(Header), Attrs(Attrs) {
[CodeGen] Generate llvm.loop.parallel_accesses instead of llvm.mem.parallel_loop_access metadata. Instead of generating llvm.mem.parallel_loop_access metadata, generate llvm.access.group on instructions and llvm.loop.parallel_accesses on loops. There is one access group per generated loop. This is clang part of D52116/r349725. Differential Revision: https://reviews.llvm.org/D52117 llvm-svn: 349823
2018-12-21 05:24:54 +08:00
LoopID =
createMetadata(Header->getContext(), Attrs, StartLoc, EndLoc, AccGroup);
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
}
[CodeGen] Pass objects that are expensive to copy by const ref. No functionality change. Found by clang-tidy's performance-unnecessary-value-param. llvm-svn: 287894
2016-11-25 00:01:20 +08:00
void LoopInfoStack::push(BasicBlock *Header, const llvm::DebugLoc &StartLoc,
const llvm::DebugLoc &EndLoc) {
Add the loop end location to the loop metadata. This additional information can be used to improve the locations when generating remarks for loops. Depends on the companion LLVM change r286227. Patch by Florian Hahn. Differential Revision: https://reviews.llvm.org/D25764 llvm-svn: 286456
2016-11-10 22:44:30 +08:00
Active.push_back(LoopInfo(Header, StagedAttrs, StartLoc, EndLoc));
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
// Clear the attributes so nested loops do not inherit them.
StagedAttrs.clear();
}
void LoopInfoStack::push(BasicBlock *Header, clang::ASTContext &Ctx,
Add a loop's debug location to its llvm.loop metadata Getting accurate locations for loops is important, because those locations are used by the frontend to generate optimization remarks. Currently, optimization remarks for loops often appear on the wrong line, often the first line of the loop body instead of the loop itself. This is confusing because that line might itself be another loop, or might be somewhere else completely if the body was an inlined function call. This happens because of the way we find the loop's starting location. First, we look for a preheader, and if we find one, and its terminator has a debug location, then we use that. Otherwise, we look for a location on an instruction in the loop header. The fallback heuristic is not bad, but will almost always find the beginning of the body, and not the loop statement itself. The preheader location search often fails because there's often not a preheader, and even when there is a preheader, depending on how it was formed, it sometimes carries the location of some preceeding code. I don't see any good theoretical way to fix this problem. On the other hand, this seems like a straightforward solution: Put the debug location in the loop's llvm.loop metadata. When emitting debug information, this commit causes us to add the debug location as an operand to each loop's llvm.loop metadata. Thus, we now generate this metadata for all loops (not just loops with optimization hints) when we're otherwise generating debug information. The remark test case changes depend on the companion LLVM commit r270771. llvm-svn: 270772
2016-05-26 05:53:24 +08:00
ArrayRef<const clang::Attr *> Attrs,
[CodeGen] Pass objects that are expensive to copy by const ref. No functionality change. Found by clang-tidy's performance-unnecessary-value-param. llvm-svn: 287894
2016-11-25 00:01:20 +08:00
const llvm::DebugLoc &StartLoc,
const llvm::DebugLoc &EndLoc) {
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
// Identify loop hint attributes from Attrs.
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
for (const auto *Attr : Attrs) {
const LoopHintAttr *LH = dyn_cast<LoopHintAttr>(Attr);
[OpenCL] Generate metadata for opencl_unroll_hint attribute Add support for opencl_unroll_hint attribute from OpenCL v2.0 s6.11.5. Reusing most of metadata generation from CGLoopInfo helper class. The code is based on Khronos OpenCL compiler: https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 Patch by Liu Yaxun (Sam)! Differential Revision: http://reviews.llvm.org/D16686 llvm-svn: 261350
2016-02-20 02:30:11 +08:00
const OpenCLUnrollHintAttr *OpenCLHint =
dyn_cast<OpenCLUnrollHintAttr>(Attr);
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
// Skip non loop hint attributes
[OpenCL] Generate metadata for opencl_unroll_hint attribute Add support for opencl_unroll_hint attribute from OpenCL v2.0 s6.11.5. Reusing most of metadata generation from CGLoopInfo helper class. The code is based on Khronos OpenCL compiler: https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 Patch by Liu Yaxun (Sam)! Differential Revision: http://reviews.llvm.org/D16686 llvm-svn: 261350
2016-02-20 02:30:11 +08:00
if (!LH && !OpenCLHint) {
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
continue;
[OpenCL] Generate metadata for opencl_unroll_hint attribute Add support for opencl_unroll_hint attribute from OpenCL v2.0 s6.11.5. Reusing most of metadata generation from CGLoopInfo helper class. The code is based on Khronos OpenCL compiler: https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 Patch by Liu Yaxun (Sam)! Differential Revision: http://reviews.llvm.org/D16686 llvm-svn: 261350
2016-02-20 02:30:11 +08:00
}
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
[OpenCL] Generate metadata for opencl_unroll_hint attribute Add support for opencl_unroll_hint attribute from OpenCL v2.0 s6.11.5. Reusing most of metadata generation from CGLoopInfo helper class. The code is based on Khronos OpenCL compiler: https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 Patch by Liu Yaxun (Sam)! Differential Revision: http://reviews.llvm.org/D16686 llvm-svn: 261350
2016-02-20 02:30:11 +08:00
LoopHintAttr::OptionType Option = LoopHintAttr::Unroll;
LoopHintAttr::LoopHintState State = LoopHintAttr::Disable;
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
unsigned ValueInt = 1;
[OpenCL] Generate metadata for opencl_unroll_hint attribute Add support for opencl_unroll_hint attribute from OpenCL v2.0 s6.11.5. Reusing most of metadata generation from CGLoopInfo helper class. The code is based on Khronos OpenCL compiler: https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 Patch by Liu Yaxun (Sam)! Differential Revision: http://reviews.llvm.org/D16686 llvm-svn: 261350
2016-02-20 02:30:11 +08:00
// Translate opencl_unroll_hint attribute argument to
// equivalent LoopHintAttr enums.
Remove trailing space sed -Ei 's/[[:space:]]+$//' include/**/*.{def,h,td} lib/**/*.{cpp,h} llvm-svn: 338291
2018-07-31 03:24:48 +08:00
// OpenCL v2.0 s6.11.5:
[OpenCL] Generate metadata for opencl_unroll_hint attribute Add support for opencl_unroll_hint attribute from OpenCL v2.0 s6.11.5. Reusing most of metadata generation from CGLoopInfo helper class. The code is based on Khronos OpenCL compiler: https://github.com/KhronosGroup/SPIR/tree/spirv-1.0 Patch by Liu Yaxun (Sam)! Differential Revision: http://reviews.llvm.org/D16686 llvm-svn: 261350
2016-02-20 02:30:11 +08:00
// 0 - full unroll (no argument).
// 1 - disable unroll.
// other positive integer n - unroll by n.
if (OpenCLHint) {
ValueInt = OpenCLHint->getUnrollHint();
if (ValueInt == 0) {
State = LoopHintAttr::Full;
} else if (ValueInt != 1) {
Option = LoopHintAttr::UnrollCount;
State = LoopHintAttr::Numeric;
}
} else if (LH) {
auto *ValueExpr = LH->getValue();
if (ValueExpr) {
llvm::APSInt ValueAPS = ValueExpr->EvaluateKnownConstInt(Ctx);
ValueInt = ValueAPS.getSExtValue();
}
Option = LH->getOption();
State = LH->getState();
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
}
switch (State) {
case LoopHintAttr::Disable:
switch (Option) {
case LoopHintAttr::Vectorize:
// Disable vectorization by specifying a width of 1.
setVectorizeWidth(1);
break;
case LoopHintAttr::Interleave:
// Disable interleaving by speciyfing a count of 1.
setInterleaveCount(1);
break;
case LoopHintAttr::Unroll:
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
setUnrollState(LoopAttributes::Disable);
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
break;
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJam:
setUnrollAndJamState(LoopAttributes::Disable);
break;
Add loop pragma for Loop Distribution Summary: This is similar to other loop pragmas like 'vectorize'. Currently it only has state values: distribute(enable) and distribute(disable). When one of these is specified the corresponding loop metadata is generated: !{!"llvm.loop.distribute.enable", i1 true/false} As a result, loop distribution will be attempted on the loop even if Loop Distribution in not enabled globally. Analogously, with 'disable' distribution can be turned off for an individual loop even when the pass is otherwise enabled. There are some slight differences compared to the existing loop pragmas. 1. There is no 'assume_safety' variant which makes its handling slightly different from 'vectorize'/'interleave'. 2. Unlike the existing loop pragmas, it does not have a corresponding numeric pragma like 'vectorize' -> 'vectorize_width'. So for the consistency checks in CheckForIncompatibleAttributes we don't need to check it against other pragmas. We just need to check for duplicates of the same pragma. Reviewers: rsmith, dexonsmith, aaron.ballman Subscribers: bob.wilson, cfe-commits, hfinkel Differential Revision: http://reviews.llvm.org/D19403 llvm-svn: 272656
2016-06-14 20:04:26 +08:00
case LoopHintAttr::Distribute:
setDistributeState(false);
break;
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineDisabled:
setPipelineDisabled(true);
break;
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::UnrollCount:
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJamCount:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::VectorizeWidth:
case LoopHintAttr::InterleaveCount:
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineInitiationInterval:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
llvm_unreachable("Options cannot be disabled.");
break;
}
break;
case LoopHintAttr::Enable:
switch (Option) {
case LoopHintAttr::Vectorize:
case LoopHintAttr::Interleave:
setVectorizeEnable(true);
break;
case LoopHintAttr::Unroll:
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
setUnrollState(LoopAttributes::Enable);
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
break;
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJam:
setUnrollAndJamState(LoopAttributes::Enable);
break;
Add loop pragma for Loop Distribution Summary: This is similar to other loop pragmas like 'vectorize'. Currently it only has state values: distribute(enable) and distribute(disable). When one of these is specified the corresponding loop metadata is generated: !{!"llvm.loop.distribute.enable", i1 true/false} As a result, loop distribution will be attempted on the loop even if Loop Distribution in not enabled globally. Analogously, with 'disable' distribution can be turned off for an individual loop even when the pass is otherwise enabled. There are some slight differences compared to the existing loop pragmas. 1. There is no 'assume_safety' variant which makes its handling slightly different from 'vectorize'/'interleave'. 2. Unlike the existing loop pragmas, it does not have a corresponding numeric pragma like 'vectorize' -> 'vectorize_width'. So for the consistency checks in CheckForIncompatibleAttributes we don't need to check it against other pragmas. We just need to check for duplicates of the same pragma. Reviewers: rsmith, dexonsmith, aaron.ballman Subscribers: bob.wilson, cfe-commits, hfinkel Differential Revision: http://reviews.llvm.org/D19403 llvm-svn: 272656
2016-06-14 20:04:26 +08:00
case LoopHintAttr::Distribute:
setDistributeState(true);
break;
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::UnrollCount:
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJamCount:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::VectorizeWidth:
case LoopHintAttr::InterleaveCount:
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineDisabled:
case LoopHintAttr::PipelineInitiationInterval:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
llvm_unreachable("Options cannot enabled.");
break;
}
break;
case LoopHintAttr::AssumeSafety:
switch (Option) {
case LoopHintAttr::Vectorize:
case LoopHintAttr::Interleave:
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
// Apply "llvm.mem.parallel_loop_access" metadata to load/stores.
setParallel(true);
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
setVectorizeEnable(true);
break;
case LoopHintAttr::Unroll:
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJam:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::UnrollCount:
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJamCount:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::VectorizeWidth:
case LoopHintAttr::InterleaveCount:
Add loop pragma for Loop Distribution Summary: This is similar to other loop pragmas like 'vectorize'. Currently it only has state values: distribute(enable) and distribute(disable). When one of these is specified the corresponding loop metadata is generated: !{!"llvm.loop.distribute.enable", i1 true/false} As a result, loop distribution will be attempted on the loop even if Loop Distribution in not enabled globally. Analogously, with 'disable' distribution can be turned off for an individual loop even when the pass is otherwise enabled. There are some slight differences compared to the existing loop pragmas. 1. There is no 'assume_safety' variant which makes its handling slightly different from 'vectorize'/'interleave'. 2. Unlike the existing loop pragmas, it does not have a corresponding numeric pragma like 'vectorize' -> 'vectorize_width'. So for the consistency checks in CheckForIncompatibleAttributes we don't need to check it against other pragmas. We just need to check for duplicates of the same pragma. Reviewers: rsmith, dexonsmith, aaron.ballman Subscribers: bob.wilson, cfe-commits, hfinkel Differential Revision: http://reviews.llvm.org/D19403 llvm-svn: 272656
2016-06-14 20:04:26 +08:00
case LoopHintAttr::Distribute:
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineDisabled:
case LoopHintAttr::PipelineInitiationInterval:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
llvm_unreachable("Options cannot be used to assume mem safety.");
break;
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
}
break;
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
case LoopHintAttr::Full:
switch (Option) {
case LoopHintAttr::Unroll:
setUnrollState(LoopAttributes::Full);
break;
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJam:
setUnrollAndJamState(LoopAttributes::Full);
break;
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
case LoopHintAttr::Vectorize:
case LoopHintAttr::Interleave:
case LoopHintAttr::UnrollCount:
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJamCount:
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
case LoopHintAttr::VectorizeWidth:
case LoopHintAttr::InterleaveCount:
Add loop pragma for Loop Distribution Summary: This is similar to other loop pragmas like 'vectorize'. Currently it only has state values: distribute(enable) and distribute(disable). When one of these is specified the corresponding loop metadata is generated: !{!"llvm.loop.distribute.enable", i1 true/false} As a result, loop distribution will be attempted on the loop even if Loop Distribution in not enabled globally. Analogously, with 'disable' distribution can be turned off for an individual loop even when the pass is otherwise enabled. There are some slight differences compared to the existing loop pragmas. 1. There is no 'assume_safety' variant which makes its handling slightly different from 'vectorize'/'interleave'. 2. Unlike the existing loop pragmas, it does not have a corresponding numeric pragma like 'vectorize' -> 'vectorize_width'. So for the consistency checks in CheckForIncompatibleAttributes we don't need to check it against other pragmas. We just need to check for duplicates of the same pragma. Reviewers: rsmith, dexonsmith, aaron.ballman Subscribers: bob.wilson, cfe-commits, hfinkel Differential Revision: http://reviews.llvm.org/D19403 llvm-svn: 272656
2016-06-14 20:04:26 +08:00
case LoopHintAttr::Distribute:
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineDisabled:
case LoopHintAttr::PipelineInitiationInterval:
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
llvm_unreachable("Options cannot be used with 'full' hint.");
break;
}
break;
case LoopHintAttr::Numeric:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
switch (Option) {
case LoopHintAttr::VectorizeWidth:
setVectorizeWidth(ValueInt);
break;
case LoopHintAttr::InterleaveCount:
setInterleaveCount(ValueInt);
break;
case LoopHintAttr::UnrollCount:
setUnrollCount(ValueInt);
break;
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJamCount:
setUnrollAndJamCount(ValueInt);
break;
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineInitiationInterval:
setPipelineInitiationInterval(ValueInt);
break;
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::Unroll:
[UnrollAndJam] Add unroll_and_jam pragma handling This adds support for the unroll_and_jam pragma, to go with the recently added unroll and jam pass. The name of the pragma is the same as is used in the Intel compiler, and most of the code works the same as for unroll. #pragma clang loop unroll_and_jam has been separated into a different patch. This part adds #pragma unroll_and_jam with an optional count, and #pragma no_unroll_and_jam to disable the transform. Differential Revision: https://reviews.llvm.org/D47267 llvm-svn: 338566
2018-08-01 22:36:12 +08:00
case LoopHintAttr::UnrollAndJam:
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
case LoopHintAttr::Vectorize:
case LoopHintAttr::Interleave:
Add loop pragma for Loop Distribution Summary: This is similar to other loop pragmas like 'vectorize'. Currently it only has state values: distribute(enable) and distribute(disable). When one of these is specified the corresponding loop metadata is generated: !{!"llvm.loop.distribute.enable", i1 true/false} As a result, loop distribution will be attempted on the loop even if Loop Distribution in not enabled globally. Analogously, with 'disable' distribution can be turned off for an individual loop even when the pass is otherwise enabled. There are some slight differences compared to the existing loop pragmas. 1. There is no 'assume_safety' variant which makes its handling slightly different from 'vectorize'/'interleave'. 2. Unlike the existing loop pragmas, it does not have a corresponding numeric pragma like 'vectorize' -> 'vectorize_width'. So for the consistency checks in CheckForIncompatibleAttributes we don't need to check it against other pragmas. We just need to check for duplicates of the same pragma. Reviewers: rsmith, dexonsmith, aaron.ballman Subscribers: bob.wilson, cfe-commits, hfinkel Differential Revision: http://reviews.llvm.org/D19403 llvm-svn: 272656
2016-06-14 20:04:26 +08:00
case LoopHintAttr::Distribute:
Add two new pragmas for controlling software pipelining optimizations. This patch adds #pragma clang loop pipeline and #pragma clang loop pipeline_initiation_interval for debugging or reducing compile time purposes. It is possible to disable SWP for concrete loops to save compilation time or to find bugs by not doing SWP to certain loops. It is possible to set value of initiation interval to concrete number to save compilation time by not doing extra pipeliner passes or to check created schedule for specific initiation interval. Patch by Alexey Lapshin. llvm-svn: 350414
2019-01-05 01:20:00 +08:00
case LoopHintAttr::PipelineDisabled:
Add new llvm.loop.unroll.enable metadata for use with "#pragma unroll". This change adds the new unroll metadata "llvm.loop.unroll.enable" which directs the optimizer to unroll a loop fully if the trip count is known at compile time, and unroll partially if the trip count is not known at compile time. This differs from "llvm.loop.unroll.full" which explicitly does not unroll a loop if the trip count is not known at compile time With this change "#pragma unroll" generates "llvm.loop.unroll.enable" rather than "llvm.loop.unroll.full" metadata. This changes the semantics of "#pragma unroll" slightly to mean "unroll aggressively (fully or partially)" rather than "unroll fully or not at all". The motivating example for this change was some internal code with a loop marked with "#pragma unroll" which only sometimes had a compile-time trip count depending on template magic. When the trip count was a compile-time constant, everything works as expected and the loop is fully unrolled. However, when the trip count was not a compile-time constant the "#pragma unroll" explicitly disabled unrolling of the loop(!). Removing "#pragma unroll" caused the loop to be unrolled partially which was desirable from a performance perspective. llvm-svn: 244467
2015-08-11 01:29:39 +08:00
llvm_unreachable("Options cannot be assigned a value.");
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
break;
}
Add assume_safety option for pragma loop vectorize and interleave. Specifying #pragma clang loop vectorize(assume_safety) on a loop adds the mem.parallel_loop_access metadata to each load/store operation in the loop. This metadata tells loop access analysis (LAA) to skip memory dependency checking. llvm-svn: 239572
2015-06-12 07:23:17 +08:00
break;
}
}
Use CGLoopInfo to emit metadata for loop hint pragmas. When ‘#pragma clang loop vectorize(assume_safety)’ was specified on a loop other loop hints were lost. The problem is that CGLoopInfo attaches metadata differently than EmitCondBrHints in CGStmt. For do-loops CGLoopInfo attaches metadata to the br in the body block and for while and for loops, the inc block. EmitCondBrHints on the other hand always attaches data to the br in the cond block. When specifying assume_safety CGLoopInfo emits an empty llvm.loop metadata shadowing the metadata in the cond block. Loop transformations like rotate and unswitch would then eliminate the cond block and its non-empty metadata. This patch unifies both approaches for adding metadata and modifies the existing safety tests to include non-assume_safety loop hints. llvm-svn: 243315
2015-07-28 04:10:20 +08:00
/// Stage the attributes.
Add the loop end location to the loop metadata. This additional information can be used to improve the locations when generating remarks for loops. Depends on the companion LLVM change r286227. Patch by Florian Hahn. Differential Revision: https://reviews.llvm.org/D25764 llvm-svn: 286456
2016-11-10 22:44:30 +08:00
push(Header, StartLoc, EndLoc);
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
}
void LoopInfoStack::pop() {
assert(!Active.empty() && "No active loops to pop");
Active.pop_back();
}
void LoopInfoStack::InsertHelper(Instruction *I) const {
[CodeGen] Generate llvm.loop.parallel_accesses instead of llvm.mem.parallel_loop_access metadata. Instead of generating llvm.mem.parallel_loop_access metadata, generate llvm.access.group on instructions and llvm.loop.parallel_accesses on loops. There is one access group per generated loop. This is clang part of D52116/r349725. Differential Revision: https://reviews.llvm.org/D52117 llvm-svn: 349823
2018-12-21 05:24:54 +08:00
if (I->mayReadOrWriteMemory()) {
SmallVector<Metadata *, 4> AccessGroups;
for (const LoopInfo &AL : Active) {
// Here we assume that every loop that has an access group is parallel.
if (MDNode *Group = AL.getAccessGroup())
AccessGroups.push_back(Group);
}
MDNode *UnionMD = nullptr;
if (AccessGroups.size() == 1)
UnionMD = cast<MDNode>(AccessGroups[0]);
else if (AccessGroups.size() >= 2)
UnionMD = MDNode::get(I->getContext(), AccessGroups);
I->setMetadata("llvm.access.group", UnionMD);
}
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
if (!hasInfo())
return;
const LoopInfo &L = getInfo();
if (!L.getLoopID())
return;
[TI removal] Test predicate rather than casting to detect a terminator and use the range based successor API. llvm-svn: 344730
2018-10-18 16:16:20 +08:00
if (I->isTerminator()) {
for (BasicBlock *Succ : successors(I))
if (Succ == L.getHeader()) {
I->setMetadata(llvm::LLVMContext::MD_loop, L.getLoopID());
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
break;
}
return;
}
}