maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp

4398 lines
182 KiB
C++
Raw Normal View History

[OPENMP] Initial codegen for '#pragma omp parallel' llvm-svn: 208077
2014-05-06 18:08:46 +08:00
//===--- CGStmtOpenMP.cpp - Emit LLVM Code from Statements ----------------===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This contains code to emit OpenMP nodes as LLVM code.
//
//===----------------------------------------------------------------------===//
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
#include "CGCleanup.h"
[OPENMP] Initial codegen for '#pragma omp parallel' llvm-svn: 208077
2014-05-06 18:08:46 +08:00
#include "CGOpenMPRuntime.h"
#include "CodeGenFunction.h"
#include "CodeGenModule.h"
[cleanup] Re-sort *all* #include lines with llvm/utils/sort_includes.py Sorry for the noise, I managed to miss a bunch of recent regressions of include orderings here. This should actually sort all the includes for Clang. Again, no functionality changed, this is just a mechanical cleanup that I try to run periodically to keep the #include lines as regular as possible across the project. llvm-svn: 225979
2015-01-14 19:29:14 +08:00
#include "TargetInfo.h"
[OPENMP] Initial codegen for '#pragma omp parallel' llvm-svn: 208077
2014-05-06 18:08:46 +08:00
#include "clang/AST/Stmt.h"
#include "clang/AST/StmtOpenMP.h"
[OPENMP 4.5] Initial support for data members in 'linear' clause. OpenMP 4.5 allows to privatize data members of current class in member functions. Patch adds initial support for privatization of data members in 'linear' clause, no codegen support. llvm-svn: 262578
2016-03-03 11:52:24 +08:00
#include "clang/AST/DeclOpenMP.h"
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
#include "llvm/IR/CallSite.h"
[OPENMP] Initial codegen for '#pragma omp parallel' llvm-svn: 208077
2014-05-06 18:08:46 +08:00
using namespace clang;
using namespace CodeGen;
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
namespace {
/// Lexical scope for OpenMP executable constructs, that handles correct codegen
/// for captured expressions.
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
class OMPLexicalScope : public CodeGenFunction::LexicalScope {
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
void emitPreInitStmt(CodeGenFunction &CGF, const OMPExecutableDirective &S) {
for (const auto *C : S.clauses()) {
if (auto *CPI = OMPClauseWithPreInit::get(C)) {
if (auto *PreInit = cast_or_null<DeclStmt>(CPI->getPreInitStmt())) {
[OPENMP 4.5] Initial support for data members in 'linear' clause. OpenMP 4.5 allows to privatize data members of current class in member functions. Patch adds initial support for privatization of data members in 'linear' clause, no codegen support. llvm-svn: 262578
2016-03-03 11:52:24 +08:00
for (const auto *I : PreInit->decls()) {
if (!I->hasAttr<OMPCaptureNoInitAttr>())
CGF.EmitVarDecl(cast<VarDecl>(*I));
else {
CodeGenFunction::AutoVarEmission Emission =
CGF.EmitAutoVarAlloca(cast<VarDecl>(*I));
CGF.EmitAutoVarCleanups(Emission);
}
}
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
}
}
}
}
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
CodeGenFunction::OMPPrivateScope InlinedShareds;
static bool isCapturedVar(CodeGenFunction &CGF, const VarDecl *VD) {
return CGF.LambdaCaptureFields.lookup(VD) ||
(CGF.CapturedStmtInfo && CGF.CapturedStmtInfo->lookup(VD)) ||
(CGF.CurCodeDecl && isa<BlockDecl>(CGF.CurCodeDecl));
}
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
public:
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope(CodeGenFunction &CGF, const OMPExecutableDirective &S,
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
bool AsInlined = false, bool EmitPreInitStmt = true)
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
: CodeGenFunction::LexicalScope(CGF, S.getSourceRange()),
InlinedShareds(CGF) {
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
if (EmitPreInitStmt)
emitPreInitStmt(CGF, S);
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
if (AsInlined) {
if (S.hasAssociatedStmt()) {
auto *CS = cast<CapturedStmt>(S.getAssociatedStmt());
for (auto &C : CS->captures()) {
if (C.capturesVariable() || C.capturesVariableByCopy()) {
auto *VD = C.getCapturedVar();
DeclRefExpr DRE(const_cast<VarDecl *>(VD),
isCapturedVar(CGF, VD) ||
(CGF.CapturedStmtInfo &&
InlinedShareds.isGlobalVarCaptured(VD)),
VD->getType().getNonReferenceType(), VK_LValue,
SourceLocation());
InlinedShareds.addPrivate(VD, [&CGF, &DRE]() -> Address {
return CGF.EmitLValue(&DRE).getAddress();
});
}
}
(void)InlinedShareds.Privatize();
}
}
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
}
};
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
/// Lexical scope for OpenMP parallel construct, that handles correct codegen
/// for captured expressions.
class OMPParallelScope final : public OMPLexicalScope {
bool EmitPreInitStmt(const OMPExecutableDirective &S) {
OpenMPDirectiveKind Kind = S.getDirectiveKind();
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
return !(isOpenMPTargetExecutionDirective(Kind) ||
isOpenMPLoopBoundSharingDirective(Kind)) &&
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
isOpenMPParallelDirective(Kind);
}
public:
OMPParallelScope(CodeGenFunction &CGF, const OMPExecutableDirective &S)
: OMPLexicalScope(CGF, S,
/*AsInlined=*/false,
/*EmitPreInitStmt=*/EmitPreInitStmt(S)) {}
};
[OpenMP] Codegen support for 'target teams' on the host. This patch adds support for codegen of 'target teams' on the host. This combined directive has two captured statements, one for the 'teams' region, and the other for the 'parallel'. This target teams region is offloaded using the __tgt_target_teams() call. The patch sets the number of teams as an argument to this call. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29084 llvm-svn: 293005
2017-01-25 10:18:43 +08:00
/// Lexical scope for OpenMP teams construct, that handles correct codegen
/// for captured expressions.
class OMPTeamsScope final : public OMPLexicalScope {
bool EmitPreInitStmt(const OMPExecutableDirective &S) {
OpenMPDirectiveKind Kind = S.getDirectiveKind();
return !isOpenMPTargetExecutionDirective(Kind) &&
isOpenMPTeamsDirective(Kind);
}
public:
OMPTeamsScope(CodeGenFunction &CGF, const OMPExecutableDirective &S)
: OMPLexicalScope(CGF, S,
/*AsInlined=*/false,
/*EmitPreInitStmt=*/EmitPreInitStmt(S)) {}
};
[OPENMP] Remove extra code transformation. For better support of some specific GNU extensions some extra transformation of AST nodes were introduced. These transformations are very hard to handle. The code is improved in handling of these extensions by using captured expressions construct. llvm-svn: 264709
2016-03-29 16:58:54 +08:00
/// Private scope for OpenMP loop-based directives, that supports capturing
/// of used expression from loop statement.
class OMPLoopScope : public CodeGenFunction::RunCleanupsScope {
void emitPreInitStmt(CodeGenFunction &CGF, const OMPLoopDirective &S) {
if (auto *LD = dyn_cast<OMPLoopDirective>(&S)) {
if (auto *PreInits = cast_or_null<DeclStmt>(LD->getPreInits())) {
for (const auto *I : PreInits->decls())
CGF.EmitVarDecl(cast<VarDecl>(*I));
}
}
}
public:
OMPLoopScope(CodeGenFunction &CGF, const OMPLoopDirective &S)
: CodeGenFunction::RunCleanupsScope(CGF) {
emitPreInitStmt(CGF, S);
}
};
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
} // namespace
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
llvm::Value *CodeGenFunction::getTypeSize(QualType Ty) {
auto &C = getContext();
llvm::Value *Size = nullptr;
auto SizeInChars = C.getTypeSizeInChars(Ty);
if (SizeInChars.isZero()) {
// getTypeSizeInChars() returns 0 for a VLA.
while (auto *VAT = C.getAsVariableArrayType(Ty)) {
llvm::Value *ArraySize;
std::tie(ArraySize, Ty) = getVLASize(VAT);
Size = Size ? Builder.CreateNUWMul(Size, ArraySize) : ArraySize;
}
SizeInChars = C.getTypeSizeInChars(Ty);
if (SizeInChars.isZero())
return llvm::ConstantInt::get(SizeTy, /*V=*/0);
Size = Builder.CreateNUWMul(Size, CGM.getSize(SizeInChars));
} else
Size = CGM.getSize(SizeInChars);
return Size;
}
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
void CodeGenFunction::GenerateOpenMPCapturedVars(
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
const CapturedStmt &S, SmallVectorImpl<llvm::Value *> &CapturedVars) {
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
const RecordDecl *RD = S.getCapturedRecordDecl();
auto CurField = RD->field_begin();
auto CurCap = S.captures().begin();
for (CapturedStmt::const_capture_init_iterator I = S.capture_init_begin(),
E = S.capture_init_end();
I != E; ++I, ++CurField, ++CurCap) {
if (CurField->hasCapturedVLAType()) {
auto VAT = CurField->getCapturedVLAType();
[OpenMP] Target directive host codegen. This patch implements the outlining for offloading functions for code annotated with the OpenMP target directive. It uses a temporary naming of the outlined functions that will have to be updated later on once target side codegen and registration of offloading libraries is implemented - the naming needs to be made unique in the produced library. llvm-svn: 249148
2015-10-03 00:14:20 +08:00
auto *Val = VLASizeMap[VAT->getSizeExpr()];
CapturedVars.push_back(Val);
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
} else if (CurCap->capturesThis())
CapturedVars.push_back(CXXThisValue);
Re-apply r272900 - [OpenMP] Cast captures by copy when passed to fork call so that they are compatible to what the runtime library expects. An issue in one of the regression tests was fixed for 32-bit hosts. llvm-svn: 272931
2016-06-17 02:39:34 +08:00
else if (CurCap->capturesVariableByCopy()) {
llvm::Value *CV =
EmitLoadOfLValue(EmitLValue(*I), SourceLocation()).getScalarVal();
// If the field is not a pointer, we need to save the actual value
// and load it as a void pointer.
if (!CurField->getType()->isAnyPointerType()) {
auto &Ctx = getContext();
auto DstAddr = CreateMemTemp(
Ctx.getUIntPtrType(),
Twine(CurCap->getCapturedVar()->getName()) + ".casted");
LValue DstLV = MakeAddrLValue(DstAddr, Ctx.getUIntPtrType());
auto *SrcAddrVal = EmitScalarConversion(
DstAddr.getPointer(), Ctx.getPointerType(Ctx.getUIntPtrType()),
Ctx.getPointerType(CurField->getType()), SourceLocation());
LValue SrcLV =
MakeNaturalAlignAddrLValue(SrcAddrVal, CurField->getType());
// Store the value using the source type pointer.
EmitStoreThroughLValue(RValue::get(CV), SrcLV);
// Load the value using the destination type pointer.
CV = EmitLoadOfLValue(DstLV, SourceLocation()).getScalarVal();
}
CapturedVars.push_back(CV);
} else {
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
assert(CurCap->capturesVariable() && "Expected capture by reference.");
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
CapturedVars.push_back(EmitLValue(*I).getAddress().getPointer());
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
}
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
}
}
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
static Address castValueFromUintptr(CodeGenFunction &CGF, QualType DstType,
StringRef Name, LValue AddrLV,
bool isReferenceType = false) {
ASTContext &Ctx = CGF.getContext();
auto *CastedPtr = CGF.EmitScalarConversion(
AddrLV.getAddress().getPointer(), Ctx.getUIntPtrType(),
Ctx.getPointerType(DstType), SourceLocation());
auto TmpAddr =
CGF.MakeNaturalAlignAddrLValue(CastedPtr, Ctx.getPointerType(DstType))
.getAddress();
// If we are dealing with references we need to return the address of the
// reference instead of the reference of the value.
if (isReferenceType) {
QualType RefType = Ctx.getLValueReferenceType(DstType);
auto *RefVal = TmpAddr.getPointer();
TmpAddr = CGF.CreateMemTemp(RefType, Twine(Name) + ".ref");
auto TmpLVal = CGF.MakeAddrLValue(TmpAddr, RefType);
[CodeGen][ObjC] Do not call objc_storeStrong when initializing a constexpr variable. When compiling a constexpr NSString initialized with an objective-c string literal, CodeGen emits objc_storeStrong on an uninitialized alloca, which causes a crash. This patch folds the code in EmitScalarInit into EmitStoreThroughLValue and fixes the crash by calling objc_retain on the string instead of using objc_storeStrong. rdar://problem/28562009 Differential Revision: https://reviews.llvm.org/D25547 llvm-svn: 284516
2016-10-19 03:05:41 +08:00
CGF.EmitStoreThroughLValue(RValue::get(RefVal), TmpLVal, /*isInit*/ true);
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
}
return TmpAddr;
}
[OPENMP] Fix for PR32333: Crash in call of outlined Function. If the type of the captured variable is a pointer(s) to variably modified type, this type was not processed correctly. Need to drill into the type, find the innermost variably modified array type and convert it to canonical parameter type. llvm-svn: 299868
2017-04-11 03:16:45 +08:00
static QualType getCanonicalParamType(ASTContext &C, QualType T) {
if (T->isLValueReferenceType()) {
return C.getLValueReferenceType(
getCanonicalParamType(C, T.getNonReferenceType()),
/*SpelledAsLValue=*/false);
}
if (T->isPointerType())
return C.getPointerType(getCanonicalParamType(C, T->getPointeeType()));
return C.getCanonicalParamType(T);
}
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
namespace {
/// Contains required data for proper outlined function codegen.
struct FunctionOptions {
/// Captured statement for which the function is generated.
const CapturedStmt *S = nullptr;
/// true if cast to/from UIntPtr is required for variables captured by
/// value.
bool UIntPtrCastRequired = true;
/// true if only casted argumefnts must be registered as local args or VLA
/// sizes.
bool RegisterCastedArgsOnly = false;
/// Name of the generated function.
StringRef FunctionName;
explicit FunctionOptions(const CapturedStmt *S, bool UIntPtrCastRequired,
bool RegisterCastedArgsOnly,
StringRef FunctionName)
: S(S), UIntPtrCastRequired(UIntPtrCastRequired),
RegisterCastedArgsOnly(UIntPtrCastRequired && RegisterCastedArgsOnly),
FunctionName(FunctionName) {}
};
}
static std::pair<llvm::Function *, bool> emitOutlinedFunctionPrologue(
CodeGenFunction &CGF, FunctionArgList &Args,
llvm::DenseMap<const Decl *, std::pair<const VarDecl *, Address>>
&LocalAddrs,
llvm::DenseMap<const Decl *, std::pair<const Expr *, llvm::Value *>>
&VLASizes,
llvm::Value *&CXXThisValue, const FunctionOptions &FO) {
const CapturedDecl *CD = FO.S->getCapturedDecl();
const RecordDecl *RD = FO.S->getCapturedRecordDecl();
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
assert(CD->hasBody() && "missing CapturedDecl body");
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
CXXThisValue = nullptr;
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// Build the argument list.
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
CodeGenModule &CGM = CGF.CGM;
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
ASTContext &Ctx = CGM.getContext();
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
bool HasUIntPtrArgs = false;
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
Args.append(CD->param_begin(),
std::next(CD->param_begin(), CD->getContextParamPosition()));
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
auto I = FO.S->captures().begin();
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
for (auto *FD : RD->fields()) {
QualType ArgType = FD->getType();
IdentifierInfo *II = nullptr;
VarDecl *CapVar = nullptr;
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
// If this is a capture by copy and the type is not a pointer, the outlined
// function argument type should be uintptr and the value properly casted to
// uintptr. This is necessary given that the runtime library is only able to
// deal with pointers. We can pass in the same way the VLA type sizes to the
// outlined function.
Re-apply r272900 - [OpenMP] Cast captures by copy when passed to fork call so that they are compatible to what the runtime library expects. An issue in one of the regression tests was fixed for 32-bit hosts. llvm-svn: 272931
2016-06-17 02:39:34 +08:00
if ((I->capturesVariableByCopy() && !ArgType->isAnyPointerType()) ||
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
I->capturesVariableArrayType()) {
HasUIntPtrArgs = true;
if (FO.UIntPtrCastRequired)
ArgType = Ctx.getUIntPtrType();
}
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
if (I->capturesVariable() || I->capturesVariableByCopy()) {
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
CapVar = I->getCapturedVar();
II = CapVar->getIdentifier();
} else if (I->capturesThis())
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
II = &Ctx.Idents.get("this");
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
else {
assert(I->capturesVariableArrayType());
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
II = &Ctx.Idents.get("vla");
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
}
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
if (ArgType->isVariablyModifiedType())
ArgType = getCanonicalParamType(Ctx, ArgType.getNonReferenceType());
Args.push_back(ImplicitParamDecl::Create(Ctx, /*DC=*/nullptr,
[DebugInfo] Add kind of ImplicitParamDecl for emission of FlagObjectPointer. Summary: If the first parameter of the function is the ImplicitParamDecl, codegen automatically marks it as an implicit argument with `this` or `self` pointer. Added internal kind of the ImplicitParamDecl to separate 'this', 'self', 'vtt' and other implicit parameters from other kind of parameters. Reviewers: rjmccall, aaron.ballman Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D33735 llvm-svn: 305075
2017-06-09 21:40:18 +08:00
FD->getLocation(), II, ArgType,
ImplicitParamDecl::Other));
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
++I;
}
Args.append(
std::next(CD->param_begin(), CD->getContextParamPosition() + 1),
CD->param_end());
// Create the function declaration.
FunctionType::ExtInfo ExtInfo;
const CGFunctionInfo &FuncInfo =
Preserve ExtParameterInfos into CGFunctionInfo. As part of this, make the function-arrangement interfaces a little simpler and more semantic. NFC. llvm-svn: 263191
2016-03-11 12:30:31 +08:00
CGM.getTypes().arrangeBuiltinFunctionDeclaration(Ctx.VoidTy, Args);
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
llvm::FunctionType *FuncLLVMTy = CGM.getTypes().GetFunctionType(FuncInfo);
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
llvm::Function *F =
llvm::Function::Create(FuncLLVMTy, llvm::GlobalValue::InternalLinkage,
FO.FunctionName, &CGM.getModule());
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
CGM.SetInternalFunctionAttributes(CD, F, FuncInfo);
if (CD->isNothrow())
F->addFnAttr(llvm::Attribute::NoUnwind);
// Generate the function.
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
CGF.StartFunction(CD, Ctx.VoidTy, F, FuncInfo, Args, CD->getLocation(),
CD->getBody()->getLocStart());
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
unsigned Cnt = CD->getContextParamPosition();
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
I = FO.S->captures().begin();
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
for (auto *FD : RD->fields()) {
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
// If we are capturing a pointer by copy we don't need to do anything, just
// use the value that we get from the arguments.
if (I->capturesVariableByCopy() && FD->getType()->isAnyPointerType()) {
[OpenMP] Add support for mapping array sections through pointer references. Summary: This patch fixes a bug in the map of array sections whose base is a reference to a pointer. The existing mapping support was not prepared to deal with it, causing the compiler to crash. Mapping a reference to a pointer enjoys the same characteristics of a regular pointer, i.e., it is passed by value. Therefore, the reference has to be materialized in the target region. Reviewers: hfinkel, carlo.bertolli, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22690 llvm-svn: 276933
2016-07-28 06:49:49 +08:00
const VarDecl *CurVD = I->getCapturedVar();
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
Address LocalAddr = CGF.GetAddrOfLocalVar(Args[Cnt]);
[OpenMP] Add support for mapping array sections through pointer references. Summary: This patch fixes a bug in the map of array sections whose base is a reference to a pointer. The existing mapping support was not prepared to deal with it, causing the compiler to crash. Mapping a reference to a pointer enjoys the same characteristics of a regular pointer, i.e., it is passed by value. Therefore, the reference has to be materialized in the target region. Reviewers: hfinkel, carlo.bertolli, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22690 llvm-svn: 276933
2016-07-28 06:49:49 +08:00
// If the variable is a reference we need to materialize it here.
if (CurVD->getType()->isReferenceType()) {
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
Address RefAddr = CGF.CreateMemTemp(
CurVD->getType(), CGM.getPointerAlign(), ".materialized_ref");
CGF.EmitStoreOfScalar(LocalAddr.getPointer(), RefAddr,
/*Volatile=*/false, CurVD->getType());
[OpenMP] Add support for mapping array sections through pointer references. Summary: This patch fixes a bug in the map of array sections whose base is a reference to a pointer. The existing mapping support was not prepared to deal with it, causing the compiler to crash. Mapping a reference to a pointer enjoys the same characteristics of a regular pointer, i.e., it is passed by value. Therefore, the reference has to be materialized in the target region. Reviewers: hfinkel, carlo.bertolli, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22690 llvm-svn: 276933
2016-07-28 06:49:49 +08:00
LocalAddr = RefAddr;
}
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
if (!FO.RegisterCastedArgsOnly)
LocalAddrs.insert({Args[Cnt], {CurVD, LocalAddr}});
Remove use of builtin comma operator. Cleanup for upcoming Clang warning -Wcomma. No functionality change intended. llvm-svn: 261271
2016-02-19 06:34:54 +08:00
++Cnt;
++I;
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
continue;
}
[CodeGen] Propagate LValueBaseInfo instead of AlignmentSource The functions creating LValues propagated information about alignment source. Extend the propagated data to also include information about possible unrestricted aliasing. A new class LValueBaseInfo will contain both AlignmentSource and MayAlias info. This patch should not introduce any functional changes. Differential Revision: https://reviews.llvm.org/D33284 llvm-svn: 303358
2017-05-19 01:07:11 +08:00
LValueBaseInfo BaseInfo(AlignmentSource::Decl, false);
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
LValue ArgLVal = CGF.MakeAddrLValue(CGF.GetAddrOfLocalVar(Args[Cnt]),
Args[Cnt]->getType(), BaseInfo);
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
if (FD->hasCapturedVLAType()) {
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
if (FO.UIntPtrCastRequired) {
ArgLVal = CGF.MakeAddrLValue(castValueFromUintptr(CGF, FD->getType(),
Args[Cnt]->getName(),
ArgLVal),
FD->getType(), BaseInfo);
}
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
auto *ExprArg =
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
CGF.EmitLoadOfLValue(ArgLVal, SourceLocation()).getScalarVal();
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
auto VAT = FD->getCapturedVLAType();
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
VLASizes.insert({Args[Cnt], {VAT->getSizeExpr(), ExprArg}});
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
} else if (I->capturesVariable()) {
auto *Var = I->getCapturedVar();
QualType VarTy = Var->getType();
Address ArgAddr = ArgLVal.getAddress();
if (!VarTy->isReferenceType()) {
Fix for PR30639: CGDebugInfo Null dereference with OpenMP array access, by Erich Keane OpenMP creates a variable array type with a a null size-expr. The Debug generation failed to due to this. This patch corrects the openmp implementation, updates the tests, and adds a new one for this condition. Differential Revision: https://reviews.llvm.org/D25373 llvm-svn: 284110
2016-10-13 17:52:46 +08:00
if (ArgLVal.getType()->isLValueReferenceType()) {
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
ArgAddr = CGF.EmitLoadOfReference(
Fix for PR30639: CGDebugInfo Null dereference with OpenMP array access, by Erich Keane OpenMP creates a variable array type with a a null size-expr. The Debug generation failed to due to this. This patch corrects the openmp implementation, updates the tests, and adds a new one for this condition. Differential Revision: https://reviews.llvm.org/D25373 llvm-svn: 284110
2016-10-13 17:52:46 +08:00
ArgAddr, ArgLVal.getType()->castAs<ReferenceType>());
[OPENMP] Fixed capturing of VLA variables. After some changes in codegen capturing of VLA variables in OpenMP regions was broken, causing compiler crash. Patch fixes this issue. llvm-svn: 286103
2016-11-07 19:16:04 +08:00
} else if (!VarTy->isVariablyModifiedType() || !VarTy->isPointerType()) {
Fix for PR30639: CGDebugInfo Null dereference with OpenMP array access, by Erich Keane OpenMP creates a variable array type with a a null size-expr. The Debug generation failed to due to this. This patch corrects the openmp implementation, updates the tests, and adds a new one for this condition. Differential Revision: https://reviews.llvm.org/D25373 llvm-svn: 284110
2016-10-13 17:52:46 +08:00
assert(ArgLVal.getType()->isPointerType());
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
ArgAddr = CGF.EmitLoadOfPointer(
Fix for PR30639: CGDebugInfo Null dereference with OpenMP array access, by Erich Keane OpenMP creates a variable array type with a a null size-expr. The Debug generation failed to due to this. This patch corrects the openmp implementation, updates the tests, and adds a new one for this condition. Differential Revision: https://reviews.llvm.org/D25373 llvm-svn: 284110
2016-10-13 17:52:46 +08:00
ArgAddr, ArgLVal.getType()->castAs<PointerType>());
}
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
}
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
if (!FO.RegisterCastedArgsOnly) {
LocalAddrs.insert(
{Args[Cnt],
{Var, Address(ArgAddr.getPointer(), Ctx.getDeclAlign(Var))}});
}
[OpenMP] Update target directive codegen to use 4.5 implicit data mappings. Summary: This patch implements the 4.5 specification for the implicit data maps. OpenMP 4.5 specification changes the default way data is captured into a target region. All the non-aggregate kinds are passed by value by default. This required activating the capturing by value during SEMA for the target region. All the non-aggregate values that can be encoded in the size of a pointer are properly casted and forwarded to the runtime library. On top of fixing the previous weird behavior for mapping pointers in nested data regions (an explicit map was always required), this also improves performance as the number of allocations/transactions to the device per non-aggregate map are reduced from two to only one - instead of passing a reference and the value, only the value passed. Explicit maps will be added later on once firstprivate, private, and map clauses' SEMA and parsing are available. Reviewers: hfinkel, rjmccall, ABataev Subscribers: cfe-commits, carlo.bertolli Differential Revision: http://reviews.llvm.org/D14940 llvm-svn: 254521
2015-12-03 01:44:43 +08:00
} else if (I->capturesVariableByCopy()) {
assert(!FD->getType()->isAnyPointerType() &&
"Not expecting a captured pointer.");
auto *Var = I->getCapturedVar();
QualType VarTy = Var->getType();
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
LocalAddrs.insert(
{Args[Cnt],
{Var,
FO.UIntPtrCastRequired
? castValueFromUintptr(CGF, FD->getType(), Args[Cnt]->getName(),
ArgLVal, VarTy->isReferenceType())
: ArgLVal.getAddress()}});
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
} else {
// If 'this' is captured, load it into CXXThisValue.
assert(I->capturesThis());
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
CXXThisValue = CGF.EmitLoadOfLValue(ArgLVal, Args[Cnt]->getLocation())
.getScalarVal();
LocalAddrs.insert({Args[Cnt], {nullptr, ArgLVal.getAddress()}});
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
}
Remove use of builtin comma operator. Cleanup for upcoming Clang warning -Wcomma. No functionality change intended. llvm-svn: 261271
2016-02-19 06:34:54 +08:00
++Cnt;
++I;
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
}
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
return {F, HasUIntPtrArgs};
}
llvm::Function *
CodeGenFunction::GenerateOpenMPCapturedStmtFunction(const CapturedStmt &S) {
assert(
CapturedStmtInfo &&
"CapturedStmtInfo should be set when generating the captured function");
const CapturedDecl *CD = S.getCapturedDecl();
// Build the argument list.
bool NeedWrapperFunction =
getDebugInfo() &&
CGM.getCodeGenOpts().getDebugInfo() >= codegenoptions::LimitedDebugInfo;
FunctionArgList Args;
llvm::DenseMap<const Decl *, std::pair<const VarDecl *, Address>> LocalAddrs;
llvm::DenseMap<const Decl *, std::pair<const Expr *, llvm::Value *>> VLASizes;
FunctionOptions FO(&S, !NeedWrapperFunction, /*RegisterCastedArgsOnly=*/false,
CapturedStmtInfo->getHelperName());
llvm::Function *F;
bool HasUIntPtrArgs;
std::tie(F, HasUIntPtrArgs) = emitOutlinedFunctionPrologue(
*this, Args, LocalAddrs, VLASizes, CXXThisValue, FO);
for (const auto &LocalAddrPair : LocalAddrs) {
if (LocalAddrPair.second.first) {
setAddrOfLocalVar(LocalAddrPair.second.first,
LocalAddrPair.second.second);
}
}
for (const auto &VLASizePair : VLASizes)
VLASizeMap[VLASizePair.second.first] = VLASizePair.second.second;
[PGO] Instrument only base constructors and destructors. Constructors and destructors may be represented by several functions in IR. Only base structors correspond to source code, others are small pieces of code and eventually call the base variant. In this case instrumentation of non-base structors has little sense, this fix remove it. Now profile data of a declaration corresponds to exactly one function in IR, it agrees with the current logic of the profile data loading. This change fixes PR24996. Differential Revision: http://reviews.llvm.org/D15158 llvm-svn: 254876
2015-12-06 22:32:39 +08:00
PGO.assignRegionCounters(GlobalDecl(CD), F);
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
CapturedStmtInfo->EmitBody(*this, CD->getBody());
FinishFunction(CD->getBodyRBrace());
[OPENMP][DEBUG] Generate second function with correct arg types. Currently, if the some of the parameters are captured by value, this argument is converted to uintptr_t type and thus we loosing the debug info about real type of the argument (captured variable): ``` void @.outlined_function.(uintptr %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.outlined_function.(uintptr %a.casted.val) ... ``` To resolve this problem, in debug mode a speciall external wrapper function is generated, that calls the outlined function with the correct parameters types: ``` void @.wrapper.(uintptr %par) { %a = alloca i32 %cast = bitcast i32* %a to uintptr* store uintptr %par, uintptr *%cast %a.val = load i32, i32* %a call void @.outlined_function.(i32 %a) ret void } void @.outlined_function.(i32 %par); ... %a = alloca i32 %a.casted = alloca uintptr %cast = bitcast uintptr* %a.casted to i32* %a.val = load i32, i32 *%a store i32 %a.val, i32 *%cast %a.casted.val = load uintptr, uintptr* %a.casted call void @.wrapper.(uintptr %a.casted.val) ... ``` llvm-svn: 306697
2017-06-30 00:43:05 +08:00
if (!NeedWrapperFunction || !HasUIntPtrArgs)
return F;
FunctionOptions WrapperFO(&S, /*UIntPtrCastRequired=*/true,
/*RegisterCastedArgsOnly=*/true,
".nondebug_wrapper.");
CodeGenFunction WrapperCGF(CGM, /*suppressNewContext=*/true);
WrapperCGF.disableDebugInfo();
Args.clear();
LocalAddrs.clear();
VLASizes.clear();
llvm::Function *WrapperF =
emitOutlinedFunctionPrologue(WrapperCGF, Args, LocalAddrs, VLASizes,
WrapperCGF.CXXThisValue, WrapperFO).first;
LValueBaseInfo BaseInfo(AlignmentSource::Decl, false);
llvm::SmallVector<llvm::Value *, 4> CallArgs;
for (const auto *Arg : Args) {
llvm::Value *CallArg;
auto I = LocalAddrs.find(Arg);
if (I != LocalAddrs.end()) {
LValue LV =
WrapperCGF.MakeAddrLValue(I->second.second, Arg->getType(), BaseInfo);
CallArg = WrapperCGF.EmitLoadOfScalar(LV, SourceLocation());
} else {
auto EI = VLASizes.find(Arg);
if (EI != VLASizes.end())
CallArg = EI->second.second;
else {
LValue LV = WrapperCGF.MakeAddrLValue(WrapperCGF.GetAddrOfLocalVar(Arg),
Arg->getType(), BaseInfo);
CallArg = WrapperCGF.EmitLoadOfScalar(LV, SourceLocation());
}
}
CallArgs.emplace_back(CallArg);
}
WrapperCGF.Builder.CreateCall(F, CallArgs);
WrapperCGF.FinishFunction();
return WrapperF;
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
}
[OPENMP] Initial codegen for '#pragma omp parallel' llvm-svn: 208077
2014-05-06 18:08:46 +08:00
//===----------------------------------------------------------------------===//
// OpenMP Directive Emission
//===----------------------------------------------------------------------===//
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
void CodeGenFunction::EmitOMPAggregateAssign(
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
Address DestAddr, Address SrcAddr, QualType OriginalType,
const llvm::function_ref<void(Address, Address)> &CopyGen) {
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
// Perform element-by-element initialization.
QualType ElementTy;
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// Drill down to the base element type on both arrays.
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
auto ArrayTy = OriginalType->getAsArrayTypeUnsafe();
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
auto NumElements = emitArrayLength(ArrayTy, ElementTy, DestAddr);
SrcAddr = Builder.CreateElementBitCast(SrcAddr, DestAddr.getElementType());
auto SrcBegin = SrcAddr.getPointer();
auto DestBegin = DestAddr.getPointer();
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
// Cast from pointer to array type to pointer to single element.
auto DestEnd = Builder.CreateGEP(DestBegin, NumElements);
// The basic structure here is a while-do loop.
auto BodyBB = createBasicBlock("omp.arraycpy.body");
auto DoneBB = createBasicBlock("omp.arraycpy.done");
auto IsEmpty =
Builder.CreateICmpEQ(DestBegin, DestEnd, "omp.arraycpy.isempty");
Builder.CreateCondBr(IsEmpty, DoneBB, BodyBB);
// Enter the loop body, making that address the current address.
auto EntryBB = Builder.GetInsertBlock();
EmitBlock(BodyBB);
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
CharUnits ElementSize = getContext().getTypeSizeInChars(ElementTy);
llvm::PHINode *SrcElementPHI =
Builder.CreatePHI(SrcBegin->getType(), 2, "omp.arraycpy.srcElementPast");
SrcElementPHI->addIncoming(SrcBegin, EntryBB);
Address SrcElementCurrent =
Address(SrcElementPHI,
SrcAddr.getAlignment().alignmentOfArrayElement(ElementSize));
llvm::PHINode *DestElementPHI =
Builder.CreatePHI(DestBegin->getType(), 2, "omp.arraycpy.destElementPast");
DestElementPHI->addIncoming(DestBegin, EntryBB);
Address DestElementCurrent =
Address(DestElementPHI,
DestAddr.getAlignment().alignmentOfArrayElement(ElementSize));
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
// Emit copy.
CopyGen(DestElementCurrent, SrcElementCurrent);
// Shift the address forward by one element.
auto DestElementNext = Builder.CreateConstGEP1_32(
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
DestElementPHI, /*Idx0=*/1, "omp.arraycpy.dest.element");
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
auto SrcElementNext = Builder.CreateConstGEP1_32(
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
SrcElementPHI, /*Idx0=*/1, "omp.arraycpy.src.element");
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
// Check whether we've reached the end.
auto Done =
Builder.CreateICmpEQ(DestElementNext, DestEnd, "omp.arraycpy.done");
Builder.CreateCondBr(Done, DoneBB, BodyBB);
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
DestElementPHI->addIncoming(DestElementNext, Builder.GetInsertBlock());
SrcElementPHI->addIncoming(SrcElementNext, Builder.GetInsertBlock());
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
// Done.
EmitBlock(DoneBB, /*IsFinished=*/true);
}
[OPENMP] Codegen for 'firstprivate' clause. This patch generates some helper variables that used as private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by copy using values of the original variables (with the copy constructor, if any). For arrays, initializator is generated for single element and in the codegen procedure this initial value is automatically propagated between all elements of the private copy. In outlined function, references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables an implicit barier is generated by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D5140 llvm-svn: 219306
2014-10-08 22:01:46 +08:00
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
/// Check if the combiner is a call to UDR combiner and if it is so return the
/// UDR decl used for reduction.
static const OMPDeclareReductionDecl *
getReductionInit(const Expr *ReductionOp) {
if (auto *CE = dyn_cast<CallExpr>(ReductionOp))
if (auto *OVE = dyn_cast<OpaqueValueExpr>(CE->getCallee()))
if (auto *DRE =
dyn_cast<DeclRefExpr>(OVE->getSourceExpr()->IgnoreImpCasts()))
if (auto *DRD = dyn_cast<OMPDeclareReductionDecl>(DRE->getDecl()))
return DRD;
return nullptr;
}
static void emitInitWithReductionInitializer(CodeGenFunction &CGF,
const OMPDeclareReductionDecl *DRD,
const Expr *InitOp,
Address Private, Address Original,
QualType Ty) {
if (DRD->getInitializer()) {
std::pair<llvm::Function *, llvm::Function *> Reduction =
CGF.CGM.getOpenMPRuntime().getUserDefinedReduction(DRD);
auto *CE = cast<CallExpr>(InitOp);
auto *OVE = cast<OpaqueValueExpr>(CE->getCallee());
const Expr *LHS = CE->getArg(/*Arg=*/0)->IgnoreParenImpCasts();
const Expr *RHS = CE->getArg(/*Arg=*/1)->IgnoreParenImpCasts();
auto *LHSDRE = cast<DeclRefExpr>(cast<UnaryOperator>(LHS)->getSubExpr());
auto *RHSDRE = cast<DeclRefExpr>(cast<UnaryOperator>(RHS)->getSubExpr());
CodeGenFunction::OMPPrivateScope PrivateScope(CGF);
PrivateScope.addPrivate(cast<VarDecl>(LHSDRE->getDecl()),
[=]() -> Address { return Private; });
PrivateScope.addPrivate(cast<VarDecl>(RHSDRE->getDecl()),
[=]() -> Address { return Original; });
(void)PrivateScope.Privatize();
RValue Func = RValue::get(Reduction.second);
CodeGenFunction::OpaqueValueMapping Map(CGF, OVE, Func);
CGF.EmitIgnoredExpr(InitOp);
} else {
llvm::Constant *Init = CGF.CGM.EmitNullConstant(Ty);
auto *GV = new llvm::GlobalVariable(
CGF.CGM.getModule(), Init->getType(), /*isConstant=*/true,
llvm::GlobalValue::PrivateLinkage, Init, ".init");
LValue LV = CGF.MakeNaturalAlignAddrLValue(GV, Ty);
RValue InitRVal;
switch (CGF.getEvaluationKind(Ty)) {
case TEK_Scalar:
InitRVal = CGF.EmitLoadOfLValue(LV, SourceLocation());
break;
case TEK_Complex:
InitRVal =
RValue::getComplex(CGF.EmitLoadOfComplex(LV, SourceLocation()));
break;
case TEK_Aggregate:
InitRVal = RValue::getAggregate(LV.getAddress());
break;
}
OpaqueValueExpr OVE(SourceLocation(), Ty, VK_RValue);
CodeGenFunction::OpaqueValueMapping OpaqueMap(CGF, &OVE, InitRVal);
CGF.EmitAnyExprToMem(&OVE, Private, Ty.getQualifiers(),
/*IsInitializer=*/false);
}
}
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
/// \brief Emit initialization of arrays of complex types.
/// \param DestAddr Address of the array.
/// \param Type Type of array.
/// \param Init Initial expression of array.
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
/// \param SrcAddr Address of the original array.
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
static void EmitOMPAggregateInit(CodeGenFunction &CGF, Address DestAddr,
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
QualType Type, const Expr *Init,
Address SrcAddr = Address::invalid()) {
auto *DRD = getReductionInit(Init);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
// Perform element-by-element initialization.
QualType ElementTy;
// Drill down to the base element type on both arrays.
auto ArrayTy = Type->getAsArrayTypeUnsafe();
auto NumElements = CGF.emitArrayLength(ArrayTy, ElementTy, DestAddr);
DestAddr =
CGF.Builder.CreateElementBitCast(DestAddr, DestAddr.getElementType());
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
if (DRD)
SrcAddr =
CGF.Builder.CreateElementBitCast(SrcAddr, DestAddr.getElementType());
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
llvm::Value *SrcBegin = nullptr;
if (DRD)
SrcBegin = SrcAddr.getPointer();
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
auto DestBegin = DestAddr.getPointer();
// Cast from pointer to array type to pointer to single element.
auto DestEnd = CGF.Builder.CreateGEP(DestBegin, NumElements);
// The basic structure here is a while-do loop.
auto BodyBB = CGF.createBasicBlock("omp.arrayinit.body");
auto DoneBB = CGF.createBasicBlock("omp.arrayinit.done");
auto IsEmpty =
CGF.Builder.CreateICmpEQ(DestBegin, DestEnd, "omp.arrayinit.isempty");
CGF.Builder.CreateCondBr(IsEmpty, DoneBB, BodyBB);
// Enter the loop body, making that address the current address.
auto EntryBB = CGF.Builder.GetInsertBlock();
CGF.EmitBlock(BodyBB);
CharUnits ElementSize = CGF.getContext().getTypeSizeInChars(ElementTy);
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
llvm::PHINode *SrcElementPHI = nullptr;
Address SrcElementCurrent = Address::invalid();
if (DRD) {
SrcElementPHI = CGF.Builder.CreatePHI(SrcBegin->getType(), 2,
"omp.arraycpy.srcElementPast");
SrcElementPHI->addIncoming(SrcBegin, EntryBB);
SrcElementCurrent =
Address(SrcElementPHI,
SrcAddr.getAlignment().alignmentOfArrayElement(ElementSize));
}
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
llvm::PHINode *DestElementPHI = CGF.Builder.CreatePHI(
DestBegin->getType(), 2, "omp.arraycpy.destElementPast");
DestElementPHI->addIncoming(DestBegin, EntryBB);
Address DestElementCurrent =
Address(DestElementPHI,
DestAddr.getAlignment().alignmentOfArrayElement(ElementSize));
// Emit copy.
{
CodeGenFunction::RunCleanupsScope InitScope(CGF);
[OPENMP] Fix crash on initialization of classes with no init clause in declare reductions. If reduction clause is applied to instance of class with user-defined reduction operation without initialization clause, it may cause a crash. Patch fixes this issue. llvm-svn: 267695
2016-04-27 19:38:05 +08:00
if (DRD && (DRD->getInitializer() || !Init)) {
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
emitInitWithReductionInitializer(CGF, DRD, Init, DestElementCurrent,
SrcElementCurrent, ElementTy);
} else
CGF.EmitAnyExprToMem(Init, DestElementCurrent, ElementTy.getQualifiers(),
/*IsInitializer=*/false);
}
if (DRD) {
// Shift the address forward by one element.
auto SrcElementNext = CGF.Builder.CreateConstGEP1_32(
SrcElementPHI, /*Idx0=*/1, "omp.arraycpy.dest.element");
SrcElementPHI->addIncoming(SrcElementNext, CGF.Builder.GetInsertBlock());
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
}
// Shift the address forward by one element.
auto DestElementNext = CGF.Builder.CreateConstGEP1_32(
DestElementPHI, /*Idx0=*/1, "omp.arraycpy.dest.element");
// Check whether we've reached the end.
auto Done =
CGF.Builder.CreateICmpEQ(DestElementNext, DestEnd, "omp.arraycpy.done");
CGF.Builder.CreateCondBr(Done, DoneBB, BodyBB);
DestElementPHI->addIncoming(DestElementNext, CGF.Builder.GetInsertBlock());
// Done.
CGF.EmitBlock(DoneBB, /*IsFinished=*/true);
}
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
void CodeGenFunction::EmitOMPCopy(QualType OriginalType, Address DestAddr,
Address SrcAddr, const VarDecl *DestVD,
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
const VarDecl *SrcVD, const Expr *Copy) {
if (OriginalType->isArrayType()) {
auto *BO = dyn_cast<BinaryOperator>(Copy);
if (BO && BO->getOpcode() == BO_Assign) {
// Perform simple memcpy for simple copying.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
EmitAggregateAssign(DestAddr, SrcAddr, OriginalType);
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
} else {
// For arrays with complex element types perform element by element
// copying.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
EmitOMPAggregateAssign(
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
DestAddr, SrcAddr, OriginalType,
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
[this, Copy, SrcVD, DestVD](Address DestElement, Address SrcElement) {
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
// Working with the single array element, so have to remap
// destination and source variables to corresponding array
// elements.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
CodeGenFunction::OMPPrivateScope Remap(*this);
Remap.addPrivate(DestVD, [DestElement]() -> Address {
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
return DestElement;
});
Remap.addPrivate(
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
SrcVD, [SrcElement]() -> Address { return SrcElement; });
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
(void)Remap.Privatize();
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
EmitIgnoredExpr(Copy);
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
});
}
} else {
// Remap pseudo source variable to private copy.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
CodeGenFunction::OMPPrivateScope Remap(*this);
Remap.addPrivate(SrcVD, [SrcAddr]() -> Address { return SrcAddr; });
Remap.addPrivate(DestVD, [DestAddr]() -> Address { return DestAddr; });
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
(void)Remap.Privatize();
// Emit copying of the whole variable.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
EmitIgnoredExpr(Copy);
[OPENMP] Codegen for 'firstprivate' clause. This patch generates some helper variables that used as private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by copy using values of the original variables (with the copy constructor, if any). For arrays, initializator is generated for single element and in the codegen procedure this initial value is automatically propagated between all elements of the private copy. In outlined function, references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables an implicit barier is generated by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D5140 llvm-svn: 219306
2014-10-08 22:01:46 +08:00
}
}
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
bool CodeGenFunction::EmitOMPFirstprivateClause(const OMPExecutableDirective &D,
OMPPrivateScope &PrivateScope) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return false;
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
bool FirstprivateIsLastprivate = false;
llvm::DenseSet<const VarDecl *> Lastprivates;
for (const auto *C : D.getClausesOfKind<OMPLastprivateClause>()) {
for (const auto *D : C->varlists())
Lastprivates.insert(
cast<VarDecl>(cast<DeclRefExpr>(D)->getDecl())->getCanonicalDecl());
}
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
llvm::DenseSet<const VarDecl *> EmittedAsFirstprivate;
[OPENMP] Fixed codegen for firstprivate vars in standalone worksharing directives. If firstprivate variable is is captured by value in outlined region and then used as firstprivate variable in inner worksharing directive, the copy for this firstprivate variable was not created. Fixed this bug. llvm-svn: 270536
2016-05-24 15:40:12 +08:00
CGCapturedStmtInfo CapturesInfo(cast<CapturedStmt>(*D.getAssociatedStmt()));
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPFirstprivateClause>()) {
[OPENMP] Codegen for 'firstprivate' clause. This patch generates some helper variables that used as private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by copy using values of the original variables (with the copy constructor, if any). For arrays, initializator is generated for single element and in the codegen procedure this initial value is automatically propagated between all elements of the private copy. In outlined function, references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables an implicit barier is generated by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D5140 llvm-svn: 219306
2014-10-08 22:01:46 +08:00
auto IRef = C->varlist_begin();
auto InitsRef = C->inits().begin();
for (auto IInit : C->private_copies()) {
Code improvements in OpenMP CodeGen. This patch makes class OMPPrivateScope a common class for all private variables. Reworked processing of firstprivate variables (now it is based on OMPPrivateScope too). llvm-svn: 219486
2014-10-10 17:48:26 +08:00
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
[OPENMP] Pass scalar firstprivate vars by value. For better performance and to unify code with offloading part we pass scalar firstprivate values by value, instead of by reference. It will remove some extra copying operations. llvm-svn: 269751
2016-05-17 16:55:33 +08:00
bool ThisFirstprivateIsLastprivate =
Lastprivates.count(OrigVD->getCanonicalDecl()) > 0;
[OPENMP] Fixed codegen for firstprivate vars in standalone worksharing directives. If firstprivate variable is is captured by value in outlined region and then used as firstprivate variable in inner worksharing directive, the copy for this firstprivate variable was not created. Fixed this bug. llvm-svn: 270536
2016-05-24 15:40:12 +08:00
auto *CapFD = CapturesInfo.lookup(OrigVD);
[OPENMP] Pass scalar firstprivate vars by value. For better performance and to unify code with offloading part we pass scalar firstprivate values by value, instead of by reference. It will remove some extra copying operations. llvm-svn: 269751
2016-05-17 16:55:33 +08:00
auto *FD = CapturedStmtInfo->lookup(OrigVD);
[OPENMP] Fixed codegen for firstprivate vars in standalone worksharing directives. If firstprivate variable is is captured by value in outlined region and then used as firstprivate variable in inner worksharing directive, the copy for this firstprivate variable was not created. Fixed this bug. llvm-svn: 270536
2016-05-24 15:40:12 +08:00
if (!ThisFirstprivateIsLastprivate && FD && (FD == CapFD) &&
[OPENMP] Pass scalar firstprivate vars by value. For better performance and to unify code with offloading part we pass scalar firstprivate values by value, instead of by reference. It will remove some extra copying operations. llvm-svn: 269751
2016-05-17 16:55:33 +08:00
!FD->getType()->isReferenceType()) {
EmittedAsFirstprivate.insert(OrigVD->getCanonicalDecl());
++IRef;
++InitsRef;
continue;
}
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
FirstprivateIsLastprivate =
[OPENMP] Pass scalar firstprivate vars by value. For better performance and to unify code with offloading part we pass scalar firstprivate values by value, instead of by reference. It will remove some extra copying operations. llvm-svn: 269751
2016-05-17 16:55:33 +08:00
FirstprivateIsLastprivate || ThisFirstprivateIsLastprivate;
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
if (EmittedAsFirstprivate.insert(OrigVD->getCanonicalDecl()).second) {
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(IInit)->getDecl());
auto *VDInit = cast<VarDecl>(cast<DeclRefExpr>(*InitsRef)->getDecl());
bool IsRegistered;
[OPENMP] Pass scalar firstprivate vars by value. For better performance and to unify code with offloading part we pass scalar firstprivate values by value, instead of by reference. It will remove some extra copying operations. llvm-svn: 269751
2016-05-17 16:55:33 +08:00
DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
/*RefersToEnclosingVariableOrCapture=*/FD != nullptr,
(*IRef)->getType(), VK_LValue, (*IRef)->getExprLoc());
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
Address OriginalAddr = EmitLValue(&DRE).getAddress();
[OPENMP] Fix for PR27463: Privatizing struct fields with array type causes code generation failure. The codegen part of firstprivate clause for member decls used type of original variable without skipping reference type from OMPCapturedExprDecl. Patch fixes this problem. llvm-svn: 267125
2016-04-22 17:05:03 +08:00
QualType Type = VD->getType();
[OPENMP] Fixed codegen for copying/initialization of array variables/parameters. This modification generates proper copyin/initialization sequences for array variables/parameters. Before they were considered as pointers, not arrays. llvm-svn: 237691
2015-05-19 20:31:28 +08:00
if (Type->isArrayType()) {
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
// Emit VarDecl with copy init for arrays.
// Get the address of the original variable captured in current
// captured region.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
auto Emission = EmitAutoVarAlloca(*VD);
auto *Init = VD->getInit();
if (!isa<CXXConstructExpr>(Init) || isTrivialInitializer(Init)) {
// Perform simple memcpy.
EmitAggregateAssign(Emission.getAllocatedAddress(), OriginalAddr,
[OPENMP] Fixed codegen for copying/initialization of array variables/parameters. This modification generates proper copyin/initialization sequences for array variables/parameters. Before they were considered as pointers, not arrays. llvm-svn: 237691
2015-05-19 20:31:28 +08:00
Type);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
} else {
EmitOMPAggregateAssign(
[OPENMP] Fixed codegen for copying/initialization of array variables/parameters. This modification generates proper copyin/initialization sequences for array variables/parameters. Before they were considered as pointers, not arrays. llvm-svn: 237691
2015-05-19 20:31:28 +08:00
Emission.getAllocatedAddress(), OriginalAddr, Type,
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
[this, VDInit, Init](Address DestElement,
Address SrcElement) {
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
// Clean up any temporaries needed by the initialization.
RunCleanupsScope InitScope(*this);
// Emit initialization for single element.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
setAddrOfLocalVar(VDInit, SrcElement);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
EmitAnyExprToMem(Init, DestElement,
Init->getType().getQualifiers(),
/*IsInitializer*/ false);
LocalDeclMap.erase(VDInit);
});
}
EmitAutoVarCleanups(Emission);
return Emission.getAllocatedAddress();
});
} else {
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
// Emit private VarDecl with copy init.
// Remap temp VDInit variable to the address of the original
// variable
// (for proper handling of captured global variables).
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
setAddrOfLocalVar(VDInit, OriginalAddr);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
EmitDecl(*VD);
LocalDeclMap.erase(VDInit);
return GetAddrOfLocalVar(VD);
});
}
assert(IsRegistered &&
"firstprivate var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
}
Remove use of builtin comma operator. Cleanup for upcoming Clang warning -Wcomma. No functionality change intended. llvm-svn: 261271
2016-02-19 06:34:54 +08:00
++IRef;
++InitsRef;
[OPENMP] Codegen for 'firstprivate' clause. This patch generates some helper variables that used as private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by copy using values of the original variables (with the copy constructor, if any). For arrays, initializator is generated for single element and in the codegen procedure this initial value is automatically propagated between all elements of the private copy. In outlined function, references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables an implicit barier is generated by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D5140 llvm-svn: 219306
2014-10-08 22:01:46 +08:00
}
}
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
return FirstprivateIsLastprivate && !EmittedAsFirstprivate.empty();
[OPENMP] Codegen for 'firstprivate' clause. This patch generates some helper variables that used as private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by copy using values of the original variables (with the copy constructor, if any). For arrays, initializator is generated for single element and in the codegen procedure this initial value is automatically propagated between all elements of the private copy. In outlined function, references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables an implicit barier is generated by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D5140 llvm-svn: 219306
2014-10-08 22:01:46 +08:00
}
[OPENMP] Codegen for 'private' clause in 'parallel' directive. This patch generates some helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by default (with the default constructor, if any). In outlined function references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables and implicit barier is set by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D4752 llvm-svn: 220262
2014-10-21 11:16:40 +08:00
void CodeGenFunction::EmitOMPPrivateClause(
const OMPExecutableDirective &D,
CodeGenFunction::OMPPrivateScope &PrivateScope) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return;
[OPENMP] Codegen for 'private' clause in 'for' directive. This patch generates helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'for' directive. These generated variables are initialized by default (with the default constructor, if any). In OpenMP region references to original variables are replaced by the references to these private helper variables. Differential Revision: http://reviews.llvm.org/D9106 llvm-svn: 235503
2015-04-22 20:24:45 +08:00
llvm::DenseSet<const VarDecl *> EmittedAsPrivate;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPPrivateClause>()) {
[OPENMP] Codegen for 'private' clause in 'parallel' directive. This patch generates some helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by default (with the default constructor, if any). In outlined function references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables and implicit barier is set by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D4752 llvm-svn: 220262
2014-10-21 11:16:40 +08:00
auto IRef = C->varlist_begin();
for (auto IInit : C->private_copies()) {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
[OPENMP] Codegen for 'private' clause in 'for' directive. This patch generates helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'for' directive. These generated variables are initialized by default (with the default constructor, if any). In OpenMP region references to original variables are replaced by the references to these private helper variables. Differential Revision: http://reviews.llvm.org/D9106 llvm-svn: 235503
2015-04-22 20:24:45 +08:00
if (EmittedAsPrivate.insert(OrigVD->getCanonicalDecl()).second) {
auto VD = cast<VarDecl>(cast<DeclRefExpr>(IInit)->getDecl());
bool IsRegistered =
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
PrivateScope.addPrivate(OrigVD, [&]() -> Address {
[OPENMP] Codegen for 'private' clause in 'for' directive. This patch generates helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'for' directive. These generated variables are initialized by default (with the default constructor, if any). In OpenMP region references to original variables are replaced by the references to these private helper variables. Differential Revision: http://reviews.llvm.org/D9106 llvm-svn: 235503
2015-04-22 20:24:45 +08:00
// Emit private VarDecl with copy init.
EmitDecl(*VD);
return GetAddrOfLocalVar(VD);
});
assert(IsRegistered && "private var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
}
[OPENMP] Codegen for 'private' clause in 'parallel' directive. This patch generates some helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'parallel' directive. These generated variables are initialized by default (with the default constructor, if any). In outlined function references to original variables are replaced by the references to these private helper variables. At the end of the initialization of the private variables and implicit barier is set by calling __kmpc_barrier(...) runtime function to be sure that all threads were initialized using original values of the variables. Differential Revision: http://reviews.llvm.org/D4752 llvm-svn: 220262
2014-10-21 11:16:40 +08:00
++IRef;
}
}
}
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
bool CodeGenFunction::EmitOMPCopyinClause(const OMPExecutableDirective &D) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return false;
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
// threadprivate_var1 = master_threadprivate_var1;
// operator=(threadprivate_var2, master_threadprivate_var2);
// ...
// __kmpc_barrier(&loc, global_tid);
llvm::DenseSet<const VarDecl *> CopiedVars;
llvm::BasicBlock *CopyBegin = nullptr, *CopyEnd = nullptr;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPCopyinClause>()) {
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
auto IRef = C->varlist_begin();
auto ISrcRef = C->source_exprs().begin();
auto IDestRef = C->destination_exprs().begin();
for (auto *AssignOp : C->assignment_ops()) {
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
[OPENMP] Fixed codegen for copying/initialization of array variables/parameters. This modification generates proper copyin/initialization sequences for array variables/parameters. Before they were considered as pointers, not arrays. llvm-svn: 237691
2015-05-19 20:31:28 +08:00
QualType Type = VD->getType();
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
if (CopiedVars.insert(VD->getCanonicalDecl()).second) {
[OpenMP] Add capture for threadprivate variables used in copyin clause if TLS is enabled in OpenMP code generation. llvm-svn: 243277
2015-07-28 00:38:06 +08:00
// Get the address of the master variable. If we are emitting code with
// TLS support, the address is passed from the master as field in the
// captured declaration.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
Address MasterAddr = Address::invalid();
[OpenMP] Add capture for threadprivate variables used in copyin clause if TLS is enabled in OpenMP code generation. llvm-svn: 243277
2015-07-28 00:38:06 +08:00
if (getLangOpts().OpenMPUseTLS &&
getContext().getTargetInfo().isTLSSupported()) {
assert(CapturedStmtInfo->lookup(VD) &&
"Copyin threadprivates should have been captured!");
DeclRefExpr DRE(const_cast<VarDecl *>(VD), true, (*IRef)->getType(),
VK_LValue, (*IRef)->getExprLoc());
MasterAddr = EmitLValue(&DRE).getAddress();
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
LocalDeclMap.erase(VD);
[OpenMP] Add capture for threadprivate variables used in copyin clause if TLS is enabled in OpenMP code generation. llvm-svn: 243277
2015-07-28 00:38:06 +08:00
} else {
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
MasterAddr =
Address(VD->isStaticLocal() ? CGM.getStaticLocalDeclAddress(VD)
: CGM.GetAddrOfGlobal(VD),
getContext().getDeclAlign(VD));
[OpenMP] Add capture for threadprivate variables used in copyin clause if TLS is enabled in OpenMP code generation. llvm-svn: 243277
2015-07-28 00:38:06 +08:00
}
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
// Get the address of the threadprivate variable.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
Address PrivateAddr = EmitLValue(*IRef).getAddress();
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
if (CopiedVars.size() == 1) {
// At first check if current thread is a master thread. If it is, no
// need to copy data.
CopyBegin = createBasicBlock("copyin.not.master");
CopyEnd = createBasicBlock("copyin.not.master.end");
Builder.CreateCondBr(
Builder.CreateICmpNE(
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
Builder.CreatePtrToInt(MasterAddr.getPointer(), CGM.IntPtrTy),
Builder.CreatePtrToInt(PrivateAddr.getPointer(), CGM.IntPtrTy)),
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
CopyBegin, CopyEnd);
EmitBlock(CopyBegin);
}
auto *SrcVD = cast<VarDecl>(cast<DeclRefExpr>(*ISrcRef)->getDecl());
auto *DestVD = cast<VarDecl>(cast<DeclRefExpr>(*IDestRef)->getDecl());
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
EmitOMPCopy(Type, PrivateAddr, MasterAddr, DestVD, SrcVD, AssignOp);
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
}
++IRef;
++ISrcRef;
++IDestRef;
}
}
if (CopyEnd) {
// Exit out of copying procedure for non-master thread.
EmitBlock(CopyEnd, /*IsFinished=*/true);
return true;
}
return false;
}
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
bool CodeGenFunction::EmitOMPLastprivateClauseInit(
const OMPExecutableDirective &D, OMPPrivateScope &PrivateScope) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return false;
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
bool HasAtLeastOneLastprivate = false;
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
llvm::DenseSet<const VarDecl *> SIMDLCVs;
if (isOpenMPSimdDirective(D.getDirectiveKind())) {
auto *LoopDirective = cast<OMPLoopDirective>(&D);
for (auto *C : LoopDirective->counters()) {
SIMDLCVs.insert(
cast<VarDecl>(cast<DeclRefExpr>(C)->getDecl())->getCanonicalDecl());
}
}
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
llvm::DenseSet<const VarDecl *> AlreadyEmittedVars;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPLastprivateClause>()) {
[OPENMP] Fixed codegen for firstprivate variables, also marked as lastprivate. In some rare cases shared copies of lastprivate/firstprivate variables were not updated after the loop directive. llvm-svn: 237243
2015-05-13 18:23:02 +08:00
HasAtLeastOneLastprivate = true;
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
if (isOpenMPTaskLoopDirective(D.getDirectiveKind()))
break;
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
auto IRef = C->varlist_begin();
auto IDestRef = C->destination_exprs().begin();
for (auto *IInit : C->private_copies()) {
// Keep the address of the original variable for future update at the end
// of the loop.
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
// Taskloops do not require additional initialization, it is done in
// runtime support library.
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
if (AlreadyEmittedVars.insert(OrigVD->getCanonicalDecl()).second) {
auto *DestVD = cast<VarDecl>(cast<DeclRefExpr>(*IDestRef)->getDecl());
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
PrivateScope.addPrivate(DestVD, [this, OrigVD, IRef]() -> Address {
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
DeclRefExpr DRE(
const_cast<VarDecl *>(OrigVD),
/*RefersToEnclosingVariableOrCapture=*/CapturedStmtInfo->lookup(
OrigVD) != nullptr,
(*IRef)->getType(), VK_LValue, (*IRef)->getExprLoc());
return EmitLValue(&DRE).getAddress();
});
// Check if the variable is also a firstprivate: in this case IInit is
// not generated. Initialization of this variable will happen in codegen
// for 'firstprivate' clause.
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (IInit && !SIMDLCVs.count(OrigVD->getCanonicalDecl())) {
[OPENMP] Fixed codegen for firstprivate variables, also marked as lastprivate. In some rare cases shared copies of lastprivate/firstprivate variables were not updated after the loop directive. llvm-svn: 237243
2015-05-13 18:23:02 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(IInit)->getDecl());
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
bool IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
// Emit private VarDecl with copy init.
EmitDecl(*VD);
return GetAddrOfLocalVar(VD);
});
[OPENMP] Fixed codegen for firstprivate variables, also marked as lastprivate. In some rare cases shared copies of lastprivate/firstprivate variables were not updated after the loop directive. llvm-svn: 237243
2015-05-13 18:23:02 +08:00
assert(IsRegistered &&
"lastprivate var already registered as private");
(void)IsRegistered;
}
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
}
Remove use of builtin comma operator. Cleanup for upcoming Clang warning -Wcomma. No functionality change intended. llvm-svn: 261271
2016-02-19 06:34:54 +08:00
++IRef;
++IDestRef;
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
}
}
return HasAtLeastOneLastprivate;
}
void CodeGenFunction::EmitOMPLastprivateClauseFinal(
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
const OMPExecutableDirective &D, bool NoFinals,
llvm::Value *IsLastIterCond) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return;
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
// Emit following code:
// if (<IsLastIterCond>) {
// orig_var1 = private_orig_var1;
// ...
// orig_varn = private_orig_varn;
// }
[OPENMP] Support lastprivate clause in omp simd directive. Added codegen for lastprivate clauses within simd loop-based directives. llvm-svn: 239813
2015-06-16 21:14:42 +08:00
llvm::BasicBlock *ThenBB = nullptr;
llvm::BasicBlock *DoneBB = nullptr;
if (IsLastIterCond) {
ThenBB = createBasicBlock(".omp.lastprivate.then");
DoneBB = createBasicBlock(".omp.lastprivate.done");
Builder.CreateCondBr(IsLastIterCond, ThenBB, DoneBB);
EmitBlock(ThenBB);
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
llvm::DenseSet<const VarDecl *> AlreadyEmittedVars;
llvm::DenseMap<const VarDecl *, const Expr *> LoopCountersAndUpdates;
[OPENMP] Fixed codegen for lastprivate LCV in worksharing constructs. If loop control variable in a worksharing construct is marked as lastprivate, we should copy last calculated value of private counter back to original variable. llvm-svn: 237879
2015-05-21 15:59:51 +08:00
if (auto *LoopDirective = dyn_cast<OMPLoopDirective>(&D)) {
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
auto IC = LoopDirective->counters().begin();
for (auto F : LoopDirective->finals()) {
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
auto *D =
cast<VarDecl>(cast<DeclRefExpr>(*IC)->getDecl())->getCanonicalDecl();
if (NoFinals)
AlreadyEmittedVars.insert(D);
else
LoopCountersAndUpdates[D] = F;
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
++IC;
[OPENMP] Fixed codegen for lastprivate LCV in worksharing constructs. If loop control variable in a worksharing construct is marked as lastprivate, we should copy last calculated value of private counter back to original variable. llvm-svn: 237879
2015-05-21 15:59:51 +08:00
}
}
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
for (const auto *C : D.getClausesOfKind<OMPLastprivateClause>()) {
auto IRef = C->varlist_begin();
auto ISrcRef = C->source_exprs().begin();
auto IDestRef = C->destination_exprs().begin();
for (auto *AssignOp : C->assignment_ops()) {
auto *PrivateVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
QualType Type = PrivateVD->getType();
auto *CanonicalVD = PrivateVD->getCanonicalDecl();
if (AlreadyEmittedVars.insert(CanonicalVD).second) {
// If lastprivate variable is a loop control variable for loop-based
// directive, update its value before copyin back to original
// variable.
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (auto *FinalExpr = LoopCountersAndUpdates.lookup(CanonicalVD))
EmitIgnoredExpr(FinalExpr);
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
auto *SrcVD = cast<VarDecl>(cast<DeclRefExpr>(*ISrcRef)->getDecl());
auto *DestVD = cast<VarDecl>(cast<DeclRefExpr>(*IDestRef)->getDecl());
// Get the address of the original variable.
Address OriginalAddr = GetAddrOfLocalVar(DestVD);
// Get the address of the private variable.
Address PrivateAddr = GetAddrOfLocalVar(PrivateVD);
if (auto RefTy = PrivateVD->getType()->getAs<ReferenceType>())
PrivateAddr =
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
Address(Builder.CreateLoad(PrivateAddr),
getNaturalTypeAlignment(RefTy->getPointeeType()));
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
EmitOMPCopy(Type, OriginalAddr, PrivateAddr, DestVD, SrcVD, AssignOp);
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
}
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
++IRef;
++ISrcRef;
++IDestRef;
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
}
[OPENMP 4.5] Codegen for member decls in 'lastprivate' clause. OpenMP 4.5 allows to privatize non-static member decls in non-static member functions. Patch captures such decls by reference in general (for bitfields, by value) and then operates with this capture. For bitfields, at the end of codegen for lastprivates original bitfield is updated with the value of captured copy. llvm-svn: 261824
2016-02-25 13:25:57 +08:00
if (auto *PostUpdate = C->getPostUpdateExpr())
EmitIgnoredExpr(PostUpdate);
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
}
[OPENMP] Fix codegen for lastprivate loop counters. Patch fixes bug with codegen for lastprivate loop counters. Also it may improve performance for lastprivates calculations in some cases. llvm-svn: 261209
2016-02-18 21:48:15 +08:00
if (IsLastIterCond)
[OPENMP] Support lastprivate clause in omp simd directive. Added codegen for lastprivate clauses within simd loop-based directives. llvm-svn: 239813
2015-06-16 21:14:42 +08:00
EmitBlock(DoneBB, /*IsFinished=*/true);
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
}
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
static Address castToBase(CodeGenFunction &CGF, QualType BaseTy, QualType ElTy,
LValue BaseLV, llvm::Value *Addr) {
Address Tmp = Address::invalid();
Address TopTmp = Address::invalid();
Address MostTopTmp = Address::invalid();
BaseTy = BaseTy.getNonReferenceType();
while ((BaseTy->isPointerType() || BaseTy->isReferenceType()) &&
!CGF.getContext().hasSameType(BaseTy, ElTy)) {
Tmp = CGF.CreateMemTemp(BaseTy);
if (TopTmp.isValid())
CGF.Builder.CreateStore(Tmp.getPointer(), TopTmp);
else
MostTopTmp = Tmp;
TopTmp = Tmp;
BaseTy = BaseTy->getPointeeType();
}
llvm::Type *Ty = BaseLV.getPointer()->getType();
if (Tmp.isValid())
Ty = Tmp.getElementType();
Addr = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(Addr, Ty);
if (Tmp.isValid()) {
CGF.Builder.CreateStore(Addr, Tmp);
return MostTopTmp;
}
return Address(Addr, BaseLV.getAlignment());
}
static LValue loadToBegin(CodeGenFunction &CGF, QualType BaseTy, QualType ElTy,
LValue BaseLV) {
BaseTy = BaseTy.getNonReferenceType();
while ((BaseTy->isPointerType() || BaseTy->isReferenceType()) &&
!CGF.getContext().hasSameType(BaseTy, ElTy)) {
if (auto *PtrTy = BaseTy->getAs<PointerType>())
BaseLV = CGF.EmitLoadOfPointerLValue(BaseLV.getAddress(), PtrTy);
else {
BaseLV = CGF.EmitLoadOfReferenceLValue(BaseLV.getAddress(),
BaseTy->castAs<ReferenceType>());
}
BaseTy = BaseTy->getPointeeType();
}
return CGF.MakeAddrLValue(
Address(
CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
BaseLV.getPointer(), CGF.ConvertTypeForMem(ElTy)->getPointerTo()),
BaseLV.getAlignment()),
[CodeGen] Propagate LValueBaseInfo instead of AlignmentSource The functions creating LValues propagated information about alignment source. Extend the propagated data to also include information about possible unrestricted aliasing. A new class LValueBaseInfo will contain both AlignmentSource and MayAlias info. This patch should not introduce any functional changes. Differential Revision: https://reviews.llvm.org/D33284 llvm-svn: 303358
2017-05-19 01:07:11 +08:00
BaseLV.getType(), BaseLV.getBaseInfo());
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
}
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
void CodeGenFunction::EmitOMPReductionClauseInit(
const OMPExecutableDirective &D,
CodeGenFunction::OMPPrivateScope &PrivateScope) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPReductionClause>()) {
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
auto ILHS = C->lhs_exprs().begin();
auto IRHS = C->rhs_exprs().begin();
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
auto IPriv = C->privates().begin();
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
auto IRed = C->reduction_ops().begin();
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
for (auto IRef : C->varlists()) {
auto *LHSVD = cast<VarDecl>(cast<DeclRefExpr>(*ILHS)->getDecl());
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
auto *RHSVD = cast<VarDecl>(cast<DeclRefExpr>(*IRHS)->getDecl());
auto *PrivateVD = cast<VarDecl>(cast<DeclRefExpr>(*IPriv)->getDecl());
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
auto *DRD = getReductionInit(*IRed);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
if (auto *OASE = dyn_cast<OMPArraySectionExpr>(IRef)) {
auto *Base = OASE->getBase()->IgnoreParenImpCasts();
while (auto *TempOASE = dyn_cast<OMPArraySectionExpr>(Base))
Base = TempOASE->getBase()->IgnoreParenImpCasts();
while (auto *TempASE = dyn_cast<ArraySubscriptExpr>(Base))
Base = TempASE->getBase()->IgnoreParenImpCasts();
auto *DE = cast<DeclRefExpr>(Base);
auto *OrigVD = cast<VarDecl>(DE->getDecl());
auto OASELValueLB = EmitOMPArraySectionExpr(OASE);
auto OASELValueUB =
EmitOMPArraySectionExpr(OASE, /*IsLowerBound=*/false);
auto OriginalBaseLValue = EmitLValue(DE);
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
LValue BaseLValue =
loadToBegin(*this, OrigVD->getType(), OASELValueLB.getType(),
OriginalBaseLValue);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
// Store the address of the original variable associated with the LHS
// implicit variable.
Remove unused lambda captures. NFC llvm-svn: 291939
2017-01-14 02:55:32 +08:00
PrivateScope.addPrivate(LHSVD, [OASELValueLB]() -> Address {
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
return OASELValueLB.getAddress();
});
// Emit reduction copy.
bool IsRegistered = PrivateScope.addPrivate(
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
OrigVD, [this, OrigVD, PrivateVD, BaseLValue, OASELValueLB,
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
OASELValueUB, OriginalBaseLValue, DRD, IRed]() -> Address {
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
// Emit VarDecl with copy init for arrays.
// Get the address of the original variable captured in current
// captured region.
auto *Size = Builder.CreatePtrDiff(OASELValueUB.getPointer(),
OASELValueLB.getPointer());
Size = Builder.CreateNUWAdd(
Size, llvm::ConstantInt::get(Size->getType(), /*V=*/1));
CodeGenFunction::OpaqueValueMapping OpaqueMap(
*this, cast<OpaqueValueExpr>(
getContext()
.getAsVariableArrayType(PrivateVD->getType())
->getSizeExpr()),
RValue::get(Size));
EmitVariablyModifiedType(PrivateVD->getType());
auto Emission = EmitAutoVarAlloca(*PrivateVD);
auto Addr = Emission.getAllocatedAddress();
auto *Init = PrivateVD->getInit();
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
EmitOMPAggregateInit(*this, Addr, PrivateVD->getType(),
DRD ? *IRed : Init,
OASELValueLB.getAddress());
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
EmitAutoVarCleanups(Emission);
// Emit private VarDecl with reduction init.
auto *Offset = Builder.CreatePtrDiff(BaseLValue.getPointer(),
OASELValueLB.getPointer());
auto *Ptr = Builder.CreateGEP(Addr.getPointer(), Offset);
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
return castToBase(*this, OrigVD->getType(),
OASELValueLB.getType(), OriginalBaseLValue,
Ptr);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
});
assert(IsRegistered && "private var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
PrivateScope.addPrivate(RHSVD, [this, PrivateVD]() -> Address {
return GetAddrOfLocalVar(PrivateVD);
});
} else if (auto *ASE = dyn_cast<ArraySubscriptExpr>(IRef)) {
auto *Base = ASE->getBase()->IgnoreParenImpCasts();
while (auto *TempASE = dyn_cast<ArraySubscriptExpr>(Base))
Base = TempASE->getBase()->IgnoreParenImpCasts();
auto *DE = cast<DeclRefExpr>(Base);
auto *OrigVD = cast<VarDecl>(DE->getDecl());
auto ASELValue = EmitLValue(ASE);
auto OriginalBaseLValue = EmitLValue(DE);
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
LValue BaseLValue = loadToBegin(
*this, OrigVD->getType(), ASELValue.getType(), OriginalBaseLValue);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
// Store the address of the original variable associated with the LHS
// implicit variable.
Remove unused lambda captures. NFC llvm-svn: 291939
2017-01-14 02:55:32 +08:00
PrivateScope.addPrivate(
LHSVD, [ASELValue]() -> Address { return ASELValue.getAddress(); });
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
// Emit reduction copy.
bool IsRegistered = PrivateScope.addPrivate(
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
OrigVD, [this, OrigVD, PrivateVD, BaseLValue, ASELValue,
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
OriginalBaseLValue, DRD, IRed]() -> Address {
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
// Emit private VarDecl with reduction init.
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
AutoVarEmission Emission = EmitAutoVarAlloca(*PrivateVD);
auto Addr = Emission.getAllocatedAddress();
[OPENMP] Fix crash on initialization of classes with no init clause in declare reductions. If reduction clause is applied to instance of class with user-defined reduction operation without initialization clause, it may cause a crash. Patch fixes this issue. llvm-svn: 267695
2016-04-27 19:38:05 +08:00
if (DRD && (DRD->getInitializer() || !PrivateVD->hasInit())) {
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
emitInitWithReductionInitializer(*this, DRD, *IRed, Addr,
ASELValue.getAddress(),
ASELValue.getType());
} else
EmitAutoVarInit(Emission);
EmitAutoVarCleanups(Emission);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
auto *Offset = Builder.CreatePtrDiff(BaseLValue.getPointer(),
ASELValue.getPointer());
auto *Ptr = Builder.CreateGEP(Addr.getPointer(), Offset);
[OPENMP 4.0] Fixed support of array sections/array subscripts. Codegen for array sections/array subscripts worked only for expressions with arrays as base. Patch fixes codegen for bases with pointer/reference types. llvm-svn: 259776
2016-02-04 19:27:03 +08:00
return castToBase(*this, OrigVD->getType(), ASELValue.getType(),
OriginalBaseLValue, Ptr);
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
});
assert(IsRegistered && "private var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
PrivateScope.addPrivate(RHSVD, [this, PrivateVD, RHSVD]() -> Address {
return Builder.CreateElementBitCast(
GetAddrOfLocalVar(PrivateVD), ConvertTypeForMem(RHSVD->getType()),
"rhs.begin");
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
});
} else {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(IRef)->getDecl());
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
QualType Type = PrivateVD->getType();
if (getContext().getAsArrayType(Type)) {
// Store the address of the original variable associated with the LHS
// implicit variable.
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
CapturedStmtInfo->lookup(OrigVD) != nullptr,
IRef->getType(), VK_LValue, IRef->getExprLoc());
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
Address OriginalAddr = EmitLValue(&DRE).getAddress();
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
PrivateScope.addPrivate(LHSVD, [this, &OriginalAddr,
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
LHSVD]() -> Address {
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
OriginalAddr = Builder.CreateElementBitCast(
OriginalAddr, ConvertTypeForMem(LHSVD->getType()), "lhs.begin");
return OriginalAddr;
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
});
bool IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
if (Type->isVariablyModifiedType()) {
CodeGenFunction::OpaqueValueMapping OpaqueMap(
*this, cast<OpaqueValueExpr>(
getContext()
.getAsVariableArrayType(PrivateVD->getType())
->getSizeExpr()),
RValue::get(
getTypeSize(OrigVD->getType().getNonReferenceType())));
EmitVariablyModifiedType(Type);
}
auto Emission = EmitAutoVarAlloca(*PrivateVD);
auto Addr = Emission.getAllocatedAddress();
auto *Init = PrivateVD->getInit();
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
EmitOMPAggregateInit(*this, Addr, PrivateVD->getType(),
DRD ? *IRed : Init, OriginalAddr);
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
EmitAutoVarCleanups(Emission);
return Emission.getAllocatedAddress();
});
assert(IsRegistered && "private var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
PrivateScope.addPrivate(RHSVD, [this, PrivateVD, RHSVD]() -> Address {
return Builder.CreateElementBitCast(
GetAddrOfLocalVar(PrivateVD),
ConvertTypeForMem(RHSVD->getType()), "rhs.begin");
});
} else {
// Store the address of the original variable associated with the LHS
// implicit variable.
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
Address OriginalAddr = Address::invalid();
PrivateScope.addPrivate(LHSVD, [this, OrigVD, IRef,
&OriginalAddr]() -> Address {
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
CapturedStmtInfo->lookup(OrigVD) != nullptr,
IRef->getType(), VK_LValue, IRef->getExprLoc());
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
OriginalAddr = EmitLValue(&DRE).getAddress();
return OriginalAddr;
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
});
// Emit reduction copy.
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
bool IsRegistered = PrivateScope.addPrivate(
OrigVD, [this, PrivateVD, OriginalAddr, DRD, IRed]() -> Address {
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
// Emit private VarDecl with reduction init.
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
AutoVarEmission Emission = EmitAutoVarAlloca(*PrivateVD);
auto Addr = Emission.getAllocatedAddress();
[OPENMP] Fix crash on initialization of classes with no init clause in declare reductions. If reduction clause is applied to instance of class with user-defined reduction operation without initialization clause, it may cause a crash. Patch fixes this issue. llvm-svn: 267695
2016-04-27 19:38:05 +08:00
if (DRD && (DRD->getInitializer() || !PrivateVD->hasInit())) {
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
emitInitWithReductionInitializer(*this, DRD, *IRed, Addr,
OriginalAddr,
PrivateVD->getType());
} else
EmitAutoVarInit(Emission);
EmitAutoVarCleanups(Emission);
return Addr;
[OPENMP 4.5] Allow arrays in 'reduction' clause. OpenMP 4.5, alogn with array sections, allows to use variables of array type in reductions. llvm-svn: 258804
2016-01-26 20:20:39 +08:00
});
assert(IsRegistered && "private var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
PrivateScope.addPrivate(RHSVD, [this, PrivateVD]() -> Address {
return GetAddrOfLocalVar(PrivateVD);
});
}
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
}
Remove use of builtin comma operator. Cleanup for upcoming Clang warning -Wcomma. No functionality change intended. llvm-svn: 261271
2016-02-19 06:34:54 +08:00
++ILHS;
++IRHS;
++IPriv;
[OPENMP 4.0] Use 'declare reduction' constructs in 'reduction' clauses. OpenMP 4.0 allows to define custom reduction operations using '#pragma omp declare reduction' construct. Patch allows to use this custom defined reduction operations in 'reduction' clauses. llvm-svn: 263701
2016-03-17 18:19:46 +08:00
++IRed;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
}
}
}
void CodeGenFunction::EmitOMPReductionClauseFinal(
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
const OMPExecutableDirective &D, const OpenMPDirectiveKind ReductionKind) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return;
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
llvm::SmallVector<const Expr *, 8> Privates;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
llvm::SmallVector<const Expr *, 8> LHSExprs;
llvm::SmallVector<const Expr *, 8> RHSExprs;
llvm::SmallVector<const Expr *, 8> ReductionOps;
bool HasAtLeastOneReduction = false;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPReductionClause>()) {
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
HasAtLeastOneReduction = true;
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
Privates.append(C->privates().begin(), C->privates().end());
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
LHSExprs.append(C->lhs_exprs().begin(), C->lhs_exprs().end());
RHSExprs.append(C->rhs_exprs().begin(), C->rhs_exprs().end());
ReductionOps.append(C->reduction_ops().begin(), C->reduction_ops().end());
}
if (HasAtLeastOneReduction) {
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
bool WithNowait = D.getSingleClause<OMPNowaitClause>() ||
isOpenMPParallelDirective(D.getDirectiveKind()) ||
D.getDirectiveKind() == OMPD_simd;
bool SimpleReduction = D.getDirectiveKind() == OMPD_simd;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// Emit nowait reduction if nowait clause is present or directive is a
// parallel directive (it always has implicit barrier).
CGM.getOpenMPRuntime().emitReduction(
[OPENMP 4.1] Codegen for array sections/subscripts in 'reduction' clause. OpenMP 4.1 adds support for array sections/subscripts in 'reduction' clause. Patch adds codegen for this feature. llvm-svn: 249672
2015-10-08 17:10:53 +08:00
*this, D.getLocEnd(), Privates, LHSExprs, RHSExprs, ReductionOps,
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
{WithNowait, SimpleReduction, ReductionKind});
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
}
}
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
static void emitPostUpdateForReductionClause(
CodeGenFunction &CGF, const OMPExecutableDirective &D,
const llvm::function_ref<llvm::Value *(CodeGenFunction &)> &CondGen) {
if (!CGF.HaveInsertPoint())
return;
llvm::BasicBlock *DoneBB = nullptr;
for (const auto *C : D.getClausesOfKind<OMPReductionClause>()) {
if (auto *PostUpdate = C->getPostUpdateExpr()) {
if (!DoneBB) {
if (auto *Cond = CondGen(CGF)) {
// If the first post-update expression is found, emit conditional
// block if it was requested.
auto *ThenBB = CGF.createBasicBlock(".omp.reduction.pu");
DoneBB = CGF.createBasicBlock(".omp.reduction.pu.done");
CGF.Builder.CreateCondBr(Cond, ThenBB, DoneBB);
CGF.EmitBlock(ThenBB);
}
}
CGF.EmitIgnoredExpr(PostUpdate);
}
}
if (DoneBB)
CGF.EmitBlock(DoneBB, /*IsFinished=*/true);
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
namespace {
/// Codegen lambda for appending distribute lower and upper bounds to outlined
/// parallel function. This is necessary for combined constructs such as
/// 'distribute parallel for'
typedef llvm::function_ref<void(CodeGenFunction &,
const OMPExecutableDirective &,
llvm::SmallVectorImpl<llvm::Value *> &)>
CodeGenBoundParametersTy;
} // anonymous namespace
static void emitCommonOMPParallelDirective(
CodeGenFunction &CGF, const OMPExecutableDirective &S,
OpenMPDirectiveKind InnermostKind, const RegionCodeGenTy &CodeGen,
const CodeGenBoundParametersTy &CodeGenBoundParameters) {
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
const CapturedStmt *CS = S.getCapturedStmt(OMPD_parallel);
auto OutlinedFn = CGF.CGM.getOpenMPRuntime().emitParallelOutlinedFunction(
S, *CS->getCapturedDecl()->param_begin(), InnermostKind, CodeGen);
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (const auto *NumThreadsClause = S.getSingleClause<OMPNumThreadsClause>()) {
[OPENMP] Codegen for 'num_threads' clause in 'parallel' directive. This patch generates call to "kmpc_push_num_threads(ident_t *loc, kmp_int32 global_tid, kmp_int32 num_threads);" library function before calling "kmpc_fork_call" each time there is an associated "num_threads" clause in the "omp parallel" directive. Differential Revision: http://reviews.llvm.org/D5145 llvm-svn: 219599
2014-10-13 16:23:51 +08:00
CodeGenFunction::RunCleanupsScope NumThreadsScope(CGF);
auto NumThreads = CGF.EmitScalarExpr(NumThreadsClause->getNumThreads(),
/*IgnoreResultAssign*/ true);
[OPENMP] Rename methods of OpenMPRuntime class. NFC. llvm-svn: 230470
2015-02-25 16:32:46 +08:00
CGF.CGM.getOpenMPRuntime().emitNumThreadsClause(
[OPENMP] Codegen for 'num_threads' clause in 'parallel' directive. This patch generates call to "kmpc_push_num_threads(ident_t *loc, kmp_int32 global_tid, kmp_int32 num_threads);" library function before calling "kmpc_fork_call" each time there is an associated "num_threads" clause in the "omp parallel" directive. Differential Revision: http://reviews.llvm.org/D5145 llvm-svn: 219599
2014-10-13 16:23:51 +08:00
CGF, NumThreads, NumThreadsClause->getLocStart());
}
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (const auto *ProcBindClause = S.getSingleClause<OMPProcBindClause>()) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
CodeGenFunction::RunCleanupsScope ProcBindScope(CGF);
[OPENMP] Codegen for 'proc_bind' clause (4.0). Adds emission of the code for 'proc_bind(master|close|spread)' clause: call void @__kmpc_push_proc_bind(<loc>, i32 thread_id, i32 4|3|2) llvm-svn: 240018
2015-06-18 21:40:03 +08:00
CGF.CGM.getOpenMPRuntime().emitProcBindClause(
CGF, ProcBindClause->getProcBindKind(), ProcBindClause->getLocStart());
}
[OPENMP] Codegen for 'if' clause in 'task' directive. If condition evaluates to true, the code executes task by calling @__kmpc_omp_task() runtime function. If condition evaluates to false, the code executes serial version of the code by executing the following code: call void @__kmpc_omp_task_begin_if0(<loc>, <threadid>, <task_t_ptr, returned by @__kmpc_omp_task_alloc()>); proxy_task_entry(<gtid>, <task_t_ptr, returned by @__kmpc_omp_task_alloc()>); call void @__kmpc_omp_task_complete_if0(<loc>, <threadid>, <task_t_ptr, returned by @__kmpc_omp_task_alloc()>); Also it checks if the condition is constant and if it is constant it evaluates its value and then generates either parallel version of the code (if the condition evaluates to true), or the serial version of the code (if the condition evaluates to false). Differential Revision: http://reviews.llvm.org/D9143 llvm-svn: 235507
2015-04-22 21:57:31 +08:00
const Expr *IfCond = nullptr;
[OPENMP 4.1] Codegen for extended format of 'if' clause. Fixed codegen for extended format of 'if' clauses with special 'directive-name-modifier' + ast-print tests for extended format of 'if' clause. llvm-svn: 246748
2015-09-03 16:45:56 +08:00
for (const auto *C : S.getClausesOfKind<OMPIfClause>()) {
if (C->getNameModifier() == OMPD_unknown ||
C->getNameModifier() == OMPD_parallel) {
IfCond = C->getCondition();
break;
}
[OPENMP] Codegen for 'if' clause in 'task' directive. If condition evaluates to true, the code executes task by calling @__kmpc_omp_task() runtime function. If condition evaluates to false, the code executes serial version of the code by executing the following code: call void @__kmpc_omp_task_begin_if0(<loc>, <threadid>, <task_t_ptr, returned by @__kmpc_omp_task_alloc()>); proxy_task_entry(<gtid>, <task_t_ptr, returned by @__kmpc_omp_task_alloc()>); call void @__kmpc_omp_task_complete_if0(<loc>, <threadid>, <task_t_ptr, returned by @__kmpc_omp_task_alloc()>); Also it checks if the condition is constant and if it is constant it evaluates its value and then generates either parallel version of the code (if the condition evaluates to true), or the serial version of the code (if the condition evaluates to false). Differential Revision: http://reviews.llvm.org/D9143 llvm-svn: 235507
2015-04-22 21:57:31 +08:00
}
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
OMPParallelScope Scope(CGF, S);
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
llvm::SmallVector<llvm::Value *, 16> CapturedVars;
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
// Combining 'distribute' with 'for' requires sharing each 'distribute' chunk
// lower and upper bounds with the pragma 'for' chunking mechanism.
// The following lambda takes care of appending the lower and upper bound
// parameters when necessary
CodeGenBoundParameters(CGF, S, CapturedVars);
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
CGF.GenerateOpenMPCapturedVars(*CS, CapturedVars);
[OPENMP] Rename methods of OpenMPRuntime class. NFC. llvm-svn: 230470
2015-02-25 16:32:46 +08:00
CGF.CGM.getOpenMPRuntime().emitParallelCall(CGF, S.getLocStart(), OutlinedFn,
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
CapturedVars, IfCond);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
static void emitEmptyBoundParameters(CodeGenFunction &,
const OMPExecutableDirective &,
llvm::SmallVectorImpl<llvm::Value *> &) {}
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
void CodeGenFunction::EmitOMPParallelDirective(const OMPParallelDirective &S) {
// Emit parallel region as a standalone region.
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
OMPPrivateScope PrivateScope(CGF);
[OPENMP] Codegen for 'copyin' clause in 'parallel' directive. Emits the following code for the clause at the beginning of the outlined function for implicit threads: if (<not a master thread>) { ... <thread local copy of var> = <master thread local copy of var>; ... } <sync point>; Checking for a non-master thread is performed by comparing of the address of the thread local variable with the address of the master's variable. Master thread always uses original variables, so you always know the address of the variable in the master thread. Differential Revision: http://reviews.llvm.org/D9026 llvm-svn: 235075
2015-04-16 13:39:01 +08:00
bool Copyins = CGF.EmitOMPCopyinClause(S);
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
(void)CGF.EmitOMPFirstprivateClause(S, PrivateScope);
if (Copyins) {
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
// Emit implicit barrier to synchronize threads and avoid data races on
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
// propagation master's thread values of threadprivate variables to local
// instances of that variables of all other implicit threads.
[OPENMP] Emit __kmpc_cancel_barrier() and code for 'cancellation point' only if 'cancel' is found. Patch improves codegen for OpenMP constructs. If the OpenMP region does not have internal 'cancel' construct, a call to 'void __kmpc_barrier()' runtime function is generated for all implicit/explicit barriers. If the region has inner 'cancel' directive, then ``` if (__kmpc_cancel_barrier()) exit from outer construct; ``` code is generated. Also, the code for 'canellation point' directive is not generated if parent directive does not have 'cancel' directive. llvm-svn: 247681
2015-09-15 20:52:43 +08:00
CGF.CGM.getOpenMPRuntime().emitBarrierCall(
CGF, S.getLocStart(), OMPD_unknown, /*EmitChecks=*/false,
/*ForceSimpleCall=*/true);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
}
CGF.EmitOMPPrivateClause(S, PrivateScope);
CGF.EmitOMPReductionClauseInit(S, PrivateScope);
(void)PrivateScope.Privatize();
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
CGF.EmitOMPReductionClauseFinal(S, /*ReductionKind=*/OMPD_parallel);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
};
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
emitCommonOMPParallelDirective(*this, S, OMPD_parallel, CodeGen,
emitEmptyBoundParameters);
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
emitPostUpdateForReductionClause(
*this, S, [](CodeGenFunction &) -> llvm::Value * { return nullptr; });
[OPENMP] Initial codegen for '#pragma omp parallel' llvm-svn: 208077
2014-05-06 18:08:46 +08:00
}
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
void CodeGenFunction::EmitOMPLoopBody(const OMPLoopDirective &D,
JumpDest LoopExit) {
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
RunCleanupsScope BodyScope(*this);
// Update counters values on current iteration.
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
for (auto I : D.updates()) {
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
EmitIgnoredExpr(I);
}
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
// Update the linear variables.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPLinearClause>()) {
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
for (auto *U : C->updates())
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
EmitIgnoredExpr(U);
}
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// On a continue in the body, jump to the end.
[OPENMP] Small refactoring of EmitOMPSimdLoop helper routine. No functional changes intended. Renamed EmitOMPSimdLoop to EmitOMPInnerLoop, I plan to re-use it to emit inner loop in the future patches for CodeGen of the worksharing loop directives (omp for, omp for simd). llvm-svn: 219195
2014-10-07 16:57:09 +08:00
auto Continue = getJumpDestInCurrentScope("omp.body.continue");
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
BreakContinueStack.push_back(BreakContinue(LoopExit, Continue));
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// Emit loop body.
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
EmitStmt(D.getBody());
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// The end (updates/cleanups).
EmitBlock(Continue.getBlock());
BreakContinueStack.pop_back();
}
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
void CodeGenFunction::EmitOMPInnerLoop(
const Stmt &S, bool RequiresCleanup, const Expr *LoopCond,
const Expr *IncExpr,
[OPENMP] Codegen for 'ordered' directive. Add codegen for 'ordered' directive: __kmpc_ordered(ident_t *, gtid); <associated statement>; __kmpc_end_ordered(ident_t *, gtid); Also for 'for' directives with the dynamic scheduling and an 'ordered' clause added a call to '__kmpc_dispatch_fini_(4|8)[u]()' function after increment expression for loop control variable: while(__kmpc_dispatch_next(&LB, &UB)) { idx = LB; while (idx <= UB) { BODY; ++idx; __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only. } // inner loop } Differential Revision: http://reviews.llvm.org/D9070 llvm-svn: 235496
2015-04-22 19:15:40 +08:00
const llvm::function_ref<void(CodeGenFunction &)> &BodyGen,
const llvm::function_ref<void(CodeGenFunction &)> &PostIncGen) {
[OPENMP] Small refactoring of EmitOMPSimdLoop helper routine. No functional changes intended. Renamed EmitOMPSimdLoop to EmitOMPInnerLoop, I plan to re-use it to emit inner loop in the future patches for CodeGen of the worksharing loop directives (omp for, omp for simd). llvm-svn: 219195
2014-10-07 16:57:09 +08:00
auto LoopExit = getJumpDestInCurrentScope("omp.inner.for.end");
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// Start the loop with a block that tests the condition.
[OPENMP] Small refactoring of EmitOMPSimdLoop helper routine. No functional changes intended. Renamed EmitOMPSimdLoop to EmitOMPInnerLoop, I plan to re-use it to emit inner loop in the future patches for CodeGen of the worksharing loop directives (omp for, omp for simd). llvm-svn: 219195
2014-10-07 16:57:09 +08:00
auto CondBlock = createBasicBlock("omp.inner.for.cond");
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
EmitBlock(CondBlock);
Add the loop end location to the loop metadata. This additional information can be used to improve the locations when generating remarks for loops. Depends on the companion LLVM change r286227. Patch by Florian Hahn. Differential Revision: https://reviews.llvm.org/D25764 llvm-svn: 286456
2016-11-10 22:44:30 +08:00
const SourceRange &R = S.getSourceRange();
LoopStack.push(CondBlock, SourceLocToDebugLoc(R.getBegin()),
SourceLocToDebugLoc(R.getEnd()));
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// If there are any cleanups between here and the loop-exit scope,
// create a block to stage a loop exit along.
auto ExitBlock = LoopExit.getBlock();
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
if (RequiresCleanup)
[OPENMP] Small refactoring of EmitOMPSimdLoop helper routine. No functional changes intended. Renamed EmitOMPSimdLoop to EmitOMPInnerLoop, I plan to re-use it to emit inner loop in the future patches for CodeGen of the worksharing loop directives (omp for, omp for simd). llvm-svn: 219195
2014-10-07 16:57:09 +08:00
ExitBlock = createBasicBlock("omp.inner.for.cond.cleanup");
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
[OPENMP] Small refactoring of EmitOMPSimdLoop helper routine. No functional changes intended. Renamed EmitOMPSimdLoop to EmitOMPInnerLoop, I plan to re-use it to emit inner loop in the future patches for CodeGen of the worksharing loop directives (omp for, omp for simd). llvm-svn: 219195
2014-10-07 16:57:09 +08:00
auto LoopBody = createBasicBlock("omp.inner.for.body");
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
// Emit condition.
InstrProf: Stop using RegionCounter outside of CodeGenPGO (NFC) The RegionCounter type does a lot of legwork, but most of it is only meaningful within the implementation of CodeGenPGO. The uses elsewhere in CodeGen generally just want to increment or read counters, so do that directly. llvm-svn: 235664
2015-04-24 07:06:47 +08:00
EmitBranchOnBoolExpr(LoopCond, LoopBody, ExitBlock, getProfileCount(&S));
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
if (ExitBlock != LoopExit.getBlock()) {
EmitBlock(ExitBlock);
EmitBranchThroughCleanup(LoopExit);
}
EmitBlock(LoopBody);
InstrProf: Stop using RegionCounter outside of CodeGenPGO (NFC) The RegionCounter type does a lot of legwork, but most of it is only meaningful within the implementation of CodeGenPGO. The uses elsewhere in CodeGen generally just want to increment or read counters, so do that directly. llvm-svn: 235664
2015-04-24 07:06:47 +08:00
incrementProfileCounter(&S);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// Create a block for the increment.
[OPENMP] Small refactoring of EmitOMPSimdLoop helper routine. No functional changes intended. Renamed EmitOMPSimdLoop to EmitOMPInnerLoop, I plan to re-use it to emit inner loop in the future patches for CodeGen of the worksharing loop directives (omp for, omp for simd). llvm-svn: 219195
2014-10-07 16:57:09 +08:00
auto Continue = getJumpDestInCurrentScope("omp.inner.for.inc");
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
BreakContinueStack.push_back(BreakContinue(LoopExit, Continue));
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
BodyGen(*this);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
// Emit "IV = IV + 1" and a back-edge to the condition block.
EmitBlock(Continue.getBlock());
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
EmitIgnoredExpr(IncExpr);
[OPENMP] Codegen for 'ordered' directive. Add codegen for 'ordered' directive: __kmpc_ordered(ident_t *, gtid); <associated statement>; __kmpc_end_ordered(ident_t *, gtid); Also for 'for' directives with the dynamic scheduling and an 'ordered' clause added a call to '__kmpc_dispatch_fini_(4|8)[u]()' function after increment expression for loop control variable: while(__kmpc_dispatch_next(&LB, &UB)) { idx = LB; while (idx <= UB) { BODY; ++idx; __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only. } // inner loop } Differential Revision: http://reviews.llvm.org/D9070 llvm-svn: 235496
2015-04-22 19:15:40 +08:00
PostIncGen(*this);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
BreakContinueStack.pop_back();
EmitBranch(CondBlock);
LoopStack.pop();
// Emit the fall-through block.
EmitBlock(LoopExit.getBlock());
}
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
void CodeGenFunction::EmitOMPLinearClauseInit(const OMPLoopDirective &D) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return;
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
// Emit inits for the linear variables.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPLinearClause>()) {
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
for (auto *Init : C->inits()) {
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(Init)->getDecl());
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
if (auto *Ref = dyn_cast<DeclRefExpr>(VD->getInit()->IgnoreImpCasts())) {
AutoVarEmission Emission = EmitAutoVarAlloca(*VD);
auto *OrigVD = cast<VarDecl>(Ref->getDecl());
DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
CapturedStmtInfo->lookup(OrigVD) != nullptr,
VD->getInit()->getType(), VK_LValue,
VD->getInit()->getExprLoc());
EmitExprAsInit(&DRE, VD, MakeAddrLValue(Emission.getAllocatedAddress(),
VD->getType()),
/*capturedByInit=*/false);
EmitAutoVarCleanups(Emission);
} else
EmitVarDecl(*VD);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
}
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
// Emit the linear steps for the linear clauses.
// If a step is not constant, it is pre-calculated before the loop.
if (auto CS = cast_or_null<BinaryOperator>(C->getCalcStep()))
if (auto SaveRef = cast<DeclRefExpr>(CS->getLHS())) {
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
EmitVarDecl(*cast<VarDecl>(SaveRef->getDecl()));
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
// Emit calculation of the linear step.
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
EmitIgnoredExpr(CS);
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
}
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
}
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
void CodeGenFunction::EmitOMPLinearClauseFinal(
const OMPLoopDirective &D,
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
const llvm::function_ref<llvm::Value *(CodeGenFunction &)> &CondGen) {
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (!HaveInsertPoint())
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
return;
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
llvm::BasicBlock *DoneBB = nullptr;
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
// Emit the final values of the linear variables.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPLinearClause>()) {
[OPENMP] Code cleanup for capturing of variables in OpenMP regions. llvm-svn: 236821
2015-05-08 18:41:21 +08:00
auto IC = C->varlist_begin();
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
for (auto *F : C->finals()) {
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
if (!DoneBB) {
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (auto *Cond = CondGen(*this)) {
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
// If the first post-update expression is found, emit conditional
// block if it was requested.
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
auto *ThenBB = createBasicBlock(".omp.linear.pu");
DoneBB = createBasicBlock(".omp.linear.pu.done");
Builder.CreateCondBr(Cond, ThenBB, DoneBB);
EmitBlock(ThenBB);
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
}
}
[OPENMP] Code cleanup for capturing of variables in OpenMP regions. llvm-svn: 236821
2015-05-08 18:41:21 +08:00
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IC)->getDecl());
DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
CapturedStmtInfo->lookup(OrigVD) != nullptr,
[OPENMP] Code cleanup for capturing of variables in OpenMP regions. llvm-svn: 236821
2015-05-08 18:41:21 +08:00
(*IC)->getType(), VK_LValue, (*IC)->getExprLoc());
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
Address OrigAddr = EmitLValue(&DRE).getAddress();
CodeGenFunction::OMPPrivateScope VarScope(*this);
VarScope.addPrivate(OrigVD, [OrigAddr]() -> Address { return OrigAddr; });
[OPENMP] Code cleanup for capturing of variables in OpenMP regions. llvm-svn: 236821
2015-05-08 18:41:21 +08:00
(void)VarScope.Privatize();
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
EmitIgnoredExpr(F);
[OPENMP] Code cleanup for capturing of variables in OpenMP regions. llvm-svn: 236821
2015-05-08 18:41:21 +08:00
++IC;
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
}
[OPENMP 4.5] Codegen for data members in 'linear' clause. OpenMP 4.5 allows to use data members in private clauses. Patch adds codegen support for 'linear' clause. llvm-svn: 263002
2016-03-09 17:49:00 +08:00
if (auto *PostUpdate = C->getPostUpdateExpr())
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
EmitIgnoredExpr(PostUpdate);
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
}
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
if (DoneBB)
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
EmitBlock(DoneBB, /*IsFinished=*/true);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
}
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
static void emitAlignedClause(CodeGenFunction &CGF,
const OMPExecutableDirective &D) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!CGF.HaveInsertPoint())
return;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *Clause : D.getClausesOfKind<OMPAlignedClause>()) {
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
unsigned ClauseAlignment = 0;
if (auto AlignmentExpr = Clause->getAlignment()) {
auto AlignmentCI =
cast<llvm::ConstantInt>(CGF.EmitScalarExpr(AlignmentExpr));
ClauseAlignment = static_cast<unsigned>(AlignmentCI->getZExtValue());
[OPENMP] Codegen of the ‘aligned’ clause for the ‘omp simd’ directive. Differential Revision: http://reviews.llvm.org/D5499 llvm-svn: 218660
2014-09-30 13:29:28 +08:00
}
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
for (auto E : Clause->varlists()) {
unsigned Alignment = ClauseAlignment;
if (Alignment == 0) {
// OpenMP [2.8.1, Description]
// If no optional parameter is specified, implementation-defined default
// alignments for SIMD instructions on the target platforms are assumed.
Alignment =
[OPENMP] Introduced type trait "__builtin_omp_required_simd_align" for default simd alignment. Adds type trait "__builtin_omp_required_simd_align" after discussions here http://reviews.llvm.org/D9894 Differential Revision: http://reviews.llvm.org/D10597 llvm-svn: 241237
2015-07-02 11:40:19 +08:00
CGF.getContext()
.toCharUnitsFromBits(CGF.getContext().getOpenMPDefaultSimdAlign(
E->getType()->getPointeeType()))
.getQuantity();
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
}
assert((Alignment == 0 || llvm::isPowerOf2_32(Alignment)) &&
"alignment is not power of 2");
if (Alignment != 0) {
llvm::Value *PtrValue = CGF.EmitScalarExpr(E);
CGF.EmitAlignmentAssumption(PtrValue, Alignment);
}
[OPENMP] Codegen of the ‘aligned’ clause for the ‘omp simd’ directive. Differential Revision: http://reviews.llvm.org/D5499 llvm-svn: 218660
2014-09-30 13:29:28 +08:00
}
}
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
void CodeGenFunction::EmitOMPPrivateLoopCounters(
const OMPLoopDirective &S, CodeGenFunction::OMPPrivateScope &LoopScope) {
if (!HaveInsertPoint())
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
return;
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
auto I = S.private_counters().begin();
for (auto *E : S.counters()) {
[OPENMP 4.1] Allow references in init expression for loop-based constructs. OpenMP 4.1 allows to use variables with reference types in private clauses and, therefore, in init expressions of the cannonical loop forms. llvm-svn: 244209
2015-08-06 20:30:57 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
auto *PrivateVD = cast<VarDecl>(cast<DeclRefExpr>(*I)->getDecl());
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
(void)LoopScope.addPrivate(VD, [&]() -> Address {
Code improvements in OpenMP CodeGen. This patch makes class OMPPrivateScope a common class for all private variables. Reworked processing of firstprivate variables (now it is based on OMPPrivateScope too). llvm-svn: 219486
2014-10-10 17:48:26 +08:00
// Emit var without initialization.
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (!LocalDeclMap.count(PrivateVD)) {
auto VarEmission = EmitAutoVarAlloca(*PrivateVD);
EmitAutoVarCleanups(VarEmission);
}
DeclRefExpr DRE(const_cast<VarDecl *>(PrivateVD),
/*RefersToEnclosingVariableOrCapture=*/false,
(*I)->getType(), VK_LValue, (*I)->getExprLoc());
return EmitLValue(&DRE).getAddress();
Code improvements in OpenMP CodeGen. This patch makes class OMPPrivateScope a common class for all private variables. Reworked processing of firstprivate variables (now it is based on OMPPrivateScope too). llvm-svn: 219486
2014-10-10 17:48:26 +08:00
});
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (LocalDeclMap.count(VD) || CapturedStmtInfo->lookup(VD) ||
VD->hasGlobalStorage()) {
(void)LoopScope.addPrivate(PrivateVD, [&]() -> Address {
DeclRefExpr DRE(const_cast<VarDecl *>(VD),
LocalDeclMap.count(VD) || CapturedStmtInfo->lookup(VD),
E->getType(), VK_LValue, E->getExprLoc());
return EmitLValue(&DRE).getAddress();
});
}
[OPENMP 4.1] Allow references in init expression for loop-based constructs. OpenMP 4.1 allows to use variables with reference types in private clauses and, therefore, in init expressions of the cannonical loop forms. llvm-svn: 244209
2015-08-06 20:30:57 +08:00
++I;
Code improvements in OpenMP CodeGen. This patch makes class OMPPrivateScope a common class for all private variables. Reworked processing of firstprivate variables (now it is based on OMPPrivateScope too). llvm-svn: 219486
2014-10-10 17:48:26 +08:00
}
}
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
static void emitPreCond(CodeGenFunction &CGF, const OMPLoopDirective &S,
const Expr *Cond, llvm::BasicBlock *TrueBlock,
llvm::BasicBlock *FalseBlock, uint64_t TrueCount) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!CGF.HaveInsertPoint())
return;
[OPENMP] Fox for http://llvm.org/PR23663: OpenMP crash Destroy RuntimeCleanupScope before generation of termination instruction in parallel loop precondition. llvm-svn: 239524
2015-06-11 18:53:56 +08:00
{
CodeGenFunction::OMPPrivateScope PreCondScope(CGF);
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
CGF.EmitOMPPrivateLoopCounters(S, PreCondScope);
[OPENMP] Fox for http://llvm.org/PR23663: OpenMP crash Destroy RuntimeCleanupScope before generation of termination instruction in parallel loop precondition. llvm-svn: 239524
2015-06-11 18:53:56 +08:00
(void)PreCondScope.Privatize();
// Get initial values of real counters.
[OPENMP] Fix for http://llvm.org/PR24371: Assert failure compiling blender 2.75. blender uses statements expression in condition of the loop under control of the '#pragma omp parallel for'. This condition is used several times in different expressions required for codegen of the loop directive. If there are some variables defined in statement expression, it fires an assert during codegen because of redefinition of the same variables. We have to rebuild several expression to be sure that all variables are unique. llvm-svn: 245041
2015-08-14 20:25:37 +08:00
for (auto I : S.inits()) {
[OPENMP] Fox for http://llvm.org/PR23663: OpenMP crash Destroy RuntimeCleanupScope before generation of termination instruction in parallel loop precondition. llvm-svn: 239524
2015-06-11 18:53:56 +08:00
CGF.EmitIgnoredExpr(I);
}
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
}
// Check that loop is executed at least one time.
CGF.EmitBranchOnBoolExpr(Cond, TrueBlock, FalseBlock, TrueCount);
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
void CodeGenFunction::EmitOMPLinearClause(
const OMPLoopDirective &D, CodeGenFunction::OMPPrivateScope &PrivateScope) {
if (!HaveInsertPoint())
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
return;
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
llvm::DenseSet<const VarDecl *> SIMDLCVs;
if (isOpenMPSimdDirective(D.getDirectiveKind())) {
auto *LoopDirective = cast<OMPLoopDirective>(&D);
for (auto *C : LoopDirective->counters()) {
SIMDLCVs.insert(
cast<VarDecl>(cast<DeclRefExpr>(C)->getDecl())->getCanonicalDecl());
}
}
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : D.getClausesOfKind<OMPLinearClause>()) {
[OPENMP 4.1] Allow variables with reference types in private clauses. OpenMP 4.1 allows to use variables with reference types in all private clauses (private, firstprivate, lastprivate, linear etc.). Patch allows to use such variables and fixes codegen for linear variables with reference types. llvm-svn: 245268
2015-08-18 14:47:21 +08:00
auto CurPrivate = C->privates().begin();
[OPENMP] Simplified iteration over clauses, NFC. llvm-svn: 235838
2015-04-27 16:00:32 +08:00
for (auto *E : C->varlists()) {
[OPENMP 4.1] Allow variables with reference types in private clauses. OpenMP 4.1 allows to use variables with reference types in all private clauses (private, firstprivate, lastprivate, linear etc.). Patch allows to use such variables and fixes codegen for linear variables with reference types. llvm-svn: 245268
2015-08-18 14:47:21 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
auto *PrivateVD =
cast<VarDecl>(cast<DeclRefExpr>(*CurPrivate)->getDecl());
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (!SIMDLCVs.count(VD->getCanonicalDecl())) {
bool IsRegistered = PrivateScope.addPrivate(VD, [&]() -> Address {
// Emit private VarDecl with copy init.
EmitVarDecl(*PrivateVD);
return GetAddrOfLocalVar(PrivateVD);
});
assert(IsRegistered && "linear var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
} else
EmitVarDecl(*PrivateVD);
[OPENMP 4.1] Allow variables with reference types in private clauses. OpenMP 4.1 allows to use variables with reference types in all private clauses (private, firstprivate, lastprivate, linear etc.). Patch allows to use such variables and fixes codegen for linear variables with reference types. llvm-svn: 245268
2015-08-18 14:47:21 +08:00
++CurPrivate;
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
}
}
}
[OPENMP 4.1] Add codegen for 'simdlen' clause. Add emission of metadata for simd loops in presence of 'simdlen' clause. If 'simdlen' clause is provided without 'safelen' clause, the vectorizer width for the loop is set to value of 'simdlen' clause + all read/write ops in loop are marked with '!llvm.mem.parallel_loop_access' metadata. If 'simdlen' clause is provided along with 'safelen' clause, the vectorizer width for the loop is set to value of 'simdlen' clause + all read/write ops in loop are not marked with '!llvm.mem.parallel_loop_access' metadata. If 'safelen' clause is provided without 'simdlen' clause, the vectorizer width for the loop is set to value of 'safelen' clause + all read/write ops in loop are not marked with '!llvm.mem.parallel_loop_access' metadata. llvm-svn: 245697
2015-08-21 20:19:04 +08:00
static void emitSimdlenSafelenClause(CodeGenFunction &CGF,
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
const OMPExecutableDirective &D,
bool IsMonotonic) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!CGF.HaveInsertPoint())
return;
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (const auto *C = D.getSingleClause<OMPSimdlenClause>()) {
[OPENMP 4.1] Add codegen for 'simdlen' clause. Add emission of metadata for simd loops in presence of 'simdlen' clause. If 'simdlen' clause is provided without 'safelen' clause, the vectorizer width for the loop is set to value of 'simdlen' clause + all read/write ops in loop are marked with '!llvm.mem.parallel_loop_access' metadata. If 'simdlen' clause is provided along with 'safelen' clause, the vectorizer width for the loop is set to value of 'simdlen' clause + all read/write ops in loop are not marked with '!llvm.mem.parallel_loop_access' metadata. If 'safelen' clause is provided without 'simdlen' clause, the vectorizer width for the loop is set to value of 'safelen' clause + all read/write ops in loop are not marked with '!llvm.mem.parallel_loop_access' metadata. llvm-svn: 245697
2015-08-21 20:19:04 +08:00
RValue Len = CGF.EmitAnyExpr(C->getSimdlen(), AggValueSlot::ignored(),
/*ignoreResult=*/true);
llvm::ConstantInt *Val = cast<llvm::ConstantInt>(Len.getScalarVal());
CGF.LoopStack.setVectorizeWidth(Val->getZExtValue());
// In presence of finite 'safelen', it may be unsafe to mark all
// the memory instructions parallel, because loop-carried
// dependences of 'safelen' iterations are possible.
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
if (!IsMonotonic)
CGF.LoopStack.setParallel(!D.getSingleClause<OMPSafelenClause>());
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
} else if (const auto *C = D.getSingleClause<OMPSafelenClause>()) {
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
RValue Len = CGF.EmitAnyExpr(C->getSafelen(), AggValueSlot::ignored(),
/*ignoreResult=*/true);
llvm::ConstantInt *Val = cast<llvm::ConstantInt>(Len.getScalarVal());
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
CGF.LoopStack.setVectorizeWidth(Val->getZExtValue());
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
// In presence of finite 'safelen', it may be unsafe to mark all
// the memory instructions parallel, because loop-carried
// dependences of 'safelen' iterations are possible.
CGF.LoopStack.setParallel(false);
}
}
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
void CodeGenFunction::EmitOMPSimdInit(const OMPLoopDirective &D,
bool IsMonotonic) {
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
// Walk clauses and process safelen/lastprivate.
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
LoopStack.setParallel(!IsMonotonic);
Make the variable names match the name of the metadata they control. Rename Vectorizer to Vectorize and VectorizeUnroll to InterleaveCount. llvm-svn: 242241
2015-07-15 07:03:09 +08:00
LoopStack.setVectorizeEnable(true);
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
emitSimdlenSafelenClause(*this, D, IsMonotonic);
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
}
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
void CodeGenFunction::EmitOMPSimdFinal(
const OMPLoopDirective &D,
const llvm::function_ref<llvm::Value *(CodeGenFunction &)> &CondGen) {
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (!HaveInsertPoint())
return;
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
llvm::BasicBlock *DoneBB = nullptr;
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
auto IC = D.counters().begin();
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
auto IPC = D.private_counters().begin();
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
for (auto F : D.finals()) {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>((*IC))->getDecl());
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
auto *PrivateVD = cast<VarDecl>(cast<DeclRefExpr>((*IPC))->getDecl());
auto *CED = dyn_cast<OMPCapturedExprDecl>(OrigVD);
if (LocalDeclMap.count(OrigVD) || CapturedStmtInfo->lookup(OrigVD) ||
OrigVD->hasGlobalStorage() || CED) {
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
if (!DoneBB) {
if (auto *Cond = CondGen(*this)) {
// If the first post-update expression is found, emit conditional
// block if it was requested.
auto *ThenBB = createBasicBlock(".omp.final.then");
DoneBB = createBasicBlock(".omp.final.done");
Builder.CreateCondBr(Cond, ThenBB, DoneBB);
EmitBlock(ThenBB);
}
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
Address OrigAddr = Address::invalid();
if (CED)
OrigAddr = EmitLValue(CED->getInit()->IgnoreImpCasts()).getAddress();
else {
DeclRefExpr DRE(const_cast<VarDecl *>(PrivateVD),
/*RefersToEnclosingVariableOrCapture=*/false,
(*IPC)->getType(), VK_LValue, (*IPC)->getExprLoc());
OrigAddr = EmitLValue(&DRE).getAddress();
}
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
OMPPrivateScope VarScope(*this);
VarScope.addPrivate(OrigVD,
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
[OrigAddr]() -> Address { return OrigAddr; });
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
(void)VarScope.Privatize();
EmitIgnoredExpr(F);
}
++IC;
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
++IPC;
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
}
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
if (DoneBB)
EmitBlock(DoneBB, /*IsFinished=*/true);
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
static void emitOMPLoopBodyWithStopPoint(CodeGenFunction &CGF,
const OMPLoopDirective &S,
CodeGenFunction::JumpDest LoopExit) {
CGF.EmitOMPLoopBody(S, LoopExit);
CGF.EmitStopPoint(&S);
Fix -Wpedantic about extra semicolons in CGStmtOpenMP.cpp llvm-svn: 301564
2017-04-28 01:02:25 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
void CodeGenFunction::EmitOMPSimdDirective(const OMPSimdDirective &S) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
[OPENMP] Remove extra code transformation. For better support of some specific GNU extensions some extra transformation of AST nodes were introduced. These transformations are very hard to handle. The code is improved in handling of these extensions by using captured expressions construct. llvm-svn: 264709
2016-03-29 16:58:54 +08:00
OMPLoopScope PreInitScope(CGF, S);
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
// if (PreCond) {
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
// for (IV in 0..LastIteration) BODY;
// <Final counter/linear vars updates>;
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
// }
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
//
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
// Emit: if (PreCond) - begin.
// If the condition constant folds and can be elided, avoid emitting the
// whole loop.
bool CondConstant;
llvm::BasicBlock *ContBlock = nullptr;
if (CGF.ConstantFoldsToSimpleInteger(S.getPreCond(), CondConstant)) {
if (!CondConstant)
return;
} else {
auto *ThenBlock = CGF.createBasicBlock("simd.if.then");
ContBlock = CGF.createBasicBlock("simd.if.end");
InstrProf: Stop using RegionCounter outside of CodeGenPGO (NFC) The RegionCounter type does a lot of legwork, but most of it is only meaningful within the implementation of CodeGenPGO. The uses elsewhere in CodeGen generally just want to increment or read counters, so do that directly. llvm-svn: 235664
2015-04-24 07:06:47 +08:00
emitPreCond(CGF, S, S.getPreCond(), ThenBlock, ContBlock,
CGF.getProfileCount(&S));
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
CGF.EmitBlock(ThenBlock);
InstrProf: Stop using RegionCounter outside of CodeGenPGO (NFC) The RegionCounter type does a lot of legwork, but most of it is only meaningful within the implementation of CodeGenPGO. The uses elsewhere in CodeGen generally just want to increment or read counters, so do that directly. llvm-svn: 235664
2015-04-24 07:06:47 +08:00
CGF.incrementProfileCounter(&S);
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
}
[OPENMP] CodeGen of the 'linear' clause for the 'omp simd' directive. The linear variable is privatized (similar to 'private') and its value on current iteration is calculated, similar to the loop counter variables. Differential revision: http://reviews.llvm.org/D8375 llvm-svn: 232890
2015-03-21 18:12:56 +08:00
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
// Emit the loop iteration variable.
const Expr *IVExpr = S.getIterationVariable();
const VarDecl *IVDecl = cast<VarDecl>(cast<DeclRefExpr>(IVExpr)->getDecl());
CGF.EmitVarDecl(*IVDecl);
CGF.EmitIgnoredExpr(S.getInit());
// Emit the iterations count variable.
// If it is not a variable, Sema decided to calculate iterations count on
[OPENMP] Fixed codegen for lastprivate LCV in worksharing constructs. If loop control variable in a worksharing construct is marked as lastprivate, we should copy last calculated value of private counter back to original variable. llvm-svn: 237879
2015-05-21 15:59:51 +08:00
// each iteration (e.g., it is foldable into a constant).
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
if (auto LIExpr = dyn_cast<DeclRefExpr>(S.getLastIteration())) {
CGF.EmitVarDecl(*cast<VarDecl>(LIExpr->getDecl()));
// Emit calculation of the iterations count.
CGF.EmitIgnoredExpr(S.getCalcLastIteration());
}
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
CGF.EmitOMPSimdInit(S);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
emitAlignedClause(CGF, S);
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
CGF.EmitOMPLinearClauseInit(S);
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
{
OMPPrivateScope LoopScope(CGF);
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
CGF.EmitOMPPrivateLoopCounters(S, LoopScope);
CGF.EmitOMPLinearClause(S, LoopScope);
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
CGF.EmitOMPPrivateClause(S, LoopScope);
[OPENMP] Supported reduction clause in omp simd construct. The following code is generated for reduction clause within 'omp simd' loop construct: #pragma omp simd reduction(op:var) for (...) <body> alloca priv_var priv_var = <initial reduction value>; <loop_start>: <body> // references to original 'var' are replaced by 'priv_var' <loop_end>: var op= priv_var; llvm-svn: 239881
2015-06-17 14:21:39 +08:00
CGF.EmitOMPReductionClauseInit(S, LoopScope);
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
bool HasLastprivateClause =
CGF.EmitOMPLastprivateClauseInit(S, LoopScope);
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
(void)LoopScope.Privatize();
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
CGF.EmitOMPInnerLoop(S, LoopScope.requiresCleanups(), S.getCond(),
S.getInc(),
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
[&S](CodeGenFunction &CGF) {
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
CGF.EmitOMPLoopBody(S, JumpDest());
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
CGF.EmitStopPoint(&S);
},
[](CodeGenFunction &) {});
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
CGF.EmitOMPSimdFinal(
S, [](CodeGenFunction &) -> llvm::Value * { return nullptr; });
[OPENMP] Support lastprivate clause in omp simd directive. Added codegen for lastprivate clauses within simd loop-based directives. llvm-svn: 239813
2015-06-16 21:14:42 +08:00
// Emit final copy of the lastprivate variables at the end of loops.
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
if (HasLastprivateClause)
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
CGF.EmitOMPLastprivateClauseFinal(S, /*NoFinals=*/true);
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
CGF.EmitOMPReductionClauseFinal(S, /*ReductionKind=*/OMPD_simd);
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
emitPostUpdateForReductionClause(
CGF, S, [](CodeGenFunction &) -> llvm::Value * { return nullptr; });
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
CGF.EmitOMPLinearClauseFinal(
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
S, [](CodeGenFunction &) -> llvm::Value * { return nullptr; });
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
// Emit: if (PreCond) - end.
if (ContBlock) {
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitBranch(ContBlock);
CGF.EmitBlock(ContBlock, true);
[OPENMP] Loop collapsing and codegen for 'omp simd' directive. This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature. Differential Revision: http://reviews.llvm.org/D5184 llvm-svn: 218743
2014-10-01 14:03:56 +08:00
}
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP 4.0] Fixed codegen for 'cancellation point' construct. Generate the next code for 'cancellation point': if (__kmpc_cancellationpoint()) { __kmpc_cancel_barrier(); <exit construct>; } llvm-svn: 241336
2015-07-03 17:56:58 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_simd, CodeGen);
This patch adds a helper class (CGLoopInfo) for marking memory instructions with llvm.mem.parallel_loop_access metadata. It also adds a simple initial version of codegen for pragma omp simd (it will change in the future to support all the clauses). Differential revision: http://reviews.llvm.org/D3644 llvm-svn: 209411
2014-05-22 16:54:05 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
void CodeGenFunction::EmitOMPOuterLoop(
bool DynamicOrOrdered, bool IsMonotonic, const OMPLoopDirective &S,
CodeGenFunction::OMPPrivateScope &LoopScope,
const CodeGenFunction::OMPLoopArguments &LoopArgs,
const CodeGenFunction::CodeGenLoopTy &CodeGenLoop,
const CodeGenFunction::CodeGenOrderedTy &CodeGenOrdered) {
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
auto &RT = CGM.getOpenMPRuntime();
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
const Expr *IVExpr = S.getIterationVariable();
const unsigned IVSize = getContext().getTypeSize(IVExpr->getType());
const bool IVSigned = IVExpr->getType()->hasSignedIntegerRepresentation();
auto LoopExit = getJumpDestInCurrentScope("omp.dispatch.end");
// Start the loop with a block that tests the condition.
auto CondBlock = createBasicBlock("omp.dispatch.cond");
EmitBlock(CondBlock);
Add the loop end location to the loop metadata. This additional information can be used to improve the locations when generating remarks for loops. Depends on the companion LLVM change r286227. Patch by Florian Hahn. Differential Revision: https://reviews.llvm.org/D25764 llvm-svn: 286456
2016-11-10 22:44:30 +08:00
const SourceRange &R = S.getSourceRange();
LoopStack.push(CondBlock, SourceLocToDebugLoc(R.getBegin()),
SourceLocToDebugLoc(R.getEnd()));
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
llvm::Value *BoolCondVal = nullptr;
[OPENMP] Fix codegen for ordered loop directives. loops with ordered clause must be generated the same way as dynamic loops, but with static scheduleing. llvm-svn: 237788
2015-05-20 21:12:48 +08:00
if (!DynamicOrOrdered) {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
// UB = min(UB, GlobalUB) or
// UB = min(UB, PrevUB) for combined loop sharing constructs (e.g.
// 'distribute parallel for')
EmitIgnoredExpr(LoopArgs.EUB);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
// IV = LB
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
EmitIgnoredExpr(LoopArgs.Init);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
// IV < UB
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
BoolCondVal = EvaluateExprAsBool(LoopArgs.Cond);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
} else {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
BoolCondVal =
RT.emitForNext(*this, S.getLocStart(), IVSize, IVSigned, LoopArgs.IL,
LoopArgs.LB, LoopArgs.UB, LoopArgs.ST);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
}
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
// If there are any cleanups between here and the loop-exit scope,
// create a block to stage a loop exit along.
auto ExitBlock = LoopExit.getBlock();
if (LoopScope.requiresCleanups())
ExitBlock = createBasicBlock("omp.dispatch.cleanup");
auto LoopBody = createBasicBlock("omp.dispatch.body");
Builder.CreateCondBr(BoolCondVal, LoopBody, ExitBlock);
if (ExitBlock != LoopExit.getBlock()) {
EmitBlock(ExitBlock);
EmitBranchThroughCleanup(LoopExit);
}
EmitBlock(LoopBody);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
// Emit "IV = LB" (in case of static schedule, we have already calculated new
// LB for loop condition and emitted it above).
[OPENMP] Fix codegen for ordered loop directives. loops with ordered clause must be generated the same way as dynamic loops, but with static scheduleing. llvm-svn: 237788
2015-05-20 21:12:48 +08:00
if (DynamicOrOrdered)
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
EmitIgnoredExpr(LoopArgs.Init);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
// Create a block for the increment.
auto Continue = getJumpDestInCurrentScope("omp.dispatch.inc");
BreakContinueStack.push_back(BreakContinue(LoopExit, Continue));
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
// Generate !llvm.loop.parallel metadata for loads and stores for loops
// with dynamic/guided scheduling and without ordered clause.
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
if (!isOpenMPSimdDirective(S.getDirectiveKind()))
LoopStack.setParallel(!IsMonotonic);
else
EmitOMPSimdInit(S, IsMonotonic);
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
[OPENMP] Codegen for 'ordered' directive. Add codegen for 'ordered' directive: __kmpc_ordered(ident_t *, gtid); <associated statement>; __kmpc_end_ordered(ident_t *, gtid); Also for 'for' directives with the dynamic scheduling and an 'ordered' clause added a call to '__kmpc_dispatch_fini_(4|8)[u]()' function after increment expression for loop control variable: while(__kmpc_dispatch_next(&LB, &UB)) { idx = LB; while (idx <= UB) { BODY; ++idx; __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only. } // inner loop } Differential Revision: http://reviews.llvm.org/D9070 llvm-svn: 235496
2015-04-22 19:15:40 +08:00
SourceLocation Loc = S.getLocStart();
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
// when 'distribute' is not combined with a 'for':
// while (idx <= UB) { BODY; ++idx; }
// when 'distribute' is combined with a 'for'
// (e.g. 'distribute parallel for')
// while (idx <= UB) { <CodeGen rest of pragma>; idx += ST; }
EmitOMPInnerLoop(
S, LoopScope.requiresCleanups(), LoopArgs.Cond, LoopArgs.IncExpr,
[&S, LoopExit, &CodeGenLoop](CodeGenFunction &CGF) {
CodeGenLoop(CGF, S, LoopExit);
},
[IVSize, IVSigned, Loc, &CodeGenOrdered](CodeGenFunction &CGF) {
CodeGenOrdered(CGF, Loc, IVSize, IVSigned);
});
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
EmitBlock(Continue.getBlock());
BreakContinueStack.pop_back();
[OPENMP] Fix codegen for ordered loop directives. loops with ordered clause must be generated the same way as dynamic loops, but with static scheduleing. llvm-svn: 237788
2015-05-20 21:12:48 +08:00
if (!DynamicOrOrdered) {
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
// Emit "LB = LB + Stride", "UB = UB + Stride".
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
EmitIgnoredExpr(LoopArgs.NextLB);
EmitIgnoredExpr(LoopArgs.NextUB);
[OPENMP] CodeGen - 'omp for' with dynamic schedule kinds. Differential Revision: http://reviews.llvm.org/D7138 llvm-svn: 232036
2015-03-12 21:37:50 +08:00
}
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
EmitBranch(CondBlock);
LoopStack.pop();
// Emit the fall-through block.
EmitBlock(LoopExit.getBlock());
// Tell the runtime we are done.
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
auto &&CodeGen = [DynamicOrOrdered, &S](CodeGenFunction &CGF) {
if (!DynamicOrOrdered)
CGF.CGM.getOpenMPRuntime().emitForStaticFinish(CGF, S.getLocEnd());
};
OMPCancelStack.emitExit(*this, S.getDirectiveKind(), CodeGen);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
}
void CodeGenFunction::EmitOMPForOuterLoop(
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
const OpenMPScheduleTy &ScheduleKind, bool IsMonotonic,
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
const OMPLoopDirective &S, OMPPrivateScope &LoopScope, bool Ordered,
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
const OMPLoopArguments &LoopArgs,
const CodeGenDispatchBoundsTy &CGDispatchBounds) {
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
auto &RT = CGM.getOpenMPRuntime();
// Dynamic scheduling of the outer loop (dynamic, guided, auto, runtime).
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
const bool DynamicOrOrdered =
Ordered || RT.isDynamic(ScheduleKind.Schedule);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
assert((Ordered ||
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
!RT.isStaticNonchunked(ScheduleKind.Schedule,
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
LoopArgs.Chunk != nullptr)) &&
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
"static non-chunked schedule does not need outer loop");
// Emit outer loop.
//
// OpenMP [2.7.1, Loop Construct, Description, table 2-1]
// When schedule(dynamic,chunk_size) is specified, the iterations are
// distributed to threads in the team in chunks as the threads request them.
// Each thread executes a chunk of iterations, then requests another chunk,
// until no chunks remain to be distributed. Each chunk contains chunk_size
// iterations, except for the last chunk to be distributed, which may have
// fewer iterations. When no chunk_size is specified, it defaults to 1.
//
// When schedule(guided,chunk_size) is specified, the iterations are assigned
// to threads in the team in chunks as the executing threads request them.
// Each thread executes a chunk of iterations, then requests another chunk,
// until no chunks remain to be assigned. For a chunk_size of 1, the size of
// each chunk is proportional to the number of unassigned iterations divided
// by the number of threads in the team, decreasing to 1. For a chunk_size
// with value k (greater than 1), the size of each chunk is determined in the
// same way, with the restriction that the chunks do not contain fewer than k
// iterations (except for the last chunk to be assigned, which may have fewer
// than k iterations).
//
// When schedule(auto) is specified, the decision regarding scheduling is
// delegated to the compiler and/or runtime system. The programmer gives the
// implementation the freedom to choose any possible mapping of iterations to
// threads in the team.
//
// When schedule(runtime) is specified, the decision regarding scheduling is
// deferred until run time, and the schedule and chunk size are taken from the
// run-sched-var ICV. If the ICV is set to auto, the schedule is
// implementation defined
//
// while(__kmpc_dispatch_next(&LB, &UB)) {
// idx = LB;
// while (idx <= UB) { BODY; ++idx;
// __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only.
// } // inner loop
// }
//
// OpenMP [2.7.1, Loop Construct, Description, table 2-1]
// When schedule(static, chunk_size) is specified, iterations are divided into
// chunks of size chunk_size, and the chunks are assigned to the threads in
// the team in a round-robin fashion in the order of the thread number.
//
// while(UB = min(UB, GlobalUB), idx = LB, idx < UB) {
// while (idx <= UB) { BODY; ++idx; } // inner loop
// LB = LB + ST;
// UB = UB + ST;
// }
//
const Expr *IVExpr = S.getIterationVariable();
const unsigned IVSize = getContext().getTypeSize(IVExpr->getType());
const bool IVSigned = IVExpr->getType()->hasSignedIntegerRepresentation();
if (DynamicOrOrdered) {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
auto DispatchBounds = CGDispatchBounds(*this, S, LoopArgs.LB, LoopArgs.UB);
llvm::Value *LBVal = DispatchBounds.first;
llvm::Value *UBVal = DispatchBounds.second;
CGOpenMPRuntime::DispatchRTInput DipatchRTInputValues = {LBVal, UBVal,
LoopArgs.Chunk};
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
RT.emitForDispatchInit(*this, S.getLocStart(), ScheduleKind, IVSize,
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
IVSigned, Ordered, DipatchRTInputValues);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
} else {
RT.emitForStaticInit(*this, S.getLocStart(), ScheduleKind, IVSize, IVSigned,
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
Ordered, LoopArgs.IL, LoopArgs.LB, LoopArgs.UB,
LoopArgs.ST, LoopArgs.Chunk);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
auto &&CodeGenOrdered = [Ordered](CodeGenFunction &CGF, SourceLocation Loc,
const unsigned IVSize,
const bool IVSigned) {
if (Ordered) {
CGF.CGM.getOpenMPRuntime().emitForOrderedIterationEnd(CGF, Loc, IVSize,
IVSigned);
}
};
OMPLoopArguments OuterLoopArgs(LoopArgs.LB, LoopArgs.UB, LoopArgs.ST,
LoopArgs.IL, LoopArgs.Chunk, LoopArgs.EUB);
OuterLoopArgs.IncExpr = S.getInc();
OuterLoopArgs.Init = S.getInit();
OuterLoopArgs.Cond = S.getCond();
OuterLoopArgs.NextLB = S.getNextLowerBound();
OuterLoopArgs.NextUB = S.getNextUpperBound();
EmitOMPOuterLoop(DynamicOrOrdered, IsMonotonic, S, LoopScope, OuterLoopArgs,
emitOMPLoopBodyWithStopPoint, CodeGenOrdered);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
static void emitEmptyOrdered(CodeGenFunction &, SourceLocation Loc,
const unsigned IVSize, const bool IVSigned) {}
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
void CodeGenFunction::EmitOMPDistributeOuterLoop(
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
OpenMPDistScheduleClauseKind ScheduleKind, const OMPLoopDirective &S,
OMPPrivateScope &LoopScope, const OMPLoopArguments &LoopArgs,
const CodeGenLoopTy &CodeGenLoopContent) {
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
auto &RT = CGM.getOpenMPRuntime();
// Emit outer loop.
// Same behavior as a OMPForOuterLoop, except that schedule cannot be
// dynamic
//
const Expr *IVExpr = S.getIterationVariable();
const unsigned IVSize = getContext().getTypeSize(IVExpr->getType());
const bool IVSigned = IVExpr->getType()->hasSignedIntegerRepresentation();
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
RT.emitDistributeStaticInit(*this, S.getLocStart(), ScheduleKind, IVSize,
IVSigned, /* Ordered = */ false, LoopArgs.IL,
LoopArgs.LB, LoopArgs.UB, LoopArgs.ST,
LoopArgs.Chunk);
// for combined 'distribute' and 'for' the increment expression of distribute
// is store in DistInc. For 'distribute' alone, it is in Inc.
Expr *IncExpr;
if (isOpenMPLoopBoundSharingDirective(S.getDirectiveKind()))
IncExpr = S.getDistInc();
else
IncExpr = S.getInc();
// this routine is shared by 'omp distribute parallel for' and
// 'omp distribute': select the right EUB expression depending on the
// directive
OMPLoopArguments OuterLoopArgs;
OuterLoopArgs.LB = LoopArgs.LB;
OuterLoopArgs.UB = LoopArgs.UB;
OuterLoopArgs.ST = LoopArgs.ST;
OuterLoopArgs.IL = LoopArgs.IL;
OuterLoopArgs.Chunk = LoopArgs.Chunk;
OuterLoopArgs.EUB = isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedEnsureUpperBound()
: S.getEnsureUpperBound();
OuterLoopArgs.IncExpr = IncExpr;
OuterLoopArgs.Init = isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedInit()
: S.getInit();
OuterLoopArgs.Cond = isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedCond()
: S.getCond();
OuterLoopArgs.NextLB = isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedNextLowerBound()
: S.getNextLowerBound();
OuterLoopArgs.NextUB = isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedNextUpperBound()
: S.getNextUpperBound();
EmitOMPOuterLoop(/* DynamicOrOrdered = */ false, /* IsMonotonic = */ false, S,
LoopScope, OuterLoopArgs, CodeGenLoopContent,
emitEmptyOrdered);
}
/// Emit a helper variable and return corresponding lvalue.
static LValue EmitOMPHelperVar(CodeGenFunction &CGF,
const DeclRefExpr *Helper) {
auto VDecl = cast<VarDecl>(Helper->getDecl());
CGF.EmitVarDecl(*VDecl);
return CGF.EmitLValue(Helper);
}
static std::pair<LValue, LValue>
emitDistributeParallelForInnerBounds(CodeGenFunction &CGF,
const OMPExecutableDirective &S) {
const OMPLoopDirective &LS = cast<OMPLoopDirective>(S);
LValue LB =
EmitOMPHelperVar(CGF, cast<DeclRefExpr>(LS.getLowerBoundVariable()));
LValue UB =
EmitOMPHelperVar(CGF, cast<DeclRefExpr>(LS.getUpperBoundVariable()));
// When composing 'distribute' with 'for' (e.g. as in 'distribute
// parallel for') we need to use the 'distribute'
// chunk lower and upper bounds rather than the whole loop iteration
// space. These are parameters to the outlined function for 'parallel'
// and we copy the bounds of the previous schedule into the
// the current ones.
LValue PrevLB = CGF.EmitLValue(LS.getPrevLowerBoundVariable());
LValue PrevUB = CGF.EmitLValue(LS.getPrevUpperBoundVariable());
llvm::Value *PrevLBVal = CGF.EmitLoadOfScalar(PrevLB, SourceLocation());
PrevLBVal = CGF.EmitScalarConversion(
PrevLBVal, LS.getPrevLowerBoundVariable()->getType(),
LS.getIterationVariable()->getType(), SourceLocation());
llvm::Value *PrevUBVal = CGF.EmitLoadOfScalar(PrevUB, SourceLocation());
PrevUBVal = CGF.EmitScalarConversion(
PrevUBVal, LS.getPrevUpperBoundVariable()->getType(),
LS.getIterationVariable()->getType(), SourceLocation());
CGF.EmitStoreOfScalar(PrevLBVal, LB);
CGF.EmitStoreOfScalar(PrevUBVal, UB);
return {LB, UB};
}
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
/// if the 'for' loop has a dispatch schedule (e.g. dynamic, guided) then
/// we need to use the LB and UB expressions generated by the worksharing
/// code generation support, whereas in non combined situations we would
/// just emit 0 and the LastIteration expression
/// This function is necessary due to the difference of the LB and UB
/// types for the RT emission routines for 'for_static_init' and
/// 'for_dispatch_init'
static std::pair<llvm::Value *, llvm::Value *>
emitDistributeParallelForDispatchBounds(CodeGenFunction &CGF,
const OMPExecutableDirective &S,
Address LB, Address UB) {
const OMPLoopDirective &LS = cast<OMPLoopDirective>(S);
const Expr *IVExpr = LS.getIterationVariable();
// when implementing a dynamic schedule for a 'for' combined with a
// 'distribute' (e.g. 'distribute parallel for'), the 'for' loop
// is not normalized as each team only executes its own assigned
// distribute chunk
QualType IteratorTy = IVExpr->getType();
llvm::Value *LBVal = CGF.EmitLoadOfScalar(LB, /*Volatile=*/false, IteratorTy,
SourceLocation());
llvm::Value *UBVal = CGF.EmitLoadOfScalar(UB, /*Volatile=*/false, IteratorTy,
SourceLocation());
return {LBVal, UBVal};
Fix -Wpedantic about extra semicolons in CGStmtOpenMP.cpp llvm-svn: 301564
2017-04-28 01:02:25 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
static void emitDistributeParallelForDistributeInnerBoundParams(
CodeGenFunction &CGF, const OMPExecutableDirective &S,
llvm::SmallVectorImpl<llvm::Value *> &CapturedVars) {
const auto &Dir = cast<OMPLoopDirective>(S);
LValue LB =
CGF.EmitLValue(cast<DeclRefExpr>(Dir.getCombinedLowerBoundVariable()));
auto LBCast = CGF.Builder.CreateIntCast(
CGF.Builder.CreateLoad(LB.getAddress()), CGF.SizeTy, /*isSigned=*/false);
CapturedVars.push_back(LBCast);
LValue UB =
CGF.EmitLValue(cast<DeclRefExpr>(Dir.getCombinedUpperBoundVariable()));
auto UBCast = CGF.Builder.CreateIntCast(
CGF.Builder.CreateLoad(UB.getAddress()), CGF.SizeTy, /*isSigned=*/false);
CapturedVars.push_back(UBCast);
Fix -Wpedantic about extra semicolons in CGStmtOpenMP.cpp llvm-svn: 301564
2017-04-28 01:02:25 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
static void
emitInnerParallelForWhenCombined(CodeGenFunction &CGF,
const OMPLoopDirective &S,
CodeGenFunction::JumpDest LoopExit) {
auto &&CGInlinedWorksharingLoop = [&S](CodeGenFunction &CGF,
PrePostActionTy &) {
CGF.EmitOMPWorksharingLoop(S, S.getPrevEnsureUpperBound(),
emitDistributeParallelForInnerBounds,
emitDistributeParallelForDispatchBounds);
};
emitCommonOMPParallelDirective(
CGF, S, OMPD_for, CGInlinedWorksharingLoop,
emitDistributeParallelForDistributeInnerBoundParams);
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
}
Resubmission of http://reviews.llvm.org/D21564 after fixes. [OpenMP] Initial implementation of parse and sema for composite pragma 'distribute parallel for' This patch is an initial implementation for #distribute parallel for. The main differences that affect other pragmas are: The implementation of 'distribute parallel for' requires blocking of the associated loop, where blocks are "distributed" to different teams and iterations within each block are scheduled to parallel threads within each team. To implement blocking, sema creates two additional worksharing directive fields that are used to pass the team assigned block lower and upper bounds through the outlined function resulting from 'parallel'. In this way, scheduling for 'for' to threads can use those bounds. As a consequence of blocking, the stride of 'distribute' is not 1 but it is equal to the blocking size. This is returned by the runtime and sema prepares a DistIncrExpr variable to hold that value. As a consequence of blocking, the global upper bound (EnsureUpperBound) expression of the 'for' is not the original loop upper bound (e.g. in for(i = 0 ; i < N; i++) this is 'N') but it is the team-assigned block upper bound. Sema creates a new expression holding the calculation of the actual upper bound for 'for' as UB = min(UB, PrevUB), where UB is the loop upper bound, and PrevUB is the team-assigned block upper bound. llvm-svn: 273884
2016-06-27 22:55:37 +08:00
void CodeGenFunction::EmitOMPDistributeParallelForDirective(
const OMPDistributeParallelForDirective &S) {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
CGF.EmitOMPDistributeLoop(S, emitInnerParallelForWhenCombined,
S.getDistInc());
};
Resubmission of http://reviews.llvm.org/D21564 after fixes. [OpenMP] Initial implementation of parse and sema for composite pragma 'distribute parallel for' This patch is an initial implementation for #distribute parallel for. The main differences that affect other pragmas are: The implementation of 'distribute parallel for' requires blocking of the associated loop, where blocks are "distributed" to different teams and iterations within each block are scheduled to parallel threads within each team. To implement blocking, sema creates two additional worksharing directive fields that are used to pass the team assigned block lower and upper bounds through the outlined function resulting from 'parallel'. In this way, scheduling for 'for' to threads can use those bounds. As a consequence of blocking, the stride of 'distribute' is not 1 but it is equal to the blocking size. This is returned by the runtime and sema prepares a DistIncrExpr variable to hold that value. As a consequence of blocking, the global upper bound (EnsureUpperBound) expression of the 'for' is not the original loop upper bound (e.g. in for(i = 0 ; i < N; i++) this is 'N') but it is the team-assigned block upper bound. Sema creates a new expression holding the calculation of the actual upper bound for 'for' as UB = min(UB, PrevUB), where UB is the loop upper bound, and PrevUB is the team-assigned block upper bound. llvm-svn: 273884
2016-06-27 22:55:37 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
OMPCancelStackRAII CancelRegion(*this, OMPD_distribute_parallel_for,
/*HasCancel=*/false);
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_distribute, CodeGen,
/*HasCancel=*/false);
Resubmission of http://reviews.llvm.org/D21564 after fixes. [OpenMP] Initial implementation of parse and sema for composite pragma 'distribute parallel for' This patch is an initial implementation for #distribute parallel for. The main differences that affect other pragmas are: The implementation of 'distribute parallel for' requires blocking of the associated loop, where blocks are "distributed" to different teams and iterations within each block are scheduled to parallel threads within each team. To implement blocking, sema creates two additional worksharing directive fields that are used to pass the team assigned block lower and upper bounds through the outlined function resulting from 'parallel'. In this way, scheduling for 'for' to threads can use those bounds. As a consequence of blocking, the stride of 'distribute' is not 1 but it is equal to the blocking size. This is returned by the runtime and sema prepares a DistIncrExpr variable to hold that value. As a consequence of blocking, the global upper bound (EnsureUpperBound) expression of the 'for' is not the original loop upper bound (e.g. in for(i = 0 ; i < N; i++) this is 'N') but it is the team-assigned block upper bound. Sema creates a new expression holding the calculation of the actual upper bound for 'for' as UB = min(UB, PrevUB), where UB is the loop upper bound, and PrevUB is the team-assigned block upper bound. llvm-svn: 273884
2016-06-27 22:55:37 +08:00
}
[OpenMP] Sema and parse for 'distribute parallel for simd' Summary: This patch is an implementation of sema and parsing for the OpenMP composite pragma 'distribute parallel for simd'. Differential Revision: http://reviews.llvm.org/D21977 llvm-svn: 274530
2016-07-05 13:00:15 +08:00
void CodeGenFunction::EmitOMPDistributeParallelForSimdDirective(
const OMPDistributeParallelForSimdDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_distribute_parallel_for_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'distribute simd' pragma Summary: This patch is an implementation of sema and parsing for the OpenMP composite pragma 'distribute simd'. Differential Revision: http://reviews.llvm.org/D22007 llvm-svn: 274604
2016-07-06 12:45:38 +08:00
void CodeGenFunction::EmitOMPDistributeSimdDirective(
const OMPDistributeSimdDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_distribute_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'target parallel for simd' pragma This patch is to implement sema and parsing for 'target parallel for simd' pragma. Differential Revision: http://reviews.llvm.org/D22096 llvm-svn: 275365
2016-07-14 10:54:56 +08:00
void CodeGenFunction::EmitOMPTargetParallelForSimdDirective(
const OMPTargetParallelForSimdDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_target_parallel_for_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'target simd' pragma This patch is to implement sema and parsing for 'target simd' pragma. Differential Revision: https://reviews.llvm.org/D22479 llvm-svn: 276203
2016-07-21 06:57:10 +08:00
void CodeGenFunction::EmitOMPTargetSimdDirective(
const OMPTargetSimdDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_target_simd, [&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'teams distribute' pragma This patch is to implement sema and parsing for 'teams distribute' pragma. Differential Revision: https://reviews.llvm.org/D23189 llvm-svn: 277818
2016-08-05 22:37:37 +08:00
void CodeGenFunction::EmitOMPTeamsDistributeDirective(
const OMPTeamsDistributeDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_teams_distribute,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
Re-apply patch r279045. llvm-svn: 285066
2016-10-25 20:50:55 +08:00
void CodeGenFunction::EmitOMPTeamsDistributeSimdDirective(
const OMPTeamsDistributeSimdDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_teams_distribute_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'teams distribute parallel for simd' pragma This patch is to implement sema and parsing for 'teams distribute parallel for simd' pragma. Differential Revision: https://reviews.llvm.org/D27084 llvm-svn: 288294
2016-12-01 07:51:03 +08:00
void CodeGenFunction::EmitOMPTeamsDistributeParallelForSimdDirective(
const OMPTeamsDistributeParallelForSimdDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_teams_distribute_parallel_for_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
Re-apply patch r279045. llvm-svn: 285066
2016-10-25 20:50:55 +08:00
[OpenMP] Sema and parsing for 'teams distribute parallel for' pragma This patch is to implement sema and parsing for 'teams distribute parallel for' pragma. Differential Revision: https://reviews.llvm.org/D27345 llvm-svn: 289179
2016-12-09 11:24:30 +08:00
void CodeGenFunction::EmitOMPTeamsDistributeParallelForDirective(
const OMPTeamsDistributeParallelForDirective &S) {
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_teams_distribute_parallel_for,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'target teams distribute' pragma This patch is to implement sema and parsing for 'target teams distribute' pragma. Differential Revision: https://reviews.llvm.org/D28015 llvm-svn: 290508
2016-12-25 12:52:54 +08:00
void CodeGenFunction::EmitOMPTargetTeamsDistributeDirective(
const OMPTargetTeamsDistributeDirective &S) {
Fix format. NFC llvm-svn: 290673
2016-12-29 01:57:07 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_target_teams_distribute,
[OpenMP] Sema and parsing for 'target teams distribute' pragma This patch is to implement sema and parsing for 'target teams distribute' pragma. Differential Revision: https://reviews.llvm.org/D28015 llvm-svn: 290508
2016-12-25 12:52:54 +08:00
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
Fix format. NFC llvm-svn: 290673
2016-12-29 01:57:07 +08:00
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
[OpenMP] Sema and parsing for 'target teams distribute' pragma This patch is to implement sema and parsing for 'target teams distribute' pragma. Differential Revision: https://reviews.llvm.org/D28015 llvm-svn: 290508
2016-12-25 12:52:54 +08:00
});
}
[OpenMP] Sema and parsing for 'target teams distribute parallel for’ pragma This patch is to implement sema and parsing for 'target teams distribute parallel for’ pragma. Differential Revision: https://reviews.llvm.org/D28160 llvm-svn: 290725
2016-12-30 06:16:30 +08:00
void CodeGenFunction::EmitOMPTargetTeamsDistributeParallelForDirective(
const OMPTargetTeamsDistributeParallelForDirective &S) {
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_target_teams_distribute_parallel_for,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OpenMP] Sema and parsing for 'target teams distribute parallel for simd’ pragma This patch is to implement sema and parsing for 'target teams distribute parallel for simd’ pragma. Differential Revision: https://reviews.llvm.org/D28202 llvm-svn: 290862
2017-01-03 13:23:48 +08:00
void CodeGenFunction::EmitOMPTargetTeamsDistributeParallelForSimdDirective(
const OMPTargetTeamsDistributeParallelForSimdDirective &S) {
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_target_teams_distribute_parallel_for_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
[OpenMP] Sema and parsing for 'target teams distribute simd’ pragma This patch is to implement sema and parsing for 'target teams distribute simd’ pragma. Differential Revision: https://reviews.llvm.org/D28252 llvm-svn: 291579
2017-01-11 02:08:18 +08:00
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
void CodeGenFunction::EmitOMPTargetTeamsDistributeSimdDirective(
const OMPTargetTeamsDistributeSimdDirective &S) {
CGM.getOpenMPRuntime().emitInlinedDirective(
*this, OMPD_target_teams_distribute_simd,
[&S](CodeGenFunction &CGF, PrePostActionTy &) {
[OpenMP] Sema and parsing for 'target teams distribute parallel for simd’ pragma This patch is to implement sema and parsing for 'target teams distribute parallel for simd’ pragma. Differential Revision: https://reviews.llvm.org/D28202 llvm-svn: 290862
2017-01-03 13:23:48 +08:00
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
});
}
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
namespace {
struct ScheduleKindModifiersTy {
OpenMPScheduleClauseKind Kind;
OpenMPScheduleClauseModifier M1;
OpenMPScheduleClauseModifier M2;
ScheduleKindModifiersTy(OpenMPScheduleClauseKind Kind,
OpenMPScheduleClauseModifier M1,
OpenMPScheduleClauseModifier M2)
: Kind(Kind), M1(M1), M2(M2) {}
};
} // namespace
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
bool CodeGenFunction::EmitOMPWorksharingLoop(
const OMPLoopDirective &S, Expr *EUB,
const CodeGenLoopBoundsTy &CodeGenLoopBounds,
const CodeGenDispatchBoundsTy &CGDispatchBounds) {
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Emit the loop iteration variable.
auto IVExpr = cast<DeclRefExpr>(S.getIterationVariable());
auto IVDecl = cast<VarDecl>(IVExpr->getDecl());
EmitVarDecl(*IVDecl);
// Emit the iterations count variable.
// If it is not a variable, Sema decided to calculate iterations count on each
// iteration (e.g., it is foldable into a constant).
if (auto LIExpr = dyn_cast<DeclRefExpr>(S.getLastIteration())) {
EmitVarDecl(*cast<VarDecl>(LIExpr->getDecl()));
// Emit calculation of the iterations count.
EmitIgnoredExpr(S.getCalcLastIteration());
}
auto &RT = CGM.getOpenMPRuntime();
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
bool HasLastprivateClause;
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Check pre-condition.
{
[OPENMP] Remove extra code transformation. For better support of some specific GNU extensions some extra transformation of AST nodes were introduced. These transformations are very hard to handle. The code is improved in handling of these extensions by using captured expressions construct. llvm-svn: 264709
2016-03-29 16:58:54 +08:00
OMPLoopScope PreInitScope(*this, S);
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Skip the entire loop if we don't meet the precondition.
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
// If the condition constant folds and can be elided, avoid emitting the
// whole loop.
bool CondConstant;
llvm::BasicBlock *ContBlock = nullptr;
if (ConstantFoldsToSimpleInteger(S.getPreCond(), CondConstant)) {
if (!CondConstant)
return false;
} else {
auto *ThenBlock = createBasicBlock("omp.precond.then");
ContBlock = createBasicBlock("omp.precond.end");
emitPreCond(*this, S, S.getPreCond(), ThenBlock, ContBlock,
InstrProf: Stop using RegionCounter outside of CodeGenPGO (NFC) The RegionCounter type does a lot of legwork, but most of it is only meaningful within the implementation of CodeGenPGO. The uses elsewhere in CodeGen generally just want to increment or read counters, so do that directly. llvm-svn: 235664
2015-04-24 07:06:47 +08:00
getProfileCount(&S));
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
EmitBlock(ThenBlock);
InstrProf: Stop using RegionCounter outside of CodeGenPGO (NFC) The RegionCounter type does a lot of legwork, but most of it is only meaningful within the implementation of CodeGenPGO. The uses elsewhere in CodeGen generally just want to increment or read counters, so do that directly. llvm-svn: 235664
2015-04-24 07:06:47 +08:00
incrementProfileCounter(&S);
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
}
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
[OPENMP 4.5] Codegen for dacross loop synchronization constructs. OpenMP 4.5 adds support for doacross loop synchronization. Patch implements codegen for this construct. llvm-svn: 270690
2016-05-25 20:36:08 +08:00
bool Ordered = false;
if (auto *OrderedClause = S.getSingleClause<OMPOrderedClause>()) {
if (OrderedClause->getNumForLoops())
RT.emitDoacrossInit(*this, S);
else
Ordered = true;
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
llvm::DenseSet<const Expr *> EmittedFinals;
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
emitAlignedClause(*this, S);
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
EmitOMPLinearClauseInit(S);
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
// Emit helper vars inits.
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
std::pair<LValue, LValue> Bounds = CodeGenLoopBounds(*this, S);
LValue LB = Bounds.first;
LValue UB = Bounds.second;
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
LValue ST =
EmitOMPHelperVar(*this, cast<DeclRefExpr>(S.getStrideVariable()));
LValue IL =
EmitOMPHelperVar(*this, cast<DeclRefExpr>(S.getIsLastIterVariable()));
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Emit 'then' code.
{
OMPPrivateScope LoopScope(*this);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
if (EmitOMPFirstprivateClause(S, LoopScope)) {
// Emit implicit barrier to synchronize threads and avoid data races on
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
// initialization of firstprivate variables and post-update of
// lastprivate variables.
[OPENMP] Emit __kmpc_cancel_barrier() and code for 'cancellation point' only if 'cancel' is found. Patch improves codegen for OpenMP constructs. If the OpenMP region does not have internal 'cancel' construct, a call to 'void __kmpc_barrier()' runtime function is generated for all implicit/explicit barriers. If the region has inner 'cancel' directive, then ``` if (__kmpc_cancel_barrier()) exit from outer construct; ``` code is generated. Also, the code for 'canellation point' directive is not generated if parent directive does not have 'cancel' directive. llvm-svn: 247681
2015-09-15 20:52:43 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(
*this, S.getLocStart(), OMPD_unknown, /*EmitChecks=*/false,
/*ForceSimpleCall=*/true);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
}
[OPENMP] Codegen for 'private' clause in 'for' directive. This patch generates helper variables which used as a private copies of the corresponding original variables inside an OpenMP 'for' directive. These generated variables are initialized by default (with the default constructor, if any). In OpenMP region references to original variables are replaced by the references to these private helper variables. Differential Revision: http://reviews.llvm.org/D9106 llvm-svn: 235503
2015-04-22 20:24:45 +08:00
EmitOMPPrivateClause(S, LoopScope);
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
HasLastprivateClause = EmitOMPLastprivateClauseInit(S, LoopScope);
[OPENMP] Codegen for 'reduction' clause in 'for' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { *(Type0*)lhs[0] = ReductionOperation0(*(Type0*)lhs[0], *(Type0*)rhs[0]); ... *(Type<n>-1*)lhs[<n>-1] = ReductionOperation<n>-1(*(Type<n>-1*)lhs[<n>-1], *(Type<n>-1*)rhs[<n>-1]); } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n>-1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: <LHSExprs>[0] = ReductionOperation0(*<LHSExprs>[0], *<RHSExprs>[0]); ... <LHSExprs>[<n>-1] = ReductionOperation<n>-1(*<LHSExprs>[<n>-1], *<RHSExprs>[<n>-1]); __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: Atomic(<LHSExprs>[0] = ReductionOperation0(*<LHSExprs>[0], *<RHSExprs>[0])); ... Atomic(<LHSExprs>[<n>-1] = ReductionOperation<n>-1(*<LHSExprs>[<n>-1], *<RHSExprs>[<n>-1])); break; default:; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D9139 llvm-svn: 235506
2015-04-22 21:43:03 +08:00
EmitOMPReductionClauseInit(S, LoopScope);
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
EmitOMPPrivateLoopCounters(S, LoopScope);
EmitOMPLinearClause(S, LoopScope);
[OPENMP] Enable codegen of the ‘private’ clause for ‘omp simd’ directive llvm-svn: 232353
2015-03-16 15:14:41 +08:00
(void)LoopScope.Privatize();
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Detect the loop schedule kind and chunk.
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
llvm::Value *Chunk = nullptr;
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
OpenMPScheduleTy ScheduleKind;
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
if (auto *C = S.getSingleClause<OMPScheduleClause>()) {
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
ScheduleKind.Schedule = C->getScheduleKind();
ScheduleKind.M1 = C->getFirstScheduleModifier();
ScheduleKind.M2 = C->getSecondScheduleModifier();
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
if (const auto *Ch = C->getChunkSize()) {
Chunk = EmitScalarExpr(Ch);
Chunk = EmitScalarConversion(Chunk, Ch->getType(),
S.getIterationVariable()->getType(),
S.getLocStart());
}
}
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
const unsigned IVSize = getContext().getTypeSize(IVExpr->getType());
const bool IVSigned = IVExpr->getType()->hasSignedIntegerRepresentation();
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
// OpenMP 4.5, 2.7.1 Loop Construct, Description.
// If the static schedule kind is specified or if the ordered clause is
// specified, and if no monotonic modifier is specified, the effect will
// be as if the monotonic modifier was specified.
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
if (RT.isStaticNonchunked(ScheduleKind.Schedule,
[OPENMP] Fix codegen for ordered loop directives. loops with ordered clause must be generated the same way as dynamic loops, but with static scheduleing. llvm-svn: 237788
2015-05-20 21:12:48 +08:00
/* Chunked */ Chunk != nullptr) &&
!Ordered) {
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
if (isOpenMPSimdDirective(S.getDirectiveKind()))
EmitOMPSimdInit(S, /*IsMonotonic=*/true);
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// OpenMP [2.7.1, Loop Construct, Description, table 2-1]
// When no chunk_size is specified, the iteration space is divided into
// chunks that are approximately equal in size, and at most one chunk is
// distributed to each thread. Note that the size of the chunks is
// unspecified in this case.
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
RT.emitForStaticInit(*this, S.getLocStart(), ScheduleKind,
IVSize, IVSigned, Ordered,
IL.getAddress(), LB.getAddress(),
UB.getAddress(), ST.getAddress());
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
auto LoopExit =
getJumpDestInCurrentScope(createBasicBlock("omp.loop.exit"));
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// UB = min(UB, GlobalUB);
EmitIgnoredExpr(S.getEnsureUpperBound());
// IV = LB;
EmitIgnoredExpr(S.getInit());
// while (idx <= UB) { BODY; ++idx; }
[OPENMP] Remove last iteration separation for loop-based constructs. Previously the last iteration for simd loop-based OpenMP constructs were generated as a separate code. This feature is not required and codegen is simplified. llvm-svn: 239810
2015-06-16 19:59:36 +08:00
EmitOMPInnerLoop(S, LoopScope.requiresCleanups(), S.getCond(),
S.getInc(),
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
[&S, LoopExit](CodeGenFunction &CGF) {
CGF.EmitOMPLoopBody(S, LoopExit);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitStopPoint(&S);
[OPENMP] Codegen for 'ordered' directive. Add codegen for 'ordered' directive: __kmpc_ordered(ident_t *, gtid); <associated statement>; __kmpc_end_ordered(ident_t *, gtid); Also for 'for' directives with the dynamic scheduling and an 'ordered' clause added a call to '__kmpc_dispatch_fini_(4|8)[u]()' function after increment expression for loop control variable: while(__kmpc_dispatch_next(&LB, &UB)) { idx = LB; while (idx <= UB) { BODY; ++idx; __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only. } // inner loop } Differential Revision: http://reviews.llvm.org/D9070 llvm-svn: 235496
2015-04-22 19:15:40 +08:00
},
[](CodeGenFunction &) {});
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
EmitBlock(LoopExit.getBlock());
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Tell the runtime we are done.
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF) {
CGF.CGM.getOpenMPRuntime().emitForStaticFinish(CGF, S.getLocEnd());
};
OMPCancelStack.emitExit(*this, S.getDirectiveKind(), CodeGen);
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
} else {
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
const bool IsMonotonic =
Ordered || ScheduleKind.Schedule == OMPC_SCHEDULE_static ||
ScheduleKind.Schedule == OMPC_SCHEDULE_unknown ||
ScheduleKind.M1 == OMPC_SCHEDULE_MODIFIER_monotonic ||
ScheduleKind.M2 == OMPC_SCHEDULE_MODIFIER_monotonic;
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
// Emit the outer loop, which requests its work chunk [LB..UB] from
// runtime and runs the inner loop to process it.
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
const OMPLoopArguments LoopArguments(LB.getAddress(), UB.getAddress(),
ST.getAddress(), IL.getAddress(),
Chunk, EUB);
[OPENMP 4.5] Codegen for 'schedule' clause with monotonic/nonmonotonic modifiers. OpenMP 4.5 adds support for monotonic/nonmonotonic modifiers in 'schedule' clause. Add codegen for these modifiers. llvm-svn: 256666
2015-12-31 14:52:34 +08:00
EmitOMPForOuterLoop(ScheduleKind, IsMonotonic, S, LoopScope, Ordered,
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
LoopArguments, CGDispatchBounds);
Support ‘omp for’ with static chunked schedule kind. Differential Revision: http://reviews.llvm.org/D7006 llvm-svn: 226795
2015-01-22 16:49:35 +08:00
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
if (isOpenMPSimdDirective(S.getDirectiveKind())) {
EmitOMPSimdFinal(S,
[&](CodeGenFunction &CGF) -> llvm::Value * {
return CGF.Builder.CreateIsNotNull(
CGF.EmitLoadOfScalar(IL, S.getLocStart()));
});
}
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
EmitOMPReductionClauseFinal(
S, /*ReductionKind=*/isOpenMPSimdDirective(S.getDirectiveKind())
? /*Parallel and Simd*/ OMPD_parallel_for_simd
: /*Parallel only*/ OMPD_parallel);
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// Emit post-update of the reduction variables if IsLastIter != 0.
emitPostUpdateForReductionClause(
*this, S, [&](CodeGenFunction &CGF) -> llvm::Value * {
return CGF.Builder.CreateIsNotNull(
CGF.EmitLoadOfScalar(IL, S.getLocStart()));
});
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
// Emit final copy of the lastprivate variables if IsLastIter != 0.
if (HasLastprivateClause)
EmitOMPLastprivateClauseFinal(
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
S, isOpenMPSimdDirective(S.getDirectiveKind()),
Builder.CreateIsNotNull(EmitLoadOfScalar(IL, S.getLocStart())));
[OPENMP] Add support for 'omp for simd' directive. Added codegen for combined 'omp for simd' directives, that is a combination of 'omp for' directive followed by 'omp simd' directive. Includes support for all clauses. llvm-svn: 239990
2015-06-18 12:45:29 +08:00
}
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
EmitOMPLinearClauseFinal(S, [&](CodeGenFunction &CGF) -> llvm::Value * {
[OPENMP 4.5] Codegen for data members in 'linear' clause OpenMP 4.5 allows privatization of non-static data members in OpenMP constructs. Patch adds proper codegen support for data members in 'linear' clause llvm-svn: 263003
2016-03-09 17:49:09 +08:00
return CGF.Builder.CreateIsNotNull(
CGF.EmitLoadOfScalar(IL, S.getLocStart()));
});
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// We're now done with the loop, so jump to the continuation block.
[OPENMP] Fix use of unsigned counters in loops with zero trip count. Patch fixes bugs in codegen for loops with unsigned counters and zero trip count. Previously preconditions for all loops were built using logic (Upper - Lower) > 0. But if the loop is a loop with zero trip count, then Upper - Lower is < 0 only for signed integer, for unsigned we're running into an underflow situation. In this patch we're using original Lower<Upper condition to check that loop body can be executed at least once. Also this allows to skip code generation for loops, if it is known that preconditions for the loop are always false. Differential Revision: http://reviews.llvm.org/D9103 llvm-svn: 235500
2015-04-22 19:59:37 +08:00
if (ContBlock) {
EmitBranch(ContBlock);
EmitBlock(ContBlock, true);
}
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
}
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
return HasLastprivateClause;
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
/// The following two functions generate expressions for the loop lower
/// and upper bounds in case of static and dynamic (dispatch) schedule
/// of the associated 'for' or 'distribute' loop.
static std::pair<LValue, LValue>
emitForLoopBounds(CodeGenFunction &CGF, const OMPExecutableDirective &S) {
const OMPLoopDirective &LS = cast<OMPLoopDirective>(S);
LValue LB =
EmitOMPHelperVar(CGF, cast<DeclRefExpr>(LS.getLowerBoundVariable()));
LValue UB =
EmitOMPHelperVar(CGF, cast<DeclRefExpr>(LS.getUpperBoundVariable()));
return {LB, UB};
}
/// When dealing with dispatch schedules (e.g. dynamic, guided) we do not
/// consider the lower and upper bound expressions generated by the
/// worksharing loop support, but we use 0 and the iteration space size as
/// constants
static std::pair<llvm::Value *, llvm::Value *>
emitDispatchForLoopBounds(CodeGenFunction &CGF, const OMPExecutableDirective &S,
Address LB, Address UB) {
const OMPLoopDirective &LS = cast<OMPLoopDirective>(S);
const Expr *IVExpr = LS.getIterationVariable();
const unsigned IVSize = CGF.getContext().getTypeSize(IVExpr->getType());
llvm::Value *LBVal = CGF.Builder.getIntN(IVSize, 0);
llvm::Value *UBVal = CGF.EmitScalarExpr(LS.getLastIteration());
return {LBVal, UBVal};
}
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
void CodeGenFunction::EmitOMPForDirective(const OMPForDirective &S) {
[OPENMP] Codegen for 'lastprivate' clause in 'for' directive. #pragma omp for lastprivate(<var>) for (i = a; i < b; ++b) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <lastprivate_var> = alloca <type> <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <var> = <lastprivate_var> ; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. Differential Revision: http://reviews.llvm.org/D8658 llvm-svn: 235074
2015-04-16 12:54:05 +08:00
bool HasLastprivates = false;
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S, &HasLastprivates](CodeGenFunction &CGF,
PrePostActionTy &) {
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
OMPCancelStackRAII CancelRegion(CGF, OMPD_for, S.hasCancel());
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
HasLastprivates = CGF.EmitOMPWorksharingLoop(S, S.getEnsureUpperBound(),
emitForLoopBounds,
emitDispatchForLoopBounds);
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
};
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
{
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_for, CodeGen,
S.hasCancel());
}
First patch with codegen of the 'omp for' directive. It implements the simplest case, which is used when no chunk_size is specified in the schedule(static) or no 'schedule' clause is specified - the iteration space is divided by the library into chunks that are approximately equal in size, and at most one chunk is distributed to each thread. In this case, we do not need an outer loop in each thread - each thread requests once which iterations range it should handle (using __kmpc_for_static_init runtime call) and then runs the inner loop on this range. Differential Revision: http://reviews.llvm.org/D5865 llvm-svn: 224233
2014-12-15 15:07:06 +08:00
// Emit an implicit barrier at the end.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (!S.getSingleClause<OMPNowaitClause>() || HasLastprivates) {
[OPENMP] Improved codegen for implicit/explicit 'barrier' constructs. Replace boolean IsExplicit parameter of OpenMPRuntime::emitBarrierCall() method by OpenMPDirectiveKind Kind for better compatibility with the runtime library. Also add processing of 'nowait' clause on worksharing directives. Differential Revision: http://reviews.llvm.org/D8659 llvm-svn: 233511
2015-03-30 12:30:22 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(*this, S.getLocStart(), OMPD_for);
}
[OPENMP] Initial support for '#pragma omp for' (fixed incompatibility with MSVC). llvm-svn: 211140
2014-06-18 12:14:57 +08:00
}
[OPENMP] Initial support for 'sections' directive. llvm-svn: 211685
2014-06-25 19:44:49 +08:00
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
void CodeGenFunction::EmitOMPForSimdDirective(const OMPForSimdDirective &S) {
bool HasLastprivates = false;
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S, &HasLastprivates](CodeGenFunction &CGF,
PrePostActionTy &) {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
HasLastprivates = CGF.EmitOMPWorksharingLoop(S, S.getEnsureUpperBound(),
emitForLoopBounds,
emitDispatchForLoopBounds);
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
};
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
{
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_simd, CodeGen);
}
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
// Emit an implicit barrier at the end.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (!S.getSingleClause<OMPNowaitClause>() || HasLastprivates) {
[OPENMP] Code reformatting for omp simd codegen, NFC. llvm-svn: 239889
2015-06-17 15:45:51 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(*this, S.getLocStart(), OMPD_for);
}
Parsing/Sema of directive omp for simd llvm-svn: 218029
2014-09-18 13:12:34 +08:00
}
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
static LValue createSectionLVal(CodeGenFunction &CGF, QualType Ty,
const Twine &Name,
llvm::Value *Init = nullptr) {
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
auto LVal = CGF.MakeAddrLValue(CGF.CreateMemTemp(Ty, Name), Ty);
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
if (Init)
[CodeGen][ObjC] Do not call objc_storeStrong when initializing a constexpr variable. When compiling a constexpr NSString initialized with an objective-c string literal, CodeGen emits objc_storeStrong on an uninitialized alloca, which causes a crash. This patch folds the code in EmitScalarInit into EmitStoreThroughLValue and fixes the crash by calling objc_retain on the string instead of using objc_storeStrong. rdar://problem/28562009 Differential Revision: https://reviews.llvm.org/D25547 llvm-svn: 284516
2016-10-19 03:05:41 +08:00
CGF.EmitStoreThroughLValue(RValue::get(Init), LVal, /*isInit*/ true);
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
return LVal;
}
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
void CodeGenFunction::EmitSections(const OMPExecutableDirective &S) {
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
auto *Stmt = cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt();
auto *CS = dyn_cast<CompoundStmt>(Stmt);
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
bool HasLastprivates = false;
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S, Stmt, CS, &HasLastprivates](CodeGenFunction &CGF,
PrePostActionTy &) {
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
auto &C = CGF.CGM.getContext();
auto KmpInt32Ty = C.getIntTypeForBitwidth(/*DestWidth=*/32, /*Signed=*/1);
// Emit helper vars inits.
LValue LB = createSectionLVal(CGF, KmpInt32Ty, ".omp.sections.lb.",
CGF.Builder.getInt32(0));
auto *GlobalUBVal = CS != nullptr ? CGF.Builder.getInt32(CS->size() - 1)
: CGF.Builder.getInt32(0);
LValue UB =
createSectionLVal(CGF, KmpInt32Ty, ".omp.sections.ub.", GlobalUBVal);
LValue ST = createSectionLVal(CGF, KmpInt32Ty, ".omp.sections.st.",
CGF.Builder.getInt32(1));
LValue IL = createSectionLVal(CGF, KmpInt32Ty, ".omp.sections.il.",
CGF.Builder.getInt32(0));
// Loop counter.
LValue IV = createSectionLVal(CGF, KmpInt32Ty, ".omp.sections.iv.");
OpaqueValueExpr IVRefExpr(S.getLocStart(), KmpInt32Ty, VK_LValue);
CodeGenFunction::OpaqueValueMapping OpaqueIV(CGF, &IVRefExpr, IV);
OpaqueValueExpr UBRefExpr(S.getLocStart(), KmpInt32Ty, VK_LValue);
CodeGenFunction::OpaqueValueMapping OpaqueUB(CGF, &UBRefExpr, UB);
// Generate condition for loop.
BinaryOperator Cond(&IVRefExpr, &UBRefExpr, BO_LE, C.BoolTy, VK_RValue,
Encapsulate FPOptions and use it consistently Sema holds the current FPOptions which is adjusted by 'pragma STDC FP_CONTRACT'. This then gets propagated into expression nodes as they are built. This encapsulates FPOptions so that this propagation happens opaquely rather than directly with the fp_contractable on/off bit. This allows controlled transitioning of fp_contractable to a ternary value (off, on, fast). It will also allow adding more fast-math flags later. This is toward moving fp-contraction=fast from an LLVM TargetOption to a FastMathFlag in order to fix PR25721. Differential Revision: https://reviews.llvm.org/D31166 llvm-svn: 298877
2017-03-28 03:17:25 +08:00
OK_Ordinary, S.getLocStart(), FPOptions());
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
// Increment for loop counter.
UnaryOperator Inc(&IVRefExpr, UO_PreInc, KmpInt32Ty, VK_RValue, OK_Ordinary,
S.getLocStart());
auto BodyGen = [Stmt, CS, &S, &IV](CodeGenFunction &CGF) {
// Iterate through all sections and emit a switch construct:
// switch (IV) {
// case 0:
// <SectionStmt[0]>;
// break;
// ...
// case <NumSection> - 1:
// <SectionStmt[<NumSection> - 1]>;
// break;
// }
// .omp.sections.exit:
auto *ExitBB = CGF.createBasicBlock(".omp.sections.exit");
auto *SwitchStmt = CGF.Builder.CreateSwitch(
CGF.EmitLoadOfLValue(IV, S.getLocStart()).getScalarVal(), ExitBB,
CS == nullptr ? 1 : CS->size());
if (CS) {
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
unsigned CaseNumber = 0;
Switch users of the 'for (StmtRange range = stmt->children(); range; ++range)‘ pattern to range for loops. The pattern was born out of the lack of range-based for loops in C++98 and is somewhat obscure. No functionality change intended. llvm-svn: 241300
2015-07-03 05:03:14 +08:00
for (auto *SubStmt : CS->children()) {
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
auto CaseBB = CGF.createBasicBlock(".omp.sections.case");
CGF.EmitBlock(CaseBB);
SwitchStmt->addCase(CGF.Builder.getInt32(CaseNumber), CaseBB);
Switch users of the 'for (StmtRange range = stmt->children(); range; ++range)‘ pattern to range for loops. The pattern was born out of the lack of range-based for loops in C++98 and is somewhat obscure. No functionality change intended. llvm-svn: 241300
2015-07-03 05:03:14 +08:00
CGF.EmitStmt(SubStmt);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitBranch(ExitBB);
Switch users of the 'for (StmtRange range = stmt->children(); range; ++range)‘ pattern to range for loops. The pattern was born out of the lack of range-based for loops in C++98 and is somewhat obscure. No functionality change intended. llvm-svn: 241300
2015-07-03 05:03:14 +08:00
++CaseNumber;
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
}
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
} else {
auto CaseBB = CGF.createBasicBlock(".omp.sections.case");
CGF.EmitBlock(CaseBB);
SwitchStmt->addCase(CGF.Builder.getInt32(0), CaseBB);
CGF.EmitStmt(Stmt);
CGF.EmitBranch(ExitBB);
[OPENMP] Codegen for 'firstprivate' clause in 'sections' directive. If there are 2 or more sections in a 'section' directive the following code is generated: <init for firstprivates> @__kmpc_cancel_barrier();// To avoid data race in firstprivate init @__kmpc_for_static_init_4(); <BODY for sections directive> @__kmpc_for_static_fini() If there is only one section, the following code is generated: if (@__kmpc_single()) { <init for firstprivates> @__kmpc_end_single(); } @__kmpc_cancel_barrier(); // To avoid data race in firstprivate init Differential Revision: http://reviews.llvm.org/D9214 llvm-svn: 235691
2015-04-24 11:37:03 +08:00
}
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CGF.EmitBlock(ExitBB, /*IsFinished=*/true);
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
};
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CodeGenFunction::OMPPrivateScope LoopScope(CGF);
if (CGF.EmitOMPFirstprivateClause(S, LoopScope)) {
[OPENMP] Codegen for 'lastprivate' clause in 'sections' directive. #pragma omp sections lastprivate(<var>) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <init for lastprivates>; <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <final copy for lastprivate>; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. If there is only one section, there is no special code generation, original shared variables are used + barrier is emitted at the end of the directive. Differential Revision: http://reviews.llvm.org/D9240 llvm-svn: 235834
2015-04-27 12:34:03 +08:00
// Emit implicit barrier to synchronize threads and avoid data races on
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
// initialization of firstprivate variables and post-update of lastprivate
// variables.
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CGF.CGM.getOpenMPRuntime().emitBarrierCall(
CGF, S.getLocStart(), OMPD_unknown, /*EmitChecks=*/false,
/*ForceSimpleCall=*/true);
[OPENMP] Codegen for 'lastprivate' clause in 'sections' directive. #pragma omp sections lastprivate(<var>) <BODY>; This construct is translated into something like: <last_iter> = alloca i32 <init for lastprivates>; <last_iter> = 0 ; No initializer for simple variables or a default constructor is called for objects. ; For arrays perform element by element initialization by the call of the default constructor. ... OMP_FOR_START(...,<last_iter>, ..); sets <last_iter> to 1 if this is the last iteration. <BODY> ... OMP_FOR_END if (<last_iter> != 0) { <final copy for lastprivate>; Update original variable with the lastprivate value. } call __kmpc_cancel_barrier() ; an implicit barrier to avoid possible data race. If there is only one section, there is no special code generation, original shared variables are used + barrier is emitted at the end of the directive. Differential Revision: http://reviews.llvm.org/D9240 llvm-svn: 235834
2015-04-27 12:34:03 +08:00
}
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CGF.EmitOMPPrivateClause(S, LoopScope);
HasLastprivates = CGF.EmitOMPLastprivateClauseInit(S, LoopScope);
CGF.EmitOMPReductionClauseInit(S, LoopScope);
(void)LoopScope.Privatize();
// Emit static non-chunked loop.
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
OpenMPScheduleTy ScheduleKind;
ScheduleKind.Schedule = OMPC_SCHEDULE_static;
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CGF.CGM.getOpenMPRuntime().emitForStaticInit(
[OPENMP 4.5] Add codegen support in runtime for '[non]monotonic' schedule modifiers. Runtime library expects some additional data in schedule argument for loop-based directives, that have additional schedule modifiers 'monotonic|nonmonotonic'. llvm-svn: 269035
2016-05-10 17:57:36 +08:00
CGF, S.getLocStart(), ScheduleKind, /*IVSize=*/32,
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
/*IVSigned=*/true, /*Ordered=*/false, IL.getAddress(), LB.getAddress(),
UB.getAddress(), ST.getAddress());
// UB = min(UB, GlobalUB);
auto *UBVal = CGF.EmitLoadOfScalar(UB, S.getLocStart());
auto *MinUBGlobalUB = CGF.Builder.CreateSelect(
CGF.Builder.CreateICmpSLT(UBVal, GlobalUBVal), UBVal, GlobalUBVal);
CGF.EmitStoreOfScalar(MinUBGlobalUB, UB);
// IV = LB;
CGF.EmitStoreOfScalar(CGF.EmitLoadOfScalar(LB, S.getLocStart()), IV);
// while (idx <= UB) { BODY; ++idx; }
CGF.EmitOMPInnerLoop(S, /*RequiresCleanup=*/false, &Cond, &Inc, BodyGen,
[](CodeGenFunction &) {});
// Tell the runtime we are done.
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF) {
CGF.CGM.getOpenMPRuntime().emitForStaticFinish(CGF, S.getLocEnd());
};
CGF.OMPCancelStack.emitExit(CGF, S.getDirectiveKind(), CodeGen);
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
CGF.EmitOMPReductionClauseFinal(S, /*ReductionKind=*/OMPD_parallel);
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// Emit post-update of the reduction variables if IsLastIter != 0.
emitPostUpdateForReductionClause(
CGF, S, [&](CodeGenFunction &CGF) -> llvm::Value * {
return CGF.Builder.CreateIsNotNull(
CGF.EmitLoadOfScalar(IL, S.getLocStart()));
});
[OPENMP] Codegen for 'firstprivate' clause in 'sections' directive. If there are 2 or more sections in a 'section' directive the following code is generated: <init for firstprivates> @__kmpc_cancel_barrier();// To avoid data race in firstprivate init @__kmpc_for_static_init_4(); <BODY for sections directive> @__kmpc_for_static_fini() If there is only one section, the following code is generated: if (@__kmpc_single()) { <init for firstprivates> @__kmpc_end_single(); } @__kmpc_cancel_barrier(); // To avoid data race in firstprivate init Differential Revision: http://reviews.llvm.org/D9214 llvm-svn: 235691
2015-04-24 11:37:03 +08:00
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
// Emit final copy of the lastprivate variables if IsLastIter != 0.
if (HasLastprivates)
CGF.EmitOMPLastprivateClauseFinal(
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
S, /*NoFinals=*/false,
CGF.Builder.CreateIsNotNull(
CGF.EmitLoadOfScalar(IL, S.getLocStart())));
[OPENMP] Initial codegen for 'parallel sections' directive. Emits code for outlined 'parallel' directive with the implicitly inlined 'sections' directive: ... call __kmpc_fork_call(..., outlined_function, ...); ... define internal void outlined_function(...) { <code for implicit sections directive>; } Differential Revision: http://reviews.llvm.org/D8997 llvm-svn: 234849
2015-04-14 11:29:22 +08:00
};
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
bool HasCancel = false;
if (auto *OSD = dyn_cast<OMPSectionsDirective>(&S))
HasCancel = OSD->hasCancel();
else if (auto *OPSD = dyn_cast<OMPParallelSectionsDirective>(&S))
HasCancel = OPSD->hasCancel();
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
OMPCancelStackRAII CancelRegion(*this, S.getDirectiveKind(), HasCancel);
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_sections, CodeGen,
HasCancel);
// Emit barrier for lastprivates only if 'sections' directive has 'nowait'
// clause. Otherwise the barrier will be generated by the codegen for the
// directive.
if (HasLastprivates && S.getSingleClause<OMPNowaitClause>()) {
[OPENMP] Codegen for 'firstprivate' clause in 'sections' directive. If there are 2 or more sections in a 'section' directive the following code is generated: <init for firstprivates> @__kmpc_cancel_barrier();// To avoid data race in firstprivate init @__kmpc_for_static_init_4(); <BODY for sections directive> @__kmpc_for_static_fini() If there is only one section, the following code is generated: if (@__kmpc_single()) { <init for firstprivates> @__kmpc_end_single(); } @__kmpc_cancel_barrier(); // To avoid data race in firstprivate init Differential Revision: http://reviews.llvm.org/D9214 llvm-svn: 235691
2015-04-24 11:37:03 +08:00
// Emit implicit barrier to synchronize threads and avoid data races on
// initialization of firstprivate variables.
[OPENMP] Generalize codegen for 'sections'-based directive. If 'sections' directive has only one sub-section, the code for 'single'-based directive was emitted. Removed this codegen, because it causes crashes in different cases. llvm-svn: 258495
2016-01-22 16:56:50 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(*this, S.getLocStart(),
OMPD_unknown);
[OPENMP] Codegen for 'firstprivate' clause in 'sections' directive. If there are 2 or more sections in a 'section' directive the following code is generated: <init for firstprivates> @__kmpc_cancel_barrier();// To avoid data race in firstprivate init @__kmpc_for_static_init_4(); <BODY for sections directive> @__kmpc_for_static_fini() If there is only one section, the following code is generated: if (@__kmpc_single()) { <init for firstprivates> @__kmpc_end_single(); } @__kmpc_cancel_barrier(); // To avoid data race in firstprivate init Differential Revision: http://reviews.llvm.org/D9214 llvm-svn: 235691
2015-04-24 11:37:03 +08:00
}
[OPENMP] Initial codegen for 'parallel sections' directive. Emits code for outlined 'parallel' directive with the implicitly inlined 'sections' directive: ... call __kmpc_fork_call(..., outlined_function, ...); ... define internal void outlined_function(...) { <code for implicit sections directive>; } Differential Revision: http://reviews.llvm.org/D8997 llvm-svn: 234849
2015-04-14 11:29:22 +08:00
}
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
[OPENMP] Initial codegen for 'parallel sections' directive. Emits code for outlined 'parallel' directive with the implicitly inlined 'sections' directive: ... call __kmpc_fork_call(..., outlined_function, ...); ... define internal void outlined_function(...) { <code for implicit sections directive>; } Differential Revision: http://reviews.llvm.org/D8997 llvm-svn: 234849
2015-04-14 11:29:22 +08:00
void CodeGenFunction::EmitOMPSectionsDirective(const OMPSectionsDirective &S) {
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
{
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
EmitSections(S);
}
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
// Emit an implicit barrier at the end.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (!S.getSingleClause<OMPNowaitClause>()) {
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(*this, S.getLocStart(),
OMPD_sections);
[OPENMP] Improved codegen for implicit/explicit 'barrier' constructs. Replace boolean IsExplicit parameter of OpenMPRuntime::emitBarrierCall() method by OpenMPDirectiveKind Kind for better compatibility with the runtime library. Also add processing of 'nowait' clause on worksharing directives. Differential Revision: http://reviews.llvm.org/D8659 llvm-svn: 233511
2015-03-30 12:30:22 +08:00
}
[OPENMP] Initial support for 'sections' directive. llvm-svn: 211685
2014-06-25 19:44:49 +08:00
}
[OPENMP] Initial codegen for 'omp sections' and 'omp section' directives. If only one section is found in the sections region, it is emitted just like single region. Otherwise it is emitted as a static non-chunked loop. #pragma omp sections { #pragma omp section {1} ... #pragma omp section {n} } is translated to something like i32 <iter_var> i32 <last_iter> = 0 i32 <lower_bound> = 0 i32 <upper_bound> = n-1 i32 <stride> = 1 call void @__kmpc_for_static_init_4(<loc>, i32 <gtid>, i32 34/*static non-chunked*/, i32* <last_iter>, i32* <lower_bound>, i32* <upper_bound>, i32* <stride>, i32 1/*increment always 1*/, i32 1/*chunk always 1*/) <upper_bound> = min(<upper_bound>, n-1) <iter_var> = <lb> check: br <iter_var> <= <upper_bound>, label cont, label exit continue: switch (IV) { case 0: {1}; break; ... case <NumSection> - 1: {n}; break; } ++<iter_var> br label check exit: call void @__kmpc_for_static_fini(<loc>, i32 <gtid>) Differential Revision: http://reviews.llvm.org/D8244 llvm-svn: 232021
2015-03-12 16:53:29 +08:00
void CodeGenFunction::EmitOMPSectionDirective(const OMPSectionDirective &S) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Emit __kmpc_cancel_barrier() and code for 'cancellation point' only if 'cancel' is found. Patch improves codegen for OpenMP constructs. If the OpenMP region does not have internal 'cancel' construct, a call to 'void __kmpc_barrier()' runtime function is generated for all implicit/explicit barriers. If the region has inner 'cancel' directive, then ``` if (__kmpc_cancel_barrier()) exit from outer construct; ``` code is generated. Also, the code for 'canellation point' directive is not generated if parent directive does not have 'cancel' directive. llvm-svn: 247681
2015-09-15 20:52:43 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_section, CodeGen,
S.hasCancel());
[OPENMP] Initial parsing and sema analysis for 'section' directive. llvm-svn: 211767
2014-06-26 16:21:58 +08:00
}
[OPENMP] Initial codegen for 'single' directive. This patch emits the following code for the single directive: #pragma omp single <body> <----> if(__kmpc_single(...)) { <body> __kmpc_end_single(...); } Differential Revision: http://reviews.llvm.org/D7045 llvm-svn: 228275
2015-02-05 14:35:41 +08:00
void CodeGenFunction::EmitOMPSingleDirective(const OMPSingleDirective &S) {
[OPENMP] Codegen for 'copyprivate' clause ('single' directive). If there is at least one 'copyprivate' clause is associated with the single directive, the following code is generated: ``` i32 did_it = 0; \\ for 'copyprivate' clause if(__kmpc_single(ident_t *, gtid)) { SingleOpGen(); __kmpc_end_single(ident_t *, gtid); did_it = 1; \\ for 'copyprivate' clause } <copyprivate_list>[0] = &var0; ... <copyprivate_list>[n] = &varn; call __kmpc_copyprivate(ident_t *, gtid, <copyprivate_list_size>, <copyprivate_list>, <copy_func>, did_it); ... void<copy_func>(void *LHSArg, void *RHSArg) { Dst = (void * [n])(LHSArg); Src = (void * [n])(RHSArg); Dst[0] = Src[0]; ... Dst[n] = Src[n]; } ``` All list items from all 'copyprivate' clauses are gathered into single <copyprivate list> (<copyprivate_list_size> is a size in bytes of this list) and <copy_func> is used to propagate values of private or threadprivate variables from the 'single' region to other implicit threads from outer 'parallel' region. Differential Revision: http://reviews.llvm.org/D8410 llvm-svn: 232932
2015-03-23 14:18:07 +08:00
llvm::SmallVector<const Expr *, 8> CopyprivateVars;
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
llvm::SmallVector<const Expr *, 8> DestExprs;
[OPENMP] Codegen for 'copyprivate' clause ('single' directive). If there is at least one 'copyprivate' clause is associated with the single directive, the following code is generated: ``` i32 did_it = 0; \\ for 'copyprivate' clause if(__kmpc_single(ident_t *, gtid)) { SingleOpGen(); __kmpc_end_single(ident_t *, gtid); did_it = 1; \\ for 'copyprivate' clause } <copyprivate_list>[0] = &var0; ... <copyprivate_list>[n] = &varn; call __kmpc_copyprivate(ident_t *, gtid, <copyprivate_list_size>, <copyprivate_list>, <copy_func>, did_it); ... void<copy_func>(void *LHSArg, void *RHSArg) { Dst = (void * [n])(LHSArg); Src = (void * [n])(RHSArg); Dst[0] = Src[0]; ... Dst[n] = Src[n]; } ``` All list items from all 'copyprivate' clauses are gathered into single <copyprivate list> (<copyprivate_list_size> is a size in bytes of this list) and <copy_func> is used to propagate values of private or threadprivate variables from the 'single' region to other implicit threads from outer 'parallel' region. Differential Revision: http://reviews.llvm.org/D8410 llvm-svn: 232932
2015-03-23 14:18:07 +08:00
llvm::SmallVector<const Expr *, 8> SrcExprs;
llvm::SmallVector<const Expr *, 8> AssignmentOps;
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
// Check if there are any 'copyprivate' clauses associated with this
[OPENMP] Remove extra sync barriers for 'firstprivate' clause. Sync barrier will be emitted after generation of firstprivate variables only if one of the firstprivate vars is used in lastprivate clause. llvm-svn: 260877
2016-02-15 16:07:17 +08:00
// 'single' construct.
[OPENMP] Codegen for 'copyprivate' clause ('single' directive). If there is at least one 'copyprivate' clause is associated with the single directive, the following code is generated: ``` i32 did_it = 0; \\ for 'copyprivate' clause if(__kmpc_single(ident_t *, gtid)) { SingleOpGen(); __kmpc_end_single(ident_t *, gtid); did_it = 1; \\ for 'copyprivate' clause } <copyprivate_list>[0] = &var0; ... <copyprivate_list>[n] = &varn; call __kmpc_copyprivate(ident_t *, gtid, <copyprivate_list_size>, <copyprivate_list>, <copy_func>, did_it); ... void<copy_func>(void *LHSArg, void *RHSArg) { Dst = (void * [n])(LHSArg); Src = (void * [n])(RHSArg); Dst[0] = Src[0]; ... Dst[n] = Src[n]; } ``` All list items from all 'copyprivate' clauses are gathered into single <copyprivate list> (<copyprivate_list_size> is a size in bytes of this list) and <copy_func> is used to propagate values of private or threadprivate variables from the 'single' region to other implicit threads from outer 'parallel' region. Differential Revision: http://reviews.llvm.org/D8410 llvm-svn: 232932
2015-03-23 14:18:07 +08:00
// Build a list of copyprivate variables along with helper expressions
// (<source>, <destination>, <destination>=<source> expressions)
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : S.getClausesOfKind<OMPCopyprivateClause>()) {
[OPENMP] Codegen for 'copyprivate' clause ('single' directive). If there is at least one 'copyprivate' clause is associated with the single directive, the following code is generated: ``` i32 did_it = 0; \\ for 'copyprivate' clause if(__kmpc_single(ident_t *, gtid)) { SingleOpGen(); __kmpc_end_single(ident_t *, gtid); did_it = 1; \\ for 'copyprivate' clause } <copyprivate_list>[0] = &var0; ... <copyprivate_list>[n] = &varn; call __kmpc_copyprivate(ident_t *, gtid, <copyprivate_list_size>, <copyprivate_list>, <copy_func>, did_it); ... void<copy_func>(void *LHSArg, void *RHSArg) { Dst = (void * [n])(LHSArg); Src = (void * [n])(RHSArg); Dst[0] = Src[0]; ... Dst[n] = Src[n]; } ``` All list items from all 'copyprivate' clauses are gathered into single <copyprivate list> (<copyprivate_list_size> is a size in bytes of this list) and <copy_func> is used to propagate values of private or threadprivate variables from the 'single' region to other implicit threads from outer 'parallel' region. Differential Revision: http://reviews.llvm.org/D8410 llvm-svn: 232932
2015-03-23 14:18:07 +08:00
CopyprivateVars.append(C->varlists().begin(), C->varlists().end());
[OPENMP] Fixed codegen for arrays in 'copyprivate' clause. Fixed a bug with codegen of variables with array types specified in 'copyprivate' clause of 'single' directive. Differential Revision: http://reviews.llvm.org/D8914 llvm-svn: 234856
2015-04-14 13:11:24 +08:00
DestExprs.append(C->destination_exprs().begin(),
C->destination_exprs().end());
[OPENMP] Codegen for 'copyprivate' clause ('single' directive). If there is at least one 'copyprivate' clause is associated with the single directive, the following code is generated: ``` i32 did_it = 0; \\ for 'copyprivate' clause if(__kmpc_single(ident_t *, gtid)) { SingleOpGen(); __kmpc_end_single(ident_t *, gtid); did_it = 1; \\ for 'copyprivate' clause } <copyprivate_list>[0] = &var0; ... <copyprivate_list>[n] = &varn; call __kmpc_copyprivate(ident_t *, gtid, <copyprivate_list_size>, <copyprivate_list>, <copy_func>, did_it); ... void<copy_func>(void *LHSArg, void *RHSArg) { Dst = (void * [n])(LHSArg); Src = (void * [n])(RHSArg); Dst[0] = Src[0]; ... Dst[n] = Src[n]; } ``` All list items from all 'copyprivate' clauses are gathered into single <copyprivate list> (<copyprivate_list_size> is a size in bytes of this list) and <copy_func> is used to propagate values of private or threadprivate variables from the 'single' region to other implicit threads from outer 'parallel' region. Differential Revision: http://reviews.llvm.org/D8410 llvm-svn: 232932
2015-03-23 14:18:07 +08:00
SrcExprs.append(C->source_exprs().begin(), C->source_exprs().end());
AssignmentOps.append(C->assignment_ops().begin(),
C->assignment_ops().end());
}
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
// Emit code for 'single' region along with 'copyprivate' clauses
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
Action.Enter(CGF);
OMPPrivateScope SingleScope(CGF);
(void)CGF.EmitOMPFirstprivateClause(S, SingleScope);
CGF.EmitOMPPrivateClause(S, SingleScope);
(void)SingleScope.Privatize();
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
};
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
{
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Improved handling of pseudo-captured expressions in OpenMP. Expressions inside 'schedule'|'dist_schedule' clause must be captured in combined directives to avoid possible crash during codegen. Patch improves handling of such constructs llvm-svn: 260954
2016-02-16 19:18:12 +08:00
CGM.getOpenMPRuntime().emitSingleRegion(*this, CodeGen, S.getLocStart(),
CopyprivateVars, DestExprs,
SrcExprs, AssignmentOps);
}
// Emit an implicit barrier at the end (to avoid data race on firstprivate
// init or if no 'nowait' clause was specified and no 'copyprivate' clause).
[OPENMP 4.5] Codegen support for data members in 'firstprivate' clause. Added codegen for captured data members in non-static member functions. llvm-svn: 261089
2016-02-17 21:19:37 +08:00
if (!S.getSingleClause<OMPNowaitClause>() && CopyprivateVars.empty()) {
[OPENMP] Codegen for 'firstprivate' clause in 'single' directive. Emit the following code for 'single' directive with 'firtstprivate' clause: if (@__kmpc_single()) { <init for firstprivates> @__kmpc_end_single(); } @__kmpc_cancel_barrier(); // To avoid data race in firstprivate init Differential Revision: http://reviews.llvm.org/D9223 llvm-svn: 235694
2015-04-24 12:21:15 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(
*this, S.getLocStart(),
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
S.getSingleClause<OMPNowaitClause>() ? OMPD_unknown : OMPD_single);
[OPENMP] Improved codegen for implicit/explicit 'barrier' constructs. Replace boolean IsExplicit parameter of OpenMPRuntime::emitBarrierCall() method by OpenMPDirectiveKind Kind for better compatibility with the runtime library. Also add processing of 'nowait' clause on worksharing directives. Differential Revision: http://reviews.llvm.org/D8659 llvm-svn: 233511
2015-03-30 12:30:22 +08:00
}
[OPENMP] Initial parsing and sema analysis for 'single' directive. llvm-svn: 211774
2014-06-26 20:05:45 +08:00
}
[OPENMP] Codegen for 'omp master' directive Patch adds 2 library functions to OpenMPRuntime class - int32 kmpc_master(ident_t *, int32 gtid) and void kmpc_end_master(ident_t *, int32 gtid); For 'omp master' directive the next code is generated: if (__kmpc_master(loc, gtid)) { <Associated structured block>; __kmpc_end_master(log, gtid); } Differential Revision: http://reviews.llvm.org/D6473 llvm-svn: 223342
2014-12-04 15:23:53 +08:00
void CodeGenFunction::EmitOMPMasterDirective(const OMPMasterDirective &S) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
Action.Enter(CGF);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGM.getOpenMPRuntime().emitMasterRegion(*this, CodeGen, S.getLocStart());
[OPENMP] Parsing/Sema analysis of directive 'master' llvm-svn: 213237
2014-07-17 16:54:58 +08:00
}
[OPENMP] Codegen for 'omp critical' directive. This patch adds codegen for constructs: #pragma omp critical [name] <body> It generates global variable ".gomp_critical_user_[name].var" of type int32[8]. Then it generates library call "kmpc_critical(loc, gtid, .gomp_critical_user_[name].var)", code for <body> statement and final call "kmpc_end_critical(loc, gtid, .gomp_critical_user_[name].var)". Differential Revision: http://reviews.llvm.org/D5202 llvm-svn: 218239
2014-09-22 18:01:53 +08:00
void CodeGenFunction::EmitOMPCriticalDirective(const OMPCriticalDirective &S) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
Action.Enter(CGF);
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
};
[OPENMP 4.5] Codegen for 'hint' clause of 'critical' directive OpenMP 4.5 defines 'hint' clause for 'critical' directive. Patch adds codegen for this clause. llvm-svn: 255639
2015-12-15 18:55:09 +08:00
Expr *Hint = nullptr;
if (auto *HintClause = S.getSingleClause<OMPHintClause>())
Hint = HintClause->getHint();
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP 4.5] Codegen for 'hint' clause of 'critical' directive OpenMP 4.5 defines 'hint' clause for 'critical' directive. Patch adds codegen for this clause. llvm-svn: 255639
2015-12-15 18:55:09 +08:00
CGM.getOpenMPRuntime().emitCriticalRegion(*this,
S.getDirectiveName().getAsString(),
CodeGen, S.getLocStart(), Hint);
[OPENMP] Parsing/Sema of the OpenMP directive 'critical'. llvm-svn: 213510
2014-07-21 17:42:05 +08:00
}
[OPENMP] Initial codegen for 'parallel for' directive. Allows generation of combined 'parallel for' directive that represents 'parallel' region with internal implicit 'for' worksharing region. Differential Revision: http://reviews.llvm.org/D8631 llvm-svn: 234722
2015-04-13 13:28:11 +08:00
void CodeGenFunction::EmitOMPParallelForDirective(
const OMPParallelForDirective &S) {
// Emit directive as a combined directive that consists of two implicit
// directives: 'parallel' with 'for' directive.
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
OMPCancelStackRAII CancelRegion(CGF, OMPD_parallel_for, S.hasCancel());
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
CGF.EmitOMPWorksharingLoop(S, S.getEnsureUpperBound(), emitForLoopBounds,
emitDispatchForLoopBounds);
[OPENMP] Initial codegen for 'parallel for' directive. Allows generation of combined 'parallel for' directive that represents 'parallel' region with internal implicit 'for' worksharing region. Differential Revision: http://reviews.llvm.org/D8631 llvm-svn: 234722
2015-04-13 13:28:11 +08:00
};
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
emitCommonOMPParallelDirective(*this, S, OMPD_for, CodeGen,
emitEmptyBoundParameters);
[OPENMP] Added initial support for 'omp parallel for'. llvm-svn: 212453
2014-07-07 21:01:15 +08:00
}
[OPENMP] Parsing/Sema of directive omp parallel for simd llvm-svn: 218299
2014-09-23 17:33:00 +08:00
void CodeGenFunction::EmitOMPParallelForSimdDirective(
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
const OMPParallelForSimdDirective &S) {
// Emit directive as a combined directive that consists of two implicit
// directives: 'parallel' with 'for' directive.
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
CGF.EmitOMPWorksharingLoop(S, S.getEnsureUpperBound(), emitForLoopBounds,
emitDispatchForLoopBounds);
[OPENMP] Add support for 'omp parallel for' directive. Codegen for this directive is a combined codegen for 'omp parallel' region with 'omp for simd' region inside. Clauses are supported. llvm-svn: 240006
2015-06-18 18:10:12 +08:00
};
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
emitCommonOMPParallelDirective(*this, S, OMPD_simd, CodeGen,
emitEmptyBoundParameters);
[OPENMP] Parsing/Sema of directive omp parallel for simd llvm-svn: 218299
2014-09-23 17:33:00 +08:00
}
[OPENMP] Parsing and sema analysis for 'omp parallel sections' directive. llvm-svn: 212516
2014-07-08 16:12:03 +08:00
void CodeGenFunction::EmitOMPParallelSectionsDirective(
[OPENMP] Initial codegen for 'parallel sections' directive. Emits code for outlined 'parallel' directive with the implicitly inlined 'sections' directive: ... call __kmpc_fork_call(..., outlined_function, ...); ... define internal void outlined_function(...) { <code for implicit sections directive>; } Differential Revision: http://reviews.llvm.org/D8997 llvm-svn: 234849
2015-04-14 11:29:22 +08:00
const OMPParallelSectionsDirective &S) {
// Emit directive as a combined directive that consists of two implicit
// directives: 'parallel' with 'sections' directive.
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
CGF.EmitSections(S);
};
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
emitCommonOMPParallelDirective(*this, S, OMPD_sections, CodeGen,
emitEmptyBoundParameters);
[OPENMP] Parsing and sema analysis for 'omp parallel sections' directive. llvm-svn: 212516
2014-07-08 16:12:03 +08:00
}
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
void CodeGenFunction::EmitOMPTaskBasedDirective(const OMPExecutableDirective &S,
const RegionCodeGenTy &BodyGen,
const TaskGenTy &TaskGen,
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
OMPTaskDataTy &Data) {
[OPENMP] Initial codegen for 'omp task' directive. The task region is emmitted in several steps: Emit a call to kmp_task_t *__kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry). Here task_entry is a pointer to the function: kmp_int32 .omp_task_entry.(kmp_int32 gtid, kmp_task_t *tt) { TaskFunction(gtid, tt->part_id, tt->shareds); return 0; } Copy a list of shared variables to field shareds of the resulting structure kmp_task_t returned by the previous call (if any). Copy a pointer to destructions function to field destructions of the resulting structure kmp_task_t. Emit a call to kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task), where new_task is a resulting structure from previous items. Differential Revision: http://reviews.llvm.org/D7560 llvm-svn: 231762
2015-03-10 15:28:44 +08:00
// Emit outlined function for task construct.
auto CS = cast<CapturedStmt>(S.getAssociatedStmt());
auto *I = CS->getCapturedDecl()->param_begin();
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
auto *PartId = std::next(I);
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
auto *TaskT = std::next(I, 4);
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
// Check if the task is final
if (const auto *Clause = S.getSingleClause<OMPFinalClause>()) {
// If the condition constant folds and can be elided, try to avoid emitting
// the condition and the dead arm of the if/else.
auto *Cond = Clause->getCondition();
bool CondConstant;
if (ConstantFoldsToSimpleInteger(Cond, CondConstant))
Data.Final.setInt(CondConstant);
else
Data.Final.setPointer(EvaluateExprAsBool(Cond));
} else {
// By default the task is not final.
Data.Final.setInt(/*IntVal=*/false);
}
[OPENMP 4.5] Initial codegen for 'priority' clause in task-based directives. OpenMP 4.5 supports clause 'priority' in task-based directives. Patch adds initial codegen support for this clause in codegen. llvm-svn: 269050
2016-05-10 20:21:02 +08:00
// Check if the task has 'priority' clause.
if (const auto *Clause = S.getSingleClause<OMPPriorityClause>()) {
auto *Prio = Clause->getPriority();
[OPENMP] Improved processing of 'priority' clause, NFC. Removed some old comments + improved handling of 'priority' clause value during codegen after comments from Richard Smith. llvm-svn: 275945
2016-07-19 12:21:09 +08:00
Data.Priority.setInt(/*IntVal=*/true);
[OPENMP 4.5] Fixed codegen for 'priority' and destructors in task-based directives. 'kmp_task_t' record type added a new field for 'priority' clause and changed the representation of pointer to destructors for privates used within loop-based directives. Old representation: typedef struct kmp_task { /* GEH: Shouldn't this be aligned somehow? */ void *shareds; /**< pointer to block of pointers to shared vars */ kmp_routine_entry_t routine; /**< pointer to routine to call for executing task */ kmp_int32 part_id; /**< part id for the task */ kmp_routine_entry_t destructors; /* pointer to function to invoke deconstructors of firstprivate C++ objects */ /* private vars */ } kmp_task_t; New representation: typedef struct kmp_task { /* GEH: Shouldn't this be aligned somehow? */ void *shareds; /**< pointer to block of pointers to shared vars */ kmp_routine_entry_t routine; /**< pointer to routine to call for executing task */ kmp_int32 part_id; /**< part id for the task */ kmp_cmplrdata_t data1; /* Two known optional additions: destructors and priority */ kmp_cmplrdata_t data2; /* Process destructors first, priority second */ /* future data */ /* private vars */ } kmp_task_t; Also excessive initialization of 'destructors' fields to 'null' was removed from codegen if it is known that no destructors shal be used. Currently a special bit is used in 'kmp_tasking_flags_t' bitfields ('destructors_thunk' bitfield). llvm-svn: 271201
2016-05-30 17:06:50 +08:00
Data.Priority.setPointer(EmitScalarConversion(
EmitScalarExpr(Prio), Prio->getType(),
getContext().getIntTypeForBitwidth(/*DestWidth=*/32, /*Signed=*/1),
Prio->getExprLoc()));
[OPENMP 4.5] Initial codegen for 'priority' clause in task-based directives. OpenMP 4.5 supports clause 'priority' in task-based directives. Patch adds initial codegen support for this clause in codegen. llvm-svn: 269050
2016-05-10 20:21:02 +08:00
}
[OPENMP] Initial codegen for 'omp task' directive. The task region is emmitted in several steps: Emit a call to kmp_task_t *__kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry). Here task_entry is a pointer to the function: kmp_int32 .omp_task_entry.(kmp_int32 gtid, kmp_task_t *tt) { TaskFunction(gtid, tt->part_id, tt->shareds); return 0; } Copy a list of shared variables to field shareds of the resulting structure kmp_task_t returned by the previous call (if any). Copy a pointer to destructions function to field destructions of the resulting structure kmp_task_t. Emit a call to kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task), where new_task is a resulting structure from previous items. Differential Revision: http://reviews.llvm.org/D7560 llvm-svn: 231762
2015-03-10 15:28:44 +08:00
// The first function argument for tasks is a thread id, the second one is a
// part id (0 for tied tasks, >=0 for untied task).
[OPENMP] Codegen for 'firstprivate' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform initialization of all firstprivate fields (by simple copying for POD data, copy constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); CopyConstructor(new_task->privates.var1, *new_task->shareds.var1_ref); new_task->shareds.var1_ref = &new_task->privates.var1; ... CopyConstructor(new_task->privates.varn, *new_task->shareds.varn_ref); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9370 llvm-svn: 236479
2015-05-05 12:05:12 +08:00
llvm::DenseSet<const VarDecl *> EmittedAsPrivate;
[OPENMP] Codegen for 'private' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform default initialization of all private fields (no initialization for POD data, default constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); DefaultConstructor(new_task->privates.var1); new_task->shareds.var1_ref = &new_task->privates.var1; ... DefaultConstructor(new_task->privates.varn); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9322 llvm-svn: 236207
2015-04-30 14:51:57 +08:00
// Get list of private variables.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : S.getClausesOfKind<OMPPrivateClause>()) {
[OPENMP] Codegen for 'private' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform default initialization of all private fields (no initialization for POD data, default constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); DefaultConstructor(new_task->privates.var1); new_task->shareds.var1_ref = &new_task->privates.var1; ... DefaultConstructor(new_task->privates.varn); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9322 llvm-svn: 236207
2015-04-30 14:51:57 +08:00
auto IRef = C->varlist_begin();
for (auto *IInit : C->private_copies()) {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
if (EmittedAsPrivate.insert(OrigVD->getCanonicalDecl()).second) {
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
Data.PrivateVars.push_back(*IRef);
Data.PrivateCopies.push_back(IInit);
[OPENMP] Codegen for 'private' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform default initialization of all private fields (no initialization for POD data, default constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); DefaultConstructor(new_task->privates.var1); new_task->shareds.var1_ref = &new_task->privates.var1; ... DefaultConstructor(new_task->privates.varn); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9322 llvm-svn: 236207
2015-04-30 14:51:57 +08:00
}
++IRef;
}
}
[OPENMP] Codegen for 'firstprivate' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform initialization of all firstprivate fields (by simple copying for POD data, copy constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); CopyConstructor(new_task->privates.var1, *new_task->shareds.var1_ref); new_task->shareds.var1_ref = &new_task->privates.var1; ... CopyConstructor(new_task->privates.varn, *new_task->shareds.varn_ref); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9370 llvm-svn: 236479
2015-05-05 12:05:12 +08:00
EmittedAsPrivate.clear();
// Get list of firstprivate variables.
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
for (const auto *C : S.getClausesOfKind<OMPFirstprivateClause>()) {
[OPENMP] Codegen for 'firstprivate' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform initialization of all firstprivate fields (by simple copying for POD data, copy constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); CopyConstructor(new_task->privates.var1, *new_task->shareds.var1_ref); new_task->shareds.var1_ref = &new_task->privates.var1; ... CopyConstructor(new_task->privates.varn, *new_task->shareds.varn_ref); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9370 llvm-svn: 236479
2015-05-05 12:05:12 +08:00
auto IRef = C->varlist_begin();
auto IElemInitRef = C->inits().begin();
for (auto *IInit : C->private_copies()) {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
if (EmittedAsPrivate.insert(OrigVD->getCanonicalDecl()).second) {
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
Data.FirstprivateVars.push_back(*IRef);
Data.FirstprivateCopies.push_back(IInit);
Data.FirstprivateInits.push_back(*IElemInitRef);
[OPENMP] Codegen for 'firstprivate' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform initialization of all firstprivate fields (by simple copying for POD data, copy constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); CopyConstructor(new_task->privates.var1, *new_task->shareds.var1_ref); new_task->shareds.var1_ref = &new_task->privates.var1; ... CopyConstructor(new_task->privates.varn, *new_task->shareds.varn_ref); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9370 llvm-svn: 236479
2015-05-05 12:05:12 +08:00
}
Remove use of builtin comma operator. Cleanup for upcoming Clang warning -Wcomma. No functionality change intended. llvm-svn: 261271
2016-02-19 06:34:54 +08:00
++IRef;
++IElemInitRef;
[OPENMP] Codegen for 'firstprivate' clause in 'task' directive. For tasks codegen for private/firstprivate variables are different rather than for other directives. 1. Build an internal structure of privates for each private variable: struct .kmp_privates_t. { Ty1 var1; ... Tyn varn; }; 2. Add a new field to kmp_task_t type with list of privates. struct kmp_task_t { void * shareds; kmp_routine_entry_t routine; kmp_int32 part_id; kmp_routine_entry_t destructors; .kmp_privates_t. privates; }; 3. Create a function with destructors calls for all privates after end of task region. kmp_int32 .omp_task_destructor.(kmp_int32 gtid, kmp_task_t *tt) { ~Destructor(&tt->privates.var1); ... ~Destructor(&tt->privates.varn); return 0; } 4. Perform initialization of all firstprivate fields (by simple copying for POD data, copy constructor calls for classes) + provide address of a destructor function after kmpc_omp_task_alloc() and before kmpc_omp_task() calls. kmp_task_t *new_task = __kmpc_omp_task_alloc(ident_t *, kmp_int32 gtid, kmp_int32 flags, size_t sizeof_kmp_task_t, size_t sizeof_shareds, kmp_routine_entry_t *task_entry); CopyConstructor(new_task->privates.var1, *new_task->shareds.var1_ref); new_task->shareds.var1_ref = &new_task->privates.var1; ... CopyConstructor(new_task->privates.varn, *new_task->shareds.varn_ref); new_task->shareds.varn_ref = &new_task->privates.varn; new_task->destructors = .omp_task_destructor.; kmp_int32 __kmpc_omp_task(ident_t *, kmp_int32 gtid, kmp_task_t *new_task) Differential Revision: http://reviews.llvm.org/D9370 llvm-svn: 236479
2015-05-05 12:05:12 +08:00
}
}
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
// Get list of lastprivate variables (for taskloops).
llvm::DenseMap<const VarDecl *, const DeclRefExpr *> LastprivateDstsOrigs;
for (const auto *C : S.getClausesOfKind<OMPLastprivateClause>()) {
auto IRef = C->varlist_begin();
auto ID = C->destination_exprs().begin();
for (auto *IInit : C->private_copies()) {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
if (EmittedAsPrivate.insert(OrigVD->getCanonicalDecl()).second) {
Data.LastprivateVars.push_back(*IRef);
Data.LastprivateCopies.push_back(IInit);
}
LastprivateDstsOrigs.insert(
{cast<VarDecl>(cast<DeclRefExpr>(*ID)->getDecl()),
cast<DeclRefExpr>(*IRef)});
++IRef;
++ID;
}
}
[OPENMP] Codegen for 'depend' clause (OpenMP 4.0). If task directive has associated 'depend' clause then function kmp_int32 __kmpc_omp_task_with_deps ( ident_t *loc_ref, kmp_int32 gtid, kmp_task_t * new_task, kmp_int32 ndeps, kmp_depend_info_t *dep_list,kmp_int32 ndeps_noalias, kmp_depend_info_t *noalias_dep_list) must be called instead of __kmpc_omp_task(). If this directive has associated 'if' clause then also before a call of kmpc_omp_task_begin_if0() a function void __kmpc_omp_wait_deps ( ident_t *loc_ref, kmp_int32 gtid, kmp_int32 ndeps, kmp_depend_info_t *dep_list, kmp_int32 ndeps_noalias, kmp_depend_info_t *noalias_dep_list) must be called. Array sections are not supported yet. llvm-svn: 240532
2015-06-24 19:01:36 +08:00
// Build list of dependences.
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
for (const auto *C : S.getClausesOfKind<OMPDependClause>())
for (auto *IRef : C->varlists())
Data.Dependences.push_back(std::make_pair(C->getDependencyKind(), IRef));
Remove unused lambda captures. NFC llvm-svn: 291939
2017-01-14 02:55:32 +08:00
auto &&CodeGen = [&Data, CS, &BodyGen, &LastprivateDstsOrigs](
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
CodeGenFunction &CGF, PrePostActionTy &Action) {
[OPENMP] Prepare codegen for privates in tasks for non-capturing of privates in CapturedStmt. Reworked codegen for privates in tasks: call @kmpc_omp_task_alloc(); ... call @kmpc_omp_task(task_proxy); void map_privates(.privates_rec. *privs, type1 ** priv1_ref, ..., typen **privn_ref) { *priv1_ref = &privs->private1; ... *privn_ref = &privs->privaten; ret void } i32 task_entry(i32 ThreadId, i32 PartId, void* privs, void (void*, ...) map_privates, shareds* captures) { type1 **priv1; ... typen **privn; call map_privates(privs, priv1, ..., privn); <Task body with priv1, .., privn instead of the captured variables>. ret i32 } i32 task_proxy(i32 ThreadId, kmp_task_t_with_privates *tt) { call task_entry(ThreadId, tt->task_data.PartId, &tt->privates, map_privates, tt->task_data.shareds); } llvm-svn: 238010
2015-05-22 16:56:35 +08:00
// Set proper addresses for generated private copies.
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
OMPPrivateScope Scope(CGF);
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
if (!Data.PrivateVars.empty() || !Data.FirstprivateVars.empty() ||
!Data.LastprivateVars.empty()) {
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
auto *CopyFn = CGF.Builder.CreateLoad(
CGF.GetAddrOfLocalVar(CS->getCapturedDecl()->getParam(3)));
auto *PrivatesPtr = CGF.Builder.CreateLoad(
CGF.GetAddrOfLocalVar(CS->getCapturedDecl()->getParam(2)));
// Map privates.
llvm::SmallVector<std::pair<const VarDecl *, Address>, 16> PrivatePtrs;
llvm::SmallVector<llvm::Value *, 16> CallArgs;
CallArgs.push_back(PrivatesPtr);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
for (auto *E : Data.PrivateVars) {
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
Address PrivatePtr = CGF.CreateMemTemp(
CGF.getContext().getPointerType(E->getType()), ".priv.ptr.addr");
PrivatePtrs.push_back(std::make_pair(VD, PrivatePtr));
CallArgs.push_back(PrivatePtr.getPointer());
}
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
for (auto *E : Data.FirstprivateVars) {
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
Address PrivatePtr =
CGF.CreateMemTemp(CGF.getContext().getPointerType(E->getType()),
".firstpriv.ptr.addr");
PrivatePtrs.push_back(std::make_pair(VD, PrivatePtr));
CallArgs.push_back(PrivatePtr.getPointer());
Revert "[OPENMP] Allow runtime insert its own code inside OpenMP regions." This reverts commit 3ee791165100607178073f14531a0dc90c622b36. llvm-svn: 264570
2016-03-28 18:12:03 +08:00
}
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
for (auto *E : Data.LastprivateVars) {
auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
Address PrivatePtr =
CGF.CreateMemTemp(CGF.getContext().getPointerType(E->getType()),
".lastpriv.ptr.addr");
PrivatePtrs.push_back(std::make_pair(VD, PrivatePtr));
CallArgs.push_back(PrivatePtr.getPointer());
}
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
CGF.EmitRuntimeCall(CopyFn, CallArgs);
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
for (auto &&Pair : LastprivateDstsOrigs) {
auto *OrigVD = cast<VarDecl>(Pair.second->getDecl());
DeclRefExpr DRE(
const_cast<VarDecl *>(OrigVD),
/*RefersToEnclosingVariableOrCapture=*/CGF.CapturedStmtInfo->lookup(
OrigVD) != nullptr,
Pair.second->getType(), VK_LValue, Pair.second->getExprLoc());
Scope.addPrivate(Pair.first, [&CGF, &DRE]() {
return CGF.EmitLValue(&DRE).getAddress();
});
}
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
for (auto &&Pair : PrivatePtrs) {
Address Replacement(CGF.Builder.CreateLoad(Pair.second),
CGF.getContext().getDeclAlign(Pair.first));
Scope.addPrivate(Pair.first, [Replacement]() { return Replacement; });
Revert "[OPENMP] Allow runtime insert its own code inside OpenMP regions." Reverting because of failed tests. llvm-svn: 264577
2016-03-28 20:58:34 +08:00
}
}
[OPENMP] Codegen for untied tasks. If the untied clause is present on a task construct, any thread in the team can resume the task region after a suspension. Patch adds proper codegen for untied tasks. llvm-svn: 266853
2016-04-20 12:01:36 +08:00
(void)Scope.Privatize();
Action.Enter(CGF);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
BodyGen(CGF);
[OPENMP] Prepare codegen for privates in tasks for non-capturing of privates in CapturedStmt. Reworked codegen for privates in tasks: call @kmpc_omp_task_alloc(); ... call @kmpc_omp_task(task_proxy); void map_privates(.privates_rec. *privs, type1 ** priv1_ref, ..., typen **privn_ref) { *priv1_ref = &privs->private1; ... *privn_ref = &privs->privaten; ret void } i32 task_entry(i32 ThreadId, i32 PartId, void* privs, void (void*, ...) map_privates, shareds* captures) { type1 **priv1; ... typen **privn; call map_privates(privs, priv1, ..., privn); <Task body with priv1, .., privn instead of the captured variables>. ret i32 } i32 task_proxy(i32 ThreadId, kmp_task_t_with_privates *tt) { call task_entry(ThreadId, tt->task_data.PartId, &tt->privates, map_privates, tt->task_data.shareds); } llvm-svn: 238010
2015-05-22 16:56:35 +08:00
};
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
auto *OutlinedFn = CGM.getOpenMPRuntime().emitTaskOutlinedFunction(
S, *I, *PartId, *TaskT, S.getDirectiveKind(), CodeGen, Data.Tied,
Data.NumberOfParts);
OMPLexicalScope Scope(*this, S);
TaskGen(*this, OutlinedFn, Data);
}
void CodeGenFunction::EmitOMPTaskDirective(const OMPTaskDirective &S) {
// Emit outlined function for task construct.
auto CS = cast<CapturedStmt>(S.getAssociatedStmt());
auto CapturedStruct = GenerateCapturedStmtArgument(*CS);
[OPENMP] Prepare codegen for privates in tasks for non-capturing of privates in CapturedStmt. Reworked codegen for privates in tasks: call @kmpc_omp_task_alloc(); ... call @kmpc_omp_task(task_proxy); void map_privates(.privates_rec. *privs, type1 ** priv1_ref, ..., typen **privn_ref) { *priv1_ref = &privs->private1; ... *privn_ref = &privs->privaten; ret void } i32 task_entry(i32 ThreadId, i32 PartId, void* privs, void (void*, ...) map_privates, shareds* captures) { type1 **priv1; ... typen **privn; call map_privates(privs, priv1, ..., privn); <Task body with priv1, .., privn instead of the captured variables>. ret i32 } i32 task_proxy(i32 ThreadId, kmp_task_t_with_privates *tt) { call task_entry(ThreadId, tt->task_data.PartId, &tt->privates, map_privates, tt->task_data.shareds); } llvm-svn: 238010
2015-05-22 16:56:35 +08:00
auto SharedsTy = getContext().getRecordType(CS->getCapturedRecordDecl());
const Expr *IfCond = nullptr;
[OPENMP 4.1] Codegen for extended format of 'if' clause. Fixed codegen for extended format of 'if' clauses with special 'directive-name-modifier' + ast-print tests for extended format of 'if' clause. llvm-svn: 246748
2015-09-03 16:45:56 +08:00
for (const auto *C : S.getClausesOfKind<OMPIfClause>()) {
if (C->getNameModifier() == OMPD_unknown ||
C->getNameModifier() == OMPD_task) {
IfCond = C->getCondition();
break;
}
[OPENMP] Prepare codegen for privates in tasks for non-capturing of privates in CapturedStmt. Reworked codegen for privates in tasks: call @kmpc_omp_task_alloc(); ... call @kmpc_omp_task(task_proxy); void map_privates(.privates_rec. *privs, type1 ** priv1_ref, ..., typen **privn_ref) { *priv1_ref = &privs->private1; ... *privn_ref = &privs->privaten; ret void } i32 task_entry(i32 ThreadId, i32 PartId, void* privs, void (void*, ...) map_privates, shareds* captures) { type1 **priv1; ... typen **privn; call map_privates(privs, priv1, ..., privn); <Task body with priv1, .., privn instead of the captured variables>. ret i32 } i32 task_proxy(i32 ThreadId, kmp_task_t_with_privates *tt) { call task_entry(ThreadId, tt->task_data.PartId, &tt->privates, map_privates, tt->task_data.shareds); } llvm-svn: 238010
2015-05-22 16:56:35 +08:00
}
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
OMPTaskDataTy Data;
// Check if we should emit tied or untied task.
Data.Tied = !S.getSingleClause<OMPUntiedClause>();
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
auto &&BodyGen = [CS](CodeGenFunction &CGF, PrePostActionTy &) {
CGF.EmitStmt(CS->getCapturedStmt());
};
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
auto &&TaskGen = [&S, SharedsTy, CapturedStruct,
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
IfCond](CodeGenFunction &CGF, llvm::Value *OutlinedFn,
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
const OMPTaskDataTy &Data) {
CGF.CGM.getOpenMPRuntime().emitTaskCall(CGF, S.getLocStart(), S, OutlinedFn,
SharedsTy, CapturedStruct, IfCond,
Data);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
};
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
EmitOMPTaskBasedDirective(S, BodyGen, TaskGen, Data);
[OPENMP] Parsing and sema analysis for 'omp task' directive. llvm-svn: 212804
2014-07-11 19:25:16 +08:00
}
[OPENMP] Codegen for 'taskyield' directive For 'taskyield' directive emit call to kmp_int32 __kmpc_omp_taskyield(ident_t *, kmp_int32 global_tid, int end_part); runtime function call with end_part arg set to 0 (it is ignored). Differential Revision: http://reviews.llvm.org/D7047 llvm-svn: 228272
2015-02-05 13:57:51 +08:00
void CodeGenFunction::EmitOMPTaskyieldDirective(
const OMPTaskyieldDirective &S) {
[OPENMP] Rename methods of OpenMPRuntime class. NFC. llvm-svn: 230470
2015-02-25 16:32:46 +08:00
CGM.getOpenMPRuntime().emitTaskyieldCall(*this, S.getLocStart());
[OPENMP] Initial parsing and sema analysis of 'taskyield' directive. llvm-svn: 213355
2014-07-18 15:47:19 +08:00
}
[OPENMP] Codegen for 'omp barrier' directive. Adds generation of call to "i32 kmpc_cancel_barrier(ident_t *, i32)" libcall for explicitly specified barriers (OMP_IDENT_BARRIER_EXPL flag is added to "flags" field of "ident_t" structure). Also this patch replaces all calls to "kmpc_barrier" function by calls of "__kmpc_cancel_barrier" function which provides additional functionality for OpenMP 4.0. Also, library specific enum OpenMPLocationFlags moved to private section of CGOpenMPRuntime class to make it more independent from library implementation. Differential Revision: http://reviews.llvm.org/D6447 llvm-svn: 223444
2014-12-05 12:09:23 +08:00
void CodeGenFunction::EmitOMPBarrierDirective(const OMPBarrierDirective &S) {
[OPENMP] Improved codegen for implicit/explicit 'barrier' constructs. Replace boolean IsExplicit parameter of OpenMPRuntime::emitBarrierCall() method by OpenMPDirectiveKind Kind for better compatibility with the runtime library. Also add processing of 'nowait' clause on worksharing directives. Differential Revision: http://reviews.llvm.org/D8659 llvm-svn: 233511
2015-03-30 12:30:22 +08:00
CGM.getOpenMPRuntime().emitBarrierCall(*this, S.getLocStart(), OMPD_barrier);
[OPENMP] Initial parsing and sema analysis for 'barrier' directive. llvm-svn: 213360
2014-07-18 17:11:51 +08:00
}
[OPENMP] Codegen for 'taskwait' directive. Emit the following code for 'taskwait' directive within tied task: call i32 @__kmpc_omp_taskwait(<loc>, i32 <thread_id>); Differential Revision: http://reviews.llvm.org/D9245 llvm-svn: 235836
2015-04-27 13:22:09 +08:00
void CodeGenFunction::EmitOMPTaskwaitDirective(const OMPTaskwaitDirective &S) {
CGM.getOpenMPRuntime().emitTaskwaitCall(*this, S.getLocStart());
[OPENMP] Initial parsing and sema analysis for 'taskwait' directive. llvm-svn: 213363
2014-07-18 18:17:07 +08:00
}
[OPENMP] Support for '#pragma omp taskgroup' directive. Added parsing, sema analysis and codegen for '#pragma omp taskgroup' directive (OpenMP 4.0). The code for directive is generated the following way: #pragma omp taskgroup <body> void __kmpc_taskgroup(<loc>, thread_id); <body> void __kmpc_end_taskgroup(<loc>, thread_id); llvm-svn: 240011
2015-06-18 20:14:09 +08:00
void CodeGenFunction::EmitOMPTaskgroupDirective(
const OMPTaskgroupDirective &S) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
Action.Enter(CGF);
[OPENMP] Support for '#pragma omp taskgroup' directive. Added parsing, sema analysis and codegen for '#pragma omp taskgroup' directive (OpenMP 4.0). The code for directive is generated the following way: #pragma omp taskgroup <body> void __kmpc_taskgroup(<loc>, thread_id); <body> void __kmpc_end_taskgroup(<loc>, thread_id); llvm-svn: 240011
2015-06-18 20:14:09 +08:00
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP] Support for '#pragma omp taskgroup' directive. Added parsing, sema analysis and codegen for '#pragma omp taskgroup' directive (OpenMP 4.0). The code for directive is generated the following way: #pragma omp taskgroup <body> void __kmpc_taskgroup(<loc>, thread_id); <body> void __kmpc_end_taskgroup(<loc>, thread_id); llvm-svn: 240011
2015-06-18 20:14:09 +08:00
CGM.getOpenMPRuntime().emitTaskgroupRegion(*this, CodeGen, S.getLocStart());
}
[OPENMP] Codegen for "omp flush" directive. For each "omp flush" directive a call to "void kmpc_flush(ident_t *, ...)" function is generated. Directive "omp flush" may have an associated list of variables to flush, but currently runtime function ignores them. So the patch generates just "call kmpc_flush(ident_t *<loc>, i32 0)". Differential Revision: http://reviews.llvm.org/D6292 llvm-svn: 222409
2014-11-20 12:34:54 +08:00
void CodeGenFunction::EmitOMPFlushDirective(const OMPFlushDirective &S) {
[OPENMP] Rename methods of OpenMPRuntime class. NFC. llvm-svn: 230470
2015-02-25 16:32:46 +08:00
CGM.getOpenMPRuntime().emitFlush(*this, [&]() -> ArrayRef<const Expr *> {
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
if (const auto *FlushClause = S.getSingleClause<OMPFlushClause>()) {
[OPENMP] Rename methods of OpenMPRuntime class. NFC. llvm-svn: 230470
2015-02-25 16:32:46 +08:00
return llvm::makeArrayRef(FlushClause->varlist_begin(),
FlushClause->varlist_end());
}
return llvm::None;
}(), S.getLocStart());
[OPENMP] Initial parsing and sema analysis for 'flush' directive. llvm-svn: 213512
2014-07-21 19:26:11 +08:00
}
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
void CodeGenFunction::EmitOMPDistributeLoop(const OMPLoopDirective &S,
const CodeGenLoopTy &CodeGenLoop,
Expr *IncExpr) {
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
// Emit the loop iteration variable.
auto IVExpr = cast<DeclRefExpr>(S.getIterationVariable());
auto IVDecl = cast<VarDecl>(IVExpr->getDecl());
EmitVarDecl(*IVDecl);
// Emit the iterations count variable.
// If it is not a variable, Sema decided to calculate iterations count on each
// iteration (e.g., it is foldable into a constant).
if (auto LIExpr = dyn_cast<DeclRefExpr>(S.getLastIteration())) {
EmitVarDecl(*cast<VarDecl>(LIExpr->getDecl()));
// Emit calculation of the iterations count.
EmitIgnoredExpr(S.getCalcLastIteration());
}
auto &RT = CGM.getOpenMPRuntime();
[OPENMP] Private, firstprivate, and lastprivate clauses for distribute, host code generation https://reviews.llvm.org/D17840 This patch enables private, firstprivate, and lastprivate clauses for the OpenMP distribute directive. Regression tests differ from the similar case of the same clauses on the for directive, by removing a reference to two global variables g and g1. This is necessary because: 1. a distribute pragma is only allowed inside a target region; 2. referring a global variable (e.g. g and g1) in a target region requires the program to enclose the variable in a "declare target" region; 3. declare target pragmas, which are used to define a declare target region, are currently unavailable in clang (patch being prepared). For this reason, I moved the global declarations into local variables. llvm-svn: 290898
2017-01-04 02:24:42 +08:00
bool HasLastprivateClause = false;
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
// Check pre-condition.
{
[OPENMP] Remove extra code transformation. For better support of some specific GNU extensions some extra transformation of AST nodes were introduced. These transformations are very hard to handle. The code is improved in handling of these extensions by using captured expressions construct. llvm-svn: 264709
2016-03-29 16:58:54 +08:00
OMPLoopScope PreInitScope(*this, S);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
// Skip the entire loop if we don't meet the precondition.
// If the condition constant folds and can be elided, avoid emitting the
// whole loop.
bool CondConstant;
llvm::BasicBlock *ContBlock = nullptr;
if (ConstantFoldsToSimpleInteger(S.getPreCond(), CondConstant)) {
if (!CondConstant)
return;
} else {
auto *ThenBlock = createBasicBlock("omp.precond.then");
ContBlock = createBasicBlock("omp.precond.end");
emitPreCond(*this, S, S.getPreCond(), ThenBlock, ContBlock,
getProfileCount(&S));
EmitBlock(ThenBlock);
incrementProfileCounter(&S);
}
// Emit 'then' code.
{
// Emit helper vars inits.
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
LValue LB = EmitOMPHelperVar(
*this, cast<DeclRefExpr>(
(isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedLowerBoundVariable()
: S.getLowerBoundVariable())));
LValue UB = EmitOMPHelperVar(
*this, cast<DeclRefExpr>(
(isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedUpperBoundVariable()
: S.getUpperBoundVariable())));
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
LValue ST =
EmitOMPHelperVar(*this, cast<DeclRefExpr>(S.getStrideVariable()));
LValue IL =
EmitOMPHelperVar(*this, cast<DeclRefExpr>(S.getIsLastIterVariable()));
OMPPrivateScope LoopScope(*this);
[OPENMP] Private, firstprivate, and lastprivate clauses for distribute, host code generation https://reviews.llvm.org/D17840 This patch enables private, firstprivate, and lastprivate clauses for the OpenMP distribute directive. Regression tests differ from the similar case of the same clauses on the for directive, by removing a reference to two global variables g and g1. This is necessary because: 1. a distribute pragma is only allowed inside a target region; 2. referring a global variable (e.g. g and g1) in a target region requires the program to enclose the variable in a "declare target" region; 3. declare target pragmas, which are used to define a declare target region, are currently unavailable in clang (patch being prepared). For this reason, I moved the global declarations into local variables. llvm-svn: 290898
2017-01-04 02:24:42 +08:00
if (EmitOMPFirstprivateClause(S, LoopScope)) {
// Emit implicit barrier to synchronize threads and avoid data races on
// initialization of firstprivate variables and post-update of
// lastprivate variables.
CGM.getOpenMPRuntime().emitBarrierCall(
*this, S.getLocStart(), OMPD_unknown, /*EmitChecks=*/false,
/*ForceSimpleCall=*/true);
}
EmitOMPPrivateClause(S, LoopScope);
HasLastprivateClause = EmitOMPLastprivateClauseInit(S, LoopScope);
[OPENMP] Fix for LCV in simd directives in explicit clauses. If loop control variable for simd-based directives is explicitly marked as linear/lastprivate in clauses, codegen for such construct would crash. Patch fixes this problem. llvm-svn: 267101
2016-04-22 11:56:56 +08:00
EmitOMPPrivateLoopCounters(S, LoopScope);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
(void)LoopScope.Privatize();
// Detect the distribute schedule kind and chunk.
llvm::Value *Chunk = nullptr;
OpenMPDistScheduleClauseKind ScheduleKind = OMPC_DIST_SCHEDULE_unknown;
if (auto *C = S.getSingleClause<OMPDistScheduleClause>()) {
ScheduleKind = C->getDistScheduleKind();
if (const auto *Ch = C->getChunkSize()) {
Chunk = EmitScalarExpr(Ch);
Chunk = EmitScalarConversion(Chunk, Ch->getType(),
S.getIterationVariable()->getType(),
S.getLocStart());
}
}
const unsigned IVSize = getContext().getTypeSize(IVExpr->getType());
const bool IVSigned = IVExpr->getType()->hasSignedIntegerRepresentation();
// OpenMP [2.10.8, distribute Construct, Description]
// If dist_schedule is specified, kind must be static. If specified,
// iterations are divided into chunks of size chunk_size, chunks are
// assigned to the teams of the league in a round-robin fashion in the
// order of the team number. When no chunk_size is specified, the
// iteration space is divided into chunks that are approximately equal
// in size, and at most one chunk is distributed to each team of the
// league. The size of the chunks is unspecified in this case.
if (RT.isStaticNonchunked(ScheduleKind,
/* Chunked */ Chunk != nullptr)) {
RT.emitDistributeStaticInit(*this, S.getLocStart(), ScheduleKind,
IVSize, IVSigned, /* Ordered = */ false,
IL.getAddress(), LB.getAddress(),
UB.getAddress(), ST.getAddress());
auto LoopExit =
getJumpDestInCurrentScope(createBasicBlock("omp.loop.exit"));
// UB = min(UB, GlobalUB);
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
EmitIgnoredExpr(isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedEnsureUpperBound()
: S.getEnsureUpperBound());
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
// IV = LB;
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
EmitIgnoredExpr(isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedInit()
: S.getInit());
Expr *Cond = isOpenMPLoopBoundSharingDirective(S.getDirectiveKind())
? S.getCombinedCond()
: S.getCond();
// for distribute alone, codegen
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
// while (idx <= UB) { BODY; ++idx; }
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
// when combined with 'for' (e.g. as in 'distribute parallel for')
// while (idx <= UB) { <CodeGen rest of pragma>; idx += ST; }
EmitOMPInnerLoop(S, LoopScope.requiresCleanups(), Cond, IncExpr,
[&S, LoopExit, &CodeGenLoop](CodeGenFunction &CGF) {
CodeGenLoop(CGF, S, LoopExit);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
},
[](CodeGenFunction &) {});
EmitBlock(LoopExit.getBlock());
// Tell the runtime we are done.
RT.emitForStaticFinish(*this, S.getLocStart());
} else {
// Emit the outer loop, which requests its work chunk [LB..UB] from
// runtime and runs the inner loop to process it.
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
const OMPLoopArguments LoopArguments = {
LB.getAddress(), UB.getAddress(), ST.getAddress(), IL.getAddress(),
Chunk};
EmitOMPDistributeOuterLoop(ScheduleKind, S, LoopScope, LoopArguments,
CodeGenLoop);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
}
[OPENMP] Private, firstprivate, and lastprivate clauses for distribute, host code generation https://reviews.llvm.org/D17840 This patch enables private, firstprivate, and lastprivate clauses for the OpenMP distribute directive. Regression tests differ from the similar case of the same clauses on the for directive, by removing a reference to two global variables g and g1. This is necessary because: 1. a distribute pragma is only allowed inside a target region; 2. referring a global variable (e.g. g and g1) in a target region requires the program to enclose the variable in a "declare target" region; 3. declare target pragmas, which are used to define a declare target region, are currently unavailable in clang (patch being prepared). For this reason, I moved the global declarations into local variables. llvm-svn: 290898
2017-01-04 02:24:42 +08:00
// Emit final copy of the lastprivate variables if IsLastIter != 0.
if (HasLastprivateClause)
EmitOMPLastprivateClauseFinal(
S, /*NoFinals=*/false,
Builder.CreateIsNotNull(
EmitLoadOfScalar(IL, S.getLocStart())));
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
}
// We're now done with the loop, so jump to the continuation block.
if (ContBlock) {
EmitBranch(ContBlock);
EmitBlock(ContBlock, true);
}
}
}
Add parse and sema of OpenMP distribute directive with all clauses except dist_schedule llvm-svn: 255498
2015-12-14 22:51:25 +08:00
void CodeGenFunction::EmitOMPDistributeDirective(
const OMPDistributeDirective &S) {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
CGF.EmitOMPDistributeLoop(S, emitOMPLoopBodyWithStopPoint, S.getInc());
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
Reapply r262741 [OPENMP] Codegen for distribute directive This patch provide basic implementation of codegen for teams directive, excluding all clauses except dist_schedule. It also fixes parts of AST reader/writer to enable correct pre-compiled header handling. http://reviews.llvm.org/D17170 llvm-svn: 262832
2016-03-08 00:04:49 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_distribute, CodeGen,
false);
Add parse and sema of OpenMP distribute directive with all clauses except dist_schedule llvm-svn: 255498
2015-12-14 22:51:25 +08:00
}
[OPENMP 4.1] Codegen for ‘simd’ clause in ‘ordered’ directive. Description. If the simd clause is specified, the ordered regions encountered by any thread will use only a single SIMD lane to execute the ordered regions in the order of the loop iterations. Restrictions. An ordered construct with the simd clause is the only OpenMP construct that can appear in the simd region. An ordered directive with ‘simd’ clause is generated as an outlined function and corresponding function call to prevent this part of code from vectorization later in backend. llvm-svn: 248772
2015-09-29 11:48:57 +08:00
static llvm::Function *emitOutlinedOrderedFunction(CodeGenModule &CGM,
const CapturedStmt *S) {
CodeGenFunction CGF(CGM, /*suppressNewContext=*/true);
CodeGenFunction::CGCapturedStmtInfo CapStmtInfo;
CGF.CapturedStmtInfo = &CapStmtInfo;
auto *Fn = CGF.GenerateOpenMPCapturedStmtFunction(*S);
Fn->addFnAttr(llvm::Attribute::NoInline);
return Fn;
}
[OPENMP] Codegen for 'ordered' directive. Add codegen for 'ordered' directive: __kmpc_ordered(ident_t *, gtid); <associated statement>; __kmpc_end_ordered(ident_t *, gtid); Also for 'for' directives with the dynamic scheduling and an 'ordered' clause added a call to '__kmpc_dispatch_fini_(4|8)[u]()' function after increment expression for loop control variable: while(__kmpc_dispatch_next(&LB, &UB)) { idx = LB; while (idx <= UB) { BODY; ++idx; __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only. } // inner loop } Differential Revision: http://reviews.llvm.org/D9070 llvm-svn: 235496
2015-04-22 19:15:40 +08:00
void CodeGenFunction::EmitOMPOrderedDirective(const OMPOrderedDirective &S) {
[OPENMP 4.5] Codegen for dacross loop synchronization constructs. OpenMP 4.5 adds support for doacross loop synchronization. Patch implements codegen for this construct. llvm-svn: 270690
2016-05-25 20:36:08 +08:00
if (!S.getAssociatedStmt()) {
for (const auto *DC : S.getClausesOfKind<OMPDependClause>())
CGM.getOpenMPRuntime().emitDoacrossOrdered(*this, DC);
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
return;
[OPENMP 4.5] Codegen for dacross loop synchronization constructs. OpenMP 4.5 adds support for doacross loop synchronization. Patch implements codegen for this construct. llvm-svn: 270690
2016-05-25 20:36:08 +08:00
}
[OPENMP 4.1] Codegen for ‘simd’ clause in ‘ordered’ directive. Description. If the simd clause is specified, the ordered regions encountered by any thread will use only a single SIMD lane to execute the ordered regions in the order of the loop iterations. Restrictions. An ordered construct with the simd clause is the only OpenMP construct that can appear in the simd region. An ordered directive with ‘simd’ clause is generated as an outlined function and corresponding function call to prevent this part of code from vectorization later in backend. llvm-svn: 248772
2015-09-29 11:48:57 +08:00
auto *C = S.getSingleClause<OMPSIMDClause>();
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S, C, this](CodeGenFunction &CGF,
PrePostActionTy &Action) {
[OPENMP 4.1] Codegen for ‘simd’ clause in ‘ordered’ directive. Description. If the simd clause is specified, the ordered regions encountered by any thread will use only a single SIMD lane to execute the ordered regions in the order of the loop iterations. Restrictions. An ordered construct with the simd clause is the only OpenMP construct that can appear in the simd region. An ordered directive with ‘simd’ clause is generated as an outlined function and corresponding function call to prevent this part of code from vectorization later in backend. llvm-svn: 248772
2015-09-29 11:48:57 +08:00
if (C) {
auto CS = cast<CapturedStmt>(S.getAssociatedStmt());
llvm::SmallVector<llvm::Value *, 16> CapturedVars;
CGF.GenerateOpenMPCapturedVars(*CS, CapturedVars);
auto *OutlinedFn = emitOutlinedOrderedFunction(CGM, CS);
CGF.EmitNounwindRuntimeCall(OutlinedFn, CapturedVars);
} else {
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
Action.Enter(CGF);
[OPENMP 4.1] Codegen for ‘simd’ clause in ‘ordered’ directive. Description. If the simd clause is specified, the ordered regions encountered by any thread will use only a single SIMD lane to execute the ordered regions in the order of the loop iterations. Restrictions. An ordered construct with the simd clause is the only OpenMP construct that can appear in the simd region. An ordered directive with ‘simd’ clause is generated as an outlined function and corresponding function call to prevent this part of code from vectorization later in backend. llvm-svn: 248772
2015-09-29 11:48:57 +08:00
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
}
[OPENMP] Codegen for 'ordered' directive. Add codegen for 'ordered' directive: __kmpc_ordered(ident_t *, gtid); <associated statement>; __kmpc_end_ordered(ident_t *, gtid); Also for 'for' directives with the dynamic scheduling and an 'ordered' clause added a call to '__kmpc_dispatch_fini_(4|8)[u]()' function after increment expression for loop control variable: while(__kmpc_dispatch_next(&LB, &UB)) { idx = LB; while (idx <= UB) { BODY; ++idx; __kmpc_dispatch_fini_(4|8)[u](); // For ordered loops only. } // inner loop } Differential Revision: http://reviews.llvm.org/D9070 llvm-svn: 235496
2015-04-22 19:15:40 +08:00
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP 4.1] Codegen for ‘simd’ clause in ‘ordered’ directive. Description. If the simd clause is specified, the ordered regions encountered by any thread will use only a single SIMD lane to execute the ordered regions in the order of the loop iterations. Restrictions. An ordered construct with the simd clause is the only OpenMP construct that can appear in the simd region. An ordered directive with ‘simd’ clause is generated as an outlined function and corresponding function call to prevent this part of code from vectorization later in backend. llvm-svn: 248772
2015-09-29 11:48:57 +08:00
CGM.getOpenMPRuntime().emitOrderedRegion(*this, CodeGen, S.getLocStart(), !C);
[OPENMP] Initial parsing and sema analysis for 'ordered' directive. llvm-svn: 213616
2014-07-22 14:45:04 +08:00
}
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
static llvm::Value *convertToScalarValue(CodeGenFunction &CGF, RValue Val,
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
QualType SrcType, QualType DestType,
SourceLocation Loc) {
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
assert(CGF.hasScalarEvaluationKind(DestType) &&
"DestType must have scalar evaluation kind.");
assert(!Val.isAggregate() && "Must be a scalar or complex.");
return Val.isScalar()
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
? CGF.EmitScalarConversion(Val.getScalarVal(), SrcType, DestType,
Loc)
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
: CGF.EmitComplexToScalarConversion(Val.getComplexVal(), SrcType,
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
DestType, Loc);
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
}
static CodeGenFunction::ComplexPairTy
convertToComplexValue(CodeGenFunction &CGF, RValue Val, QualType SrcType,
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
QualType DestType, SourceLocation Loc) {
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
assert(CGF.getEvaluationKind(DestType) == TEK_Complex &&
"DestType must have complex evaluation kind.");
CodeGenFunction::ComplexPairTy ComplexVal;
if (Val.isScalar()) {
// Convert the input element to the element type of the complex.
auto DestElementType = DestType->castAs<ComplexType>()->getElementType();
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
auto ScalarVal = CGF.EmitScalarConversion(Val.getScalarVal(), SrcType,
DestElementType, Loc);
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
ComplexVal = CodeGenFunction::ComplexPairTy(
ScalarVal, llvm::Constant::getNullValue(ScalarVal->getType()));
} else {
assert(Val.isComplex() && "Must be a scalar or complex.");
auto SrcElementType = SrcType->castAs<ComplexType>()->getElementType();
auto DestElementType = DestType->castAs<ComplexType>()->getElementType();
ComplexVal.first = CGF.EmitScalarConversion(
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
Val.getComplexVal().first, SrcElementType, DestElementType, Loc);
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
ComplexVal.second = CGF.EmitScalarConversion(
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
Val.getComplexVal().second, SrcElementType, DestElementType, Loc);
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
}
return ComplexVal;
}
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
static void emitSimpleAtomicStore(CodeGenFunction &CGF, bool IsSeqCst,
LValue LVal, RValue RVal) {
if (LVal.isGlobalReg()) {
CGF.EmitStoreThroughGlobalRegLValue(RVal, LVal);
} else {
NFC: make AtomicOrdering an enum class Summary: See LLVM change D18775 for details, this change depends on it. Reviewers: jyknight, reames Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D18776 llvm-svn: 265569
2016-04-07 01:26:42 +08:00
CGF.EmitAtomicStore(RVal, LVal,
IsSeqCst ? llvm::AtomicOrdering::SequentiallyConsistent
: llvm::AtomicOrdering::Monotonic,
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
LVal.isVolatile(), /*IsInit=*/false);
}
}
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
void CodeGenFunction::emitOMPSimpleStore(LValue LVal, RValue RVal,
QualType RValTy, SourceLocation Loc) {
switch (getEvaluationKind(LVal.getType())) {
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
case TEK_Scalar:
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
EmitStoreThroughLValue(RValue::get(convertToScalarValue(
*this, RVal, RValTy, LVal.getType(), Loc)),
LVal);
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
break;
case TEK_Complex:
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
EmitStoreOfComplex(
convertToComplexValue(*this, RVal, RValTy, LVal.getType(), Loc), LVal,
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
/*isInit=*/false);
break;
case TEK_Aggregate:
llvm_unreachable("Must be a scalar or complex.");
}
}
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
static void EmitOMPAtomicReadExpr(CodeGenFunction &CGF, bool IsSeqCst,
const Expr *X, const Expr *V,
SourceLocation Loc) {
// v = x;
assert(V->isLValue() && "V of 'omp atomic read' is not lvalue");
assert(X->isLValue() && "X of 'omp atomic read' is not lvalue");
LValue XLValue = CGF.EmitLValue(X);
LValue VLValue = CGF.EmitLValue(V);
Revert "Revert r229082 for a bit, it caused PR22577." This reverts commit r229123. It was a red herring, the bug was present without r229082. llvm-svn: 229205
2015-02-14 09:35:12 +08:00
RValue Res = XLValue.isGlobalReg()
? CGF.EmitLoadOfLValue(XLValue, Loc)
NFC: make AtomicOrdering an enum class Summary: See LLVM change D18775 for details, this change depends on it. Reviewers: jyknight, reames Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D18776 llvm-svn: 265569
2016-04-07 01:26:42 +08:00
: CGF.EmitAtomicLoad(
XLValue, Loc,
IsSeqCst ? llvm::AtomicOrdering::SequentiallyConsistent
: llvm::AtomicOrdering::Monotonic,
XLValue.isVolatile());
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
// OpenMP, 2.12.6, atomic Construct
// Any atomic construct with a seq_cst clause forces the atomically
// performed operation to include an implicit flush operation without a
// list.
if (IsSeqCst)
[OPENMP] Rename methods of OpenMPRuntime class. NFC. llvm-svn: 230470
2015-02-25 16:32:46 +08:00
CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc);
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
CGF.emitOMPSimpleStore(VLValue, Res, X->getType().getNonReferenceType(), Loc);
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
}
[OPENMP] Codegen for "#pragma omp atomic write" For global reg lvalue - use regular store through global register. For simple lvalue - use simple atomic store. For bitfields, vector element, extended vector elements - the original value of the whole storage (for vector elements) or of some aligned value (for bitfields) is atomically read, the part of this value for the given lvalue is modified and then use atomic compare-and-exchange operation to try to atomically write modified value (if it was not modified). Also, changes in this patch fix the bug for '#pragma omp atomic read' applied to extended vector elements. Differential Revision: http://reviews.llvm.org/D7369 llvm-svn: 230736
2015-02-27 14:33:30 +08:00
static void EmitOMPAtomicWriteExpr(CodeGenFunction &CGF, bool IsSeqCst,
const Expr *X, const Expr *E,
SourceLocation Loc) {
// x = expr;
assert(X->isLValue() && "X of 'omp atomic write' is not lvalue");
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
emitSimpleAtomicStore(CGF, IsSeqCst, CGF.EmitLValue(X), CGF.EmitAnyExpr(E));
[OPENMP] Codegen for "#pragma omp atomic write" For global reg lvalue - use regular store through global register. For simple lvalue - use simple atomic store. For bitfields, vector element, extended vector elements - the original value of the whole storage (for vector elements) or of some aligned value (for bitfields) is atomically read, the part of this value for the given lvalue is modified and then use atomic compare-and-exchange operation to try to atomically write modified value (if it was not modified). Also, changes in this patch fix the bug for '#pragma omp atomic read' applied to extended vector elements. Differential Revision: http://reviews.llvm.org/D7369 llvm-svn: 230736
2015-02-27 14:33:30 +08:00
// OpenMP, 2.12.6, atomic Construct
// Any atomic construct with a seq_cst clause forces the atomically
// performed operation to include an implicit flush operation without a
// list.
if (IsSeqCst)
CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc);
}
Make helper functions static. NFC. llvm-svn: 236315
2015-05-01 21:59:53 +08:00
static std::pair<bool, RValue> emitOMPAtomicRMW(CodeGenFunction &CGF, LValue X,
RValue Update,
BinaryOperatorKind BO,
llvm::AtomicOrdering AO,
bool IsXLHSInRHSPart) {
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
auto &Context = CGF.CGM.getContext();
// Allow atomicrmw only if 'x' and 'update' are integer values, lvalue for 'x'
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
// expression is simple and atomic is allowed for the given type for the
// target platform.
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
if (BO == BO_Comma || !Update.isScalar() ||
[OPENMP] Fixed atomic construct with non-integer expressions. Do not emit 'atomicrmw' instruction for simple atomic constructs with non-integer expressions. llvm-svn: 236828
2015-05-08 19:47:16 +08:00
!Update.getScalarVal()->getType()->isIntegerTy() ||
!X.isSimple() || (!isa<llvm::ConstantInt>(Update.getScalarVal()) &&
(Update.getScalarVal()->getType() !=
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
X.getAddress().getElementType())) ||
!X.getAddress().getElementType()->isIntegerTy() ||
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
!Context.getTargetInfo().hasBuiltinAtomic(
Context.getTypeSize(X.getType()), Context.toBits(X.getAlignment())))
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
return std::make_pair(false, RValue::get(nullptr));
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
llvm::AtomicRMWInst::BinOp RMWOp;
switch (BO) {
case BO_Add:
RMWOp = llvm::AtomicRMWInst::Add;
break;
case BO_Sub:
if (!IsXLHSInRHSPart)
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
return std::make_pair(false, RValue::get(nullptr));
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
RMWOp = llvm::AtomicRMWInst::Sub;
break;
case BO_And:
RMWOp = llvm::AtomicRMWInst::And;
break;
case BO_Or:
RMWOp = llvm::AtomicRMWInst::Or;
break;
case BO_Xor:
RMWOp = llvm::AtomicRMWInst::Xor;
break;
case BO_LT:
RMWOp = X.getType()->hasSignedIntegerRepresentation()
? (IsXLHSInRHSPart ? llvm::AtomicRMWInst::Min
: llvm::AtomicRMWInst::Max)
: (IsXLHSInRHSPart ? llvm::AtomicRMWInst::UMin
: llvm::AtomicRMWInst::UMax);
break;
case BO_GT:
RMWOp = X.getType()->hasSignedIntegerRepresentation()
? (IsXLHSInRHSPart ? llvm::AtomicRMWInst::Max
: llvm::AtomicRMWInst::Min)
: (IsXLHSInRHSPart ? llvm::AtomicRMWInst::UMax
: llvm::AtomicRMWInst::UMin);
break;
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
case BO_Assign:
RMWOp = llvm::AtomicRMWInst::Xchg;
break;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
case BO_Mul:
case BO_Div:
case BO_Rem:
case BO_Shl:
case BO_Shr:
case BO_LAnd:
case BO_LOr:
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
return std::make_pair(false, RValue::get(nullptr));
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
case BO_PtrMemD:
case BO_PtrMemI:
case BO_LE:
case BO_GE:
case BO_EQ:
case BO_NE:
case BO_AddAssign:
case BO_SubAssign:
case BO_AndAssign:
case BO_OrAssign:
case BO_XorAssign:
case BO_MulAssign:
case BO_DivAssign:
case BO_RemAssign:
case BO_ShlAssign:
case BO_ShrAssign:
case BO_Comma:
llvm_unreachable("Unsupported atomic update operation");
}
auto *UpdateVal = Update.getScalarVal();
if (auto *IC = dyn_cast<llvm::ConstantInt>(UpdateVal)) {
UpdateVal = CGF.Builder.CreateIntCast(
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
IC, X.getAddress().getElementType(),
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
X.getType()->hasSignedIntegerRepresentation());
}
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
auto *Res = CGF.Builder.CreateAtomicRMW(RMWOp, X.getPointer(), UpdateVal, AO);
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
return std::make_pair(true, RValue::get(Res));
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
}
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
std::pair<bool, RValue> CodeGenFunction::EmitOMPAtomicSimpleUpdateExpr(
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
LValue X, RValue E, BinaryOperatorKind BO, bool IsXLHSInRHSPart,
llvm::AtomicOrdering AO, SourceLocation Loc,
const llvm::function_ref<RValue(RValue)> &CommonGen) {
// Update expressions are allowed to have the following forms:
// x binop= expr; -> xrval + expr;
// x++, ++x -> xrval + 1;
// x--, --x -> xrval - 1;
// x = x binop expr; -> xrval binop expr
// x = expr Op x; - > expr binop xrval;
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
auto Res = emitOMPAtomicRMW(*this, X, E, BO, AO, IsXLHSInRHSPart);
if (!Res.first) {
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
if (X.isGlobalReg()) {
// Emit an update expression: 'xrval' binop 'expr' or 'expr' binop
// 'xrval'.
EmitStoreThroughLValue(CommonGen(EmitLoadOfLValue(X, Loc)), X);
} else {
// Perform compare-and-swap procedure.
EmitAtomicUpdate(X, AO, CommonGen, X.getType().isVolatileQualified());
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
}
}
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
return Res;
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
}
static void EmitOMPAtomicUpdateExpr(CodeGenFunction &CGF, bool IsSeqCst,
const Expr *X, const Expr *E,
const Expr *UE, bool IsXLHSInRHSPart,
SourceLocation Loc) {
assert(isa<BinaryOperator>(UE->IgnoreImpCasts()) &&
"Update expr in 'atomic update' must be a binary operator.");
auto *BOUE = cast<BinaryOperator>(UE->IgnoreImpCasts());
// Update expressions are allowed to have the following forms:
// x binop= expr; -> xrval + expr;
// x++, ++x -> xrval + 1;
// x--, --x -> xrval - 1;
// x = x binop expr; -> xrval binop expr
// x = expr Op x; - > expr binop xrval;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
assert(X->isLValue() && "X of 'omp atomic update' is not lvalue");
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
LValue XLValue = CGF.EmitLValue(X);
RValue ExprRValue = CGF.EmitAnyExpr(E);
NFC: make AtomicOrdering an enum class Summary: See LLVM change D18775 for details, this change depends on it. Reviewers: jyknight, reames Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D18776 llvm-svn: 265569
2016-04-07 01:26:42 +08:00
auto AO = IsSeqCst ? llvm::AtomicOrdering::SequentiallyConsistent
: llvm::AtomicOrdering::Monotonic;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
auto *LHS = cast<OpaqueValueExpr>(BOUE->getLHS()->IgnoreImpCasts());
auto *RHS = cast<OpaqueValueExpr>(BOUE->getRHS()->IgnoreImpCasts());
auto *XRValExpr = IsXLHSInRHSPart ? LHS : RHS;
auto *ERValExpr = IsXLHSInRHSPart ? RHS : LHS;
auto Gen =
[&CGF, UE, ExprRValue, XRValExpr, ERValExpr](RValue XRValue) -> RValue {
CodeGenFunction::OpaqueValueMapping MapExpr(CGF, ERValExpr, ExprRValue);
CodeGenFunction::OpaqueValueMapping MapX(CGF, XRValExpr, XRValue);
return CGF.EmitAnyExpr(UE);
};
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
(void)CGF.EmitOMPAtomicSimpleUpdateExpr(
XLValue, ExprRValue, BOUE->getOpcode(), IsXLHSInRHSPart, AO, Loc, Gen);
// OpenMP, 2.12.6, atomic Construct
// Any atomic construct with a seq_cst clause forces the atomically
// performed operation to include an implicit flush operation without a
// list.
if (IsSeqCst)
CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc);
}
static RValue convertToType(CodeGenFunction &CGF, RValue Value,
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
QualType SourceType, QualType ResType,
SourceLocation Loc) {
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
switch (CGF.getEvaluationKind(ResType)) {
case TEK_Scalar:
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
return RValue::get(
convertToScalarValue(CGF, Value, SourceType, ResType, Loc));
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
case TEK_Complex: {
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
auto Res = convertToComplexValue(CGF, Value, SourceType, ResType, Loc);
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
return RValue::getComplex(Res.first, Res.second);
}
case TEK_Aggregate:
break;
}
llvm_unreachable("Must be a scalar or complex.");
}
static void EmitOMPAtomicCaptureExpr(CodeGenFunction &CGF, bool IsSeqCst,
bool IsPostfixUpdate, const Expr *V,
const Expr *X, const Expr *E,
const Expr *UE, bool IsXLHSInRHSPart,
SourceLocation Loc) {
assert(X->isLValue() && "X of 'omp atomic capture' is not lvalue");
assert(V->isLValue() && "V of 'omp atomic capture' is not lvalue");
RValue NewVVal;
LValue VLValue = CGF.EmitLValue(V);
LValue XLValue = CGF.EmitLValue(X);
RValue ExprRValue = CGF.EmitAnyExpr(E);
NFC: make AtomicOrdering an enum class Summary: See LLVM change D18775 for details, this change depends on it. Reviewers: jyknight, reames Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D18776 llvm-svn: 265569
2016-04-07 01:26:42 +08:00
auto AO = IsSeqCst ? llvm::AtomicOrdering::SequentiallyConsistent
: llvm::AtomicOrdering::Monotonic;
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
QualType NewVValType;
if (UE) {
// 'x' is updated with some additional value.
assert(isa<BinaryOperator>(UE->IgnoreImpCasts()) &&
"Update expr in 'atomic capture' must be a binary operator.");
auto *BOUE = cast<BinaryOperator>(UE->IgnoreImpCasts());
// Update expressions are allowed to have the following forms:
// x binop= expr; -> xrval + expr;
// x++, ++x -> xrval + 1;
// x--, --x -> xrval - 1;
// x = x binop expr; -> xrval binop expr
// x = expr Op x; - > expr binop xrval;
auto *LHS = cast<OpaqueValueExpr>(BOUE->getLHS()->IgnoreImpCasts());
auto *RHS = cast<OpaqueValueExpr>(BOUE->getRHS()->IgnoreImpCasts());
auto *XRValExpr = IsXLHSInRHSPart ? LHS : RHS;
NewVValType = XRValExpr->getType();
auto *ERValExpr = IsXLHSInRHSPart ? RHS : LHS;
auto &&Gen = [&CGF, &NewVVal, UE, ExprRValue, XRValExpr, ERValExpr,
Remove unused lambda captures. NFC llvm-svn: 291939
2017-01-14 02:55:32 +08:00
IsPostfixUpdate](RValue XRValue) -> RValue {
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
CodeGenFunction::OpaqueValueMapping MapExpr(CGF, ERValExpr, ExprRValue);
CodeGenFunction::OpaqueValueMapping MapX(CGF, XRValExpr, XRValue);
RValue Res = CGF.EmitAnyExpr(UE);
NewVVal = IsPostfixUpdate ? XRValue : Res;
return Res;
};
auto Res = CGF.EmitOMPAtomicSimpleUpdateExpr(
XLValue, ExprRValue, BOUE->getOpcode(), IsXLHSInRHSPart, AO, Loc, Gen);
if (Res.first) {
// 'atomicrmw' instruction was generated.
if (IsPostfixUpdate) {
// Use old value from 'atomicrmw'.
NewVVal = Res.second;
} else {
// 'atomicrmw' does not provide new value, so evaluate it using old
// value of 'x'.
CodeGenFunction::OpaqueValueMapping MapExpr(CGF, ERValExpr, ExprRValue);
CodeGenFunction::OpaqueValueMapping MapX(CGF, XRValExpr, Res.second);
NewVVal = CGF.EmitAnyExpr(UE);
}
}
} else {
// 'x' is simply rewritten with some 'expr'.
NewVValType = X->getType().getNonReferenceType();
ExprRValue = convertToType(CGF, ExprRValue, E->getType(),
Propagate SourceLocations through to get a Loc on float_cast_overflow Summary: float_cast_overflow is the only UBSan check without a source location attached. This patch propagates SourceLocations where necessary to get them to the EmitCheck() call. Reviewers: rsmith, ABataev, rjmccall Subscribers: cfe-commits Differential Revision: http://reviews.llvm.org/D11757 llvm-svn: 244568
2015-08-11 12:19:28 +08:00
X->getType().getNonReferenceType(), Loc);
Remove unused lambda captures. NFC llvm-svn: 291939
2017-01-14 02:55:32 +08:00
auto &&Gen = [&NewVVal, ExprRValue](RValue XRValue) -> RValue {
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
NewVVal = XRValue;
return ExprRValue;
};
// Try to perform atomicrmw xchg, otherwise simple exchange.
auto Res = CGF.EmitOMPAtomicSimpleUpdateExpr(
XLValue, ExprRValue, /*BO=*/BO_Assign, /*IsXLHSInRHSPart=*/false, AO,
Loc, Gen);
if (Res.first) {
// 'atomicrmw' instruction was generated.
NewVVal = IsPostfixUpdate ? Res.second : ExprRValue;
}
}
// Emit post-update store to 'v' of old/new 'x' value.
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
CGF.emitOMPSimpleStore(VLValue, NewVVal, NewVValType, Loc);
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
// OpenMP, 2.12.6, atomic Construct
// Any atomic construct with a seq_cst clause forces the atomically
// performed operation to include an implicit flush operation without a
// list.
if (IsSeqCst)
CGF.CGM.getOpenMPRuntime().emitFlush(CGF, llvm::None, Loc);
}
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
static void EmitOMPAtomicExpr(CodeGenFunction &CGF, OpenMPClauseKind Kind,
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
bool IsSeqCst, bool IsPostfixUpdate,
const Expr *X, const Expr *V, const Expr *E,
const Expr *UE, bool IsXLHSInRHSPart,
SourceLocation Loc) {
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
switch (Kind) {
case OMPC_read:
EmitOMPAtomicReadExpr(CGF, IsSeqCst, X, V, Loc);
break;
case OMPC_write:
[OPENMP] Codegen for "#pragma omp atomic write" For global reg lvalue - use regular store through global register. For simple lvalue - use simple atomic store. For bitfields, vector element, extended vector elements - the original value of the whole storage (for vector elements) or of some aligned value (for bitfields) is atomically read, the part of this value for the given lvalue is modified and then use atomic compare-and-exchange operation to try to atomically write modified value (if it was not modified). Also, changes in this patch fix the bug for '#pragma omp atomic read' applied to extended vector elements. Differential Revision: http://reviews.llvm.org/D7369 llvm-svn: 230736
2015-02-27 14:33:30 +08:00
EmitOMPAtomicWriteExpr(CGF, IsSeqCst, X, E, Loc);
break;
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
case OMPC_unknown:
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
case OMPC_update:
[OPENMP] Codegen for 'atomic update' construct. Adds atomic update codegen for the following forms of expressions: x binop= expr; x++; ++x; x--; --x; x = x binop expr; x = expr binop x; If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted: bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: ... Differential Revision: http://reviews.llvm.org/D8536 llvm-svn: 233513
2015-03-30 13:20:59 +08:00
EmitOMPAtomicUpdateExpr(CGF, IsSeqCst, X, E, UE, IsXLHSInRHSPart, Loc);
break;
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
case OMPC_capture:
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
EmitOMPAtomicCaptureExpr(CGF, IsSeqCst, IsPostfixUpdate, V, X, E, UE,
IsXLHSInRHSPart, Loc);
break;
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
case OMPC_if:
case OMPC_final:
case OMPC_num_threads:
case OMPC_private:
case OMPC_firstprivate:
case OMPC_lastprivate:
case OMPC_reduction:
case OMPC_safelen:
[OPENMP 4.1] Initial support for 'simdlen' clause. Add parsing/sema analysis for 'simdlen' clause in simd directives. Also add check that if both 'safelen' and 'simdlen' clauses are specified, the value of 'simdlen' parameter is less than the value of 'safelen' parameter. llvm-svn: 245692
2015-08-21 19:14:16 +08:00
case OMPC_simdlen:
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
case OMPC_collapse:
case OMPC_default:
case OMPC_seq_cst:
case OMPC_shared:
case OMPC_linear:
case OMPC_aligned:
case OMPC_copyin:
case OMPC_copyprivate:
case OMPC_flush:
case OMPC_proc_bind:
case OMPC_schedule:
case OMPC_ordered:
case OMPC_nowait:
case OMPC_untied:
case OMPC_threadprivate:
[OPENMP] Initial support for 'depend' clause (4.0). Parsing and sema analysis (without support for array sections in arguments) for 'depend' clause (used in 'task' directive, OpenMP 4.0). llvm-svn: 240409
2015-06-23 22:25:19 +08:00
case OMPC_depend:
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
case OMPC_mergeable:
This patch commits OpenMP 4 target device clauses This is committed on behalf of Kelvin Li http://reviews.llvm.org/D11469?id=31227 llvm-svn: 244325
2015-08-08 00:16:36 +08:00
case OMPC_device:
[OPENMP 4.1] Add 'threads' clause for '#pragma omp ordered'. OpenMP 4.1 extends format of '#pragma omp ordered'. It adds 3 additional clauses: 'threads', 'simd' and 'depend'. If no clause is specified, the ordered construct behaves as if the threads clause had been specified. If the threads clause is specified, the threads in the team executing the loop region execute ordered regions sequentially in the order of the loop iterations. The loop region to which an ordered region without any clause or with a threads clause binds must have an ordered clause without the parameter specified on the corresponding loop directive. llvm-svn: 248569
2015-09-25 18:37:12 +08:00
case OMPC_threads:
[OPENMP 4.1] Add 'simd' clause for 'ordered' directive. Parsing and sema analysis for 'simd' clause in 'ordered' directive. Description If the simd clause is specified, the ordered regions encountered by any thread will use only a single SIMD lane to execute the ordered regions in the order of the loop iterations. Restrictions An ordered construct with the simd clause is the only OpenMP construct that can appear in the simd region llvm-svn: 248696
2015-09-28 14:39:35 +08:00
case OMPC_simd:
[OpenMP] Parsing and sema support for map clause http://reviews.llvm.org/D14134 llvm-svn: 253849
2015-11-23 13:32:03 +08:00
case OMPC_map:
[OpenMP] Parsing and sema support for num_teams clause http://reviews.llvm.org/D14802 llvm-svn: 254019
2015-11-25 04:50:12 +08:00
case OMPC_num_teams:
[OpenMP] Parsing and sema support for thread_limit clause. http://reviews.llvm.org/D15029 llvm-svn: 254207
2015-11-28 02:47:36 +08:00
case OMPC_thread_limit:
[OPENMP 4.5] Parsing/sema analysis for 'priority' clause. OpenMP 4.5 defines new clause 'priority' for 'task', 'taskloop' and 'taskloop simd' directives. Added parsing and sema analysis for 'priority' clause in 'task' and 'taskloop' directives. llvm-svn: 254398
2015-12-01 18:17:31 +08:00
case OMPC_priority:
[OPENMP 4.5] parsing/sema support for 'grainsize' clause. OpenMP 4.5 adds 'taksloop' and 'taskloop simd' directives, which have 'grainsize' clause. Patch adds parsing/sema analysis of this clause. llvm-svn: 254903
2015-12-07 20:52:51 +08:00
case OMPC_grainsize:
[OPENMP 4.5] parsing/sema support for 'nogroup' clause. OpenMP 4.5 adds 'taskloop' and 'taskloop simd' directives. These directives have new 'nogroup' clause. Patch adds basic parsing/sema support for this clause. llvm-svn: 254899
2015-12-07 18:51:44 +08:00
case OMPC_nogroup:
[OPENMP 4.5] Parsing/sema for 'num_tasks' clause. OpenMP 4.5 adds directives 'taskloop' and 'taskloop simd'. These directives support clause 'num_tasks'. Patch adds parsing/semantic analysis for this clause. llvm-svn: 255008
2015-12-08 20:06:20 +08:00
case OMPC_num_tasks:
[OPENMP 4.5] Parsing/sema for 'hint' clause of 'critical' directive. OpenMP 4.5 adds 'hint' clause to critical directive. Patch adds parsing/semantic analysis for this clause. llvm-svn: 255625
2015-12-15 16:19:24 +08:00
case OMPC_hint:
Add OpenMP dist_schedule clause to distribute directive and related regression tests. llvm-svn: 257917
2016-01-16 02:50:31 +08:00
case OMPC_dist_schedule:
[OpenMP] Parsing + sema for defaultmap clause. Summary: This patch adds parsing + sema for the defaultmap clause associated with the target directive (among others). Reviewers: ABataev Differential Revision: http://reviews.llvm.org/D16527 llvm-svn: 258817
2016-01-27 00:37:23 +08:00
case OMPC_defaultmap:
[OPENMP 4.0] Support for 'uniform' clause in 'declare simd' directive. OpenMP 4.0 defines clause 'uniform' in 'declare simd' directive: 'uniform' '(' <argument-list> ')' The uniform clause declares one or more arguments to have an invariant value for all concurrent invocations of the function in the execution of a single SIMD loop. The special this pointer can be used as if was one of the arguments to the function in any of the linear, aligned, or uniform clauses. llvm-svn: 266041
2016-04-12 13:28:34 +08:00
case OMPC_uniform:
[OpenMP] Parsing and sema support for the to clause Summary: The patch contains the parsing and sema support for the `to` clause. Patch based on the original post by Kelvin Li. Reviewers: carlo.bertolli, hfinkel, kkwli0, arpith-jacob, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D18597 llvm-svn: 270880
2016-05-27 01:39:58 +08:00
case OMPC_to:
[OpenMP] Parsing and sema support for the from clause Summary: The patch contains the parsing and sema support for the `from` clause. Patch based on the original post by Kelvin Li. Reviewers: hfinkel, carlo.bertolli, kkwli0, arpith-jacob, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D18488 llvm-svn: 270882
2016-05-27 01:49:04 +08:00
case OMPC_from:
[OpenMP] Initial implementation of parse+sema for clause use_device_ptr of 'target data' http://reviews.llvm.org/D21904 This patch is similar to the implementation of 'private' clause: it adds a list of private pointers to be used within the target data region to store the device pointers returned by the runtime. Please refer to the following document for a full description of what the runtime witll return in this case (page 10 and 11): https://github.com/clang-omp/OffloadingDesign I am happy to answer any question related to the runtime interface to help reviewing this patch. llvm-svn: 275271
2016-07-13 23:37:16 +08:00
case OMPC_use_device_ptr:
[OpenMP] Initial implementation of parse+sema for OpenMP clause 'is_device_ptr' of target http://reviews.llvm.org/D22070 llvm-svn: 275282
2016-07-14 01:16:49 +08:00
case OMPC_is_device_ptr:
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
llvm_unreachable("Clause is not allowed in 'omp atomic'.");
}
}
void CodeGenFunction::EmitOMPAtomicDirective(const OMPAtomicDirective &S) {
[OpenMP] Make the filetered clause iterator a real iterator and type safe. This replaces the filtered generic iterator with a type-specfic one based on dyn_cast instead of comparing the kind enum. This allows us to use range-based for loops and eliminates casts. No functionality change intended. llvm-svn: 246384
2015-08-30 23:12:28 +08:00
bool IsSeqCst = S.getSingleClause<OMPSeqCstClause>();
[OPENMP] CodeGen for "omp atomic read [seq_cst]" directive. "omp atomic read [seq_cst]" accepts expressions "v=x;". In this patch we perform an atomic load of "x" (using builtin atomic loading instructions or a call to "atomic_load()" for simple lvalues and "kmpc_atomic_start();load <x>;kmpc_atomic_end();" for other lvalues), convert the result of loading to type of "v" (using EmitScalarConversion() for simple types and EmitComplexToScalarConversion() for conversions from complex to scalar) and then store the result in "v".) Differential Revision: http://reviews.llvm.org/D6431 llvm-svn: 226788
2015-01-22 14:17:56 +08:00
OpenMPClauseKind Kind = OMPC_unknown;
for (auto *C : S.clauses()) {
// Find first clause (skip seq_cst clause, if it is first).
if (C->getClauseKind() != OMPC_seq_cst) {
Kind = C->getClauseKind();
break;
}
}
[OPENMP] Fix for ExprWithCleanups in 'omp atomic' constructs. This patch allows using of ExprWithCleanups expressions and other complex expressions in 'omp atomic' construct Differential Revision: http://reviews.llvm.org/D8200 llvm-svn: 231905
2015-03-11 12:48:56 +08:00
const auto *CS =
S.getAssociatedStmt()->IgnoreContainers(/*IgnoreCaptured=*/true);
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
if (const auto *EWC = dyn_cast<ExprWithCleanups>(CS)) {
[OPENMP] Fix for ExprWithCleanups in 'omp atomic' constructs. This patch allows using of ExprWithCleanups expressions and other complex expressions in 'omp atomic' construct Differential Revision: http://reviews.llvm.org/D8200 llvm-svn: 231905
2015-03-11 12:48:56 +08:00
enterFullExpression(EWC);
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
}
// Processing for statements under 'atomic capture'.
if (const auto *Compound = dyn_cast<CompoundStmt>(CS)) {
for (const auto *C : Compound->body()) {
if (const auto *EWC = dyn_cast<ExprWithCleanups>(C)) {
enterFullExpression(EWC);
}
}
}
[OPENMP] Fix for ExprWithCleanups in 'omp atomic' constructs. This patch allows using of ExprWithCleanups expressions and other complex expressions in 'omp atomic' construct Differential Revision: http://reviews.llvm.org/D8200 llvm-svn: 231905
2015-03-11 12:48:56 +08:00
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S, Kind, IsSeqCst, CS](CodeGenFunction &CGF,
PrePostActionTy &) {
[OPENMP] Fix debug info for 'atomic' construct. Debug info for statement under 'atomic' construct must point exactly to that statement, not the directive itself. llvm-svn: 255487
2015-12-14 17:26:19 +08:00
CGF.EmitStopPoint(CS);
[OPENMP] Codegen for 'atomic capture'. Adds codegen for 'atomic capture' constructs with the following forms of expressions/statements: v = x binop= expr; v = x++; v = ++x; v = x--; v = --x; v = x = x binop expr; v = x = expr binop x; {v = x; x = binop= expr;} {v = x; x++;} {v = x; ++x;} {v = x; x--;} {v = x; --x;} {x = x binop expr; v = x;} {x binop= expr; v = x;} {x++; v = x;} {++x; v = x;} {x--; v = x;} {--x; v = x;} {x = x binop expr; v = x;} {x = expr binop x; v = x;} {v = x; x = expr;} If x and expr are integer and binop is associative or x is a LHS in a RHS of the assignment expression, and atomics are allowed for type of x on the target platform atomicrmw instruction is emitted. Otherwise compare-and-swap sequence is emitted. Update of 'v' is not required to be be atomic with respect to the read or write of the 'x'. bb: ... atomic load <x> cont: <expected> = phi [ <x>, label %bb ], [ <new_failed>, %cont ] <desired> = <expected> binop <expr> <res> = cmpxchg atomic &<x>, desired, expected <new_failed> = <res>.field1; br <res>field2, label %exit, label %cont exit: atomic store <old/new x>, <v> ... Differential Revision: http://reviews.llvm.org/D9049 llvm-svn: 235573
2015-04-23 14:35:10 +08:00
EmitOMPAtomicExpr(CGF, Kind, IsSeqCst, S.isPostfixUpdate(), S.getX(),
S.getV(), S.getExpr(), S.getUpdateExpr(),
S.isXLHSInRHSPart(), S.getLocStart());
[OPENMP] Refactoring of codegen for OpenMP directives. Refactored API of OpenMPRuntime for compatibility with combined directives. Differential Revision: http://reviews.llvm.org/D8859 llvm-svn: 234564
2015-04-10 12:50:10 +08:00
};
[OPENMP] Fix for codegen of captured variables in inlined directives. Currently there is a problem with codegen of inlined directives inside lambdas, it may cause a crash during codegen because of incorrect capturing of variables. Patch fixes this problem. llvm-svn: 267677
2016-04-27 15:56:03 +08:00
OMPLexicalScope Scope(*this, S, /*AsInlined=*/true);
[OPENMP 4.0] Fixed codegen for 'cancellation point' construct. Generate the next code for 'cancellation point': if (__kmpc_cancellationpoint()) { __kmpc_cancel_barrier(); <exit construct>; } llvm-svn: 241336
2015-07-03 17:56:58 +08:00
CGM.getOpenMPRuntime().emitInlinedDirective(*this, OMPD_atomic, CodeGen);
[OPENMP] Initial parsing and sema analysis for 'atomic' directive. llvm-svn: 213639
2014-07-22 18:10:35 +08:00
}
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
static void emitCommonOMPTargetDirective(CodeGenFunction &CGF,
const OMPExecutableDirective &S,
const RegionCodeGenTy &CodeGen) {
assert(isOpenMPTargetExecutionDirective(S.getDirectiveKind()));
CodeGenModule &CGM = CGF.CGM;
[OpenMP] Target directive host codegen. This patch implements the outlining for offloading functions for code annotated with the OpenMP target directive. It uses a temporary naming of the outlined functions that will have to be updated later on once target side codegen and registration of offloading libraries is implemented - the naming needs to be made unique in the produced library. llvm-svn: 249148
2015-10-03 00:14:20 +08:00
const CapturedStmt &CS = *cast<CapturedStmt>(S.getAssociatedStmt());
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
llvm::Function *Fn = nullptr;
llvm::Constant *FnID = nullptr;
[OpenMP] Target directive host codegen. This patch implements the outlining for offloading functions for code annotated with the OpenMP target directive. It uses a temporary naming of the outlined functions that will have to be updated later on once target side codegen and registration of offloading libraries is implemented - the naming needs to be made unique in the produced library. llvm-svn: 249148
2015-10-03 00:14:20 +08:00
const Expr *IfCond = nullptr;
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
// Check for the at most one if clause associated with the target region.
for (const auto *C : S.getClausesOfKind<OMPIfClause>()) {
if (C->getNameModifier() == OMPD_unknown ||
C->getNameModifier() == OMPD_target) {
IfCond = C->getCondition();
break;
}
[OpenMP] Target directive host codegen. This patch implements the outlining for offloading functions for code annotated with the OpenMP target directive. It uses a temporary naming of the outlined functions that will have to be updated later on once target side codegen and registration of offloading libraries is implemented - the naming needs to be made unique in the produced library. llvm-svn: 249148
2015-10-03 00:14:20 +08:00
}
// Check if we have any device clause associated with the directive.
const Expr *Device = nullptr;
if (auto *C = S.getSingleClause<OMPDeviceClause>()) {
Device = C->getDevice();
}
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
// Check if we have an if clause whose conditional always evaluates to false
// or if we do not have any targets specified. If so the target region is not
// an offload entry point.
bool IsOffloadEntry = true;
if (IfCond) {
bool Val;
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
if (CGF.ConstantFoldsToSimpleInteger(IfCond, Val) && !Val)
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
IsOffloadEntry = false;
}
if (CGM.getLangOpts().OMPTargetTriples.empty())
IsOffloadEntry = false;
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
assert(CGF.CurFuncDecl && "No parent declaration for target region!");
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
StringRef ParentName;
// In case we have Ctors/Dtors we use the complete type variant to produce
// the mangling of the device outlined kernel.
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
if (auto *D = dyn_cast<CXXConstructorDecl>(CGF.CurFuncDecl))
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
ParentName = CGM.getMangledName(GlobalDecl(D, Ctor_Complete));
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
else if (auto *D = dyn_cast<CXXDestructorDecl>(CGF.CurFuncDecl))
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
ParentName = CGM.getMangledName(GlobalDecl(D, Dtor_Complete));
else
ParentName =
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
CGM.getMangledName(GlobalDecl(cast<FunctionDecl>(CGF.CurFuncDecl)));
[OpenMP] Reapply rL256842: [OpenMP] Offloading descriptor registration and device codegen. This patch attempts to fix the regressions identified when the patch was committed initially. Thanks to Michael Liao for identifying the fix in the offloading metadata generation related with side effects in evaluation of function arguments. llvm-svn: 256933
2016-01-06 21:42:12 +08:00
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
// Emit target region as a standalone region.
CGM.getOpenMPRuntime().emitTargetOutlinedFunction(S, ParentName, Fn, FnID,
IsOffloadEntry, CodeGen);
OMPLexicalScope Scope(CGF, S);
[OpenMP] Support for the if-clause on the combined directive 'target parallel'. The if-clause on the combined directive potentially applies to both the 'target' and the 'parallel' regions. Codegen'ing the if-clause on the combined directive requires additional support because the expression in the clause must be captured by the 'target' capture statement but not the 'parallel' capture statement. Note that this situation arises for other clauses such as num_threads. The OMPIfClause class inherits OMPClauseWithPreInit to support capturing of expressions in the clause. A member CaptureRegion is added to OMPClauseWithPreInit to indicate which captured statement (in this case 'target' but not 'parallel') captures these expressions. To ensure correct codegen of captured expressions in the presence of combined 'target' directives, OMPParallelScope was added to 'parallel' codegen. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28781 llvm-svn: 292437
2017-01-19 04:40:48 +08:00
llvm::SmallVector<llvm::Value *, 16> CapturedVars;
CGF.GenerateOpenMPCapturedVars(CS, CapturedVars);
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
CGM.getOpenMPRuntime().emitTargetCall(CGF, S, Fn, FnID, IfCond, Device,
[OpenMP] Target directive host codegen. This patch implements the outlining for offloading functions for code annotated with the OpenMP target directive. It uses a temporary naming of the outlined functions that will have to be updated later on once target side codegen and registration of offloading libraries is implemented - the naming needs to be made unique in the produced library. llvm-svn: 249148
2015-10-03 00:14:20 +08:00
CapturedVars);
[OPENMP] Initial parsing/sema analysis of 'target' directive. llvm-svn: 218110
2014-09-19 16:19:49 +08:00
}
[OpenMP] Refactor code that calls codegen for target regions on the device. This patch refactors code that calls codegen for target regions. Currently the codebase only supports the 'target' directive. The patch pulls out common target processing code into a static function that can be called by codegen for any target directive. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28752 llvm-svn: 292134
2017-01-16 23:26:02 +08:00
static void emitTargetRegion(CodeGenFunction &CGF, const OMPTargetDirective &S,
PrePostActionTy &Action) {
CodeGenFunction::OMPPrivateScope PrivateScope(CGF);
(void)CGF.EmitOMPFirstprivateClause(S, PrivateScope);
CGF.EmitOMPPrivateClause(S, PrivateScope);
(void)PrivateScope.Privatize();
Action.Enter(CGF);
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
}
void CodeGenFunction::EmitOMPTargetDeviceFunction(CodeGenModule &CGM,
StringRef ParentName,
const OMPTargetDirective &S) {
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
emitTargetRegion(CGF, S, Action);
};
llvm::Function *Fn;
llvm::Constant *Addr;
// Emit target region as a standalone region.
CGM.getOpenMPRuntime().emitTargetOutlinedFunction(
S, ParentName, Fn, Addr, /*IsOffloadEntry=*/true, CodeGen);
assert(Fn && Addr && "Target device function emission failed.");
}
void CodeGenFunction::EmitOMPTargetDirective(const OMPTargetDirective &S) {
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
emitTargetRegion(CGF, S, Action);
};
emitCommonOMPTargetDirective(*this, S, CodeGen);
}
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
static void emitCommonOMPTeamsDirective(CodeGenFunction &CGF,
const OMPExecutableDirective &S,
OpenMPDirectiveKind InnermostKind,
const RegionCodeGenTy &CodeGen) {
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
const CapturedStmt *CS = S.getCapturedStmt(OMPD_teams);
auto OutlinedFn = CGF.CGM.getOpenMPRuntime().emitTeamsOutlinedFunction(
S, *CS->getCapturedDecl()->param_begin(), InnermostKind, CodeGen);
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
[OpenMP] Codegen support for 'target teams' on the host. This patch adds support for codegen of 'target teams' on the host. This combined directive has two captured statements, one for the 'teams' region, and the other for the 'parallel'. This target teams region is offloaded using the __tgt_target_teams() call. The patch sets the number of teams as an argument to this call. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29084 llvm-svn: 293005
2017-01-25 10:18:43 +08:00
const OMPNumTeamsClause *NT = S.getSingleClause<OMPNumTeamsClause>();
const OMPThreadLimitClause *TL = S.getSingleClause<OMPThreadLimitClause>();
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
if (NT || TL) {
[OPENMP] Codegen for teams directive for NVPTX This patch implements the teams directive for the NVPTX backend. It is different from the host code generation path as it: Does not call kmpc_fork_teams. All necessary teams and threads are started upon touching the target region, when launching a CUDA kernel, and their execution is coordinated through sequential and parallel regions within the target region. Does not call kmpc_push_num_teams even if a num_teams of thread_limit clause is present. Setting the number of teams and the thread limit is implemented by the nvptx-related runtime. Please note that I am now passing a Clang Expr * to emitPushNumTeams instead of the originally chosen llvm::Value * type. The reason for that is that I want to avoid emitting expressions for num_teams and thread_limit if they are not needed in the target region. http://reviews.llvm.org/D17963 llvm-svn: 265304
2016-04-04 23:55:02 +08:00
Expr *NumTeams = (NT) ? NT->getNumTeams() : nullptr;
Expr *ThreadLimit = (TL) ? TL->getThreadLimit() : nullptr;
CGF.CGM.getOpenMPRuntime().emitNumTeamsClause(CGF, NumTeams, ThreadLimit,
S.getLocStart());
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
}
[OpenMP] Code generation for teams - kernel launching Summary: This patch implements the launching of a target region in the presence of a nested teams region, i.e calls tgt_target_teams with the required arguments gathered from the enclosed teams directive. The actual codegen of the region enclosed by the teams construct will be contributed in a separate patch. Reviewers: hfinkel, arpith-jacob, kkwli0, carlo.bertolli, ABataev Subscribers: cfe-commits, caomhin, fraggamuffin Differential Revision: http://reviews.llvm.org/D17019 llvm-svn: 262625
2016-03-04 00:20:23 +08:00
[OpenMP] Codegen support for 'target teams' on the host. This patch adds support for codegen of 'target teams' on the host. This combined directive has two captured statements, one for the 'teams' region, and the other for the 'parallel'. This target teams region is offloaded using the __tgt_target_teams() call. The patch sets the number of teams as an argument to this call. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29084 llvm-svn: 293005
2017-01-25 10:18:43 +08:00
OMPTeamsScope Scope(CGF, S);
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
llvm::SmallVector<llvm::Value *, 16> CapturedVars;
CGF.GenerateOpenMPCapturedVars(*CS, CapturedVars);
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
CGF.CGM.getOpenMPRuntime().emitTeamsCall(CGF, S, S.getLocStart(), OutlinedFn,
CapturedVars);
}
void CodeGenFunction::EmitOMPTeamsDirective(const OMPTeamsDirective &S) {
Fix typo in comment. NFC. llvm-svn: 289836
2016-12-16 01:55:32 +08:00
// Emit teams region as a standalone region.
[OPENMP] Allow runtime insert its own code inside OpenMP regions. Solution unifies interface of RegionCodeGenTy type to allow insert runtime-specific code before/after main codegen action defined in CGStmtOpenMP.cpp file. Runtime should not define its own RegionCodeGenTy for general OpenMP directives, but must be allowed to insert its own (required) code to support target specific codegen. llvm-svn: 264700
2016-03-29 13:34:15 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
OMPPrivateScope PrivateScope(CGF);
[OPENMP] firstprivate and private clauses of teams, host codegeneration Add code generation support for firstprivate and private clauses of teams on the host. Add extensive regression tests including lambda functions and vla testing. http://reviews.llvm.org/D17582 llvm-svn: 262663
2016-03-04 06:09:40 +08:00
(void)CGF.EmitOMPFirstprivateClause(S, PrivateScope);
CGF.EmitOMPPrivateClause(S, PrivateScope);
[OpenMP] Teams reduction on the NVPTX device. This patch implements codegen for the reduction clause on any teams construct for elementary data types. It builds on parallel reductions on the GPU. Subsequently, the team master writes to a unique location in a global memory scratchpad. The last team to do so loads and reduces this array to calculate the final result. This patch emits two helper functions that are used by the OpenMP runtime on the GPU to perform reductions across teams. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29879 llvm-svn: 295335
2017-02-17 00:48:49 +08:00
CGF.EmitOMPReductionClauseInit(S, PrivateScope);
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
(void)PrivateScope.Privatize();
CGF.EmitStmt(cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
[OpenMP] Teams reduction on the NVPTX device. This patch implements codegen for the reduction clause on any teams construct for elementary data types. It builds on parallel reductions on the GPU. Subsequently, the team master writes to a unique location in a global memory scratchpad. The last team to do so loads and reduces this array to calculate the final result. This patch emits two helper functions that are used by the OpenMP runtime on the GPU to perform reductions across teams. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29879 llvm-svn: 295335
2017-02-17 00:48:49 +08:00
CGF.EmitOMPReductionClauseFinal(S, /*ReductionKind=*/OMPD_teams);
Add code generation for teams directive inside target region llvm-svn: 262652
2016-03-04 04:34:23 +08:00
};
emitCommonOMPTeamsDirective(*this, S, OMPD_teams, CodeGen);
[OpenMP] Teams reduction on the NVPTX device. This patch implements codegen for the reduction clause on any teams construct for elementary data types. It builds on parallel reductions on the GPU. Subsequently, the team master writes to a unique location in a global memory scratchpad. The last team to do so loads and reduces this array to calculate the final result. This patch emits two helper functions that are used by the OpenMP runtime on the GPU to perform reductions across teams. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29879 llvm-svn: 295335
2017-02-17 00:48:49 +08:00
emitPostUpdateForReductionClause(
*this, S, [](CodeGenFunction &) -> llvm::Value * { return nullptr; });
[OPENMP] 'omp teams' directive basic support. Includes parsing and semantic analysis for 'omp teams' directive support from OpenMP 4.0. Adds additional analysis to 'omp target' directive with 'omp teams' directive. llvm-svn: 219385
2014-10-09 12:18:56 +08:00
}
[OPENMP 4.0] Initial support for 'omp cancellation point' construct. Add parsing and sema analysis for 'omp cancellation point' directive. llvm-svn: 241145
2015-07-01 14:57:41 +08:00
[OpenMP] Codegen support for 'target teams' on the host. This patch adds support for codegen of 'target teams' on the host. This combined directive has two captured statements, one for the 'teams' region, and the other for the 'parallel'. This target teams region is offloaded using the __tgt_target_teams() call. The patch sets the number of teams as an argument to this call. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29084 llvm-svn: 293005
2017-01-25 10:18:43 +08:00
static void emitTargetTeamsRegion(CodeGenFunction &CGF, PrePostActionTy &Action,
const OMPTargetTeamsDirective &S) {
auto *CS = S.getCapturedStmt(OMPD_teams);
Action.Enter(CGF);
auto &&CodeGen = [CS](CodeGenFunction &CGF, PrePostActionTy &) {
// TODO: Add support for clauses.
CGF.EmitStmt(CS->getCapturedStmt());
};
emitCommonOMPTeamsDirective(CGF, S, OMPD_teams, CodeGen);
}
void CodeGenFunction::EmitOMPTargetTeamsDeviceFunction(
CodeGenModule &CGM, StringRef ParentName,
const OMPTargetTeamsDirective &S) {
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
emitTargetTeamsRegion(CGF, Action, S);
};
llvm::Function *Fn;
llvm::Constant *Addr;
// Emit target region as a standalone region.
CGM.getOpenMPRuntime().emitTargetOutlinedFunction(
S, ParentName, Fn, Addr, /*IsOffloadEntry=*/true, CodeGen);
assert(Fn && Addr && "Target device function emission failed.");
}
void CodeGenFunction::EmitOMPTargetTeamsDirective(
const OMPTargetTeamsDirective &S) {
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
emitTargetTeamsRegion(CGF, Action, S);
};
emitCommonOMPTargetDirective(*this, S, CodeGen);
}
[OPENMP 4.0] Initial support for 'omp cancellation point' construct. Add parsing and sema analysis for 'omp cancellation point' directive. llvm-svn: 241145
2015-07-01 14:57:41 +08:00
void CodeGenFunction::EmitOMPCancellationPointDirective(
const OMPCancellationPointDirective &S) {
[OPENMP 4.0] Codegen for 'cancellation point' directive. The next code is generated for this construct: ``` if (__kmpc_cancellationpoint(ident_t *loc, kmp_int32 global_tid, kmp_int32 cncl_kind) != 0) <exit from outer innermost construct>; ``` llvm-svn: 241239
2015-07-02 12:17:07 +08:00
CGM.getOpenMPRuntime().emitCancellationPointCall(*this, S.getLocStart(),
S.getCancelRegion());
[OPENMP 4.0] Initial support for 'omp cancellation point' construct. Add parsing and sema analysis for 'omp cancellation point' directive. llvm-svn: 241145
2015-07-01 14:57:41 +08:00
}
[OPENMP 4.0] Initial support for 'omp cancel' construct. Implemented parsing/sema analysis + (de)serialization. llvm-svn: 241253
2015-07-02 19:25:17 +08:00
void CodeGenFunction::EmitOMPCancelDirective(const OMPCancelDirective &S) {
[OPENMP 4.0] Add 'if' clause for 'cancel' directive. Add parsing, sema analysis and codegen for 'if' clause in 'cancel' directive. llvm-svn: 247976
2015-09-18 16:07:34 +08:00
const Expr *IfCond = nullptr;
for (const auto *C : S.getClausesOfKind<OMPIfClause>()) {
if (C->getNameModifier() == OMPD_unknown ||
C->getNameModifier() == OMPD_cancel) {
IfCond = C->getCondition();
break;
}
}
CGM.getOpenMPRuntime().emitCancelCall(*this, S.getLocStart(), IfCond,
[OPENMP 4.0] Codegen for 'omp cancel' directive. Add the next codegen for 'omp cancel' directive: if (__kmpc_cancel()) { __kmpc_cancel_barrier(); <exit construct>; } llvm-svn: 241429
2015-07-06 13:50:32 +08:00
S.getCancelRegion());
[OPENMP 4.0] Initial support for 'omp cancel' construct. Implemented parsing/sema analysis + (de)serialization. llvm-svn: 241253
2015-07-02 19:25:17 +08:00
}
[OPENMP 4.0] Fixed codegen for 'cancellation point' construct. Generate the next code for 'cancellation point': if (__kmpc_cancellationpoint()) { __kmpc_cancel_barrier(); <exit construct>; } llvm-svn: 241336
2015-07-03 17:56:58 +08:00
CodeGenFunction::JumpDest
CodeGenFunction::getOMPCancelDestination(OpenMPDirectiveKind Kind) {
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
if (Kind == OMPD_parallel || Kind == OMPD_task ||
Kind == OMPD_target_parallel)
[OPENMP 4.0] Fixed codegen for 'cancellation point' construct. Generate the next code for 'cancellation point': if (__kmpc_cancellationpoint()) { __kmpc_cancel_barrier(); <exit construct>; } llvm-svn: 241336
2015-07-03 17:56:58 +08:00
return ReturnBlock;
[OPENMP] Emit __kmpc_cancel_barrier() and code for 'cancellation point' only if 'cancel' is found. Patch improves codegen for OpenMP constructs. If the OpenMP region does not have internal 'cancel' construct, a call to 'void __kmpc_barrier()' runtime function is generated for all implicit/explicit barriers. If the region has inner 'cancel' directive, then ``` if (__kmpc_cancel_barrier()) exit from outer construct; ``` code is generated. Also, the code for 'canellation point' directive is not generated if parent directive does not have 'cancel' directive. llvm-svn: 247681
2015-09-15 20:52:43 +08:00
assert(Kind == OMPD_for || Kind == OMPD_section || Kind == OMPD_sections ||
[OPENMP] Fixed codegen for 'omp cancel' construct. If 'omp cancel' construct is used in a worksharing construct it may cause hanging of the software in case if reduction clause is used. Patch fixes this problem by avoiding extra reduction processing for branches that were canceled. llvm-svn: 287227
2016-11-17 23:12:05 +08:00
Kind == OMPD_parallel_sections || Kind == OMPD_parallel_for ||
Kind == OMPD_distribute_parallel_for ||
Kind == OMPD_target_parallel_for);
return OMPCancelStack.getExitBlock();
[OPENMP 4.0] Fixed codegen for 'cancellation point' construct. Generate the next code for 'cancellation point': if (__kmpc_cancellationpoint()) { __kmpc_cancel_barrier(); <exit construct>; } llvm-svn: 241336
2015-07-03 17:56:58 +08:00
}
Commit for http://reviews.llvm.org/D10765 for OpenMP 4 target data directive parsing and sema. This commit is on behalf of Kelvin Li. llvm-svn: 242785
2015-07-21 21:44:28 +08:00
[OpenMP] Codegen for use_device_ptr clause. Summary: This patch adds support for the use_device_ptr clause. It includes changes in SEMA that could not be tested without codegen, namely, the use of the first private logic and mappable expressions support. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22691 llvm-svn: 276977
2016-07-28 22:23:26 +08:00
void CodeGenFunction::EmitOMPUseDevicePtrClause(
const OMPClause &NC, OMPPrivateScope &PrivateScope,
const llvm::DenseMap<const ValueDecl *, Address> &CaptureDeviceAddrMap) {
const auto &C = cast<OMPUseDevicePtrClause>(NC);
auto OrigVarIt = C.varlist_begin();
auto InitIt = C.inits().begin();
for (auto PvtVarIt : C.private_copies()) {
auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*OrigVarIt)->getDecl());
auto *InitVD = cast<VarDecl>(cast<DeclRefExpr>(*InitIt)->getDecl());
auto *PvtVD = cast<VarDecl>(cast<DeclRefExpr>(PvtVarIt)->getDecl());
// In order to identify the right initializer we need to match the
// declaration used by the mapping logic. In some cases we may get
// OMPCapturedExprDecl that refers to the original declaration.
const ValueDecl *MatchingVD = OrigVD;
if (auto *OED = dyn_cast<OMPCapturedExprDecl>(MatchingVD)) {
// OMPCapturedExprDecl are used to privative fields of the current
// structure.
auto *ME = cast<MemberExpr>(OED->getInit());
assert(isa<CXXThisExpr>(ME->getBase()) &&
"Base should be the current struct!");
MatchingVD = ME->getMemberDecl();
}
// If we don't have information about the current list item, move on to
// the next one.
auto InitAddrIt = CaptureDeviceAddrMap.find(MatchingVD);
if (InitAddrIt == CaptureDeviceAddrMap.end())
continue;
bool IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
// Initialize the temporary initialization variable with the address we
// get from the runtime library. We have to cast the source address
// because it is always a void *. References are materialized in the
// privatization scope, so the initialization here disregards the fact
// the original variable is a reference.
QualType AddrQTy =
getContext().getPointerType(OrigVD->getType().getNonReferenceType());
llvm::Type *AddrTy = ConvertTypeForMem(AddrQTy);
Address InitAddr = Builder.CreateBitCast(InitAddrIt->second, AddrTy);
setAddrOfLocalVar(InitVD, InitAddr);
// Emit private declaration, it will be initialized by the value we
// declaration we just added to the local declarations map.
EmitDecl(*PvtVD);
// The initialization variables reached its purpose in the emission
// ofthe previous declaration, so we don't need it anymore.
LocalDeclMap.erase(InitVD);
// Return the address of the private variable.
return GetAddrOfLocalVar(PvtVD);
});
assert(IsRegistered && "firstprivate var already registered as private");
// Silence the warning about unused variable.
(void)IsRegistered;
++OrigVarIt;
++InitIt;
}
}
Commit for http://reviews.llvm.org/D10765 for OpenMP 4 target data directive parsing and sema. This commit is on behalf of Kelvin Li. llvm-svn: 242785
2015-07-21 21:44:28 +08:00
// Generate the instructions for '#pragma omp target data' directive.
void CodeGenFunction::EmitOMPTargetDataDirective(
const OMPTargetDataDirective &S) {
[OpenMP] Codegen for use_device_ptr clause. Summary: This patch adds support for the use_device_ptr clause. It includes changes in SEMA that could not be tested without codegen, namely, the use of the first private logic and mappable expressions support. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22691 llvm-svn: 276977
2016-07-28 22:23:26 +08:00
CGOpenMPRuntime::TargetDataInfo Info(/*RequiresDevicePointerInfo=*/true);
// Create a pre/post action to signal the privatization of the device pointer.
// This action can be replaced by the OpenMP runtime code generation to
// deactivate privatization.
bool PrivatizeDevicePointers = false;
class DevicePointerPrivActionTy : public PrePostActionTy {
bool &PrivatizeDevicePointers;
public:
explicit DevicePointerPrivActionTy(bool &PrivatizeDevicePointers)
: PrePostActionTy(), PrivatizeDevicePointers(PrivatizeDevicePointers) {}
void Enter(CodeGenFunction &CGF) override {
PrivatizeDevicePointers = true;
}
};
DevicePointerPrivActionTy PrivAction(PrivatizeDevicePointers);
auto &&CodeGen = [&S, &Info, &PrivatizeDevicePointers](
CodeGenFunction &CGF, PrePostActionTy &Action) {
auto &&InnermostCodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &) {
CGF.EmitStmt(
cast<CapturedStmt>(S.getAssociatedStmt())->getCapturedStmt());
};
// Codegen that selects wheather to generate the privatization code or not.
auto &&PrivCodeGen = [&S, &Info, &PrivatizeDevicePointers,
&InnermostCodeGen](CodeGenFunction &CGF,
PrePostActionTy &Action) {
RegionCodeGenTy RCG(InnermostCodeGen);
PrivatizeDevicePointers = false;
// Call the pre-action to change the status of PrivatizeDevicePointers if
// needed.
Action.Enter(CGF);
if (PrivatizeDevicePointers) {
OMPPrivateScope PrivateScope(CGF);
// Emit all instances of the use_device_ptr clause.
for (const auto *C : S.getClausesOfKind<OMPUseDevicePtrClause>())
CGF.EmitOMPUseDevicePtrClause(*C, PrivateScope,
Info.CaptureDeviceAddrMap);
(void)PrivateScope.Privatize();
RCG(CGF);
} else
RCG(CGF);
};
// Forward the provided action to the privatization codegen.
RegionCodeGenTy PrivRCG(PrivCodeGen);
PrivRCG.setAction(Action);
// Notwithstanding the body of the region is emitted as inlined directive,
// we don't use an inline scope as changes in the references inside the
// region are expected to be visible outside, so we do not privative them.
OMPLexicalScope Scope(CGF, S);
CGF.CGM.getOpenMPRuntime().emitInlinedDirective(CGF, OMPD_target_data,
PrivRCG);
[OpenMP] Code generation for target data directive Summary: This patch adds support for the target data directive code generation. Part of the already existent functionality related with data maps is moved to a new function so that it could be reused. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: cfe-commits, fraggamuffin, caomhin Differential Revision: http://reviews.llvm.org/D17367 llvm-svn: 267811
2016-04-28 06:58:19 +08:00
};
[OpenMP] Codegen for use_device_ptr clause. Summary: This patch adds support for the use_device_ptr clause. It includes changes in SEMA that could not be tested without codegen, namely, the use of the first private logic and mappable expressions support. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22691 llvm-svn: 276977
2016-07-28 22:23:26 +08:00
RegionCodeGenTy RCG(CodeGen);
[OpenMP] Code generation for target data directive Summary: This patch adds support for the target data directive code generation. Part of the already existent functionality related with data maps is moved to a new function so that it could be reused. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: cfe-commits, fraggamuffin, caomhin Differential Revision: http://reviews.llvm.org/D17367 llvm-svn: 267811
2016-04-28 06:58:19 +08:00
// If we don't have target devices, don't bother emitting the data mapping
// code.
if (CGM.getLangOpts().OMPTargetTriples.empty()) {
[OpenMP] Codegen for use_device_ptr clause. Summary: This patch adds support for the use_device_ptr clause. It includes changes in SEMA that could not be tested without codegen, namely, the use of the first private logic and mappable expressions support. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22691 llvm-svn: 276977
2016-07-28 22:23:26 +08:00
RCG(*this);
[OpenMP] Code generation for target data directive Summary: This patch adds support for the target data directive code generation. Part of the already existent functionality related with data maps is moved to a new function so that it could be reused. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: cfe-commits, fraggamuffin, caomhin Differential Revision: http://reviews.llvm.org/D17367 llvm-svn: 267811
2016-04-28 06:58:19 +08:00
return;
}
// Check if we have any if clause associated with the directive.
const Expr *IfCond = nullptr;
if (auto *C = S.getSingleClause<OMPIfClause>())
IfCond = C->getCondition();
// Check if we have any device clause associated with the directive.
const Expr *Device = nullptr;
if (auto *C = S.getSingleClause<OMPDeviceClause>())
Device = C->getDevice();
[OpenMP] Codegen for use_device_ptr clause. Summary: This patch adds support for the use_device_ptr clause. It includes changes in SEMA that could not be tested without codegen, namely, the use of the first private logic and mappable expressions support. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: https://reviews.llvm.org/D22691 llvm-svn: 276977
2016-07-28 22:23:26 +08:00
// Set the action to signal privatization of device pointers.
RCG.setAction(PrivAction);
// Emit region code.
CGM.getOpenMPRuntime().emitTargetDataCalls(*this, S, IfCond, Device, RCG,
Info);
Commit for http://reviews.llvm.org/D10765 for OpenMP 4 target data directive parsing and sema. This commit is on behalf of Kelvin Li. llvm-svn: 242785
2015-07-21 21:44:28 +08:00
}
[OPENMP 4.5] Parsing/sema analysis for 'taskloop' directive. Adds initial parsing and semantic analysis for 'taskloop' directive. llvm-svn: 254367
2015-12-01 12:18:41 +08:00
[OpenMP] Parsing + sema for "target enter data" directive. Patch by Arpith Jacob. Thanks! llvm-svn: 258165
2016-01-20 03:15:56 +08:00
void CodeGenFunction::EmitOMPTargetEnterDataDirective(
const OMPTargetEnterDataDirective &S) {
[OpenMP] Code generation for target enter data directive Summary: This patch adds support for the target enter data directive code generation. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: cfe-commits, fraggamuffin, caomhin Differential Revision: http://reviews.llvm.org/D17368 llvm-svn: 267812
2016-04-28 07:07:29 +08:00
// If we don't have target devices, don't bother emitting the data mapping
// code.
if (CGM.getLangOpts().OMPTargetTriples.empty())
return;
// Check if we have any if clause associated with the directive.
const Expr *IfCond = nullptr;
if (auto *C = S.getSingleClause<OMPIfClause>())
IfCond = C->getCondition();
// Check if we have any device clause associated with the directive.
const Expr *Device = nullptr;
if (auto *C = S.getSingleClause<OMPDeviceClause>())
Device = C->getDevice();
[OpenMP] Codegen for target update directive. Summary: This patch implements the code generation for the `target update` directive. The implemntation relies on the logic already in place for target data standalone directives, i.e. target enter/exit data. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D20650 llvm-svn: 270886
2016-05-27 02:30:22 +08:00
CGM.getOpenMPRuntime().emitTargetDataStandAloneCall(*this, S, IfCond, Device);
[OpenMP] Parsing + sema for "target enter data" directive. Patch by Arpith Jacob. Thanks! llvm-svn: 258165
2016-01-20 03:15:56 +08:00
}
[OpenMP] Parsing + sema for "target exit data" directive. Patch by Arpith Jacob. Thanks! llvm-svn: 258177
2016-01-20 04:04:50 +08:00
void CodeGenFunction::EmitOMPTargetExitDataDirective(
const OMPTargetExitDataDirective &S) {
[OpenMP] Code generation for target exit data directive Summary: This patch adds support for the target exit data directive code generation. Given that, apart from the employed runtime call, target exit data requires the same code generation pattern as target enter data, the OpenMP codegen entry point was renamed and reused for both. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: cfe-commits, fraggamuffin, caomhin Differential Revision: http://reviews.llvm.org/D17369 llvm-svn: 267814
2016-04-28 07:14:30 +08:00
// If we don't have target devices, don't bother emitting the data mapping
// code.
if (CGM.getLangOpts().OMPTargetTriples.empty())
return;
// Check if we have any if clause associated with the directive.
const Expr *IfCond = nullptr;
if (auto *C = S.getSingleClause<OMPIfClause>())
IfCond = C->getCondition();
// Check if we have any device clause associated with the directive.
const Expr *Device = nullptr;
if (auto *C = S.getSingleClause<OMPDeviceClause>())
Device = C->getDevice();
[OpenMP] Codegen for target update directive. Summary: This patch implements the code generation for the `target update` directive. The implemntation relies on the logic already in place for target data standalone directives, i.e. target enter/exit data. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D20650 llvm-svn: 270886
2016-05-27 02:30:22 +08:00
CGM.getOpenMPRuntime().emitTargetDataStandAloneCall(*this, S, IfCond, Device);
[OpenMP] Parsing + sema for "target exit data" directive. Patch by Arpith Jacob. Thanks! llvm-svn: 258177
2016-01-20 04:04:50 +08:00
}
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
static void emitTargetParallelRegion(CodeGenFunction &CGF,
const OMPTargetParallelDirective &S,
PrePostActionTy &Action) {
// Get the captured statement associated with the 'parallel' region.
auto *CS = S.getCapturedStmt(OMPD_parallel);
Action.Enter(CGF);
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
auto &&CodeGen = [&S, CS](CodeGenFunction &CGF, PrePostActionTy &) {
CodeGenFunction::OMPPrivateScope PrivateScope(CGF);
(void)CGF.EmitOMPFirstprivateClause(S, PrivateScope);
CGF.EmitOMPPrivateClause(S, PrivateScope);
CGF.EmitOMPReductionClauseInit(S, PrivateScope);
(void)PrivateScope.Privatize();
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
// TODO: Add support for clauses.
CGF.EmitStmt(CS->getCapturedStmt());
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
CGF.EmitOMPReductionClauseFinal(S, /*ReductionKind=*/OMPD_parallel);
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
};
Recommit of [OpenMP] Initial implementation of code generation for pragma 'distribute parallel for' on host https://reviews.llvm.org/D29508 This patch makes the following additions: It abstracts away loop bound generation code from procedures associated with pragma 'for' and loops in general, in such a way that the same procedures can be used for 'distribute parallel for' without the need for a full re-implementation. It implements code generation for 'distribute parallel for' and adds regression tests. It includes tests for clauses. It is important to notice that most of the clauses are implemented as part of existing procedures. For instance, firstprivate is already implemented for 'distribute' and 'for' as separate pragmas. As the implementation of 'distribute parallel for' is based on the same procedures, then we automatically obtain implementation for such clauses without the need to add new code. However, this requires regression tests that verify correctness of produced code. llvm-svn: 301340
2017-04-26 01:52:12 +08:00
emitCommonOMPParallelDirective(CGF, S, OMPD_parallel, CodeGen,
emitEmptyBoundParameters);
[OpenMP] Parallel reduction on the NVPTX device. This patch implements codegen for the reduction clause on any parallel construct for elementary data types. An efficient implementation requires hierarchical reduction within a warp and a threadblock. It is complicated by the fact that variables declared in the stack of a CUDA thread cannot be shared with other threads. The patch creates a struct to hold reduction variables and a number of helper functions. The OpenMP runtime on the GPU implements reduction algorithms that uses these helper functions to perform reductions within a team. Variables are shared between CUDA threads using shuffle intrinsics. An implementation of reductions on the NVPTX device is substantially different to that of CPUs. However, this patch is written so that there are minimal changes to the rest of OpenMP codegen. The implemented design allows the compiler and runtime to be decoupled, i.e., the runtime does not need to know of the reduction operation(s), the type of the reduction variable(s), or the number of reductions. The design also allows reuse of host codegen, with appropriate specialization for the NVPTX device. While the patch does introduce a number of abstractions, the expected use case calls for inlining of the GPU OpenMP runtime. After inlining and optimizations in LLVM, these abstractions are unwound and performance of OpenMP reductions is comparable to CUDA-canonical code. Patch by Tian Jin in collaboration with Arpith Jacob Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D29758 llvm-svn: 295333
2017-02-17 00:20:16 +08:00
emitPostUpdateForReductionClause(
CGF, S, [](CodeGenFunction &) -> llvm::Value * { return nullptr; });
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
}
void CodeGenFunction::EmitOMPTargetParallelDeviceFunction(
CodeGenModule &CGM, StringRef ParentName,
const OMPTargetParallelDirective &S) {
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
emitTargetParallelRegion(CGF, S, Action);
};
llvm::Function *Fn;
llvm::Constant *Addr;
// Emit target region as a standalone region.
CGM.getOpenMPRuntime().emitTargetOutlinedFunction(
S, ParentName, Fn, Addr, /*IsOffloadEntry=*/true, CodeGen);
assert(Fn && Addr && "Target device function emission failed.");
}
[OpenMP] Parsing + sema for target parallel directive. Summary: This patch adds parsing + sema for the target parallel directive and its clauses along with testcases. Reviewers: ABataev Differential Revision: http://reviews.llvm.org/D16553 Rebased to current trunk and updated test cases. llvm-svn: 258832
2016-01-27 02:48:41 +08:00
void CodeGenFunction::EmitOMPTargetParallelDirective(
const OMPTargetParallelDirective &S) {
[OpenMP] Codegen support for 'target parallel' on the host. This patch adds support for codegen of 'target parallel' on the host. It is also the first combined directive that requires two or more captured statements. Support for this functionality is included in the patch. A combined directive such as 'target parallel' has two captured statements, one for the 'target' and the other for the 'parallel' region. Two captured statements are required because each has different implicit parameters (see SemaOpenMP.cpp). For example, the 'parallel' has 'global_tid' and 'bound_tid' while the 'target' does not. The patch adds support for handling multiple captured statements based on the combined directive. When codegen'ing the 'target parallel' directive, the 'target' outlined function is created using the outer captured statement and the 'parallel' outlined function is created using the inner captured statement. Reviewers: ABataev Differential Revision: https://reviews.llvm.org/D28753 llvm-svn: 292419
2017-01-19 02:18:53 +08:00
auto &&CodeGen = [&S](CodeGenFunction &CGF, PrePostActionTy &Action) {
emitTargetParallelRegion(CGF, S, Action);
};
emitCommonOMPTargetDirective(*this, S, CodeGen);
[OpenMP] Parsing + sema for target parallel directive. Summary: This patch adds parsing + sema for the target parallel directive and its clauses along with testcases. Reviewers: ABataev Differential Revision: http://reviews.llvm.org/D16553 Rebased to current trunk and updated test cases. llvm-svn: 258832
2016-01-27 02:48:41 +08:00
}
[OpenMP] Parsing + sema for target parallel for directive. Summary: This patch adds parsing + sema for the target parallel for directive along with testcases. Reviewers: ABataev Differential Revision: http://reviews.llvm.org/D16759 llvm-svn: 259654
2016-02-03 23:46:42 +08:00
void CodeGenFunction::EmitOMPTargetParallelForDirective(
const OMPTargetParallelForDirective &S) {
// TODO: codegen for target parallel for.
}
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
/// Emit a helper variable and return corresponding lvalue.
static void mapParam(CodeGenFunction &CGF, const DeclRefExpr *Helper,
const ImplicitParamDecl *PVD,
CodeGenFunction::OMPPrivateScope &Privates) {
auto *VDecl = cast<VarDecl>(Helper->getDecl());
Privates.addPrivate(
VDecl, [&CGF, PVD]() -> Address { return CGF.GetAddrOfLocalVar(PVD); });
}
void CodeGenFunction::EmitOMPTaskLoopBasedDirective(const OMPLoopDirective &S) {
assert(isOpenMPTaskLoopDirective(S.getDirectiveKind()));
// Emit outlined function for task construct.
auto CS = cast<CapturedStmt>(S.getAssociatedStmt());
auto CapturedStruct = GenerateCapturedStmtArgument(*CS);
auto SharedsTy = getContext().getRecordType(CS->getCapturedRecordDecl());
const Expr *IfCond = nullptr;
for (const auto *C : S.getClausesOfKind<OMPIfClause>()) {
if (C->getNameModifier() == OMPD_unknown ||
C->getNameModifier() == OMPD_taskloop) {
IfCond = C->getCondition();
break;
}
}
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
OMPTaskDataTy Data;
// Check if taskloop must be emitted without taskgroup.
Data.Nogroup = S.getSingleClause<OMPNogroupClause>();
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
// TODO: Check if we should emit tied or untied task.
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
Data.Tied = true;
// Set scheduling for taskloop
[OPENMP 4.5] Codegen for 'grainsize/num_tasks' clauses of 'taskloop' directive. OpenMP 4.5 defines 'taskloop' directive and 2 additional clauses 'grainsize' and 'num_tasks' for this directive. Patch adds codegen for these clauses. These clauses are generated as arguments of the '__kmpc_taskloop' libcall and are encoded the following way: void __kmpc_taskloop(ident_t *loc, int gtid, kmp_task_t *task, int if_val, kmp_uint64 *lb, kmp_uint64 *ub, kmp_int64 st, int nogroup, int sched, kmp_uint64 grainsize, void *task_dup); If 'grainsize' is specified, 'sched' argument must be set to '1' and 'grainsize' argument must be set to the value of the 'grainsize' clause. If 'num_tasks' is specified, 'sched' argument must be set to '2' and 'grainsize' argument must be set to the value of the 'num_tasks' clause. It is possible because these 2 clauses are mutually exclusive and can't be used at the same time on the same directive. If none of these clauses is specified, 'sched' argument must be set to '0'. llvm-svn: 267862
2016-04-28 17:15:06 +08:00
if (const auto* Clause = S.getSingleClause<OMPGrainsizeClause>()) {
// grainsize clause
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
Data.Schedule.setInt(/*IntVal=*/false);
Data.Schedule.setPointer(EmitScalarExpr(Clause->getGrainsize()));
[OPENMP 4.5] Codegen for 'grainsize/num_tasks' clauses of 'taskloop' directive. OpenMP 4.5 defines 'taskloop' directive and 2 additional clauses 'grainsize' and 'num_tasks' for this directive. Patch adds codegen for these clauses. These clauses are generated as arguments of the '__kmpc_taskloop' libcall and are encoded the following way: void __kmpc_taskloop(ident_t *loc, int gtid, kmp_task_t *task, int if_val, kmp_uint64 *lb, kmp_uint64 *ub, kmp_int64 st, int nogroup, int sched, kmp_uint64 grainsize, void *task_dup); If 'grainsize' is specified, 'sched' argument must be set to '1' and 'grainsize' argument must be set to the value of the 'grainsize' clause. If 'num_tasks' is specified, 'sched' argument must be set to '2' and 'grainsize' argument must be set to the value of the 'num_tasks' clause. It is possible because these 2 clauses are mutually exclusive and can't be used at the same time on the same directive. If none of these clauses is specified, 'sched' argument must be set to '0'. llvm-svn: 267862
2016-04-28 17:15:06 +08:00
} else if (const auto* Clause = S.getSingleClause<OMPNumTasksClause>()) {
// num_tasks clause
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
Data.Schedule.setInt(/*IntVal=*/true);
Data.Schedule.setPointer(EmitScalarExpr(Clause->getNumTasks()));
[OPENMP 4.5] Codegen for 'grainsize/num_tasks' clauses of 'taskloop' directive. OpenMP 4.5 defines 'taskloop' directive and 2 additional clauses 'grainsize' and 'num_tasks' for this directive. Patch adds codegen for these clauses. These clauses are generated as arguments of the '__kmpc_taskloop' libcall and are encoded the following way: void __kmpc_taskloop(ident_t *loc, int gtid, kmp_task_t *task, int if_val, kmp_uint64 *lb, kmp_uint64 *ub, kmp_int64 st, int nogroup, int sched, kmp_uint64 grainsize, void *task_dup); If 'grainsize' is specified, 'sched' argument must be set to '1' and 'grainsize' argument must be set to the value of the 'grainsize' clause. If 'num_tasks' is specified, 'sched' argument must be set to '2' and 'grainsize' argument must be set to the value of the 'num_tasks' clause. It is possible because these 2 clauses are mutually exclusive and can't be used at the same time on the same directive. If none of these clauses is specified, 'sched' argument must be set to '0'. llvm-svn: 267862
2016-04-28 17:15:06 +08:00
}
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
auto &&BodyGen = [CS, &S](CodeGenFunction &CGF, PrePostActionTy &) {
// if (PreCond) {
// for (IV in 0..LastIteration) BODY;
// <Final counter/linear vars updates>;
// }
//
// Emit: if (PreCond) - begin.
// If the condition constant folds and can be elided, avoid emitting the
// whole loop.
bool CondConstant;
llvm::BasicBlock *ContBlock = nullptr;
OMPLoopScope PreInitScope(CGF, S);
if (CGF.ConstantFoldsToSimpleInteger(S.getPreCond(), CondConstant)) {
if (!CondConstant)
return;
} else {
auto *ThenBlock = CGF.createBasicBlock("taskloop.if.then");
ContBlock = CGF.createBasicBlock("taskloop.if.end");
emitPreCond(CGF, S, S.getPreCond(), ThenBlock, ContBlock,
CGF.getProfileCount(&S));
CGF.EmitBlock(ThenBlock);
CGF.incrementProfileCounter(&S);
}
[OPENMP 4.5] Initial codegen for 'taskloop simd' directive. OpenMP 4.5 defines 'taskloop simd' directive, which is combined directive for 'taskloop' and 'simd' directives. Patch adds initial codegen support for this directive and its 2 basic clauses 'safelen' and 'simdlen'. llvm-svn: 267872
2016-04-28 20:14:51 +08:00
if (isOpenMPSimdDirective(S.getDirectiveKind()))
CGF.EmitOMPSimdInit(S);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
OMPPrivateScope LoopScope(CGF);
// Emit helper vars inits.
enum { LowerBound = 5, UpperBound, Stride, LastIter };
auto *I = CS->getCapturedDecl()->param_begin();
auto *LBP = std::next(I, LowerBound);
auto *UBP = std::next(I, UpperBound);
auto *STP = std::next(I, Stride);
auto *LIP = std::next(I, LastIter);
mapParam(CGF, cast<DeclRefExpr>(S.getLowerBoundVariable()), *LBP,
LoopScope);
mapParam(CGF, cast<DeclRefExpr>(S.getUpperBoundVariable()), *UBP,
LoopScope);
mapParam(CGF, cast<DeclRefExpr>(S.getStrideVariable()), *STP, LoopScope);
mapParam(CGF, cast<DeclRefExpr>(S.getIsLastIterVariable()), *LIP,
LoopScope);
CGF.EmitOMPPrivateLoopCounters(S, LoopScope);
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
bool HasLastprivateClause = CGF.EmitOMPLastprivateClauseInit(S, LoopScope);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
(void)LoopScope.Privatize();
// Emit the loop iteration variable.
const Expr *IVExpr = S.getIterationVariable();
const VarDecl *IVDecl = cast<VarDecl>(cast<DeclRefExpr>(IVExpr)->getDecl());
CGF.EmitVarDecl(*IVDecl);
CGF.EmitIgnoredExpr(S.getInit());
// Emit the iterations count variable.
// If it is not a variable, Sema decided to calculate iterations count on
// each iteration (e.g., it is foldable into a constant).
if (auto LIExpr = dyn_cast<DeclRefExpr>(S.getLastIteration())) {
CGF.EmitVarDecl(*cast<VarDecl>(LIExpr->getDecl()));
// Emit calculation of the iterations count.
CGF.EmitIgnoredExpr(S.getCalcLastIteration());
}
CGF.EmitOMPInnerLoop(S, LoopScope.requiresCleanups(), S.getCond(),
S.getInc(),
[&S](CodeGenFunction &CGF) {
CGF.EmitOMPLoopBody(S, JumpDest());
CGF.EmitStopPoint(&S);
},
[](CodeGenFunction &) {});
// Emit: if (PreCond) - end.
if (ContBlock) {
CGF.EmitBranch(ContBlock);
CGF.EmitBlock(ContBlock, true);
}
[OPENMP 4.5] Codegen for 'lastprivate' clauses in 'taskloop' directives. OpenMP 4.5 adds taskloop/taskloop simd directives. These directives allow to use lastprivate clause. Patch adds codegen for this clause. llvm-svn: 268618
2016-05-05 16:46:22 +08:00
// Emit final copy of the lastprivate variables if IsLastIter != 0.
if (HasLastprivateClause) {
CGF.EmitOMPLastprivateClauseFinal(
S, isOpenMPSimdDirective(S.getDirectiveKind()),
CGF.Builder.CreateIsNotNull(CGF.EmitLoadOfScalar(
CGF.GetAddrOfLocalVar(*LIP), /*Volatile=*/false,
(*LIP)->getType(), S.getLocStart())));
}
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
};
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
auto &&TaskGen = [&S, SharedsTy, CapturedStruct,
IfCond](CodeGenFunction &CGF, llvm::Value *OutlinedFn,
const OMPTaskDataTy &Data) {
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
auto &&CodeGen = [&](CodeGenFunction &CGF, PrePostActionTy &) {
OMPLoopScope PreInitScope(CGF, S);
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
CGF.CGM.getOpenMPRuntime().emitTaskLoopCall(CGF, S.getLocStart(), S,
OutlinedFn, SharedsTy,
CapturedStruct, IfCond, Data);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
};
CGF.CGM.getOpenMPRuntime().emitInlinedDirective(CGF, OMPD_taskloop,
CodeGen);
};
[OPENMP] Simplified interface for codegen of tasks, NFC. Reduced number of arguments in member functions of runtime support library for task-based directives. llvm-svn: 267863
2016-04-28 17:23:51 +08:00
EmitOMPTaskBasedDirective(S, BodyGen, TaskGen, Data);
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
}
[OPENMP 4.5] Parsing/sema analysis for 'taskloop' directive. Adds initial parsing and semantic analysis for 'taskloop' directive. llvm-svn: 254367
2015-12-01 12:18:41 +08:00
void CodeGenFunction::EmitOMPTaskLoopDirective(const OMPTaskLoopDirective &S) {
[OPENMP 4.5] Codegen for 'taskloop' directive. The taskloop construct specifies that the iterations of one or more associated loops will be executed in parallel using OpenMP tasks. The iterations are distributed across tasks created by the construct and scheduled to be executed. The next code will be generated for the taskloop directive: #pragma omp taskloop num_tasks(N) lastprivate(j) for( i=0; i<N*GRAIN*STRIDE-1; i+=STRIDE ) { int th = omp_get_thread_num(); #pragma omp atomic counter++; #pragma omp atomic th_counter[th]++; j = i; } Generated code: task = __kmpc_omp_task_alloc(NULL,gtid,1,sizeof(struct task),sizeof(struct shar),&task_entry); psh = task->shareds; psh->pth_counter = &th_counter; psh->pcounter = &counter; psh->pj = &j; task->lb = 0; task->ub = N*GRAIN*STRIDE-2; task->st = STRIDE; __kmpc_taskloop( NULL, // location gtid, // gtid task, // task structure 1, // if clause value &task->lb, // lower bound &task->ub, // upper bound STRIDE, // loop increment 0, // 1 if nogroup specified 2, // schedule type: 0-none, 1-grainsize, 2-num_tasks N, // schedule value (ignored for type 0) (void*)&__task_dup_entry // tasks duplication routine ); llvm-svn: 267395
2016-04-25 20:22:29 +08:00
EmitOMPTaskLoopBasedDirective(S);
[OPENMP 4.5] Parsing/sema analysis for 'taskloop' directive. Adds initial parsing and semantic analysis for 'taskloop' directive. llvm-svn: 254367
2015-12-01 12:18:41 +08:00
}
[OPENMP 4.5] Parsing/sema support for 'omp taskloop simd' directive. OpenMP 4.5 adds directive 'taskloop simd'. Patch adds parsing/sema analysis for 'taskloop simd' directive and its clauses. llvm-svn: 254597
2015-12-03 17:40:15 +08:00
void CodeGenFunction::EmitOMPTaskLoopSimdDirective(
const OMPTaskLoopSimdDirective &S) {
[OPENMP 4.5] Initial codegen for 'taskloop simd' directive. OpenMP 4.5 defines 'taskloop simd' directive, which is combined directive for 'taskloop' and 'simd' directives. Patch adds initial codegen support for this directive and its 2 basic clauses 'safelen' and 'simdlen'. llvm-svn: 267872
2016-04-28 20:14:51 +08:00
EmitOMPTaskLoopBasedDirective(S);
[OPENMP 4.5] Parsing/sema support for 'omp taskloop simd' directive. OpenMP 4.5 adds directive 'taskloop simd'. Patch adds parsing/sema analysis for 'taskloop simd' directive and its clauses. llvm-svn: 254597
2015-12-03 17:40:15 +08:00
}
[OpenMP] Parsing and sema support for target update directive Summary: This patch is to add parsing and sema support for `target update` directive. Support for the `to` and `from` clauses will be added by a different patch. This patch also adds support for other clauses that are already implemented upstream and apply to `target update`, e.g. `device` and `if`. This patch is based on the original post by Kelvin Li. Reviewers: hfinkel, carlo.bertolli, kkwli0, arpith-jacob, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D15944 llvm-svn: 270878
2016-05-27 01:30:50 +08:00
// Generate the instructions for '#pragma omp target update' directive.
void CodeGenFunction::EmitOMPTargetUpdateDirective(
const OMPTargetUpdateDirective &S) {
[OpenMP] Codegen for target update directive. Summary: This patch implements the code generation for the `target update` directive. The implemntation relies on the logic already in place for target data standalone directives, i.e. target enter/exit data. Reviewers: hfinkel, carlo.bertolli, arpith-jacob, kkwli0, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D20650 llvm-svn: 270886
2016-05-27 02:30:22 +08:00
// If we don't have target devices, don't bother emitting the data mapping
// code.
if (CGM.getLangOpts().OMPTargetTriples.empty())
return;
// Check if we have any if clause associated with the directive.
const Expr *IfCond = nullptr;
if (auto *C = S.getSingleClause<OMPIfClause>())
IfCond = C->getCondition();
// Check if we have any device clause associated with the directive.
const Expr *Device = nullptr;
if (auto *C = S.getSingleClause<OMPDeviceClause>())
Device = C->getDevice();
CGM.getOpenMPRuntime().emitTargetDataStandAloneCall(*this, S, IfCond, Device);
[OpenMP] Parsing and sema support for target update directive Summary: This patch is to add parsing and sema support for `target update` directive. Support for the `to` and `from` clauses will be added by a different patch. This patch also adds support for other clauses that are already implemented upstream and apply to `target update`, e.g. `device` and `if`. This patch is based on the original post by Kelvin Li. Reviewers: hfinkel, carlo.bertolli, kkwli0, arpith-jacob, ABataev Subscribers: caomhin, cfe-commits Differential Revision: http://reviews.llvm.org/D15944 llvm-svn: 270878
2016-05-27 01:30:50 +08:00
}