maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/test/OpenMP/parallel_reduction_codegen.cpp

896 lines
46 KiB
C++
Raw Normal View History

[OPENMP] -fopenmp enables OpenMP support (fix for http://llvm.org/PR23492) -fopenmp turns on OpenMP support and links libiomp5 as OpenMP library. Also there is -fopenmp={libiomp5|libgomp} option that allows to override effect of -fopenmp and link libgomp library (if -fopenmp=libgomp is specified). Differential Revision: http://reviews.llvm.org/D9736 llvm-svn: 237769
2015-05-20 12:24:19 +08:00
// RUN: %clang_cc1 -verify -fopenmp -x c++ -triple x86_64-apple-darwin10 -emit-llvm %s -o - | FileCheck %s
// RUN: %clang_cc1 -fopenmp -x c++ -std=c++11 -triple x86_64-apple-darwin10 -emit-pch -o %t %s
// RUN: %clang_cc1 -fopenmp -x c++ -triple x86_64-apple-darwin10 -std=c++11 -include-pch %t -verify %s -emit-llvm -o - | FileCheck %s
// RUN: %clang_cc1 -verify -fopenmp -x c++ -std=c++11 -DLAMBDA -triple x86_64-apple-darwin10 -emit-llvm %s -o - | FileCheck -check-prefix=LAMBDA %s
// RUN: %clang_cc1 -verify -fopenmp -x c++ -fblocks -DBLOCKS -triple x86_64-apple-darwin10 -emit-llvm %s -o - | FileCheck -check-prefix=BLOCKS %s
[OPENMP] Support for -fopenmp-simd option with compilation of simd loops only. Added support for -fopenmp-simd option that allows compilation of simd-based constructs without emission of OpenMP runtime calls. llvm-svn: 321560
2017-12-30 02:07:07 +08:00
// RUN: %clang_cc1 -verify -fopenmp-simd -x c++ -triple x86_64-apple-darwin10 -emit-llvm %s -o - | FileCheck --check-prefix SIMD-ONLY0 %s
// RUN: %clang_cc1 -fopenmp-simd -x c++ -std=c++11 -triple x86_64-apple-darwin10 -emit-pch -o %t %s
// RUN: %clang_cc1 -fopenmp-simd -x c++ -triple x86_64-apple-darwin10 -std=c++11 -include-pch %t -verify %s -emit-llvm -o - | FileCheck --check-prefix SIMD-ONLY0 %s
// RUN: %clang_cc1 -verify -fopenmp-simd -x c++ -std=c++11 -DLAMBDA -triple x86_64-apple-darwin10 -emit-llvm %s -o - | FileCheck --check-prefix SIMD-ONLY0 %s
// RUN: %clang_cc1 -verify -fopenmp-simd -x c++ -fblocks -DBLOCKS -triple x86_64-apple-darwin10 -emit-llvm %s -o - | FileCheck --check-prefix SIMD-ONLY0 %s
// SIMD-ONLY0-NOT: {{__kmpc|__tgt}}
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// expected-no-diagnostics
#ifndef HEADER
#define HEADER
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
volatile int g __attribute__((aligned(128))) = 1212;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
template <class T>
struct S {
T f;
S(T a) : f(a + g) {}
S() : f(g) {}
operator T() { return T(); }
S &operator&(const S &) { return *this; }
~S() {}
};
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
struct SS {
int a;
int b : 4;
int &c;
SS(int &d) : a(0), b(0), c(d) {
#pragma omp parallel reduction(+: a, b, c)
#ifdef LAMBDA
[&]() {
++this->a, --b, (this)->c /= 1;
#pragma omp parallel reduction(&: a, b, c)
++(this)->a, --b, this->c /= 1;
}();
#elif defined(BLOCKS)
^{
++a;
--this->b;
(this)->c /= 1;
#pragma omp parallel reduction(-: a, b, c)
++(this)->a, --b, this->c /= 1;
}();
#else
++this->a, --b, c /= 1;
#endif
}
};
template<typename T>
struct SST {
T a;
SST() : a(T()) {
#pragma omp parallel reduction(*: a)
#ifdef LAMBDA
[&]() {
[&]() {
++this->a;
#pragma omp parallel reduction(&& :a)
++(this)->a;
}();
}();
#elif defined(BLOCKS)
^{
^{
++a;
#pragma omp parallel reduction(|: a)
++(this)->a;
}();
}();
#else
++(this)->a;
#endif
}
};
// CHECK: [[SS_TY:%.+]] = type { i{{[0-9]+}}, i8
// LAMBDA: [[SS_TY:%.+]] = type { i{{[0-9]+}}, i8
// BLOCKS: [[SS_TY:%.+]] = type { i{{[0-9]+}}, i8
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK-DAG: [[S_FLOAT_TY:%.+]] = type { float }
// CHECK-DAG: [[S_INT_TY:%.+]] = type { i{{[0-9]+}} }
[OPENMP] Create non-const ident_t objects. Currently ident_t objects are created const when debug info is not enabled, but the libittnotify libray in the OpenMP runtime writes to the reserved_2 field (See __kmp_itt_region_forking in openmp/runtime/src/kmp_itt.inl). Now create ident_t objects non-const. Differential Revision: https://reviews.llvm.org/D51331 llvm-svn: 340934
2018-08-29 23:45:11 +08:00
// CHECK-DAG: [[REDUCTION_LOC:@.+]] = private unnamed_addr global %{{.+}} { i32 0, i32 18, i32 0, i32 0, i8*
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK-DAG: [[REDUCTION_LOCK:@.+]] = common global [8 x i32] zeroinitializer
[OPENMP] Fix PR34926: Fix handling of the array sections passed as function params. Codegen could crash if the array section base expression is the function parameter. llvm-svn: 315586
2017-10-12 23:18:41 +08:00
//CHECK: foo_array_sect
//CHECK: call void {{.+}}@__kmpc_fork_call(
//CHECK: ret void
void foo_array_sect(short x[1]) {
#pragma omp parallel reduction(+ : x[:])
{}
}
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
template <typename T>
T tmain() {
T t;
S<T> test;
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
SST<T> sst;
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
T t_var __attribute__((aligned(128))) = T(), t_var1 __attribute__((aligned(128)));
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
T vec[] = {1, 2};
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
S<T> s_arr[] = {1, 2};
S<T> var __attribute__((aligned(128))) (3), var1 __attribute__((aligned(128)));
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
#pragma omp parallel reduction(+:t_var) reduction(&:var) reduction(&& : var1) reduction(min: t_var1)
{
vec[0] = t_var;
s_arr[0] = var;
}
return T();
}
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
int sivar;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
int main() {
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
SS ss(sivar);
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
#ifdef LAMBDA
// LAMBDA: [[G:@.+]] = global i{{[0-9]+}} 1212,
// LAMBDA-LABEL: @main
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// LAMBDA: alloca [[SS_TY]],
// LAMBDA: alloca [[CAP_TY:%.+]],
// LAMBDA: call{{.*}} void [[OUTER_LAMBDA:@[^(]+]]([[CAP_TY]]*
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
[&]() {
// LAMBDA: define{{.*}} internal{{.*}} void [[OUTER_LAMBDA]](
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// LAMBDA: call void {{.+}} @__kmpc_fork_call({{.+}}, i32 1, {{.+}}* [[OMP_REGION:@.+]] to {{.+}}, i32* [[G]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
#pragma omp parallel reduction(+:g)
{
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// LAMBDA: define {{.+}} @{{.+}}([[SS_TY]]*
// LAMBDA: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 0
// LAMBDA: store i{{[0-9]+}} 0, i{{[0-9]+}}* %
// LAMBDA: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 1
// LAMBDA: store i8
// LAMBDA: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 2
// LAMBDA: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 0
// LAMBDA-NOT: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 1
// LAMBDA: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 2
// LAMBDA: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 4, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, [[SS_TY]]*, i32*, i32*, i32*)* [[SS_MICROTASK:@.+]] to void
// LAMBDA: [[B_REF:%.+]] = getelementptr {{.*}}[[SS_TY]], [[SS_TY]]* %{{.*}}, i32 0, i32 1
// LAMBDA: store i8 %{{.+}}, i8* [[B_REF]],
// LAMBDA: ret
// LAMBDA: define internal void [[SS_MICROTASK]](i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}}, [[SS_TY]]* %{{.+}}, i32* {{.+}}, i32* {{.+}}, i32* {{.+}})
// LAMBDA-NOT: getelementptr {{.*}}[[SS_TY]], [[SS_TY]]* %
// LAMBDA: call{{.*}} void
// LAMBDA: ret void
// LAMBDA: define internal void @{{.+}}(i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}}, [[SS_TY]]*
// LAMBDA: [[A_PRIV:%.+]] = alloca i{{[0-9]+}},
// LAMBDA: [[B_PRIV:%.+]] = alloca i{{[0-9]+}},
// LAMBDA: [[C_PRIV:%.+]] = alloca i{{[0-9]+}},
// LAMBDA: store i{{[0-9]+}} -1, i{{[0-9]+}}* [[A_PRIV]],
// LAMBDA: store i{{[0-9]+}}* [[A_PRIV]], i{{[0-9]+}}** [[REFA:%.+]],
// LAMBDA: store i{{[0-9]+}} -1, i{{[0-9]+}}* [[B_PRIV]],
// LAMBDA: store i{{[0-9]+}} -1, i{{[0-9]+}}* [[C_PRIV]],
// LAMBDA: store i{{[0-9]+}}* [[C_PRIV]], i{{[0-9]+}}** [[REFC:%.+]],
// LAMBDA: [[A_PRIV:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[REFA]],
// LAMBDA-NEXT: [[A_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[A_PRIV]],
// LAMBDA-NEXT: [[INC:%.+]] = add nsw i{{[0-9]+}} [[A_VAL]], 1
// LAMBDA-NEXT: store i{{[0-9]+}} [[INC]], i{{[0-9]+}}* [[A_PRIV]],
// LAMBDA-NEXT: [[B_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[B_PRIV]],
// LAMBDA-NEXT: [[DEC:%.+]] = add nsw i{{[0-9]+}} [[B_VAL]], -1
// LAMBDA-NEXT: store i{{[0-9]+}} [[DEC]], i{{[0-9]+}}* [[B_PRIV]],
// LAMBDA-NEXT: [[C_PRIV:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[REFC]],
// LAMBDA-NEXT: [[C_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[C_PRIV]],
// LAMBDA-NEXT: [[DIV:%.+]] = sdiv i{{[0-9]+}} [[C_VAL]], 1
// LAMBDA-NEXT: store i{{[0-9]+}} [[DIV]], i{{[0-9]+}}* [[C_PRIV]],
// LAMBDA: call i32 @__kmpc_reduce_nowait(
// LAMBDA: ret void
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// LAMBDA: define{{.*}} internal{{.*}} void [[OMP_REGION]](i32* noalias %{{.+}}, i32* noalias %{{.+}}, i32* dereferenceable(4) %{{.+}})
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// LAMBDA: [[G_PRIVATE_ADDR:%.+]] = alloca i{{[0-9]+}},
// Reduction list for runtime.
// LAMBDA: [[RED_LIST:%.+]] = alloca [1 x i8*],
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// LAMBDA: [[G_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[G_REF_ADDR:%.+]]
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
// LAMBDA: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[G_PRIVATE_ADDR]], align 128
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
g = 1;
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
// LAMBDA: store i{{[0-9]+}} 1, i{{[0-9]+}}* [[G_PRIVATE_ADDR]], align 128
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// LAMBDA: [[G_PRIVATE_ADDR_REF:%.+]] = getelementptr inbounds %{{.+}}, %{{.+}}* [[ARG:%.+]], i{{[0-9]+}} 0, i{{[0-9]+}} 0
// LAMBDA: store i{{[0-9]+}}* [[G_PRIVATE_ADDR]], i{{[0-9]+}}** [[G_PRIVATE_ADDR_REF]]
// LAMBDA: call void [[INNER_LAMBDA:@.+]](%{{.+}}* [[ARG]])
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// LAMBDA: [[G_PRIV_REF:%.+]] = getelementptr inbounds [1 x i8*], [1 x i8*]* [[RED_LIST]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// LAMBDA: [[BITCAST:%.+]] = bitcast i32* [[G_PRIVATE_ADDR]] to i8*
// LAMBDA: store i8* [[BITCAST]], i8** [[G_PRIV_REF]],
// LAMBDA: call i32 @__kmpc_reduce_nowait(
// LAMBDA: switch i32 %{{.+}}, label %[[REDUCTION_DONE:.+]] [
// LAMBDA: i32 1, label %[[CASE1:.+]]
// LAMBDA: i32 2, label %[[CASE2:.+]]
// LAMBDA: [[CASE1]]
// LAMBDA: [[G_VAL:%.+]] = load i32, i32* [[G_REF]]
// LAMBDA: [[G_PRIV_VAL:%.+]] = load i32, i32* [[G_PRIVATE_ADDR]]
// LAMBDA: [[ADD:%.+]] = add nsw i32 [[G_VAL]], [[G_PRIV_VAL]]
// LAMBDA: store i32 [[ADD]], i32* [[G_REF]]
// LAMBDA: call void @__kmpc_end_reduce_nowait(
// LAMBDA: br label %[[REDUCTION_DONE]]
// LAMBDA: [[CASE2]]
// LAMBDA: [[G_PRIV_VAL:%.+]] = load i32, i32* [[G_PRIVATE_ADDR]]
// LAMBDA: atomicrmw add i32* [[G_REF]], i32 [[G_PRIV_VAL]] monotonic
// LAMBDA: br label %[[REDUCTION_DONE]]
// LAMBDA: [[REDUCTION_DONE]]
// LAMBDA: ret void
[&]() {
// LAMBDA: define {{.+}} void [[INNER_LAMBDA]](%{{.+}}* [[ARG_PTR:%.+]])
// LAMBDA: store %{{.+}}* [[ARG_PTR]], %{{.+}}** [[ARG_PTR_REF:%.+]],
g = 2;
// LAMBDA: [[ARG_PTR:%.+]] = load %{{.+}}*, %{{.+}}** [[ARG_PTR_REF]]
// LAMBDA: [[G_PTR_REF:%.+]] = getelementptr inbounds %{{.+}}, %{{.+}}* [[ARG_PTR]], i{{[0-9]+}} 0, i{{[0-9]+}} 0
// LAMBDA: [[G_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[G_PTR_REF]]
[OPENMP] Drop type qualifiers from private variables. If the variable is marked as private in OpenMP construct, the reference to this variable should not keep type qualifiers for the original variable. Private copy is not volatile or constant, so we can use unqualified type for private copy. llvm-svn: 242133
2015-07-14 18:32:29 +08:00
// LAMBDA: store i{{[0-9]+}} 2, i{{[0-9]+}}* [[G_REF]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
}();
}
}();
return 0;
#elif defined(BLOCKS)
// BLOCKS: [[G:@.+]] = global i{{[0-9]+}} 1212,
// BLOCKS-LABEL: @main
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// BLOCKS: call
[opaque pointer types] Explicit non-pointer type for call expressions (migration for recent LLVM change to textual IR for calls) llvm-svn: 235147
2015-04-17 07:25:00 +08:00
// BLOCKS: call void {{%.+}}(i8
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
^{
// BLOCKS: define{{.*}} internal{{.*}} void {{.+}}(i8*
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// BLOCKS: call void {{.+}} @__kmpc_fork_call({{.+}}, i32 1, {{.+}}* [[OMP_REGION:@.+]] to {{.+}}, i32* [[G]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
#pragma omp parallel reduction(-:g)
{
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// BLOCKS: define{{.*}} internal{{.*}} void [[OMP_REGION]](i32* noalias %{{.+}}, i32* noalias %{{.+}}, i32* dereferenceable(4) %{{.+}})
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// BLOCKS: [[G_PRIVATE_ADDR:%.+]] = alloca i{{[0-9]+}},
// Reduction list for runtime.
// BLOCKS: [[RED_LIST:%.+]] = alloca [1 x i8*],
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// BLOCKS: [[G_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[G_REF_ADDR:%.+]]
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
// BLOCKS: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[G_PRIVATE_ADDR]], align 128
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
g = 1;
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
// BLOCKS: store i{{[0-9]+}} 1, i{{[0-9]+}}* [[G_PRIVATE_ADDR]], align 128
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// BLOCKS-NOT: [[G]]{{[[^:word:]]}}
// BLOCKS: i{{[0-9]+}}* [[G_PRIVATE_ADDR]]
// BLOCKS-NOT: [[G]]{{[[^:word:]]}}
[opaque pointer types] Explicit non-pointer type for call expressions (migration for recent LLVM change to textual IR for calls) llvm-svn: 235147
2015-04-17 07:25:00 +08:00
// BLOCKS: call void {{%.+}}(i8
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// BLOCKS: [[G_PRIV_REF:%.+]] = getelementptr inbounds [1 x i8*], [1 x i8*]* [[RED_LIST]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// BLOCKS: [[BITCAST:%.+]] = bitcast i32* [[G_PRIVATE_ADDR]] to i8*
// BLOCKS: store i8* [[BITCAST]], i8** [[G_PRIV_REF]],
// BLOCKS: call i32 @__kmpc_reduce_nowait(
// BLOCKS: switch i32 %{{.+}}, label %[[REDUCTION_DONE:.+]] [
// BLOCKS: i32 1, label %[[CASE1:.+]]
// BLOCKS: i32 2, label %[[CASE2:.+]]
// BLOCKS: [[CASE1]]
// BLOCKS: [[G_VAL:%.+]] = load i32, i32* [[G_REF]]
// BLOCKS: [[G_PRIV_VAL:%.+]] = load i32, i32* [[G_PRIVATE_ADDR]]
// BLOCKS: [[ADD:%.+]] = add nsw i32 [[G_VAL]], [[G_PRIV_VAL]]
// BLOCKS: store i32 [[ADD]], i32* [[G_REF]]
// BLOCKS: call void @__kmpc_end_reduce_nowait(
// BLOCKS: br label %[[REDUCTION_DONE]]
// BLOCKS: [[CASE2]]
// BLOCKS: [[G_PRIV_VAL:%.+]] = load i32, i32* [[G_PRIVATE_ADDR]]
// BLOCKS: atomicrmw add i32* [[G_REF]], i32 [[G_PRIV_VAL]] monotonic
// BLOCKS: br label %[[REDUCTION_DONE]]
// BLOCKS: [[REDUCTION_DONE]]
// BLOCKS: ret void
^{
// BLOCKS: define {{.+}} void {{@.+}}(i8*
g = 2;
// BLOCKS-NOT: [[G]]{{[[^:word:]]}}
[OPENMP] Drop type qualifiers from private variables. If the variable is marked as private in OpenMP construct, the reference to this variable should not keep type qualifiers for the original variable. Private copy is not volatile or constant, so we can use unqualified type for private copy. llvm-svn: 242133
2015-07-14 18:32:29 +08:00
// BLOCKS: store i{{[0-9]+}} 2, i{{[0-9]+}}*
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// BLOCKS-NOT: [[G]]{{[[^:word:]]}}
// BLOCKS: ret
}();
}
}();
return 0;
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// BLOCKS: define {{.+}} @{{.+}}([[SS_TY]]*
// BLOCKS: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 0
// BLOCKS: store i{{[0-9]+}} 0, i{{[0-9]+}}* %
// BLOCKS: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 1
// BLOCKS: store i8
// BLOCKS: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 2
// BLOCKS: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 0
// BLOCKS-NOT: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 1
// BLOCKS: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 2
// BLOCKS: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 4, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, [[SS_TY]]*, i32*, i32*, i32*)* [[SS_MICROTASK:@.+]] to void
// BLOCKS: [[B_REF:%.+]] = getelementptr {{.*}}[[SS_TY]], [[SS_TY]]* %{{.*}}, i32 0, i32 1
// BLOCKS: store i8 %{{.+}}, i8* [[B_REF]],
// BLOCKS: ret
// BLOCKS: define internal void [[SS_MICROTASK]](i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}}, [[SS_TY]]* %{{.+}}, i32* {{.+}}, i32* {{.+}}, i32* {{.+}})
// BLOCKS-NOT: getelementptr {{.*}}[[SS_TY]], [[SS_TY]]* %
// BLOCKS: call{{.*}} void
// BLOCKS: ret void
// BLOCKS: define internal void @{{.+}}(i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}}, [[SS_TY]]* %{{.+}}, i32* {{.+}}, i32* {{.+}}, i32* {{.+}})
// BLOCKS: [[A_PRIV:%.+]] = alloca i{{[0-9]+}},
// BLOCKS: [[B_PRIV:%.+]] = alloca i{{[0-9]+}},
// BLOCKS: [[C_PRIV:%.+]] = alloca i{{[0-9]+}},
// BLOCKS: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[A_PRIV]],
// BLOCKS: store i{{[0-9]+}}* [[A_PRIV]], i{{[0-9]+}}** [[REFA:%.+]],
// BLOCKS: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[B_PRIV]],
// BLOCKS: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[C_PRIV]],
// BLOCKS: store i{{[0-9]+}}* [[C_PRIV]], i{{[0-9]+}}** [[REFC:%.+]],
// BLOCKS: [[A_PRIV:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[REFA]],
// BLOCKS-NEXT: [[A_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[A_PRIV]],
// BLOCKS-NEXT: [[INC:%.+]] = add nsw i{{[0-9]+}} [[A_VAL]], 1
// BLOCKS-NEXT: store i{{[0-9]+}} [[INC]], i{{[0-9]+}}* [[A_PRIV]],
// BLOCKS-NEXT: [[B_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[B_PRIV]],
// BLOCKS-NEXT: [[DEC:%.+]] = add nsw i{{[0-9]+}} [[B_VAL]], -1
// BLOCKS-NEXT: store i{{[0-9]+}} [[DEC]], i{{[0-9]+}}* [[B_PRIV]],
// BLOCKS-NEXT: [[C_PRIV:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[REFC]],
// BLOCKS-NEXT: [[C_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[C_PRIV]],
// BLOCKS-NEXT: [[DIV:%.+]] = sdiv i{{[0-9]+}} [[C_VAL]], 1
// BLOCKS-NEXT: store i{{[0-9]+}} [[DIV]], i{{[0-9]+}}* [[C_PRIV]],
// BLOCKS: call i32 @__kmpc_reduce_nowait(
// BLOCKS: ret void
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
#else
S<float> test;
float t_var = 0, t_var1;
int vec[] = {1, 2};
S<float> s_arr[] = {1, 2};
S<float> var(3), var1;
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
float _Complex cf;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
#pragma omp parallel reduction(+:t_var) reduction(&:var) reduction(&& : var1) reduction(min: t_var1)
{
vec[0] = t_var;
s_arr[0] = var;
}
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
if (var1)
#pragma omp parallel reduction(+ : t_var) reduction(& : var) reduction(&& : var1) reduction(min : t_var1)
while (1) {
vec[0] = t_var;
s_arr[0] = var;
}
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
#pragma omp parallel reduction(+ : cf)
;
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
return tmain<int>();
#endif
}
// CHECK: define {{.*}}i{{[0-9]+}} @main()
// CHECK: [[TEST:%.+]] = alloca [[S_FLOAT_TY]],
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_FLOAT_TY_CONSTR:@.+]]([[S_FLOAT_TY]]* [[TEST]])
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// CHECK: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 6, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, [2 x i32]*, float*, [2 x [[S_FLOAT_TY]]]*, [[S_FLOAT_TY]]*, [[S_FLOAT_TY]]*, float*)* [[MAIN_MICROTASK:@.+]] to void
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
// CHECK: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 6, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, [2 x i32]*, float*, [2 x [[S_FLOAT_TY]]]*, [[S_FLOAT_TY]]*, [[S_FLOAT_TY]]*, float*)* [[MAIN_MICROTASK1:@.+]] to void
[OPENMP] Fix crash on reduction for complex variables. reworked codegen for reduction operation for complex types to avoid crash llvm-svn: 258394
2016-01-21 20:35:58 +08:00
// CHECK: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 1, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, { float, float }*)* [[MAIN_MICROTASK2:@.+]] to void
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: = call {{.*}}i{{.+}} [[TMAIN_INT:@.+]]()
// CHECK: call {{.*}} [[S_FLOAT_TY_DESTR:@.+]]([[S_FLOAT_TY]]*
// CHECK: ret
//
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// CHECK: define internal void [[MAIN_MICROTASK]](i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}},
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR_PRIV:%.+]] = alloca float,
// CHECK: [[VAR_PRIV:%.+]] = alloca [[S_FLOAT_TY]],
// CHECK: [[VAR1_PRIV:%.+]] = alloca [[S_FLOAT_TY]],
// CHECK: [[T_VAR1_PRIV:%.+]] = alloca float,
// Reduction list for runtime.
// CHECK: [[RED_LIST:%.+]] = alloca [4 x i8*],
// CHECK: store i{{[0-9]+}}* [[GTID_ADDR]], i{{[0-9]+}}** [[GTID_ADDR_ADDR:%.+]],
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// CHECK: [[T_VAR_REF:%.+]] = load float*, float** %
// CHECK: [[VAR_REF:%.+]] = load [[S_FLOAT_TY]]*, [[S_FLOAT_TY]]** %
// CHECK: [[VAR1_REF:%.+]] = load [[S_FLOAT_TY]]*, [[S_FLOAT_TY]]** %
// CHECK: [[T_VAR1_REF:%.+]] = load float*, float** %
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// For + reduction operation initial value of private variable is 0.
// CHECK: store float 0.0{{.+}}, float* [[T_VAR_PRIV]],
// For & reduction operation initial value of private variable is ones in all bits.
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_FLOAT_TY_CONSTR:@.+]]([[S_FLOAT_TY]]* [[VAR_PRIV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// For && reduction operation initial value of private variable is 1.0.
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_FLOAT_TY_CONSTR:@.+]]([[S_FLOAT_TY]]* [[VAR1_PRIV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// For min reduction operation initial value of private variable is largest repesentable value.
// CHECK: store float 0x47EFFFFFE0000000, float* [[T_VAR1_PRIV]],
// Skip checks for internal operations.
// void *RedList[<n>] = {<ReductionVars>[0], ..., <ReductionVars>[<n>-1]};
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast float* [[T_VAR_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[T_VAR_PRIV_REF]],
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 1
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[VAR_PRIV_REF]],
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR1_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 2
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR1_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[VAR1_PRIV_REF]],
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR1_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 3
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast float* [[T_VAR1_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[T_VAR1_PRIV_REF]],
// res = __kmpc_reduce_nowait(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
// CHECK: [[GTID_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[GTID_ADDR_ADDR]]
// CHECK: [[GTID:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[GTID_REF]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast [4 x i8*]* [[RED_LIST]] to i8*
// CHECK: [[RES:%.+]] = call i32 @__kmpc_reduce_nowait(%{{.+}}* [[REDUCTION_LOC]], i32 [[GTID]], i32 4, i64 32, i8* [[BITCAST]], void (i8*, i8*)* [[REDUCTION_FUNC:@.+]], [8 x i32]* [[REDUCTION_LOCK]])
// switch(res)
// CHECK: switch i32 [[RES]], label %[[RED_DONE:.+]] [
// CHECK: i32 1, label %[[CASE1:.+]]
// CHECK: i32 2, label %[[CASE2:.+]]
// CHECK: ]
// case 1:
// t_var += t_var_reduction;
// CHECK: [[T_VAR_VAL:%.+]] = load float, float* [[T_VAR_REF]],
// CHECK: [[T_VAR_PRIV_VAL:%.+]] = load float, float* [[T_VAR_PRIV]],
// CHECK: [[UP:%.+]] = fadd float [[T_VAR_VAL]], [[T_VAR_PRIV_VAL]]
// CHECK: store float [[UP]], float* [[T_VAR_REF]],
// var = var.operator &(var_reduction);
// CHECK: [[UP:%.+]] = call dereferenceable(4) [[S_FLOAT_TY]]* @{{.+}}([[S_FLOAT_TY]]* [[VAR_REF]], [[S_FLOAT_TY]]* dereferenceable(4) [[VAR_PRIV]])
// CHECK: [[BC1:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_FLOAT_TY]]* [[UP]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes (Step 1). Summary: Upstream LLVM is changing the the prototypes of the @llvm.memcpy/memmove/memset intrinsics. This change updates the Clang tests for this change. The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change removes the alignment argument in favour of placing the alignment attribute on the source and destination pointers of the memory intrinsic call. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) At this time the source and destination alignments must be the same (Step 1). Step 2 of the change, to be landed shortly, will relax that contraint and allow the source and destination to have different alignments. llvm-svn: 322964
2018-01-20 01:12:54 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// var1 = var1.operator &&(var1_reduction);
// CHECK: [[TO_FLOAT:%.+]] = call float @{{.+}}([[S_FLOAT_TY]]* [[VAR1_REF]])
// CHECK: [[VAR1_BOOL:%.+]] = fcmp une float [[TO_FLOAT]], 0.0
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[VAR1_BOOL]], label %[[TRUE:.+]], label %[[END2:.+]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[TRUE]]
// CHECK: [[TO_FLOAT:%.+]] = call float @{{.+}}([[S_FLOAT_TY]]* [[VAR1_PRIV]])
// CHECK: [[VAR1_REDUCTION_BOOL:%.+]] = fcmp une float [[TO_FLOAT]], 0.0
// CHECK: br label %[[END2]]
// CHECK: [[END2]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: [[COND_LVALUE:%.+]] = phi i1 [ false, %{{.+}} ], [ [[VAR1_REDUCTION_BOOL]], %[[TRUE]] ]
// CHECK: [[CONV:%.+]] = uitofp i1 [[COND_LVALUE]] to float
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call void @{{.+}}([[S_FLOAT_TY]]* [[COND_LVALUE:%.+]], float [[CONV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BC1:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR1_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_FLOAT_TY]]* [[COND_LVALUE]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes (Step 1). Summary: Upstream LLVM is changing the the prototypes of the @llvm.memcpy/memmove/memset intrinsics. This change updates the Clang tests for this change. The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change removes the alignment argument in favour of placing the alignment attribute on the source and destination pointers of the memory intrinsic call. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) At this time the source and destination alignments must be the same (Step 1). Step 2 of the change, to be landed shortly, will relax that contraint and allow the source and destination to have different alignments. llvm-svn: 322964
2018-01-20 01:12:54 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// t_var1 = min(t_var1, t_var1_reduction);
// CHECK: [[T_VAR1_VAL:%.+]] = load float, float* [[T_VAR1_REF]],
// CHECK: [[T_VAR1_PRIV_VAL:%.+]] = load float, float* [[T_VAR1_PRIV]],
// CHECK: [[CMP:%.+]] = fcmp olt float [[T_VAR1_VAL]], [[T_VAR1_PRIV_VAL]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[CMP]]
// CHECK: [[UP:%.+]] = phi float
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: store float [[UP]], float* [[T_VAR1_REF]],
// __kmpc_end_reduce_nowait(<loc>, <gtid>, &<lock>);
// CHECK: call void @__kmpc_end_reduce_nowait(%{{.+}}* [[REDUCTION_LOC]], i32 [[GTID]], [8 x i32]* [[REDUCTION_LOCK]])
// break;
// CHECK: br label %[[RED_DONE]]
// case 2:
// t_var += t_var_reduction;
// CHECK: load float, float* [[T_VAR_PRIV]]
// CHECK: [[T_VAR_REF_INT:%.+]] = bitcast float* [[T_VAR_REF]] to i32*
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[OLD1:%.+]] = load atomic i32, i32* [[T_VAR_REF_INT]] monotonic,
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: br label %[[CONT:.+]]
// CHECK: [[CONT]]
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[ORIG_OLD_INT:%.+]] = phi i32 [ [[OLD1]], %{{.+}} ], [ [[OLD2:%.+]], %[[CONT]] ]
// CHECK: fadd float
// CHECK: [[UP_INT:%.+]] = load i32
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR_REF_INT:%.+]] = bitcast float* [[T_VAR_REF]] to i32*
// CHECK: [[RES:%.+]] = cmpxchg i32* [[T_VAR_REF_INT]], i32 [[ORIG_OLD_INT]], i32 [[UP_INT]] monotonic monotonic
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[OLD2:%.+]] = extractvalue { i32, i1 } [[RES]], 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[SUCCESS_FAIL:%.+]] = extractvalue { i32, i1 } [[RES]], 1
// CHECK: br i1 [[SUCCESS_FAIL]], label %[[ATOMIC_DONE:.+]], label %[[CONT]]
// CHECK: [[ATOMIC_DONE]]
// var = var.operator &(var_reduction);
// CHECK: call void @__kmpc_critical(
// CHECK: [[UP:%.+]] = call dereferenceable(4) [[S_FLOAT_TY]]* @{{.+}}([[S_FLOAT_TY]]* [[VAR_REF]], [[S_FLOAT_TY]]* dereferenceable(4) [[VAR_PRIV]])
// CHECK: [[BC1:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_FLOAT_TY]]* [[UP]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes (Step 1). Summary: Upstream LLVM is changing the the prototypes of the @llvm.memcpy/memmove/memset intrinsics. This change updates the Clang tests for this change. The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change removes the alignment argument in favour of placing the alignment attribute on the source and destination pointers of the memory intrinsic call. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) At this time the source and destination alignments must be the same (Step 1). Step 2 of the change, to be landed shortly, will relax that contraint and allow the source and destination to have different alignments. llvm-svn: 322964
2018-01-20 01:12:54 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: call void @__kmpc_end_critical(
// var1 = var1.operator &&(var1_reduction);
// CHECK: call void @__kmpc_critical(
// CHECK: [[TO_FLOAT:%.+]] = call float @{{.+}}([[S_FLOAT_TY]]* [[VAR1_REF]])
// CHECK: [[VAR1_BOOL:%.+]] = fcmp une float [[TO_FLOAT]], 0.0
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[VAR1_BOOL]], label %[[TRUE:.+]], label %[[END2:.+]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[TRUE]]
// CHECK: [[TO_FLOAT:%.+]] = call float @{{.+}}([[S_FLOAT_TY]]* [[VAR1_PRIV]])
// CHECK: [[VAR1_REDUCTION_BOOL:%.+]] = fcmp une float [[TO_FLOAT]], 0.0
// CHECK: br label %[[END2]]
// CHECK: [[END2]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: [[COND_LVALUE:%.+]] = phi i1 [ false, %{{.+}} ], [ [[VAR1_REDUCTION_BOOL]], %[[TRUE]] ]
// CHECK: [[CONV:%.+]] = uitofp i1 [[COND_LVALUE]] to float
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call void @{{.+}}([[S_FLOAT_TY]]* [[COND_LVALUE:%.+]], float [[CONV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BC1:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR1_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_FLOAT_TY]]* [[COND_LVALUE]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes (Step 1). Summary: Upstream LLVM is changing the the prototypes of the @llvm.memcpy/memmove/memset intrinsics. This change updates the Clang tests for this change. The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change removes the alignment argument in favour of placing the alignment attribute on the source and destination pointers of the memory intrinsic call. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) At this time the source and destination alignments must be the same (Step 1). Step 2 of the change, to be landed shortly, will relax that contraint and allow the source and destination to have different alignments. llvm-svn: 322964
2018-01-20 01:12:54 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: call void @__kmpc_end_critical(
// t_var1 = min(t_var1, t_var1_reduction);
// CHECK: load float, float* [[T_VAR1_PRIV]]
// CHECK: [[T_VAR1_REF_INT:%.+]] = bitcast float* [[T_VAR1_REF]] to i32*
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[OLD1:%.+]] = load atomic i32, i32* [[T_VAR1_REF_INT]] monotonic,
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: br label %[[CONT:.+]]
// CHECK: [[CONT]]
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[ORIG_OLD_INT:%.+]] = phi i32 [ [[OLD1]], %{{.+}} ], [ [[OLD2:%.+]], %{{.+}} ]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[CMP:%.+]] = fcmp olt float
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[CMP]]
// CHECK: [[UP:%.+]] = phi float
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[UP_INT:%.+]] = load i32
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR1_REF_INT:%.+]] = bitcast float* [[T_VAR1_REF]] to i32*
// CHECK: [[RES:%.+]] = cmpxchg i32* [[T_VAR1_REF_INT]], i32 [[ORIG_OLD_INT]], i32 [[UP_INT]] monotonic monotonic
[OPENMP] Fixed bug in atomic update/capture/write constructs. Fixed a bug with codegen for destination atomic l-value with padding and junk in this padding bytes. llvm-svn: 237422
2015-05-15 16:36:34 +08:00
// CHECK: [[OLD2:%.+]] = extractvalue { i32, i1 } [[RES]], 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[SUCCESS_FAIL:%.+]] = extractvalue { i32, i1 } [[RES]], 1
// CHECK: br i1 [[SUCCESS_FAIL]], label %[[ATOMIC_DONE:.+]], label %[[CONT]]
// CHECK: [[ATOMIC_DONE]]
// break;
// CHECK: br label %[[RED_DONE]]
// CHECK: [[RED_DONE]]
// CHECK-DAG: call {{.*}} [[S_FLOAT_TY_DESTR]]([[S_FLOAT_TY]]* [[VAR_PRIV]])
// CHECK-DAG: call {{.*}} [[S_FLOAT_TY_DESTR]]([[S_FLOAT_TY]]*
// CHECK: ret void
// void reduce_func(void *lhs[<n>], void *rhs[<n>]) {
// *(Type0*)lhs[0] = ReductionOperation0(*(Type0*)lhs[0], *(Type0*)rhs[0]);
// ...
// *(Type<n>-1*)lhs[<n>-1] = ReductionOperation<n>-1(*(Type<n>-1*)lhs[<n>-1],
// *(Type<n>-1*)rhs[<n>-1]);
// }
// CHECK: define internal void [[REDUCTION_FUNC]](i8*, i8*)
// t_var_lhs = (float*)lhs[0];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS:%.+]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR_RHS_VOID:%.+]] = load i8*, i8** [[T_VAR_RHS_REF]],
// CHECK: [[T_VAR_RHS:%.+]] = bitcast i8* [[T_VAR_RHS_VOID]] to float*
// t_var_rhs = (float*)rhs[0];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS:%.+]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR_LHS_VOID:%.+]] = load i8*, i8** [[T_VAR_LHS_REF]],
// CHECK: [[T_VAR_LHS:%.+]] = bitcast i8* [[T_VAR_LHS_VOID]] to float*
// var_lhs = (S<float>*)lhs[1];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS]], i64 0, i64 1
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR_RHS_VOID:%.+]] = load i8*, i8** [[VAR_RHS_REF]],
// CHECK: [[VAR_RHS:%.+]] = bitcast i8* [[VAR_RHS_VOID]] to [[S_FLOAT_TY]]*
// var_rhs = (S<float>*)rhs[1];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS]], i64 0, i64 1
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR_LHS_VOID:%.+]] = load i8*, i8** [[VAR_LHS_REF]],
// CHECK: [[VAR_LHS:%.+]] = bitcast i8* [[VAR_LHS_VOID]] to [[S_FLOAT_TY]]*
// var1_lhs = (S<float>*)lhs[2];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR1_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS]], i64 0, i64 2
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR1_RHS_VOID:%.+]] = load i8*, i8** [[VAR1_RHS_REF]],
// CHECK: [[VAR1_RHS:%.+]] = bitcast i8* [[VAR1_RHS_VOID]] to [[S_FLOAT_TY]]*
// var1_rhs = (S<float>*)rhs[2];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR1_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS]], i64 0, i64 2
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR1_LHS_VOID:%.+]] = load i8*, i8** [[VAR1_LHS_REF]],
// CHECK: [[VAR1_LHS:%.+]] = bitcast i8* [[VAR1_LHS_VOID]] to [[S_FLOAT_TY]]*
// t_var1_lhs = (float*)lhs[3];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR1_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS]], i64 0, i64 3
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR1_RHS_VOID:%.+]] = load i8*, i8** [[T_VAR1_RHS_REF]],
// CHECK: [[T_VAR1_RHS:%.+]] = bitcast i8* [[T_VAR1_RHS_VOID]] to float*
// t_var1_rhs = (float*)rhs[3];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR1_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS]], i64 0, i64 3
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR1_LHS_VOID:%.+]] = load i8*, i8** [[T_VAR1_LHS_REF]],
// CHECK: [[T_VAR1_LHS:%.+]] = bitcast i8* [[T_VAR1_LHS_VOID]] to float*
// t_var_lhs += t_var_rhs;
// CHECK: [[T_VAR_LHS_VAL:%.+]] = load float, float* [[T_VAR_LHS]],
// CHECK: [[T_VAR_RHS_VAL:%.+]] = load float, float* [[T_VAR_RHS]],
// CHECK: [[UP:%.+]] = fadd float [[T_VAR_LHS_VAL]], [[T_VAR_RHS_VAL]]
// CHECK: store float [[UP]], float* [[T_VAR_LHS]],
// var_lhs = var_lhs.operator &(var_rhs);
// CHECK: [[UP:%.+]] = call dereferenceable(4) [[S_FLOAT_TY]]* @{{.+}}([[S_FLOAT_TY]]* [[VAR_LHS]], [[S_FLOAT_TY]]* dereferenceable(4) [[VAR_RHS]])
// CHECK: [[BC1:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR_LHS]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_FLOAT_TY]]* [[UP]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes (Step 1). Summary: Upstream LLVM is changing the the prototypes of the @llvm.memcpy/memmove/memset intrinsics. This change updates the Clang tests for this change. The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change removes the alignment argument in favour of placing the alignment attribute on the source and destination pointers of the memory intrinsic call. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) At this time the source and destination alignments must be the same (Step 1). Step 2 of the change, to be landed shortly, will relax that contraint and allow the source and destination to have different alignments. llvm-svn: 322964
2018-01-20 01:12:54 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// var1_lhs = var1_lhs.operator &&(var1_rhs);
// CHECK: [[TO_FLOAT:%.+]] = call float @{{.+}}([[S_FLOAT_TY]]* [[VAR1_LHS]])
// CHECK: [[VAR1_BOOL:%.+]] = fcmp une float [[TO_FLOAT]], 0.0
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[VAR1_BOOL]], label %[[TRUE:.+]], label %[[END2:.+]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[TRUE]]
// CHECK: [[TO_FLOAT:%.+]] = call float @{{.+}}([[S_FLOAT_TY]]* [[VAR1_RHS]])
// CHECK: [[VAR1_REDUCTION_BOOL:%.+]] = fcmp une float [[TO_FLOAT]], 0.0
// CHECK: br label %[[END2]]
// CHECK: [[END2]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: [[COND_LVALUE:%.+]] = phi i1 [ false, %{{.+}} ], [ [[VAR1_REDUCTION_BOOL]], %[[TRUE]] ]
// CHECK: [[CONV:%.+]] = uitofp i1 [[COND_LVALUE]] to float
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call void @{{.+}}([[S_FLOAT_TY]]* [[COND_LVALUE:%.+]], float [[CONV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BC1:%.+]] = bitcast [[S_FLOAT_TY]]* [[VAR1_LHS]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_FLOAT_TY]]* [[COND_LVALUE]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes (Step 1). Summary: Upstream LLVM is changing the the prototypes of the @llvm.memcpy/memmove/memset intrinsics. This change updates the Clang tests for this change. The @llvm.memcpy/memmove/memset intrinsics currently have an explicit argument which is required to be a constant integer. It represents the alignment of the dest (and source), and so must be the minimum of the actual alignment of the two. This change removes the alignment argument in favour of placing the alignment attribute on the source and destination pointers of the memory intrinsic call. For example, code which used to read: call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 100, i32 4, i1 false) will now read call void @llvm.memcpy.p0i8.p0i8.i32(i8* align 4 %dest, i8* align 4 %src, i32 100, i1 false) At this time the source and destination alignments must be the same (Step 1). Step 2 of the change, to be landed shortly, will relax that contraint and allow the source and destination to have different alignments. llvm-svn: 322964
2018-01-20 01:12:54 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 4 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// t_var1_lhs = min(t_var1_lhs, t_var1_rhs);
// CHECK: [[T_VAR1_LHS_VAL:%.+]] = load float, float* [[T_VAR1_LHS]],
// CHECK: [[T_VAR1_RHS_VAL:%.+]] = load float, float* [[T_VAR1_RHS]],
// CHECK: [[CMP:%.+]] = fcmp olt float [[T_VAR1_LHS_VAL]], [[T_VAR1_RHS_VAL]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[CMP]]
// CHECK: [[UP:%.+]] = phi float
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: store float [[UP]], float* [[T_VAR1_LHS]],
// CHECK: ret void
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
// CHECK: define internal void [[MAIN_MICROTASK1]](i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}},
// CHECK: [[T_VAR_PRIV:%.+]] = alloca float,
// CHECK: [[VAR_PRIV:%.+]] = alloca [[S_FLOAT_TY]],
// CHECK: [[VAR1_PRIV:%.+]] = alloca [[S_FLOAT_TY]],
// CHECK: [[T_VAR1_PRIV:%.+]] = alloca float,
// CHECK: store i{{[0-9]+}}* [[GTID_ADDR]], i{{[0-9]+}}** [[GTID_ADDR_ADDR:%.+]],
// CHECK: [[T_VAR_REF:%.+]] = load float*, float** %
// CHECK: [[VAR_REF:%.+]] = load [[S_FLOAT_TY]]*, [[S_FLOAT_TY]]** %
// CHECK: [[VAR1_REF:%.+]] = load [[S_FLOAT_TY]]*, [[S_FLOAT_TY]]** %
// CHECK: [[T_VAR1_REF:%.+]] = load float*, float** %
// For + reduction operation initial value of private variable is 0.
// CHECK: store float 0.0{{.+}}, float* [[T_VAR_PRIV]],
// For & reduction operation initial value of private variable is ones in all bits.
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_FLOAT_TY_CONSTR:@.+]]([[S_FLOAT_TY]]* [[VAR_PRIV]])
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
// For && reduction operation initial value of private variable is 1.0.
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_FLOAT_TY_CONSTR:@.+]]([[S_FLOAT_TY]]* [[VAR1_PRIV]])
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
// For min reduction operation initial value of private variable is largest repesentable value.
// CHECK: store float 0x47EFFFFFE0000000, float* [[T_VAR1_PRIV]],
// CHECK-NOT: call i32 @__kmpc_reduce
Don't emit an unreachable return block. Patch by Brad Moody. llvm-svn: 358104
2019-04-11 01:03:09 +08:00
// CHECK: }
[OPENMP] Fix for http://llvm.org/PR25878: Error compiling an OpenMP program OpenMP codegen tried to emit the code for its constructs even if it was detected as a dead-code. Added checks to ensure that the code is emitted if the code is not dead. llvm-svn: 255990
2015-12-18 15:58:25 +08:00
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: define {{.*}} i{{[0-9]+}} [[TMAIN_INT]]()
// CHECK: [[TEST:%.+]] = alloca [[S_INT_TY]],
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_INT_TY_CONSTR:@.+]]([[S_INT_TY]]* [[TEST]])
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// CHECK: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 6, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, [2 x i32]*, i32*, [2 x [[S_INT_TY]]]*, [[S_INT_TY]]*, [[S_INT_TY]]*, i32*)* [[TMAIN_MICROTASK:@.+]] to void
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: call {{.*}} [[S_INT_TY_DESTR:@.+]]([[S_INT_TY]]*
// CHECK: ret
//
[OPENMP 4.5] Codegen for data members in 'reduction' clause. OpenMP 4.5 allows to privatize non-static data members of current class in non-static member functions. Patch supports codegen for non-static data members in 'reduction' clauses. llvm-svn: 262460
2016-03-02 12:57:40 +08:00
// CHECK: define {{.+}} @{{.+}}([[SS_TY]]*
// CHECK: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 0
// CHECK: store i{{[0-9]+}} 0, i{{[0-9]+}}* %
// CHECK: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 1
// CHECK: store i8
// CHECK: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 2
// CHECK: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 0
// CHECK-NOT: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 1
// CHECK: getelementptr inbounds [[SS_TY]], [[SS_TY]]* %{{.+}}, i32 0, i32 2
// CHECK: call void (%{{.+}}*, i{{[0-9]+}}, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)*, ...) @__kmpc_fork_call(%{{.+}}* @{{.+}}, i{{[0-9]+}} 4, void (i{{[0-9]+}}*, i{{[0-9]+}}*, ...)* bitcast (void (i{{[0-9]+}}*, i{{[0-9]+}}*, [[SS_TY]]*, i{{[0-9]+}}*, i{{[0-9]+}}*, i{{[0-9]+}}*)* [[SS_MICROTASK:@.+]] to void
// CHECK: [[B_REF:%.+]] = getelementptr {{.*}}[[SS_TY]], [[SS_TY]]* %{{.*}}, i32 0, i32 1
// CHECK: store i8 %{{.+}}, i8* [[B_REF]],
// CHECK: ret
// CHECK: define internal void [[SS_MICROTASK]](i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}}, [[SS_TY]]*
// CHECK: [[A_PRIV:%.+]] = alloca i{{[0-9]+}},
// CHECK: [[B_PRIV:%.+]] = alloca i{{[0-9]+}},
// CHECK: [[C_PRIV:%.+]] = alloca i{{[0-9]+}},
// CHECK: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[A_PRIV]],
// CHECK: store i{{[0-9]+}}* [[A_PRIV]], i{{[0-9]+}}** [[REFA:%.+]],
// CHECK: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[B_PRIV]],
// CHECK: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[C_PRIV]],
// CHECK: store i{{[0-9]+}}* [[C_PRIV]], i{{[0-9]+}}** [[REFC:%.+]],
// CHECK: [[A_PRIV:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[REFA]],
// CHECK-NEXT: [[A_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[A_PRIV]],
// CHECK-NEXT: [[INC:%.+]] = add nsw i{{[0-9]+}} [[A_VAL]], 1
// CHECK-NEXT: store i{{[0-9]+}} [[INC]], i{{[0-9]+}}* [[A_PRIV]],
// CHECK-NEXT: [[B_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[B_PRIV]],
// CHECK-NEXT: [[DEC:%.+]] = add nsw i{{[0-9]+}} [[B_VAL]], -1
// CHECK-NEXT: store i{{[0-9]+}} [[DEC]], i{{[0-9]+}}* [[B_PRIV]],
// CHECK-NEXT: [[C_PRIV:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[REFC]],
// CHECK-NEXT: [[C_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[C_PRIV]],
// CHECK-NEXT: [[DIV:%.+]] = sdiv i{{[0-9]+}} [[C_VAL]], 1
// CHECK-NEXT: store i{{[0-9]+}} [[DIV]], i{{[0-9]+}}* [[C_PRIV]],
// CHECK: call i32 @__kmpc_reduce_nowait(
// CHECK: ret void
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// CHECK: define internal void [[TMAIN_MICROTASK]](i{{[0-9]+}}* noalias [[GTID_ADDR:%.+]], i{{[0-9]+}}* noalias %{{.+}},
[OPENMP] Propagate alignment from original variables to the private copies. Currently private copies of captured variables have default alignment. Patch makes private variables to have same alignment as original variables. llvm-svn: 247260
2015-09-10 17:48:30 +08:00
// CHECK: [[T_VAR_PRIV:%.+]] = alloca i{{[0-9]+}}, align 128
// CHECK: [[VAR_PRIV:%.+]] = alloca [[S_INT_TY]], align 128
// CHECK: [[VAR1_PRIV:%.+]] = alloca [[S_INT_TY]], align 128
// CHECK: [[T_VAR1_PRIV:%.+]] = alloca i{{[0-9]+}}, align 128
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// Reduction list for runtime.
// CHECK: [[RED_LIST:%.+]] = alloca [4 x i8*],
// CHECK: store i{{[0-9]+}}* [[GTID_ADDR]], i{{[0-9]+}}** [[GTID_ADDR_ADDR:%.+]],
[OPENMP] Outlined function for parallel and other regions with list of captured variables. Currently all variables used in OpenMP regions are captured into a record and passed to outlined functions in this record. It may result in some poor performance because of too complex analysis later in optimization passes. Patch makes to emit outlined functions for parallel-based regions with a list of captured variables. It reduces code for 2*n GEPs, stores and loads at least. Codegen for task-based regions remains unchanged because runtime requires that all captured variables are passed in captured record. llvm-svn: 247251
2015-09-10 16:12:02 +08:00
// CHECK: [[T_VAR_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** %
// CHECK: [[VAR_REF:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** %
// CHECK: [[VAR1_REF:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** %
// CHECK: [[T_VAR1_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** %
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// For + reduction operation initial value of private variable is 0.
// CHECK: store i{{[0-9]+}} 0, i{{[0-9]+}}* [[T_VAR_PRIV]],
// For & reduction operation initial value of private variable is ones in all bits.
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_INT_TY_CONSTR:@.+]]([[S_INT_TY]]* [[VAR_PRIV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// For && reduction operation initial value of private variable is 1.0.
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call {{.*}} [[S_INT_TY_CONSTR:@.+]]([[S_INT_TY]]* [[VAR1_PRIV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// For min reduction operation initial value of private variable is largest repesentable value.
// CHECK: store i{{[0-9]+}} 2147483647, i{{[0-9]+}}* [[T_VAR1_PRIV]],
// Skip checks for internal operations.
// void *RedList[<n>] = {<ReductionVars>[0], ..., <ReductionVars>[<n>-1]};
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast i{{[0-9]+}}* [[T_VAR_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[T_VAR_PRIV_REF]],
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 1
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast [[S_INT_TY]]* [[VAR_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[VAR_PRIV_REF]],
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR1_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 2
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast [[S_INT_TY]]* [[VAR1_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[VAR1_PRIV_REF]],
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR1_PRIV_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST]], i64 0, i64 3
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast i{{[0-9]+}}* [[T_VAR1_PRIV]] to i8*
// CHECK: store i8* [[BITCAST]], i8** [[T_VAR1_PRIV_REF]],
// res = __kmpc_reduce_nowait(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>);
[OPENMP] Codegen for 'firstprivate' clause in 'for' directive. Adds proper codegen for 'firstprivate' clause in for directive. Initially codegen for 'firstprivate' clause was implemented for 'parallel' directive only. Also this patch emits sync point only after initialization of firstprivate variables, not all private variables. This sync point is not required for privates, lastprivates etc., only for initialization of firstprivate variables. Differential Revision: http://reviews.llvm.org/D8660 llvm-svn: 234978
2015-04-15 12:52:20 +08:00
// CHECK: [[GTID_REF:%.+]] = load i{{[0-9]+}}*, i{{[0-9]+}}** [[GTID_ADDR_ADDR]]
// CHECK: [[GTID:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[GTID_REF]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BITCAST:%.+]] = bitcast [4 x i8*]* [[RED_LIST]] to i8*
// CHECK: [[RES:%.+]] = call i32 @__kmpc_reduce_nowait(%{{.+}}* [[REDUCTION_LOC]], i32 [[GTID]], i32 4, i64 32, i8* [[BITCAST]], void (i8*, i8*)* [[REDUCTION_FUNC:@.+]], [8 x i32]* [[REDUCTION_LOCK]])
// switch(res)
// CHECK: switch i32 [[RES]], label %[[RED_DONE:.+]] [
// CHECK: i32 1, label %[[CASE1:.+]]
// CHECK: i32 2, label %[[CASE2:.+]]
// CHECK: ]
// case 1:
// t_var += t_var_reduction;
// CHECK: [[T_VAR_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR_REF]],
// CHECK: [[T_VAR_PRIV_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR_PRIV]],
// CHECK: [[UP:%.+]] = add nsw i{{[0-9]+}} [[T_VAR_VAL]], [[T_VAR_PRIV_VAL]]
// CHECK: store i{{[0-9]+}} [[UP]], i{{[0-9]+}}* [[T_VAR_REF]],
// var = var.operator &(var_reduction);
// CHECK: [[UP:%.+]] = call dereferenceable(4) [[S_INT_TY]]* @{{.+}}([[S_INT_TY]]* [[VAR_REF]], [[S_INT_TY]]* dereferenceable(4) [[VAR_PRIV]])
// CHECK: [[BC1:%.+]] = bitcast [[S_INT_TY]]* [[VAR_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_INT_TY]]* [[UP]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes. Summary: This change is step three in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html Reviewers: rjmccall Subscribers: jyknight, nemanjai, nhaehnle, javed.absar, sbc100, aheejin, kbarton, fedor.sergeev, cfe-commits Differential Revision: https://reviews.llvm.org/D41677 llvm-svn: 323617
2018-01-29 01:27:45 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 128 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// var1 = var1.operator &&(var1_reduction);
// CHECK: [[TO_INT:%.+]] = call i{{[0-9]+}} @{{.+}}([[S_INT_TY]]* [[VAR1_REF]])
// CHECK: [[VAR1_BOOL:%.+]] = icmp ne i{{[0-9]+}} [[TO_INT]], 0
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[VAR1_BOOL]], label %[[TRUE:.+]], label %[[END2:.+]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[TRUE]]
// CHECK: [[TO_INT:%.+]] = call i{{[0-9]+}} @{{.+}}([[S_INT_TY]]* [[VAR1_PRIV]])
// CHECK: [[VAR1_REDUCTION_BOOL:%.+]] = icmp ne i{{[0-9]+}} [[TO_INT]], 0
// CHECK: [[END2]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: [[COND_LVALUE:%.+]] = phi i1 [ false, %{{.+}} ], [ [[VAR1_REDUCTION_BOOL]], %[[TRUE]] ]
// CHECK: [[CONV:%.+]] = zext i1 [[COND_LVALUE]] to i32
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call void @{{.+}}([[S_INT_TY]]* [[COND_LVALUE:%.+]], i32 [[CONV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BC1:%.+]] = bitcast [[S_INT_TY]]* [[VAR1_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_INT_TY]]* [[COND_LVALUE]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes. Summary: This change is step three in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html Reviewers: rjmccall Subscribers: jyknight, nemanjai, nhaehnle, javed.absar, sbc100, aheejin, kbarton, fedor.sergeev, cfe-commits Differential Revision: https://reviews.llvm.org/D41677 llvm-svn: 323617
2018-01-29 01:27:45 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 128 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// t_var1 = min(t_var1, t_var1_reduction);
// CHECK: [[T_VAR1_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR1_REF]],
// CHECK: [[T_VAR1_PRIV_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR1_PRIV]],
// CHECK: [[CMP:%.+]] = icmp slt i{{[0-9]+}} [[T_VAR1_VAL]], [[T_VAR1_PRIV_VAL]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[CMP]]
// CHECK: [[UP:%.+]] = phi i32
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: store i{{[0-9]+}} [[UP]], i{{[0-9]+}}* [[T_VAR1_REF]],
// __kmpc_end_reduce_nowait(<loc>, <gtid>, &<lock>);
// CHECK: call void @__kmpc_end_reduce_nowait(%{{.+}}* [[REDUCTION_LOC]], i32 [[GTID]], [8 x i32]* [[REDUCTION_LOCK]])
// break;
// CHECK: br label %[[RED_DONE]]
// case 2:
// t_var += t_var_reduction;
// CHECK: [[T_VAR_PRIV_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR_PRIV]]
// CHECK: atomicrmw add i32* [[T_VAR_REF]], i32 [[T_VAR_PRIV_VAL]] monotonic
// var = var.operator &(var_reduction);
// CHECK: call void @__kmpc_critical(
// CHECK: [[UP:%.+]] = call dereferenceable(4) [[S_INT_TY]]* @{{.+}}([[S_INT_TY]]* [[VAR_REF]], [[S_INT_TY]]* dereferenceable(4) [[VAR_PRIV]])
// CHECK: [[BC1:%.+]] = bitcast [[S_INT_TY]]* [[VAR_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_INT_TY]]* [[UP]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes. Summary: This change is step three in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html Reviewers: rjmccall Subscribers: jyknight, nemanjai, nhaehnle, javed.absar, sbc100, aheejin, kbarton, fedor.sergeev, cfe-commits Differential Revision: https://reviews.llvm.org/D41677 llvm-svn: 323617
2018-01-29 01:27:45 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 128 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: call void @__kmpc_end_critical(
// var1 = var1.operator &&(var1_reduction);
// CHECK: call void @__kmpc_critical(
// CHECK: [[TO_INT:%.+]] = call i{{[0-9]+}} @{{.+}}([[S_INT_TY]]* [[VAR1_REF]])
// CHECK: [[VAR1_BOOL:%.+]] = icmp ne i{{[0-9]+}} [[TO_INT]], 0
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[VAR1_BOOL]], label %[[TRUE:.+]], label %[[END2:.+]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[TRUE]]
// CHECK: [[TO_INT:%.+]] = call i{{[0-9]+}} @{{.+}}([[S_INT_TY]]* [[VAR1_PRIV]])
// CHECK: [[VAR1_REDUCTION_BOOL:%.+]] = icmp ne i{{[0-9]+}} [[TO_INT]], 0
// CHECK: br label %[[END2]]
// CHECK: [[END2]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: [[COND_LVALUE:%.+]] = phi i1 [ false, %{{.+}} ], [ [[VAR1_REDUCTION_BOOL]], %[[TRUE]] ]
// CHECK: [[CONV:%.+]] = zext i1 [[COND_LVALUE]] to i32
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call void @{{.+}}([[S_INT_TY]]* [[COND_LVALUE:%.+]], i32 [[CONV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BC1:%.+]] = bitcast [[S_INT_TY]]* [[VAR1_REF]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_INT_TY]]* [[COND_LVALUE]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes. Summary: This change is step three in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html Reviewers: rjmccall Subscribers: jyknight, nemanjai, nhaehnle, javed.absar, sbc100, aheejin, kbarton, fedor.sergeev, cfe-commits Differential Revision: https://reviews.llvm.org/D41677 llvm-svn: 323617
2018-01-29 01:27:45 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 128 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: call void @__kmpc_end_critical(
// t_var1 = min(t_var1, t_var1_reduction);
// CHECK: [[T_VAR1_PRIV_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR1_PRIV]]
// CHECK: atomicrmw min i32* [[T_VAR1_REF]], i32 [[T_VAR1_PRIV_VAL]] monotonic
// break;
// CHECK: br label %[[RED_DONE]]
// CHECK: [[RED_DONE]]
// CHECK-DAG: call {{.*}} [[S_INT_TY_DESTR]]([[S_INT_TY]]* [[VAR_PRIV]])
// CHECK-DAG: call {{.*}} [[S_INT_TY_DESTR]]([[S_INT_TY]]*
// CHECK: ret void
// void reduce_func(void *lhs[<n>], void *rhs[<n>]) {
// *(Type0*)lhs[0] = ReductionOperation0(*(Type0*)lhs[0], *(Type0*)rhs[0]);
// ...
// *(Type<n>-1*)lhs[<n>-1] = ReductionOperation<n>-1(*(Type<n>-1*)lhs[<n>-1],
// *(Type<n>-1*)rhs[<n>-1]);
// }
// CHECK: define internal void [[REDUCTION_FUNC]](i8*, i8*)
// t_var_lhs = (i{{[0-9]+}}*)lhs[0];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS:%.+]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR_RHS_VOID:%.+]] = load i8*, i8** [[T_VAR_RHS_REF]],
// CHECK: [[T_VAR_RHS:%.+]] = bitcast i8* [[T_VAR_RHS_VOID]] to i{{[0-9]+}}*
// t_var_rhs = (i{{[0-9]+}}*)rhs[0];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS:%.+]], i64 0, i64 0
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR_LHS_VOID:%.+]] = load i8*, i8** [[T_VAR_LHS_REF]],
// CHECK: [[T_VAR_LHS:%.+]] = bitcast i8* [[T_VAR_LHS_VOID]] to i{{[0-9]+}}*
// var_lhs = (S<i{{[0-9]+}}>*)lhs[1];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS]], i64 0, i64 1
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR_RHS_VOID:%.+]] = load i8*, i8** [[VAR_RHS_REF]],
// CHECK: [[VAR_RHS:%.+]] = bitcast i8* [[VAR_RHS_VOID]] to [[S_INT_TY]]*
// var_rhs = (S<i{{[0-9]+}}>*)rhs[1];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS]], i64 0, i64 1
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR_LHS_VOID:%.+]] = load i8*, i8** [[VAR_LHS_REF]],
// CHECK: [[VAR_LHS:%.+]] = bitcast i8* [[VAR_LHS_VOID]] to [[S_INT_TY]]*
// var1_lhs = (S<i{{[0-9]+}}>*)lhs[2];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR1_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS]], i64 0, i64 2
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR1_RHS_VOID:%.+]] = load i8*, i8** [[VAR1_RHS_REF]],
// CHECK: [[VAR1_RHS:%.+]] = bitcast i8* [[VAR1_RHS_VOID]] to [[S_INT_TY]]*
// var1_rhs = (S<i{{[0-9]+}}>*)rhs[2];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[VAR1_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS]], i64 0, i64 2
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[VAR1_LHS_VOID:%.+]] = load i8*, i8** [[VAR1_LHS_REF]],
// CHECK: [[VAR1_LHS:%.+]] = bitcast i8* [[VAR1_LHS_VOID]] to [[S_INT_TY]]*
// t_var1_lhs = (i{{[0-9]+}}*)lhs[3];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR1_RHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_RHS]], i64 0, i64 3
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR1_RHS_VOID:%.+]] = load i8*, i8** [[T_VAR1_RHS_REF]],
// CHECK: [[T_VAR1_RHS:%.+]] = bitcast i8* [[T_VAR1_RHS_VOID]] to i{{[0-9]+}}*
// t_var1_rhs = (i{{[0-9]+}}*)rhs[3];
Compute and preserve alignment more faithfully in IR-generation. Introduce an Address type to bundle a pointer value with an alignment. Introduce APIs on CGBuilderTy to work with Address values. Change core APIs on CGF/CGM to traffic in Address where appropriate. Require alignments to be non-zero. Update a ton of code to compute and propagate alignment information. As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment helper function to CGF and made use of it in a number of places in the expression emitter. The end result is that we should now be significantly more correct when performing operations on objects that are locally known to be under-aligned. Since alignment is not reliably tracked in the type system, there are inherent limits to this, but at least we are no longer confused by standard operations like derived-to-base conversions and array-to-pointer decay. I've also fixed a large number of bugs where we were applying the complete-object alignment to a pointer instead of the non-virtual alignment, although most of these were hidden by the very conservative approach we took with member alignment. Also, because IRGen now reliably asserts on zero alignments, we should no longer be subject to an absurd but frustrating recurring bug where an incomplete type would report a zero alignment and then we'd naively do a alignmentAtOffset on it and emit code using an alignment equal to the largest power-of-two factor of the offset. We should also now be emitting much more aggressive alignment attributes in the presence of over-alignment. In particular, field access now uses alignmentAtOffset instead of min. Several times in this patch, I had to change the existing code-generation pattern in order to more effectively use the Address APIs. For the most part, this seems to be a strict improvement, like doing pointer arithmetic with GEPs instead of ptrtoint. That said, I've tried very hard to not change semantics, but it is likely that I've failed in a few places, for which I apologize. ABIArgInfo now always carries the assumed alignment of indirect and indirect byval arguments. In order to cut down on what was already a dauntingly large patch, I changed the code to never set align attributes in the IR on non-byval indirect arguments. That is, we still generate code which assumes that indirect arguments have the given alignment, but we don't express this information to the backend except where it's semantically required (i.e. on byvals). This is likely a minor regression for those targets that did provide this information, but it'll be trivial to add it back in a later patch. I partially punted on applying this work to CGBuiltin. Please do not add more uses of the CreateDefaultAligned{Load,Store} APIs; they will be going away eventually. llvm-svn: 246985
2015-09-08 16:05:57 +08:00
// CHECK: [[T_VAR1_LHS_REF:%.+]] = getelementptr inbounds [4 x i8*], [4 x i8*]* [[RED_LIST_LHS]], i64 0, i64 3
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[T_VAR1_LHS_VOID:%.+]] = load i8*, i8** [[T_VAR1_LHS_REF]],
// CHECK: [[T_VAR1_LHS:%.+]] = bitcast i8* [[T_VAR1_LHS_VOID]] to i{{[0-9]+}}*
// t_var_lhs += t_var_rhs;
// CHECK: [[T_VAR_LHS_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR_LHS]],
// CHECK: [[T_VAR_RHS_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR_RHS]],
// CHECK: [[UP:%.+]] = add nsw i{{[0-9]+}} [[T_VAR_LHS_VAL]], [[T_VAR_RHS_VAL]]
// CHECK: store i{{[0-9]+}} [[UP]], i{{[0-9]+}}* [[T_VAR_LHS]],
// var_lhs = var_lhs.operator &(var_rhs);
// CHECK: [[UP:%.+]] = call dereferenceable(4) [[S_INT_TY]]* @{{.+}}([[S_INT_TY]]* [[VAR_LHS]], [[S_INT_TY]]* dereferenceable(4) [[VAR_RHS]])
// CHECK: [[BC1:%.+]] = bitcast [[S_INT_TY]]* [[VAR_LHS]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_INT_TY]]* [[UP]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes. Summary: This change is step three in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html Reviewers: rjmccall Subscribers: jyknight, nemanjai, nhaehnle, javed.absar, sbc100, aheejin, kbarton, fedor.sergeev, cfe-commits Differential Revision: https://reviews.llvm.org/D41677 llvm-svn: 323617
2018-01-29 01:27:45 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 128 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// var1_lhs = var1_lhs.operator &&(var1_rhs);
// CHECK: [[TO_INT:%.+]] = call i{{[0-9]+}} @{{.+}}([[S_INT_TY]]* [[VAR1_LHS]])
// CHECK: [[VAR1_BOOL:%.+]] = icmp ne i{{[0-9]+}} [[TO_INT]], 0
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[VAR1_BOOL]], label %[[TRUE:.+]], label %[[END2:.+]]
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[TRUE]]
// CHECK: [[TO_INT:%.+]] = call i{{[0-9]+}} @{{.+}}([[S_INT_TY]]* [[VAR1_RHS]])
// CHECK: [[VAR1_REDUCTION_BOOL:%.+]] = icmp ne i{{[0-9]+}} [[TO_INT]], 0
// CHECK: br label %[[END2]]
// CHECK: [[END2]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: [[COND_LVALUE:%.+]] = phi i1 [ false, %{{.+}} ], [ [[VAR1_REDUCTION_BOOL]], %[[TRUE]] ]
// CHECK: [[CONV:%.+]] = zext i1 [[COND_LVALUE]] to i32
Revert "[CodeGenCXX] Treat 'this' as noalias in constructors" This reverts commit https://reviews.llvm.org/rL344150 which causes MachineOutliner related failures on the ppc64le multistage buildbot. llvm-svn: 344526
2018-10-15 23:43:00 +08:00
// CHECK: call void @{{.+}}([[S_INT_TY]]* [[COND_LVALUE:%.+]], i32 [[CONV]])
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: [[BC1:%.+]] = bitcast [[S_INT_TY]]* [[VAR1_LHS]] to i8*
// CHECK: [[BC2:%.+]] = bitcast [[S_INT_TY]]* [[COND_LVALUE]] to i8*
Change memcpy/memove/memset to have dest and source alignment attributes. Summary: This change is step three in the series of changes to remove alignment argument from memcpy/memmove/memset in favour of alignment attributes. Steps: Step 1) Remove alignment parameter and create alignment parameter attributes for memcpy/memmove/memset. ( rL322965, rC322964, rL322963 ) Step 2) Expand the IRBuilder API to allow creation of memcpy/memmove with differing source and dest alignments. ( rL323597 ) Step 3) Update Clang to use the new IRBuilder API. Step 4) Update Polly to use the new IRBuilder API. Step 5) Update LLVM passes that create memcpy/memmove calls to use the new IRBuilder API, and those that use use MemIntrinsicInst::[get|set]Alignment() to use getDestAlignment() and getSourceAlignment() instead. Step 6) Remove the single-alignment IRBuilder API for memcpy/memmove, and the MemIntrinsicInst::[get|set]Alignment() methods. Reference http://lists.llvm.org/pipermail/llvm-dev/2015-August/089384.html http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20151109/312083.html Reviewers: rjmccall Subscribers: jyknight, nemanjai, nhaehnle, javed.absar, sbc100, aheejin, kbarton, fedor.sergeev, cfe-commits Differential Revision: https://reviews.llvm.org/D41677 llvm-svn: 323617
2018-01-29 01:27:45 +08:00
// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 128 [[BC1]], i8* align 4 [[BC2]], i64 4, i1 false)
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// t_var1_lhs = min(t_var1_lhs, t_var1_rhs);
// CHECK: [[T_VAR1_LHS_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR1_LHS]],
// CHECK: [[T_VAR1_RHS_VAL:%.+]] = load i{{[0-9]+}}, i{{[0-9]+}}* [[T_VAR1_RHS]],
// CHECK: [[CMP:%.+]] = icmp slt i{{[0-9]+}} [[T_VAR1_LHS_VAL]], [[T_VAR1_RHS_VAL]]
[OPENMP] Fixed codegen for 'reduction' clause. Fixed codegen for reduction operations min, max, && and ||. Codegen for them is quite similar and I was confused by this similarity. Also added a call to kmpc_end_reduce() in atomic part of reduction codegen (call to kmpc_end_reduce_nowait() is not required). Differential Revision: http://reviews.llvm.org/D9513 llvm-svn: 236689
2015-05-07 11:54:03 +08:00
// CHECK: br i1 [[CMP]]
// CHECK: [[UP:%.+]] = phi i32
[OPENMP] Codegen for 'reduction' clause in 'parallel' directive. Emit a code for reduction clause. Next code should be emitted for reductions: static kmp_critical_name lock = { 0 }; void reduce_func(void *lhs[<n>], void *rhs[<n>]) { ... *(Type<i> *)lhs[i] = RedOp<i>(*(Type<i> *)lhs[i], *(Type<i> *)rhs[i]); ... } ... void *RedList[<n>] = {&<RHSExprs>[0], ..., &<RHSExprs>[<n> - 1]}; switch (__kmpc_reduce{_nowait}(<loc>, <gtid>, <n>, sizeof(RedList), RedList, reduce_func, &<lock>)) { case 1: ... <LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i]); ... __kmpc_end_reduce{_nowait}(<loc>, <gtid>, &<lock>); break; case 2: ... Atomic(<LHSExprs>[i] = RedOp<i>(*<LHSExprs>[i], *<RHSExprs>[i])); ... break; default: ; } Reduction variables are a kind of a private variables, they have private copies, but initial values are chosen in accordance with the reduction operation. Differential Revision: http://reviews.llvm.org/D8915 llvm-svn: 234583
2015-04-10 18:43:45 +08:00
// CHECK: store i{{[0-9]+}} [[UP]], i{{[0-9]+}}* [[T_VAR1_LHS]],
// CHECK: ret void
#endif