According to OpenMP, 2.11.3 allocate Directive, Restrictions, C / C++,
if a list item has a static storage type, the allocator expression in
the allocator clause must be a constant expression that evaluates to
one of the predefined memory allocator values. Added check for this
restriction.
llvm-svn: 356496
Added initial codegen for the local variables with the #pragma omp
allocate directive. Instead of allocating the variables on the stack,
__kmpc_alloc|__kmpc_free functions are used for memory (de-)allocation.
llvm-svn: 356472
If the doacross lop construct is used and the loop counter is declare
outside of the loop, the compiler might crash trying to get the address
of the loop counter. Patch fixes this problem.
llvm-svn: 356198
array.
If the firstprivate variable is a reference, we may incorrectly classify
the kind of the private copy. Use the type of the private copy instead
of the original shared variable.
llvm-svn: 356098
'_openmp_teams_reductions_buffer_$_.
nvlink does not handle weak linkage correctly, same symbols with the
different sizes are reported as erroneous though the largest size must
be chosen instead. Patch fixes this problem by using Internal linkage
instead of the Common.
llvm-svn: 356072
If the variable was declared and marked as declare target, a new offload
entry with size 0 is created. But if later a definition is created and
marked as declare target, this definition is not added to the entry set
and the definition remains not mapped to the target. Patch fixes this
problem allowing to redefine the size and linkage for
previously registered declaration.
llvm-svn: 355960
If the declare target link global is used in the target region
indirectly (used in the inner parallel, teams, etc. regions), we may
miss this variable and it leads to incorrect codegen.
llvm-svn: 355858
memory.
If the variable with the constant non-scalar type is firstprivatized in
the target region, the local copy is created with the data copying.
Instead, we allocate the copy in the constant memory and avoid extra
copying in the outlined target regions. This global copy is used in the
target regions without loss of the performance.
llvm-svn: 355418
statements.
If the assembler instruction is not generated and the delayed diagnostic
is emitted, we may end up with extra warning message for variables used
in the asm statement. Since the asm statement is not built, the
variables may be left non-referenced and it may produce a warning about
a use of the non-initialized variables.
llvm-svn: 354928
This patch implements the parsing and sema support for the OpenMP
'from'-clause with potential user-defined mappers attached.
User-defined mappers are a new feature in OpenMP 5.0. A 'from'-clause
can have an explicit or implicit associated mapper, which instructs the
compiler to generate and use customized mapping functions. An example is
shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(id: struct S s) map(s, s.d[0:s.len])
struct S ss;
#pragma omp target update from(mapper(id): ss) // use the mapper with name 'id' to map ss from device
Contributed-by: Lingda Li <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D58638
llvm-svn: 354817
This patch implements the parsing and sema support for OpenMP to clause
with potential user-defined mappers attached. User defined mapper is a
new feature in OpenMP 5.0. A to/from clause can have an explicit or
implicit associated mapper, which instructs the compiler to generate and
use customized mapping functions. An example is shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(id: struct S s) map(s, s.d[0:s.len])
struct S ss;
#pragma omp target update to(mapper(id): ss) // use the mapper with name 'id' to map ss to device
Contributed-by: <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D58523
llvm-svn: 354698
Summary:
Added the ability to emit target-specific builtin assembler error
messages only in case if the function is really is going to be emitted
for the device.
Reviewers: rjmccall
Subscribers: guansong, jdoerfert, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D58243
llvm-svn: 354486
A faster way to reduce the values in teams reductions was found, the
codegen is updated to use this faster algorithm and new runtime functions.
llvm-svn: 354479
This patch implements the parsing and sema support for OpenMP map
clauses with potential user-defined mapper attached. User defined mapper
is a new feature in OpenMP 5.0. A map clause can have an explicit or
implicit associated mapper, which instructs the compiler to generate
extra data mapping. An example is shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(id: struct S s) map(s, s.d[0:s.len])
struct S ss;
#pragma omp target map(mapper(id) tofrom: ss) // use the mapper with name 'id' to map ss
Contributed-by: Lingda Li <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D58074
llvm-svn: 354347
expression is a discarded-value expression.
Summary:
We used to get this wrong in three ways:
1) During parsing, an expression-statement followed by the }) ending a
statement expression was always treated as producing the value of the
statement expression. That's wrong for ({ if (1) expr; })
2) During template instantiation, various kinds of statement (most
statements not appearing directly in a compound-statement) were not
treated as discarded-value expressions, resulting in missing volatile
loads (etc).
3) In all contexts, an expression-statement with attributes was not
treated as producing the value of the statement expression, eg
({ [[attr]] expr; }).
Also fix incorrect enforcement of OpenMP rule that directives can "only
be placed in the program at a position where ignoring or deleting the
directive would result in a program with correct syntax". In particular,
a label (be it goto, case, or default) should not affect whether
directives are permitted.
Reviewers: aaron.ballman, rjmccall
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D57984
llvm-svn: 354090
Fixed diagnostic emission for the exceptions support in case of the
compilation of OpenMP code for the devices. From now on, it uses delayed
diagnostics mechanism, previously used for CUDA only. It allow to
diagnose not allowed used of exceptions only in functions that are going
to be codegen'ed.
llvm-svn: 353542
It is important to delay the emission of the diagnostic messages for the
functions unless it is proved that the function is going to be used on
the device side. It is required to support compilation with some of the
target-specific system headers.
llvm-svn: 353540
Some of these functions take some extraneous arguments, e.g. EltSize,
Offset, which are computable from the Type and DataLayout.
Add some asserts to ensure that the computed values are consistent
with the passed-in values, in preparation for eliminating the
extraneous arguments. This also asserts that the Type is an Array for
the calls named "Array" and a Struct for the calls named "Struct".
Then, correct a couple of errors:
1. Using CreateStructGEP on an array type. (this causes the majority
of the test differences, as struct GEPs are created with i32
indices, while array GEPs are created with i64 indices)
2. Passing the wrong Offset to CreateStructGEP in TargetInfo.cpp on
x86-64 NACL (which uses 32-bit pointers).
Differential Revision: https://reviews.llvm.org/D57766
llvm-svn: 353529
The fix is to issue error messages if there are more than one
teams construct inside a target constructs.
#pragma omp target
{
#pragma omp teams
{ ... }
#pragma omp teams
{ ... }
}
llvm-svn: 353186
Summary: this commit adds support to a new dependence type introduced in OpenMP
5.0. The LLVM OpenMP RTL already supports this feature, so we only need to
modify CLANG to take advantage of them.
Differential Revision: https://reviews.llvm.org/D57576
llvm-svn: 353018
This patch implements parsing and sema for "omp declare mapper"
directive. User defined mapper, i.e., declare mapper directive, is a new
feature in OpenMP 5.0. It is introduced to extend existing map clauses
for the purpose of simplifying the copy of complex data structures
between host and device (i.e., deep copy). An example is shown below:
struct S { int len; int *d; };
#pragma omp declare mapper(struct S s) map(s, s.d[0:s.len]) // Memory region that d points to is also mapped using this mapper.
Contributed-by: Lingda Li <lildmh@gmail.com>
Differential Revision: https://reviews.llvm.org/D56326
llvm-svn: 352906
required.
Function __kmpc_push_target_tripcount should be emitted only if the
offloading entry is going to be emitted (for use in tgt_target...
functions). Otherwise, it should not be emitted.
llvm-svn: 352669
We don't need to use the predetermined data-sharing attributes for the
loop counters if the user explicitly specified correct data-sharing
attributes for such variables.
llvm-svn: 352543
According to the report, better to keep the original strict compare
operation as the loop condition with unsigned loop counters to make the
loop countable. This allows further loop transformations.
llvm-svn: 352526
With commit r351627, LLVM gained the ability to apply (existing) IPO
optimizations on indirections through callbacks, or transitive calls.
The general idea is that we use an abstraction to hide the middle man
and represent the callback call in the context of the initial caller.
It is described in more detail in the commit message of the LLVM patch
r351627, the llvm::AbstractCallSite class description, and the
language reference section on callback-metadata.
This commit enables clang to emit !callback metadata that is
understood by LLVM. It does so in three different cases:
1) For known broker functions declarations that are directly
generated, e.g., __kmpc_fork_call for the OpenMP pragma parallel.
2) For known broker functions that are identified by their name and
source location through the builtin detection, e.g.,
pthread_create from the POSIX thread API.
3) For user annotated functions that carry the "callback(callee, ...)"
attribute. The attribute has to include the name, or index, of
the callback callee and how the passed arguments can be
identified (as many as the callback callee has). See the callback
attribute documentation for detailed information.
Differential Revision: https://reviews.llvm.org/D55483
llvm-svn: 351629
Summary: Change the strategy for computing loop index variables after collapsing a loop nest via the collapse clause by replacing the expensive remainder operation with multiplications and additions.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: guansong, arphaman, cfe-commits
Differential Revision: https://reviews.llvm.org/D56413
llvm-svn: 350759
Summary: Introduce a compiler flag for cases when the user knows that the collapsed loop counter can be safely represented using at most 32 bits. This will prevent the emission of expensive mathematical operations (such as the div operation) on the iteration variable using 64 bits where 32 bit operations are sufficient.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: hfinkel, kkwli0, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D55928
llvm-svn: 350758
Summary:
There is a minor issue in how the implicit data-sharings for nested tasks are computed.
For the following example:
```
int x;
#pragma omp task shared(x)
#pragma omp task
x++;
```
We compute an implicit data-sharing of shared for `x` in the second task although I think that it should be firstprivate. Below you can find the part of the OpenMP spec that covers this example:
- // In a task generating construct, if no default clause is present, a variable for which the data-sharing attribute is not determined by the rules above and that in the enclosing context is determined to be shared by all implicit tasks bound to the current team is shared.//
- //In a task generating construct, if no default clause is present, a variable for which the data-sharing attribute is not determined by the rules above is firstprivate.//
Since each implicit-task has its own copy of `x`, we shouldn't apply the first rule.
Reviewers: ABataev
Reviewed By: ABataev
Subscribers: cfe-commits, rogfer01
Tags: #openmp
Differential Revision: https://reviews.llvm.org/D56430
llvm-svn: 350734
Each we create the target regions with the teams distribute inner
region, we can better estimate number of the teams required to execute
the target region. Function __kmpc_push_target_tripcount() is used for
purpose, which accepts device_id and the number of the iterations,
performed by the associated loop.
llvm-svn: 350571
As discussed in D56113, this patch refactors the implementation of the
const restriction for linear to reuse a function introduced by D56113.
A side effect is that, if a variable has mutable members, this
diagnostic is now skipped, and the diagnostic for the variable not
being an integer or pointer is reported instead.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D56299
llvm-svn: 350441
As discussed in D56113, this patch refactors the implementation of the
const restriction for reductions to reuse a function introduced by
D56113. A side effect is that diagnostics sometimes now say
"variable" instead of "list item" when a list item is a variable.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D56298
llvm-svn: 350440
The following appears in OpenMP 3.1 sec. 2.9.1.1 as a predetermined
data-sharing attribute:
> Variables with const-qualified type having no mutable member are
> shared.
It does not appear in OpenmP 4.0, 4.5, or 5.0. This patch removes the
implementation of that attribute when the requested OpenMP version is
greater than 3.1.
One effect of that removal is that `default(none)` affects const
variables without mutable members.
Also, without this patch, if a const variable without mutable members
was explicitly lastprivate or private, it was an error because it was
predetermined shared. Now, clang instead complains that it's const
without mutable fields, which is a more intelligible diagnostic. That
should be fine for all of the above versions because they all have
something like the following, which is quoted from OpenMP 5.0
sec. 2.19.3:
> A variable that is privatized must not have a const-qualified type
> unless it is of class type with a mutable member. This restriction does
> not apply to the firstprivate clause.
reduction and linear clauses already have separate checks for const
variables. Future patches will merge the implementations.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D56113
llvm-svn: 350439
nvvm_barrier0.
Use runtime functions instead of the direct call to the nvvm intrinsics.
It allows to prevent some dangerous LLVM optimizations, that breaks the
code for the NVPTX target.
llvm-svn: 350328
For constants with the predefined data-sharing clauses we may had
troubles with the target combined directives. It may cause compiler
crash in some corner cases.
llvm-svn: 350127
Not all %itanium_abi_triple values support TLS. Makes
OpenMP/declare_reduction_codegen.cpp, OpenMP/parallel_copyin_codegen.cpp for
%itanium_abi_triples without TLS support.
Alternatively we could pass -fnoopenmp-use-tls and tweak some of the CHECK
lines, but this seems a bit simpler.
Fixes PR40156.
Differential Revision: https://reviews.llvm.org/D56086
llvm-svn: 350067
Instead of generating llvm.mem.parallel_loop_access metadata, generate
llvm.access.group on instructions and llvm.loop.parallel_accesses on
loops. There is one access group per generated loop.
This is clang part of D52116/r349725.
Differential Revision: https://reviews.llvm.org/D52117
llvm-svn: 349823
Need to mark the loop as started when the initialization statement is
found. It is required to prevent possible incorrect loop iteraton
variable detection during template instantiation and fix the compiler
crash during the codegen.
llvm-svn: 349657
Without this patch, clang doesn't complain that X needs explicit data
sharing attributes in the following:
```
#pragma omp target teams default(none)
{
#pragma omp parallel num_threads(X)
;
}
```
However, clang does produce that complaint after the braces are
removed. With this patch, clang complains in both cases.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D55861
llvm-svn: 349635
A map clause with the close map-type-modifier is a hint to
prefer that the variables are mapped using a copy into faster
memory.
Patch by Ahsan Saghir (saghir)
Differential Revision: https://reviews.llvm.org/D55719
llvm-svn: 349551
buffer.
Seems to me, nvlink has a bug with the proper support of the weakly
linked symbols. It does not allow to define several shared memory buffer
with the different sizes even with the weak linkage. Instead we always
use 128 bytes buffer to prevent nvlink from the error message emission.
llvm-svn: 349540
Inlined runtime with the current implementation of the interwarp copy
function leads to the undefined behavior because of the not quite
correct implementation of the barriers. Start using generic
__kmpc_barier function instead of the custom made barriers.
llvm-svn: 349192
If the array section is based on pointer and this sections is mapped in
target region + then it is used in the inner parallel region, it also
must be globalized as the pointer itself is passed by value, not by
reference.
llvm-svn: 348492
Critical regions in NVPTX are the constructs, which, generally speaking,
are not supported by the NVPTX target. Instead we're using special
technique to handle the critical regions. Currently they are supported
only within the loop and all the threads in the loop must execute the
same critical region.
Inside of this special regions the regions still must be emitted as
critical, to avoid possible data races between the teams +
synchronization must use __kmpc_barrier functions.
llvm-svn: 348272
__kmpc_barrier runtime functions must be marked as convergent to prevent
some dangerous optimizations. Also, for NVPTX target all barriers must
be emitted as simple barriers.
llvm-svn: 348271
This moves everything primarily testing the functionality of -ast-dump and -ast-print into their own directory, rather than leaving the tests spread around the testing directory.
llvm-svn: 348017
Summary: This patch adds a new runtime for the SPMD deinit kernel function which replaces the previous function. The new function takes as argument the flag which signals whether the runtime is required or not. This enables the compiler to optimize out the part of the deinit function which are not needed.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D54970
llvm-svn: 347915
Fixed emission of the target regions found in the virtual functions.
Previously we may end up with the situation when those regions could be
skipped.
llvm-svn: 347793
modes.
If the region is inside target|teams|distribute region, we can emit the
locations with the correct info for execution mode and runtime mode.
Patch adds this ability to the NVPTX codegen to help the optimizer to
produce better code.
llvm-svn: 347583
For the NVPTX target default locations should be emitted as constants +
additional info must be emitted in the reserved_2 field of the ident_t
structure. The 1st bit controls the execution mode and the 2nd bit
controls use of the lightweight runtime. The combination of the bits for
Non-SPMD mode + lightweight runtime represents special undefined mode,
used outside of the target regions for orphaned directives or functions.
Should allow and additional optimization inside of the target regions.
llvm-svn: 347425
Loop-control variables with the default data-sharing attributes should
not be captured in the OpenMP region as they are private by default.
Also, default attributes should be emitted for such variables in the
inner OpenMP regions for the correct data sharing during codegen.
llvm-svn: 347409
forms of random access iterator
In OpenMP 4.5, only 4 relational operators are supported: <, <=, >,
and >=. This work is to enable support for relational operator
!= (not-equal) as one of the canonical forms.
Patch by Anh Tuyen Tran
Differential Revision: https://reviews.llvm.org/D54441
llvm-svn: 347405
This patch adjusts a test not to depend on deprecated FileCheck
behavior that permits overlapping matches within a block of CHECK-DAG
directives. Thus, this patch also removes uses of FileCheck's
-allow-deprecated-dag-overlap command-line option.
There were two issues in this test:
1. There were sets of patterns for store instructions in which a
pattern X could match a superset of a pattern Y. While X appeared
before Y, Y's intended match appeared before X's intended match. The
result was that X matched Y's intended match. Under the old
overlapping behavior, Y also matched Y's intended match. Under the
new non-overlapping behavior, Y had nothing left to match. This patch
fixes this by gathering these sets in one place and putting the most
specific patterns (Y) before the more general patterns (X).
2. The CHECK-DAG patterns involving the variables CBPADDR3 and
CBPADDR4 were the same, but there was only one match in the text, so
CBPADDR4 patterns had nothing to match under the new non-overlapping
behavior. Moreover, a preceding related series of directives had
variables (SADDR0, BPADDR0, etc.) numbered only 0 through 4, but this
series had variables numbered 0 through 5. Assuming CBPADDR4's
directives were not intended, this patch removes them.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D54765
llvm-svn: 347351
This patch adjusts a test not to depend on deprecated FileCheck
behavior that permits overlapping matches within a block of CHECK-DAG
directives. Thus, this patch also removes uses of FileCheck's
-allow-deprecated-dag-overlap command-line option.
Specifically, the FileCheck variables DBG_LOC_START, DBG_LOC_END, and
DBG_LOC_CANCEL were all set to the same value. As a result, three
TERM_DEBUG-DAG patterns, one for each variable, all matched the same
text under the old overlapping behavior. Under the new
non-overlapping behavior, that's not permitted. This patch's solution
is to replace these variables with one variable and replace these
patterns with one pattern.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D54764
llvm-svn: 347350
If lambda is used inside of the OpenMP region and captures `this`, we
should recapture it in the OpenMP region also. But we should do this
only if the OpenMP region is used in the context of the same class, just
like the lambda.
llvm-svn: 347096
reductions.
Fixed previously committed code for the reduction support in
teams/parallel constructs taking into account new design of the NVPTX
support in the compiler. Teams reduction are not fully functional yet,
it is going to be fixed in the following patches.
llvm-svn: 347081
If the statements between target|teams|distribute directives does not
require execution in master thread, like constant expressions, null
statements, simple declarations, etc., such construct can be xecuted in
SPMD mode.
llvm-svn: 346551
target|teams|distribute variables.
If the total size of the variables, declared in target|teams|distribute
regions, is less than the maximal size of shared memory available, the
buffer is allocated in the shared memory.
llvm-svn: 346507
The base pointer for the lambda mapping must point to the lambda capture
placement and pointer must point to the captured variable itself. Patch
fixes this problem.
llvm-svn: 346408
Fixed lookup for the target regions in unused virtual functions + fixed
processing of the global variables not marked as declare target but
emitted during debug info emission.
llvm-svn: 346343
Coalesced memory access requires use of the new function
`__kmpc_data_sharing_coalesced_push_stack` instead of the
`__kmpc_data_sharing_push_stack`.
llvm-svn: 345991
The previously used combination `PTR_AND_OBJ | PRIVATE` could be used for mapping of some data in Fortran. Changed it to `PTR_AND_OBJ | LITERAL`.
llvm-svn: 345982
target/teams/distribute regions.
Target/teams/distribute regions exist for all the time the kernel is
executed. Thus, if the variable is declared in their context and then
escape it, we can allocate global memory statically instead of
allocating it dynamically.
Patch captures all the globalized variables in target/teams/distribute
contexts, merges them into the records, one per each target region.
Those records are then joined into the union, one per compilation unit
(to save the global memory). Those units are organized into
2 x dimensional arrays, where the first dimension is
the number of blocks per SM and the second one is the number of SMs.
Runtime functions manage this global memory space between the executing
teams.
llvm-svn: 345978
Added support for mapping of lambdas in the target regions. It scans all
the captures by reference in the lambda, implicitly maps those variables
in the target region and then later reinstate the addresses of
references in lambda to the correct addresses of the captured|privatized
variables.
llvm-svn: 345609
being shared.
According to the standard, the variables with unspecified data-sharing
attributes in presence of `default(none)` clause must be reported to
users. Compiler did not generate error reports for the variables used in
other OpenMP regions. Patch fixes this.
llvm-svn: 345533
Summary: Iteration variable must be strictly less than the number of iterations. This fixes a bug introduced by previous patch D53448.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D53827
llvm-svn: 345527
Summary: This patch adds a new code generation path for bound sharing directives containing distribute parallel for. The new code generation scheme applies to chunked schedules on distribute and parallel for directives. The scheme simplifies the code that is being generated by eliminating the need for an outer for loop over chunks for both distribute and parallel for directives. In the case of distribute it applies to any sized chunk while in the parallel for case it only applies when chunk size is 1.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D53448
llvm-svn: 345509
Summary: This patch enables the choosing of the default schedule for parallel for loops even in non-SPMD cases.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D53443
llvm-svn: 345507
If the loop counter is not declared in the context of the loop and it is
private, such loop counters should not be captured in the outlined
regions.
llvm-svn: 345505
Summary:
This is a simple test of the parallel for code generation. It will be used to showcase the change introduced by patch D53443.
Reviewers: ABataev, caomhin
Reviewed By: ABataev
Subscribers: guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D53772
llvm-svn: 345417
According to the OpenMP standard, In a task generating construct, if no
default clause is present, a variable for which the data-sharing
attribute is not determined by the rules above is firstprivatized.
Compiler tries to implement this, but if the variable is not directly
used in the task context, this variable may not be firstprivatized.
Patch fixes this problem.
llvm-svn: 345277
Summary:
For the following code:
```
int i;
#pragma omp taskloop
for (i = 0; i < 100; ++i)
{}
#pragma omp taskloop nogroup
for (i = 0; i < 100; ++i)
{}
```
Clang emits the following LLVM IR:
```
...
call void @__kmpc_taskgroup(%struct.ident_t* @0, i32 %0)
%2 = call i8* @__kmpc_omp_task_alloc(%struct.ident_t* @0, i32 %0, i32 1, i64 80, i64 8, i32 (i32, i8*)* bitcast (i32 (i32, %struct.kmp_task_t_with_privates*)* @.omp_task_entry. to i32 (i32, i8*)*))
...
call void @__kmpc_taskloop(%struct.ident_t* @0, i32 %0, i8* %2, i32 1, i64* %8, i64* %9, i64 %13, i32 0, i32 0, i64 0, i8* null)
call void @__kmpc_end_taskgroup(%struct.ident_t* @0, i32 %0)
...
%15 = call i8* @__kmpc_omp_task_alloc(%struct.ident_t* @0, i32 %0, i32 1, i64 80, i64 8, i32 (i32, i8*)* bitcast (i32 (i32, %struct.kmp_task_t_with_privates.1*)* @.omp_task_entry..2 to i32 (i32, i8*)*))
...
call void @__kmpc_taskloop(%struct.ident_t* @0, i32 %0, i8* %15, i32 1, i64* %21, i64* %22, i64 %26, i32 0, i32 0, i64 0, i8* null)
```
The first set of instructions corresponds to the first taskloop construct. It is important to note that the implicit taskgroup region associated with the taskloop construct has been materialized in our IR: the `__kmpc_taskloop` occurs inside a taskgroup region. Note also that this taskgroup region does not exist in our second taskloop because we are using the `nogroup` clause.
The issue here is the 4th argument of the kmpc_taskloop call, starting from the end, is always a zero. Checking the LLVM OpenMP RT implementation, we see that this argument corresponds to the nogroup parameter:
```
void __kmpc_taskloop(ident_t *loc, int gtid, kmp_task_t *task, int if_val,
kmp_uint64 *lb, kmp_uint64 *ub, kmp_int64 st, int nogroup,
int sched, kmp_uint64 grainsize, void *task_dup);
```
So basically we always tell to the RT to do another taskgroup region. For the first taskloop, this means that we create two taskgroup regions. For the second example, it means that despite the fact we had a nogroup clause we are going to have a taskgroup region, so we unnecessary wait until all descendant tasks have been executed.
Reviewers: ABataev
Reviewed By: ABataev
Subscribers: rogfer01, cfe-commits
Differential Revision: https://reviews.llvm.org/D53636
llvm-svn: 345180
The compiler is crashing if we trying to post-capture the fields
implicitly captured inside of the task constructs. Seems, this kind of
processing is not supported and such fields should not be
firstprivatized.
llvm-svn: 345177
This reverts commit https://reviews.llvm.org/rL344150 which causes
MachineOutliner related failures on the ppc64le multistage buildbot.
llvm-svn: 344526
if the function has globalized variables and called in context of
target/teams/distribute regions, it does not need to globalize 32
copies of the same variables for memory coalescing, it is enough to
have just one copy, because there is parallel region.
Patch does this by adding call for `__kmpc_parallel_level` function and
checking its return value. If the code sees that the parallel level is
0, then only one variable is allocated, not 32.
llvm-svn: 344356
target/teams/distribute regions.
Previously introduced globalization scheme that uses memory coalescing
scheme may increase memory usage fr the variables that are devlared in
target/teams/distribute contexts. We don't need 32 copies of such
variables, just 1. Patch reduces memory use in this case.
llvm-svn: 344273
This is currently a clang extension and a resolution
of the defect report in the C++ Standard.
Differential Revision: https://reviews.llvm.org/D46441
llvm-svn: 344150
Added support for memory coalescing for better performance for
globalized variables. From now on all the globalized variables are
represented as arrays of 32 elements and each thread accesses these
elements using `tid & 31` as index.
llvm-svn: 344049
Fixed emission of the __kmpc_global_thread_num() so that it is not
messed up with alloca instructions anymore. Plus, fixes emission of the
__kmpc_global_thread_num() functions in the target outlined regions so
that they are not called before runtime is initialized.
llvm-svn: 343856
Worker threads fork off to the compiler generated worker function
directly after entering the kernel function. Hence, there is no
need to check whether the current thread is the master if we are
outside of a parallel region (neither SPMD nor parallel_level > 0).
Differential Revision: https://reviews.llvm.org/D52732
llvm-svn: 343618
lightweight runtime.
The datasharing flag must be set to `1` when executing SPMD-mode compatible directive with reduction|lastprivate clauses.
llvm-svn: 343492
According to OpenMP, the reduction item must be shared in parent region.
But the item can be an array section or array subscript. In this case,
we should not check for the datasharing of the base declaration.
llvm-svn: 343356
Summary: Set default schedule for parallel for loops to schedule(static, 1) when using SPMD mode on the NVPTX device offloading toolchain to ensure coalescing.
Reviewers: ABataev, Hahnfeld, caomhin
Reviewed By: ABataev
Subscribers: jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D52629
llvm-svn: 343260
Summary: For the OpenMP NVPTX toolchain choose a default distribute schedule that ensures coalescing on the GPU when in SPMD mode. This significantly increases the performance of offloaded target code and reduces the number of registers used on the GPU side.
Reviewers: ABataev, caomhin, Hahnfeld
Reviewed By: ABataev, Hahnfeld
Subscribers: Hahnfeld, jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D52434
llvm-svn: 343253
Add support for OMP5.0 requires directive and unified_address clause.
Patches to follow will include support for additional clauses.
Differential Revision: https://reviews.llvm.org/D52359
llvm-svn: 343063
declare reduction.
If the declare reduction construct with the non-dependent type is
defined in the template construct, the compiler might crash on the
template instantition. Reworked the whole instantiation scheme for the
declare reduction constructs to fix this problem correctly.
llvm-svn: 342151
Sema analysis should not mark functions as an implicit declare target,
it may break codegen. Simplified semantic analysis and removed extra
code for implicit declare target functions.
llvm-svn: 341939
Add the capability to nest multiple declare target directives
- including header files within a declare target region.
Differential Revision: https://reviews.llvm.org/D51378
Patch by Patrick Lyster
llvm-svn: 341766
context.
If the explicit template instantiation definition defined outside of the
target context, its vtable should not be marked as used. This is true
for other situations where the compiler want to emit vtables
unconditionally.
llvm-svn: 341570
'declare target'.
All the functions, referenced in implicit|explicit target regions must
be emitted during code emission for the device.
llvm-svn: 341093
If the target construct can be executed in SPMD mode + it is a loop
based directive with static scheduling, we can use lightweight runtime
support.
llvm-svn: 340953
Currently ident_t objects are created const when debug info is not
enabled, but the libittnotify libray in the OpenMP runtime writes to
the reserved_2 field (See __kmp_itt_region_forking in
openmp/runtime/src/kmp_itt.inl). Now create ident_t objects non-const.
Differential Revision: https://reviews.llvm.org/D51331
llvm-svn: 340934
If the function is actually a weak reference, it should not be marked as
deferred definition as this is only a declaration. Patch adds checks for
the definitions if they must be emitted. Otherwise, only declaration is
emitted.
llvm-svn: 340191
The compiler may produce unexpected error messages/crashes when declare
target variables were used. Patch fixes problems with the declarations
marked as declare target to or link.
llvm-svn: 339805
declare target.
According to OpenMP 5.0, variables captured in lambdas in declare target
regions must be considered as implicitly declare target.
llvm-svn: 339152
Encoding offload target triples onto comdat group key for offload initialization
code guarantees that it will be executed once per each unique combination of
offload targets.
Differential Revision: https://reviews.llvm.org/D50218
llvm-svn: 338916
offload targets.
Changed the linkage of omp_offloading.img_start.<triple> and omp_offloading.img_end.<triple> symbols from external to external weak to allow dropping of some targets during linking.
llvm-svn: 338413
No need to change the linkage, we can avoid the problem using special variable. That points to the original variable and, thus, prevent some of the optimizations that might break the compilation.
llvm-svn: 338399
The first argument for the parallel outlined functions, called as
serialized parallel regions, should be a pointer to the global thread id
that always is 0.
llvm-svn: 337957
the children.
Special internal helper expressions/statements for the OpenMP directives
should not be exposed as children, only the main substatement must be
represented as the child.
llvm-svn: 337941
This reverts commit r336467: libatomic is not available on all Linux
systems and this commit completely breaks OpenMP on them, even if there
are no atomic operations or all of them can be lowered to hardware
instructions.
See http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20180716/234816.html
for post-commit discussion.
llvm-svn: 337722
device IDs are now 64-bit integers (as opposed to 32-bit)
map flags are 64-bit long (used to be 32-bit)
mappings for partially mapped structs are now calculated at compile time and members of partially mapped structs are flagged using the MEMBER_OF field
Support for is_device_ptr on struct members was dropped - this functionality is not supported by the OpenMP standard and its implementation is technically infeasible (however, use_device_ptr on struct members works as a non-standard extension of the compiler)
llvm-svn: 337468
If the declare target link entries are created but not used, the
compiler will produce an error message. Patch improves handling of such
situations + improves checks for possibly lost declare target variables.
llvm-svn: 337207
Summary: In the SPMD case, we need to initialize the data sharing and globalization infrastructure. This covers the case when an SPMD region calls a function in a different compilation unit.
Reviewers: ABataev, carlo.bertolli, caomhin
Reviewed By: ABataev
Subscribers: Hahnfeld, jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D49188
llvm-svn: 337015
When the parsing of the functions happens inside of the declare target
region, we may erroneously mark local variables as declare target
thought they are not. This attribute can be applied only to global
variables.
llvm-svn: 336592
In generic data-sharing mode we are allowed to not globalize local
variables that escape their declaration context iff they are declared
inside of the parallel region. We can do this because L2 parallel
regions are executed sequentially and, thus, we do not need to put
shared local variables in the global memory.
llvm-svn: 336567
For some of the clauses the closing location erroneously points to the
beginning of the next clause rather than on the location of the closing
bracket of the clause.
llvm-svn: 336460
A reduction for an incomplete array type used to produce an assert
fail during codegen. Now it produces a diagnostic.
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D48735
llvm-svn: 335911
Patch tries to make better analysis of the variables that should be
globalized. From now, instead of all parallel directives it will check
only distribute parallel .. directives and check only for
firstprivte/lastprivate variables if they must be globalized.
llvm-svn: 335632
If the shuffle is required for the reduced structures/big data type,
current code may cause compiler crash because of the loading of the
aggregate values. Patch fixes this problem.
llvm-svn: 335377
parallel region.
If the current construct requires sharing of the local variable in the
inner parallel region, this variable must be globalized to avoid
runtime crash.
llvm-svn: 335285
If orphaned parallel region is found, the next code must be emitted:
```
if(__kmpc_is_spmd_exec_mode() || __kmpc_parallel_level(loc, gtid))
Serialized execution.
else if (IsMasterThread())
Prepare and signal worker.
else
Outined function call.
```
llvm-svn: 333301
if `-fopenmp-simd` is specified alone, `_OPENMP` macro should not be
defined. If `-fopenmp-simd` is specified along with the `-fopenmp`,
`_OPENMP` macro should be defined with the value `201511`.
llvm-svn: 332852
functions.
If the combined construct is specified in the declare target function
and the device code is emitted, the compiler crashes because of the
incorrectly chosen captured stmt. We should choose the innermost
captured statement, not the outermost.
llvm-svn: 332477
If the orphaned directive is executed in SPMD mode, we need to emit the
check for the SPMD mode and run the orphaned parallel directive in
sequential mode.
llvm-svn: 332467
In generic data-sharing mode we do not need to globalize
variables/parameters of reference/pointer types. They already are placed
in the global memory.
llvm-svn: 332380
Added initial support for L2 parallelism in SPMD mode. Note, though,
that the orphaned parallel directives are not currently supported in
SPMD mode.
llvm-svn: 332016
It is required to emit unique names for offloading regions ids. Required
to support compilation and linking of several compilation units.
llvm-svn: 331899
If the global variables are marked as declare target and they need
ctors/dtors, these ctors/dtors are emitted and then invoked by the
offloading runtime library. They are not explicitly used in the emitted
code and thus can be optimized out. Patch marks these functions as used,
so the optimizer cannot remove these function during the optimization
phase.
llvm-svn: 331879
The linkage of the global entries must be weak to enable support of
redefinition of the same target regions in multiple compilation units.
llvm-svn: 331768
Added initial codegen for level 2, 3 etc. parallelism. Currently, all
the second, the third etc. parallel regions will run sequentially.
llvm-svn: 331642
enabled for the host.
If the compilation for the host enables C++ exceptions, but they are not
supported by the device, we still need to allow the code with the
exception handling constructs outside of the target regions.
llvm-svn: 331372
Some symbols are not allowed to be used as names on some targets. Patch
ries to unify the emission of the names of LLVM globals so they could be
used on different targets.
llvm-svn: 331358
devices.
If the function is an instantiation|specialization of the template and
is used in the device code, the definitions of such functions should be
emitted for the device.
llvm-svn: 331261
We should not emit warning that the parameters are not marked as declare
target, these declaration are local and cannot be marked as declare
target.
llvm-svn: 331211
Emit error messages instead of compiler crashing when the target region
does not exist in the device code + fix crash when the location comes
from macros.
llvm-svn: 331195
NVPTX target.
When generating the wrapper function for the offloading region, we need
to call the outlined function and cast the arguments correctly to follow
the ABI. Usually, variables captured by value are casted to `uintptr_t`
type. But this should not performed for the variables with pointer type.
llvm-svn: 330620
Global variables marked as declare target are allowed to be used in map
clauses. Patch fixes the crash of the compiler on the declare target
variables in map clauses.
llvm-svn: 330156
Added NUW flags for all the add|mul|sub operations + replaced sdiv by udiv
as we operate on unsigned values only (addresses, converted to integers)
llvm-svn: 329411
Found via codespell -q 3 -I ../clang-whitelist.txt
Where whitelist consists of:
archtype
cas
classs
checkk
compres
definit
frome
iff
inteval
ith
lod
methode
nd
optin
ot
pres
statics
te
thru
Patch by luzpaz! (This is a subset of D44188 that applies cleanly with a few
files that have dubious fixes reverted.)
Differential revision: https://reviews.llvm.org/D44188
llvm-svn: 329399
variables.
Added emission of the offloading data sections for the variables within
declare target regions + fixes emission of the declare target variables
marked as declare target not within the declare target region.
llvm-svn: 328888
When the declare target variables are emitted for the device,
constructors|destructors for these variables must emitted and registered
by the runtime in the offloading sections.
llvm-svn: 328705
If the link clause is used on the declare target directive, the object
should be linked on target or target data directives, not during the
codegen. Patch adds support for this clause.
llvm-svn: 328544
Summary: The workers also need to initialize the global stack. The call to the initialization function needs to happen after the kernel_init() function is called by the master. This ensures that the per-team data structures of the runtime have been initialized.
Reviewers: ABataev, grokos, carlo.bertolli, caomhin
Reviewed By: ABataev
Subscribers: jholewinski, guansong, cfe-commits
Differential Revision: https://reviews.llvm.org/D44749
llvm-svn: 328219
constructs in generic mode.
Fixed codegen for distribute parallel combined constructs. We have to
pass and read the shared lower and upper bound from the distribute
region in the inner parallel region. Patch is for generic mode.
llvm-svn: 327990
If the generic codegen is enabled and private copy of the original
variable escapes the declaration context, this private copy should be
globalized just like it was the original variable.
llvm-svn: 327985
Alexey Bataev