2020-06-25 15:01:15 +08:00
// RUN: %clang_cc1 -triple x86_64-apple-macos10.7.0 -verify=expected,omp45 -fopenmp -fopenmp-version=45 -fnoopenmp-use-tls -ferror-limit 100 -o - %s
2020-08-28 03:35:36 +08:00
// RUN: %clang_cc1 -triple x86_64-apple-macos10.7.0 -verify=expected,omp5,host5 -fopenmp -fnoopenmp-use-tls -ferror-limit 100 -o - %s
// RUN: %clang_cc1 -triple x86_64-apple-macos10.7.0 -verify=expected,omp5,dev5 -fopenmp -fopenmp-is-device -fopenmp-targets=x86_64-apple-macos10.7.0 -aux-triple x86_64-apple-macos10.7.0 -fnoopenmp-use-tls -ferror-limit 100 -o - %s
2016-04-06 19:38:59 +08:00
2020-08-28 03:35:36 +08:00
// RUN: %clang_cc1 -triple x86_64-apple-macos10.7.0 -verify=expected,omp5,host5 -fopenmp-simd -fnoopenmp-use-tls -ferror-limit 100 -o - %s
// RUN: %clang_cc1 -triple x86_64-apple-macos10.7.0 -verify=expected,omp5,host5 -fopenmp-simd -fopenmp-is-device -fnoopenmp-use-tls -ferror-limit 100 -o - %s
2020-06-25 15:01:15 +08:00
// RUN: %clang_cc1 -triple x86_64-apple-macos10.7.0 -verify=expected,omp45 -fopenmp-version=45 -fopenmp-simd -fnoopenmp-use-tls -ferror-limit 100 -o - %s
2017-12-30 02:07:07 +08:00
2016-04-06 19:38:59 +08:00
# pragma omp end declare target // expected-error {{unexpected OpenMP directive '#pragma omp end declare target'}}
2019-08-17 04:15:02 +08:00
int a , b , z ; // omp5-error {{variable captured in declare target region must appear in a to clause}}
2016-04-06 19:38:59 +08:00
__thread int t ; // expected-note {{defined as threadprivate or thread local}}
2016-05-09 22:59:13 +08:00
# pragma omp declare target . // expected-error {{expected '(' after 'declare target'}}
# pragma omp declare target
2016-04-06 19:38:59 +08:00
void f ( ) ;
# pragma omp end declare target shared(a) // expected-warning {{extra tokens at the end of '#pragma omp end declare target' are ignored}}
2016-05-09 22:59:13 +08:00
[OpenMP] Overhaul `declare target` handling
This patch fixes various issues with our prior `declare target` handling
and extends it to support `omp begin declare target` as well.
This started with PR49649 in mind, trying to provide a way for users to
avoid the "ref" global use introduced for globals with internal linkage.
From there it went down the rabbit hole, e.g., all variables, even
`nohost` ones, were emitted into the device code so it was impossible to
determine if "ref" was needed late in the game (based on the name only).
To make it really useful, `begin declare target` was needed as it can
carry the `device_type`. Not emitting variables eagerly had a ripple
effect. Finally, the precedence of the (explicit) declare target list
items needed to be taken into account, that meant we cannot just look
for any declare target attribute to make a decision. This caused the
handling of functions to require fixup as well.
I tried to clean up things while I was at it, e.g., we should not "parse
declarations and defintions" as part of OpenMP parsing, this will always
break at some point. Instead, we keep track what region we are in and
act on definitions and declarations instead, this is what we do for
declare variant and other begin/end directives already.
Highlights:
- new diagnosis for restrictions specificed in the standard,
- delayed emission of globals not mentioned in an explicit
list of a declare target,
- omission of `nohost` globals on the host and `host` globals on the
device,
- no explicit parsing of declarations in-between `omp [begin] declare
variant` and the corresponding end anymore, regular parsing instead,
- precedence for explicit mentions in `declare target` lists over
implicit mentions in the declaration-definition-seq, and
- `omp allocate` declarations will now replace an earlier emitted
global, if necessary.
---
Notes:
The patch is larger than I hoped but it turns out that most changes do
on their own lead to "inconsistent states", which seem less desirable
overall.
After working through this I feel the standard should remove the
explicit declare target forms as the delayed emission is horrible.
That said, while we delay things anyway, it seems to me we check too
often for the current status even though that is often not sufficient to
act upon. There seems to be a lot of duplication that can probably be
trimmed down. Eagerly emitting some things seems pretty weak as an
argument to keep so much logic around.
---
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D101030
2021-04-22 13:57:28 +08:00
# pragma omp declare target map(a) // expected-error {{expected at least one 'to' or 'link' clause}} omp45-error {{unexpected 'map' clause, only 'to' or 'link' clauses expected}} omp5-error {{unexpected 'map' clause, only 'to', 'link' or 'device_type' clauses expected}}
2016-05-09 22:59:13 +08:00
2017-12-01 02:52:06 +08:00
# pragma omp declare target to(foo1) // expected-error {{use of undeclared identifier 'foo1'}}
# pragma omp declare target link(foo2) // expected-error {{use of undeclared identifier 'foo2'}}
[OpenMP] Overhaul `declare target` handling
This patch fixes various issues with our prior `declare target` handling
and extends it to support `omp begin declare target` as well.
This started with PR49649 in mind, trying to provide a way for users to
avoid the "ref" global use introduced for globals with internal linkage.
From there it went down the rabbit hole, e.g., all variables, even
`nohost` ones, were emitted into the device code so it was impossible to
determine if "ref" was needed late in the game (based on the name only).
To make it really useful, `begin declare target` was needed as it can
carry the `device_type`. Not emitting variables eagerly had a ripple
effect. Finally, the precedence of the (explicit) declare target list
items needed to be taken into account, that meant we cannot just look
for any declare target attribute to make a decision. This caused the
handling of functions to require fixup as well.
I tried to clean up things while I was at it, e.g., we should not "parse
declarations and defintions" as part of OpenMP parsing, this will always
break at some point. Instead, we keep track what region we are in and
act on definitions and declarations instead, this is what we do for
declare variant and other begin/end directives already.
Highlights:
- new diagnosis for restrictions specificed in the standard,
- delayed emission of globals not mentioned in an explicit
list of a declare target,
- omission of `nohost` globals on the host and `host` globals on the
device,
- no explicit parsing of declarations in-between `omp [begin] declare
variant` and the corresponding end anymore, regular parsing instead,
- precedence for explicit mentions in `declare target` lists over
implicit mentions in the declaration-definition-seq, and
- `omp allocate` declarations will now replace an earlier emitted
global, if necessary.
---
Notes:
The patch is larger than I hoped but it turns out that most changes do
on their own lead to "inconsistent states", which seem less desirable
overall.
After working through this I feel the standard should remove the
explicit declare target forms as the delayed emission is horrible.
That said, while we delay things anyway, it seems to me we check too
often for the current status even though that is often not sufficient to
act upon. There seems to be a lot of duplication that can probably be
trimmed down. Eagerly emitting some things seems pretty weak as an
argument to keep so much logic around.
---
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D101030
2021-04-22 13:57:28 +08:00
# pragma omp declare target to(f) device_type(host) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} dev5-note {{marked as 'device_type(host)' here}}
void q ( ) ;
# pragma omp declare target to(q) device_type(any) device_type(any) device_type(host) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} omp5-warning {{more than one 'device_type' clause is specified}}
2019-08-24 00:11:14 +08:00
2018-05-03 01:39:00 +08:00
void c ( ) ;
2016-04-06 19:38:59 +08:00
2017-12-13 04:08:12 +08:00
void func ( ) { } // expected-note {{'func' defined here}}
2019-08-24 00:11:14 +08:00
# pragma omp declare target link(func) allocate(a) // expected-error {{function name is not allowed in 'link' clause}} omp45-error {{unexpected 'allocate' clause, only 'to' or 'link' clauses expected}} omp5-error {{unexpected 'allocate' clause, only 'to', 'link' or 'device_type' clauses expected}}
void bar ( ) ;
void baz ( ) { bar ( ) ; }
# pragma omp declare target(bar) // omp5-warning {{declaration marked as declare target after first use, it may lead to incorrect results}}
2017-12-13 04:08:12 +08:00
2016-04-06 19:38:59 +08:00
extern int b ;
struct NonT {
int a ;
} ;
typedef int sint ;
2018-02-17 05:23:23 +08:00
template < typename T >
T bla1 ( ) { return 0 ; }
# pragma omp declare target
template < typename T >
T bla2 ( ) { return 0 ; }
# pragma omp end declare target
template < >
float bla2 ( ) { return 1.0 ; }
# pragma omp declare target
void blub2 ( ) {
bla2 < float > ( ) ;
bla2 < int > ( ) ;
}
# pragma omp end declare target
void t2 ( ) {
# pragma omp target
{
bla2 < float > ( ) ;
bla2 < long > ( ) ;
}
}
2018-09-10 10:07:09 +08:00
# pragma omp declare target
void abc ( ) ;
# pragma omp end declare target
void cba ( ) ;
# pragma omp end declare target // expected-error {{unexpected OpenMP directive '#pragma omp end declare target'}}
2018-02-17 05:23:23 +08:00
2019-08-17 04:15:02 +08:00
# pragma omp declare target
# pragma omp declare target
void def ( ) ;
# pragma omp end declare target
void fed ( ) ;
2018-09-10 10:07:09 +08:00
# pragma omp declare target
2016-04-06 19:38:59 +08:00
# pragma omp threadprivate(a) // expected-note {{defined as threadprivate or thread local}}
extern int b ;
int g ;
2017-09-13 19:12:35 +08:00
struct T {
2016-04-06 19:38:59 +08:00
int a ;
virtual int method ( ) ;
} ;
2017-09-13 19:12:35 +08:00
class VC {
2016-04-06 19:38:59 +08:00
T member ;
NonT member1 ;
public :
2017-09-13 19:12:35 +08:00
virtual int method ( ) { T a ; return 0 ; }
2016-04-06 19:38:59 +08:00
} ;
struct C {
NonT a ;
sint b ;
int method ( ) ;
int method1 ( ) ;
} ;
int C : : method1 ( ) {
return 0 ;
}
2018-05-01 02:28:08 +08:00
void foo ( int p ) {
2016-04-06 19:38:59 +08:00
a = 0 ; // expected-error {{threadprivate variables cannot be used in target constructs}}
2018-09-11 21:59:10 +08:00
b = 0 ;
2016-04-06 19:38:59 +08:00
t = 1 ; // expected-error {{threadprivate variables cannot be used in target constructs}}
C object ;
2017-09-13 19:12:35 +08:00
VC object1 ;
2016-04-06 19:38:59 +08:00
g = object . method ( ) ;
g + = object . method1 ( ) ;
2018-05-01 02:28:08 +08:00
g + = object1 . method ( ) + p ;
[OpenMP] Overhaul `declare target` handling
This patch fixes various issues with our prior `declare target` handling
and extends it to support `omp begin declare target` as well.
This started with PR49649 in mind, trying to provide a way for users to
avoid the "ref" global use introduced for globals with internal linkage.
From there it went down the rabbit hole, e.g., all variables, even
`nohost` ones, were emitted into the device code so it was impossible to
determine if "ref" was needed late in the game (based on the name only).
To make it really useful, `begin declare target` was needed as it can
carry the `device_type`. Not emitting variables eagerly had a ripple
effect. Finally, the precedence of the (explicit) declare target list
items needed to be taken into account, that meant we cannot just look
for any declare target attribute to make a decision. This caused the
handling of functions to require fixup as well.
I tried to clean up things while I was at it, e.g., we should not "parse
declarations and defintions" as part of OpenMP parsing, this will always
break at some point. Instead, we keep track what region we are in and
act on definitions and declarations instead, this is what we do for
declare variant and other begin/end directives already.
Highlights:
- new diagnosis for restrictions specificed in the standard,
- delayed emission of globals not mentioned in an explicit
list of a declare target,
- omission of `nohost` globals on the host and `host` globals on the
device,
- no explicit parsing of declarations in-between `omp [begin] declare
variant` and the corresponding end anymore, regular parsing instead,
- precedence for explicit mentions in `declare target` lists over
implicit mentions in the declaration-definition-seq, and
- `omp allocate` declarations will now replace an earlier emitted
global, if necessary.
---
Notes:
The patch is larger than I hoped but it turns out that most changes do
on their own lead to "inconsistent states", which seem less desirable
overall.
After working through this I feel the standard should remove the
explicit declare target forms as the delayed emission is horrible.
That said, while we delay things anyway, it seems to me we check too
often for the current status even though that is often not sufficient to
act upon. There seems to be a lot of duplication that can probably be
trimmed down. Eagerly emitting some things seems pretty weak as an
argument to keep so much logic around.
---
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D101030
2021-04-22 13:57:28 +08:00
f ( ) ; // dev5-error {{function with 'device_type(host)' is not available on device}}
q ( ) ;
2018-05-03 01:39:00 +08:00
c ( ) ;
2016-04-06 19:38:59 +08:00
}
2018-09-10 10:07:09 +08:00
# pragma omp declare target
2019-08-17 04:15:02 +08:00
void foo1 ( ) {
[ & ] ( ) { ( void ) ( b + z ) ; } ( ) ; // omp5-note {{variable 'z' is captured here}}
}
2016-04-06 19:38:59 +08:00
# pragma omp end declare target
2018-09-10 10:07:09 +08:00
# pragma omp end declare target
# pragma omp end declare target
2016-04-06 19:38:59 +08:00
# pragma omp end declare target // expected-error {{unexpected OpenMP directive '#pragma omp end declare target'}}
int C : : method ( ) {
return 0 ;
}
struct S {
2017-10-04 04:00:00 +08:00
# pragma omp declare target
2016-04-06 19:38:59 +08:00
int v ;
2017-10-04 04:00:00 +08:00
# pragma omp end declare target
2016-04-06 19:38:59 +08:00
} ;
int main ( int argc , char * * argv ) {
# pragma omp declare target // expected-error {{unexpected OpenMP directive '#pragma omp declare target'}}
int v ;
# pragma omp end declare target // expected-error {{unexpected OpenMP directive '#pragma omp end declare target'}}
2018-05-01 02:28:08 +08:00
foo ( v ) ;
2016-04-06 19:38:59 +08:00
return ( 0 ) ;
}
namespace {
[OpenMP] Overhaul `declare target` handling
This patch fixes various issues with our prior `declare target` handling
and extends it to support `omp begin declare target` as well.
This started with PR49649 in mind, trying to provide a way for users to
avoid the "ref" global use introduced for globals with internal linkage.
From there it went down the rabbit hole, e.g., all variables, even
`nohost` ones, were emitted into the device code so it was impossible to
determine if "ref" was needed late in the game (based on the name only).
To make it really useful, `begin declare target` was needed as it can
carry the `device_type`. Not emitting variables eagerly had a ripple
effect. Finally, the precedence of the (explicit) declare target list
items needed to be taken into account, that meant we cannot just look
for any declare target attribute to make a decision. This caused the
handling of functions to require fixup as well.
I tried to clean up things while I was at it, e.g., we should not "parse
declarations and defintions" as part of OpenMP parsing, this will always
break at some point. Instead, we keep track what region we are in and
act on definitions and declarations instead, this is what we do for
declare variant and other begin/end directives already.
Highlights:
- new diagnosis for restrictions specificed in the standard,
- delayed emission of globals not mentioned in an explicit
list of a declare target,
- omission of `nohost` globals on the host and `host` globals on the
device,
- no explicit parsing of declarations in-between `omp [begin] declare
variant` and the corresponding end anymore, regular parsing instead,
- precedence for explicit mentions in `declare target` lists over
implicit mentions in the declaration-definition-seq, and
- `omp allocate` declarations will now replace an earlier emitted
global, if necessary.
---
Notes:
The patch is larger than I hoped but it turns out that most changes do
on their own lead to "inconsistent states", which seem less desirable
overall.
After working through this I feel the standard should remove the
explicit declare target forms as the delayed emission is horrible.
That said, while we delay things anyway, it seems to me we check too
often for the current status even though that is often not sufficient to
act upon. There seems to be a lot of duplication that can probably be
trimmed down. Eagerly emitting some things seems pretty weak as an
argument to keep so much logic around.
---
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D101030
2021-04-22 13:57:28 +08:00
# pragma omp declare target
2016-04-06 19:38:59 +08:00
int x ;
[OpenMP] Overhaul `declare target` handling
This patch fixes various issues with our prior `declare target` handling
and extends it to support `omp begin declare target` as well.
This started with PR49649 in mind, trying to provide a way for users to
avoid the "ref" global use introduced for globals with internal linkage.
From there it went down the rabbit hole, e.g., all variables, even
`nohost` ones, were emitted into the device code so it was impossible to
determine if "ref" was needed late in the game (based on the name only).
To make it really useful, `begin declare target` was needed as it can
carry the `device_type`. Not emitting variables eagerly had a ripple
effect. Finally, the precedence of the (explicit) declare target list
items needed to be taken into account, that meant we cannot just look
for any declare target attribute to make a decision. This caused the
handling of functions to require fixup as well.
I tried to clean up things while I was at it, e.g., we should not "parse
declarations and defintions" as part of OpenMP parsing, this will always
break at some point. Instead, we keep track what region we are in and
act on definitions and declarations instead, this is what we do for
declare variant and other begin/end directives already.
Highlights:
- new diagnosis for restrictions specificed in the standard,
- delayed emission of globals not mentioned in an explicit
list of a declare target,
- omission of `nohost` globals on the host and `host` globals on the
device,
- no explicit parsing of declarations in-between `omp [begin] declare
variant` and the corresponding end anymore, regular parsing instead,
- precedence for explicit mentions in `declare target` lists over
implicit mentions in the declaration-definition-seq, and
- `omp allocate` declarations will now replace an earlier emitted
global, if necessary.
---
Notes:
The patch is larger than I hoped but it turns out that most changes do
on their own lead to "inconsistent states", which seem less desirable
overall.
After working through this I feel the standard should remove the
explicit declare target forms as the delayed emission is horrible.
That said, while we delay things anyway, it seems to me we check too
often for the current status even though that is often not sufficient to
act upon. There seems to be a lot of duplication that can probably be
trimmed down. Eagerly emitting some things seems pretty weak as an
argument to keep so much logic around.
---
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D101030
2021-04-22 13:57:28 +08:00
}
# pragma omp end declare target
2016-04-06 19:38:59 +08:00
2016-05-09 22:59:13 +08:00
# pragma omp declare target link(S) // expected-error {{'S' used in declare target directive is not a variable or a function name}}
# pragma omp declare target (x, x) // expected-error {{'x' appears multiple times in clauses on the same declare target directive}}
# pragma omp declare target to(x) to(x) // expected-error {{'x' appears multiple times in clauses on the same declare target directive}}
# pragma omp declare target link(x) // expected-error {{'x' must not appear in both clauses 'to' and 'link'}}
2019-08-24 00:11:14 +08:00
void bazz ( ) { }
2020-02-26 23:57:39 +08:00
# pragma omp declare target to(bazz) device_type(nohost) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} host5-note 3{{marked as 'device_type(nohost)' here}}
2019-08-24 00:11:14 +08:00
void bazzz ( ) { bazz ( ) ; }
# pragma omp declare target to(bazzz) device_type(nohost) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}}
2019-10-10 07:54:10 +08:00
void any ( ) { bazz ( ) ; } // host5-error {{function with 'device_type(nohost)' is not available on host}}
2020-02-26 23:57:39 +08:00
void host1 ( ) { bazz ( ) ; } // host5-error {{function with 'device_type(nohost)' is not available on host}}
2020-08-19 02:51:51 +08:00
# pragma omp declare target to(host1) device_type(host) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} dev5-note 3 {{marked as 'device_type(host)' here}}
2020-02-26 23:57:39 +08:00
void host2 ( ) { bazz ( ) ; } //host5-error {{function with 'device_type(nohost)' is not available on host}}
2019-08-24 00:11:14 +08:00
# pragma omp declare target to(host2)
2020-02-26 23:57:39 +08:00
void device ( ) { host1 ( ) ; } // dev5-error {{function with 'device_type(host)' is not available on device}}
2019-08-24 00:11:14 +08:00
# pragma omp declare target to(device) device_type(nohost) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} host5-note 2 {{marked as 'device_type(nohost)' here}}
2020-02-26 23:57:39 +08:00
void host3 ( ) { host1 ( ) ; } // dev5-error {{function with 'device_type(host)' is not available on device}}
2019-08-24 00:11:14 +08:00
# pragma omp declare target to(host3)
# pragma omp declare target
void any1 ( ) { any ( ) ; }
void any2 ( ) { host1 ( ) ; } // dev5-error {{function with 'device_type(host)' is not available on device}}
void any3 ( ) { device ( ) ; } // host5-error {{function with 'device_type(nohost)' is not available on host}}
void any4 ( ) { any2 ( ) ; }
# pragma omp end declare target
void any5 ( ) { any ( ) ; }
2020-08-19 02:51:51 +08:00
void any6 ( ) { host1 ( ) ; }
2019-08-24 00:11:14 +08:00
void any7 ( ) { device ( ) ; } // host5-error {{function with 'device_type(nohost)' is not available on host}}
void any8 ( ) { any2 ( ) ; }
[OpenMP] Overhaul `declare target` handling
This patch fixes various issues with our prior `declare target` handling
and extends it to support `omp begin declare target` as well.
This started with PR49649 in mind, trying to provide a way for users to
avoid the "ref" global use introduced for globals with internal linkage.
From there it went down the rabbit hole, e.g., all variables, even
`nohost` ones, were emitted into the device code so it was impossible to
determine if "ref" was needed late in the game (based on the name only).
To make it really useful, `begin declare target` was needed as it can
carry the `device_type`. Not emitting variables eagerly had a ripple
effect. Finally, the precedence of the (explicit) declare target list
items needed to be taken into account, that meant we cannot just look
for any declare target attribute to make a decision. This caused the
handling of functions to require fixup as well.
I tried to clean up things while I was at it, e.g., we should not "parse
declarations and defintions" as part of OpenMP parsing, this will always
break at some point. Instead, we keep track what region we are in and
act on definitions and declarations instead, this is what we do for
declare variant and other begin/end directives already.
Highlights:
- new diagnosis for restrictions specificed in the standard,
- delayed emission of globals not mentioned in an explicit
list of a declare target,
- omission of `nohost` globals on the host and `host` globals on the
device,
- no explicit parsing of declarations in-between `omp [begin] declare
variant` and the corresponding end anymore, regular parsing instead,
- precedence for explicit mentions in `declare target` lists over
implicit mentions in the declaration-definition-seq, and
- `omp allocate` declarations will now replace an earlier emitted
global, if necessary.
---
Notes:
The patch is larger than I hoped but it turns out that most changes do
on their own lead to "inconsistent states", which seem less desirable
overall.
After working through this I feel the standard should remove the
explicit declare target forms as the delayed emission is horrible.
That said, while we delay things anyway, it seems to me we check too
often for the current status even though that is often not sufficient to
act upon. There seems to be a lot of duplication that can probably be
trimmed down. Eagerly emitting some things seems pretty weak as an
argument to keep so much logic around.
---
Reviewed By: ABataev
Differential Revision: https://reviews.llvm.org/D101030
2021-04-22 13:57:28 +08:00
int MultiDevTy ;
# pragma omp declare target to(MultiDevTy) device_type(any) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}}
# pragma omp declare target to(MultiDevTy) device_type(host) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} omp5-error {{'device_type(host)' does not match previously specified 'device_type(any)' for the same declaration}}
# pragma omp declare target to(MultiDevTy) device_type(nohost) // omp45-error {{unexpected 'device_type' clause, only 'to' or 'link' clauses expected}} omp5-error {{'device_type(nohost)' does not match previously specified 'device_type(any)' for the same declaration}}
// TODO: Issue an error message error {{expected '#pragma omp end declare target'}} note {{to match this '#pragma omp declare target'}}
# pragma omp declare target