Commit Graph

42 Commits

Author SHA1 Message Date
Chuanqi Xu d30ca5e2e2 [C++20] [Coroutines] Implement return value optimization for get_return_object
This patch tries to implement RVO for coroutine's return object got from
get_return_object.
From [dcl.fct.def.coroutine]/p7 we could know that the return value of
get_return_object is either a reference or a prvalue. So it makes sense
to do copy elision for the return value. The return object should be
constructed directly into the storage where they would otherwise be
copied/moved to.

Test Plan: folly, check-all

Reviewed By: junparser

Differential revision: https://reviews.llvm.org/D117087
2022-02-16 13:38:00 +08:00
Chuanqi Xu c75cedc237 [Coroutines] Set presplit attribute in Clang and mlir
This fixes bug49264.

Simply, coroutine shouldn't be inlined before CoroSplit. And the marker
for pre-splited coroutine is created in CoroEarly pass, which ran after
AlwaysInliner Pass in O0 pipeline. So that the AlwaysInliner couldn't
detect it shouldn't inline a coroutine. So here is the error.

This patch set the presplit attribute in clang and mlir. So the inliner
would always detect the attribute before splitting.

Reviewed By: rjmccall, ezhulenev

Differential Revision: https://reviews.llvm.org/D115790
2022-01-05 10:25:02 +08:00
Chuanqi Xu f3d4e168db [C++20] Conform coroutine's comments in clang (NFC-ish)
The comments for coroutine in clang wrote for coroutine-TS. Now
coroutine is merged into standard. Try to conform the comments.
2021-12-24 12:41:44 +08:00
Xun Li 5faba87938 Revert "[Coroutines] Set presplit attribute in Clang instead of CoroEarly pass"
This reverts commit fa6b54c44a.
The commited patch broke mlir tests. It seems that mlir tests depend on coroutine function properties set in CoroEarly pass.
2021-04-18 17:22:28 -07:00
Xun Li fa6b54c44a [Coroutines] Set presplit attribute in Clang instead of CoroEarly pass
Presplit coroutines cannot be inlined. During AlwaysInliner we check if a function is a presplit coroutine, if so we skip inlining.
The presplit coroutine attributes are set in CoroEarly pass.
However in O0 pipeline, AlwaysInliner runs before CoroEarly, so the attribute isn't set yet and will still inline the coroutine.
This causes Clang to crash: https://bugs.llvm.org/show_bug.cgi?id=49920

To fix this, we set the attributes in the Clang front-end instead of in CoroEarly pass.

Reviewed By: rjmccall, ChuanqiXu

Differential Revision: https://reviews.llvm.org/D100282
2021-04-18 15:41:09 -07:00
Xun Li c0211e8d7d Revert "[Coroutines] Move CoroEarly pass to before AlwaysInliner"
This reverts commit 2b50f5a434.
Forgot to update the description of the commit to sync with phabricator. Going to redo the commit.
2021-04-18 15:38:19 -07:00
Xun Li 2b50f5a434 [Coroutines] Move CoroEarly pass to before AlwaysInliner
Presplit coroutines cannot be inlined. During AlwaysInliner we check if a function is a presplit coroutine, if so we skip inlining.
The presplit coroutine attributes are set in CoroEarly pass.
However in O0 pipeline, AlwaysInliner runs before CoroEarly, so the attribute isn't set yet and will still inline the coroutine.
This causes Clang to crash: https://bugs.llvm.org/show_bug.cgi?id=49920

Differential Revision: https://reviews.llvm.org/D100282
2021-04-18 14:54:04 -07:00
yifeng.dongyifeng 3a6a80b641 [Clang][Coroutine][DebugInfo] In c++ coroutine, clang will emit different debug info variables for parameters and move-parameters.
The first one is the real parameters of the coroutine function, the
other one just for copying parameters to the coroutine frame.

Considering the following c++ code:
```
struct coro {
  ...
};

coro foo(struct test & t) {
  ...
  co_await suspend_always();
    ...
    co_await suspend_always();
    ...
    co_await suspend_always();
}

int main(int argc, char *argv[]) {
  auto c = foo(...);
    c.handle.resume();
      ...
  }
```

Function foo is the standard coroutine function, and it has only
one parameter named t (ignoring this at first),
when we use the llvm code to compile this function, we can get the
following ir:

```
!2921 = distinct !DISubprogram(name: "foo", linkageName:
"_ZN6Object3fooE4test", scope: !2211, file: !45, li\
ne: 48, type: !2329, scopeLine: 48, flags: DIFlagPrototyped |
DIFlagAllCallsDescribed, spFlags: DISPFlagDefi\
nition | DISPFlagOptimized, unit: !44, declaration: !2328,
retainedNodes: !2922)
!2924 = !DILocalVariable(name: "t", arg: 2, scope: !2921, file: !45,
line: 48, type: !838)
...
!2926 = !DILocalVariable(name: "t", scope: !2921, type: !838, flags:
DIFlagArtificial)
```
We can find there are two `the same` DIVariable named t in the same
dwarf scope for foo.resume.
And when we try to use llvm-dwarfdump to dump the dwarf info of this
elf, we get the following output:

```
0x00006684:   DW_TAG_subprogram
                DW_AT_low_pc    (0x00000000004013a0)
                DW_AT_high_pc   (0x00000000004013a8)
                DW_AT_frame_base        (DW_OP_reg7 RSP)
                DW_AT_object_pointer    (0x0000669c)
                DW_AT_GNU_all_call_sites        (true)
                DW_AT_specification     (0x00005b5c "_ZN6Object3fooE4test")

0x000066a5:     DW_TAG_formal_parameter
                DW_AT_name    ("t")
                DW_AT_decl_file       ("/disk1/yifeng.dongyifeng/my_code/llvm/build/bin/coro-debug-1.cpp")
                DW_AT_decl_line       (48)
                DW_AT_type    (0x00004146 "test")

0x000066ba:     DW_TAG_variable
                  DW_AT_name    ("t")
                  DW_AT_type    (0x00004146 "test")
                  DW_AT_artificial      (true)
```
The elf also has two 't' in the same scope.
But unluckily, it might let the debugger
confused. And failed to print parameters for O0 or above.
This patch will make coroutine parameters and move
parameters use the same DIVar and try to fix the problems
that I mentioned before.

Test Plan: check-clang

Reviewed By: aprantl, jmorse

Differential Revision: https://reviews.llvm.org/D97533
2021-04-12 11:10:47 +08:00
Xun Li c7a39c833a [Coroutine][Clang] Force emit lifetime intrinsics for Coroutines
tl;dr Correct implementation of Corouintes requires having lifetime intrinsics available.

Coroutine functions are functions that can be suspended and resumed latter. To do so, data that need to stay alive after suspension must be put on the heap (i.e. the coroutine frame).
The optimizer is responsible for analyzing each AllocaInst and figure out whether it should be put on the stack or the frame.
In most cases, for data that we are unable to accurately analyze lifetime, we can just conservatively put them on the heap.
Unfortunately, there exists a few cases where certain data MUST be put on the stack, not on the heap. Without lifetime intrinsics, we are unable to correctly analyze those data's lifetime.

To dig into more details, there exists cases where at certain code points, the current coroutine frame may have already been destroyed. Hence no frame access would be allowed beyond that point.
The following is a common code pattern called "Symmetric Transfer" in coroutine:
```
auto tmp = await_suspend();
__builtin_coro_resume(tmp.address());
return;
```
In the above code example, `await_suspend()` returns a new coroutine handle, which we will obtain the address and then resume that coroutine. This essentially "transfered" from the current coroutine to a different coroutine.
During the call to `await_suspend()`, the current coroutine may be destroyed, which should be fine because we are not accessing any data afterwards.
However when LLVM is emitting IR for the above code, it needs to emit an AllocaInst for `tmp`. It will then call the `address` function on tmp. `address` function is a member function of coroutine, and there is no way for the LLVM optimizer to know that it does not capture the `tmp` pointer. So when the optimizer looks at it, it has to conservatively assume that `tmp` may escape and hence put it on the heap. Furthermore, in some cases `address` call would be inlined, which will generate a bunch of store/load instructions that move the `tmp` pointer around. Those stores will also make the compiler to think that `tmp` might escape.
To summarize, it's really difficult for the mid-end to figure out that the `tmp` data is short-lived.
I made some attempt in D98638, but it appears to be way too complex and is basically doing the same thing as inserting lifetime intrinsics in coroutines.

Also, for reference, we already force emitting lifetime intrinsics in O0 for AlwaysInliner: https://github.com/llvm/llvm-project/blob/main/llvm/lib/Passes/PassBuilder.cpp#L1893

Differential Revision: https://reviews.llvm.org/D99227
2021-03-25 13:46:20 -07:00
Jun Ma 53c2e10fb8 [Coroutines] Do not evaluate InitListExpr of a co_return
Differential Revision: https://reviews.llvm.org/D76118
2020-03-16 12:42:44 +08:00
Rui Ueyama 49a3ad21d6 Fix parameter name comments using clang-tidy. NFC.
This patch applies clang-tidy's bugprone-argument-comment tool
to LLVM, clang and lld source trees. Here is how I created this
patch:

$ git clone https://github.com/llvm/llvm-project.git
$ cd llvm-project
$ mkdir build
$ cd build
$ cmake -GNinja -DCMAKE_BUILD_TYPE=Debug \
    -DLLVM_ENABLE_PROJECTS='clang;lld;clang-tools-extra' \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=On -DLLVM_ENABLE_LLD=On \
    -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ ../llvm
$ ninja
$ parallel clang-tidy -checks='-*,bugprone-argument-comment' \
    -config='{CheckOptions: [{key: StrictMode, value: 1}]}' -fix \
    ::: ../llvm/lib/**/*.{cpp,h} ../clang/lib/**/*.{cpp,h} ../lld/**/*.{cpp,h}

llvm-svn: 366177
2019-07-16 04:46:31 +00:00
Fangrui Song 90e95bb289 Delete dead stores
llvm-svn: 365901
2019-07-12 14:04:34 +00:00
James Y Knight 8799caee8d [opaque pointer types] Trivial changes towards CallInst requiring
explicit function types.

llvm-svn: 353009
2019-02-03 21:53:49 +00:00
Richard Smith 259e1bdfdd Factor out duplication between ExprIterator and ConstExprIterator.
This also exposes a more general iterator cast mechanism suitable for
use in http://reviews.llvm.org/D56571.

llvm-svn: 352925
2019-02-01 21:58:17 +00:00
Chandler Carruth 2946cd7010 Update the file headers across all of the LLVM projects in the monorepo
to reflect the new license.

We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.

Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.

llvm-svn: 351636
2019-01-19 08:50:56 +00:00
Stephen Kelly f2ceec4811 Port getLocStart -> getBeginLoc
Reviewers: teemperor!

Subscribers: jholewinski, whisperity, jfb, cfe-commits

Differential Revision: https://reviews.llvm.org/D50350

llvm-svn: 339385
2018-08-09 21:08:08 +00:00
Brian Gesiak 12728474b3 [Coroutines] Less IR for noexcept await_resume
Summary:
In his review of https://reviews.llvm.org/D45860, @GorNishanov suggested
avoiding generating additional exception-handling IR in the case that
the resume function was marked as 'noexcept', and exceptions could not
occur. This implements that suggestion.

Test Plan: `check-clang`

Reviewers: GorNishanov, EricWF

Reviewed By: GorNishanov

Subscribers: cfe-commits, GorNishanov

Differential Revision: https://reviews.llvm.org/D47673

llvm-svn: 335422
2018-06-23 18:57:26 +00:00
Brian Gesiak ea9144e818 [Coroutines] Catch exceptions in await_resume
Summary:
http://wg21.link/P0664r2 section "Evolution/Core Issues 24" describes a
proposed change to Coroutines TS that would have any exceptions thrown
after the initial suspend point of a coroutine be caught by the handler
specified by the promise type's 'unhandled_exception' member function.
This commit provides a sample implementation of the specified behavior.

Test Plan: `check-clang`

Reviewers: GorNishanov, EricWF

Reviewed By: GorNishanov

Subscribers: cfe-commits, lewissbaker, eric_niebler

Differential Revision: https://reviews.llvm.org/D45860

llvm-svn: 331519
2018-05-04 14:02:37 +00:00
George Burgess IV 00f70bd933 Remove redundant casts. NFC
So I wrote a clang-tidy check to lint out redundant `isa`, `cast`, and
`dyn_cast`s for fun. This is a portion of what it found for clang; I
plan to do similar cleanups in LLVM and other subprojects when I find
time.

Because of the volume of changes, I explicitly avoided making any change
that wasn't highly local and obviously correct to me (e.g. we still have
a number of foo(cast<Bar>(baz)) that I didn't touch, since overloading
is a thing and the cast<Bar> did actually change the type -- just up the
class hierarchy).

I also tried to leave the types we were cast<>ing to somewhere nearby,
in cases where it wasn't locally obvious what we were dealing with
before.

llvm-svn: 326416
2018-03-01 05:43:23 +00:00
Gor Nishanov 0f33300609 [coroutines] Support coroutine-handle returning await-suspend (i.e symmetric control transfer)
Summary:
If await_suspend returns a coroutine_handle, as in the example below:
```
  coroutine_handle<> await_suspend(coroutine_handle<> h) {
    coro.promise().waiter = h;
    return coro;
  }
```
suspensionExpression processing will resume the coroutine pointed at by that handle.
Related LLVM change rL311751 makes resume calls of this kind `musttail` at any optimization level.

This enables unlimited symmetric control transfer from coroutine to coroutine without blowing up the stack.

Reviewers: GorNishanov

Reviewed By: GorNishanov

Subscribers: rsmith, EricWF, cfe-commits

Differential Revision: https://reviews.llvm.org/D37131

llvm-svn: 311762
2017-08-25 04:46:54 +00:00
Eric Fiselier e9a5e7e407 [coroutines] Evaluate the operand of void `co_return` expressions.
Summary:
Previously Clang incorrectly ignored the expression of a void `co_return`. This patch addresses that bug.

I'm not quite sure if I got the code-gen right, but this patch is at least a start.

Reviewers: rsmith, GorNishanov

Reviewed By: rsmith, GorNishanov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D36070

llvm-svn: 309545
2017-07-31 07:48:13 +00:00
Eric Fiselier 461b141a9c [coroutines] Remove pass-through operator co_await; Replace it with the input expression
Reviewers: GorNishanov, rsmith

Reviewed By: GorNishanov

Differential Revision: https://reviews.llvm.org/D34216

llvm-svn: 305498
2017-06-15 20:00:54 +00:00
Eric Fiselier cddaf8728f [coroutines] Allow co_await and co_yield expressions that return an lvalue to compile
Summary:
The title says it all.


Reviewers: GorNishanov, rsmith

Reviewed By: GorNishanov

Subscribers: rjmccall, cfe-commits

Differential Revision: https://reviews.llvm.org/D34194

llvm-svn: 305496
2017-06-15 19:43:36 +00:00
Galina Kistanova 0872d6c275 Added LLVM_FALLTHROUGH to address warning: this statement may fall through. NFC.
llvm-svn: 304649
2017-06-03 06:30:46 +00:00
Gor Nishanov 75a8ea5e88 CGCoroutine.cpp: (NFC) clang-format misplaced brace
llvm-svn: 304176
2017-05-29 21:15:31 +00:00
Gor Nishanov 33d5fd24a0 [coroutines] Add support for coroutines with non-scalar parameters
Summary:
Simple types like int are handled by LLVM Coroutines just fine.
But for non-scalar parameters we need to create copies of those parameters in the coroutine frame and make all uses of those parameters to refer to parameter copies.

Reviewers: rsmith, EricWF, GorNishanov

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D33507

llvm-svn: 303803
2017-05-24 20:09:14 +00:00
Gor Nishanov 4c2f68fd7c [coroutines] Implement correct GRO lifetime
Summary:
Sema creates a declaration for gro variable as:

auto $gro = $promise.get_return_object();

However, gro variable has to outlive coroutine frame and coroutine promise, but,
it can only be initialized after the coroutine promise was created, thus, we
split its emission in two parts: EmitGroAlloca emits an alloca and sets up
the cleanups. Later when the coroutine promise is available we initialize
the gro and set the flag that the cleanup is now active.

Duplicate of: https://reviews.llvm.org/D31670 (which arc patch refuses to apply for some reason)

Reviewers: GorNishanov, rsmith

Reviewed By: GorNishanov

Subscribers: EricWF, cfe-commits

Differential Revision: https://reviews.llvm.org/D33477

llvm-svn: 303716
2017-05-24 02:38:26 +00:00
Gor Nishanov db615dd677 [coroutines] Fix leak in CGCoroutine.cpp
FinalBB need to be emitted even when unused to make sure it is deleted

llvm-svn: 303714
2017-05-24 01:54:37 +00:00
Gor Nishanov e4f15a2bf0 [coroutines] Skip over passthrough operator co_await
https://reviews.llvm.org/D31627

llvm-svn: 303605
2017-05-23 05:25:31 +00:00
Gor Nishanov 5efc61866d [coroutines] Add emission of initial and final suspends
https://reviews.llvm.org/D31608

llvm-svn: 303603
2017-05-23 05:04:01 +00:00
Gor Nishanov 6c4530c6b5 [coroutines] Add support for deallocation elision
Wrap deallocation code with:

  if (auto *mem = coro.free()) Deallocate

When backend decides to elide allocations it will replace coro.free with nullptr to suppress deallocation code.

llvm-svn: 303599
2017-05-23 04:21:27 +00:00
Gor Nishanov 68fe6ee768 [coroutines] Replace all coro.frame builtins with an SSA value of coro.begin
SemaCoroutine forms expressions referring to the coroutine frame of the enclosing coroutine using coro.frame builtin.
During codegen, we emit llvm.coro.begin intrinsic that returns the address of the coroutine frame.
When coro.frame is emitted, we replace it with SSA value of coro.begin.

llvm-svn: 303598
2017-05-23 03:46:59 +00:00
Gor Nishanov aa6e9a99b4 [coroutines] Add support for allocation elision
Summary:
We wrap allocation code so that backend can elide it if necessary.
llvm.coro.alloc intrinsic returns true, when allocation is needed and false otherwise.

```
  %NeedAlloc = call i1 @llvm.coro.alloc(token %2)
  br i1 %NeedAlloc, label %AllocBB, label %InitBB

AllocBB:
  %5 = call i64 @llvm.coro.size.i64()
  %call = call i8* @_Znwm(i64 %5) ; operator new
  br label %InitBB

InitBB:
  %Phi = phi i8* [ null, %0 ], [ %call, %4 ]
  call i8* @llvm.coro.begin(token %2, i8* %Phi)
```

Reviewers: majnemer, EricWF

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31584

llvm-svn: 303596
2017-05-23 01:13:17 +00:00
Gor Nishanov 5b050e4a18 [coroutines] Wrap the body of the coroutine in try-catch
Summary:
If unhandled_exception member function is present in the coroutine promise,
wrap the body of the coroutine in:

```
try {
  body
} catch(...) { promise.unhandled_exception(); }
```

Reviewers: EricWF, rnk, rsmith

Reviewed By: rsmith

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31692

llvm-svn: 303583
2017-05-22 22:33:17 +00:00
Gor Nishanov 6a470689eb [coroutines] Build GRO declaration and return GRO statement
Summary:
1. build declaration of the gro local variable that keeps the result of get_return_object.
2. build return statement returning the gro variable
3. emit them during CodeGen
4. sema and CodeGen tests updated

Reviewers: EricWF, rsmith

Reviewed By: rsmith

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31646

llvm-svn: 303573
2017-05-22 20:22:23 +00:00
Gor Nishanov 818a776102 [coroutines] Add coro.end handling
Summary:
For WinEH, We add a funclet bundle to a coro.end call, so that CoroSplit in LLVM can replace it with cleanup ret and cut the rest out.
For landing pad, we add a branch to resume block if coro.end returns true.

LLVM Part: https://reviews.llvm.org/D25445

Reviewers: majnemer

Reviewed By: majnemer

Subscribers: EricWF, cfe-commits, rsmith, mehdi_amini

Differential Revision: https://reviews.llvm.org/D25444

llvm-svn: 299510
2017-04-05 04:55:03 +00:00
Gor Nishanov 63b6df4f05 [coroutines] Add cleanup for compiler injected objects/allocations in coroutine body
Summary:
* Use pushCleanup to emit freeing coroutine memory on normal and EH exits.
* Surround emitted code with CodeGenFunction::RunCleanupsScope.

Reviewers: rsmith, rnk, EricWF

Reviewed By: rnk

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D31460

llvm-svn: 299281
2017-04-01 00:22:47 +00:00
Gor Nishanov 3aa9eb38a0 [coroutines] Handle get_return_object_on_allocation_failure
Summary:
If promise_type has get_return_object_on_allocation_failure defined,
check if an allocation function returns nullptr, and if so,
return the result of get_return_object_on_allocation_failure().

Reviewers: rsmith, EricWF

Reviewed By: EricWF

Subscribers: mehdi_amini, cfe-commits

Differential Revision: https://reviews.llvm.org/D31399

llvm-svn: 298891
2017-03-27 23:36:59 +00:00
Gor Nishanov 5eb585836a [coroutines] Add codegen for await and yield expressions
Details:

Emit suspend expression which roughly looks like:

auto && x = CommonExpr();
if (!x.await_ready()) {
   llvm_coro_save();
   x.await_suspend(...);     (*)
   llvm_coro_suspend(); (**)
}
x.await_resume();
where the result of the entire expression is the result of x.await_resume()

(*) If x.await_suspend return type is bool, it allows to veto a suspend:
if (x.await_suspend(...))
   llvm_coro_suspend();
(**) llvm_coro_suspend() encodes three possible continuations as a switch instruction:

%where-to = call i8 @llvm.coro.suspend(...)
switch i8 %where-to, label %coro.ret [ ; jump to epilogue to suspend
  i8 0, label %yield.ready   ; go here when resumed
  i8 1, label %yield.cleanup ; go here when destroyed
]

llvm-svn: 298784
2017-03-26 02:18:05 +00:00
Gor Nishanov 90be1213d2 [coroutines] Add co_return statement emission
Summary:
Added co_return statement emission.

Tweaked coro-alloc.cpp test to use co_return to trigger coroutine processing instead of co_await, since this change starts emitting the body of the coroutine and await expression handling has not been upstreamed yet.

Reviewers: rsmith, majnemer, EricWF, aaron.ballman

Reviewed By: rsmith

Subscribers: majnemer, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D29979

llvm-svn: 297076
2017-03-06 21:12:54 +00:00
Gor Nishanov 8df64e940d [coroutines] Add allocation and deallocation substatements.
Summary:
SemaCoroutine: Add allocation / deallocation substatements.
CGCoroutine/Test: Emit allocation and deallocation + test.

Reviewers: rsmith

Subscribers: ABataev, EricWF, llvm-commits, mehdi_amini

Differential Revision: https://reviews.llvm.org/D25879

llvm-svn: 285306
2016-10-27 16:28:31 +00:00
Gor Nishanov 97e3b6d895 [coroutines] Adding builtins for coroutine intrinsics and backendutil support.
Summary:
With this commit simple coroutines can be created in plain C using coroutine builtins.

Reviewers: rnk, EricWF, rsmith

Subscribers: modocache, mgorny, mehdi_amini, beanz, cfe-commits

Differential Revision: https://reviews.llvm.org/D24373

llvm-svn: 283155
2016-10-03 22:44:48 +00:00