tl;dr Correct implementation of Corouintes requires having lifetime intrinsics available.
Coroutine functions are functions that can be suspended and resumed latter. To do so, data that need to stay alive after suspension must be put on the heap (i.e. the coroutine frame).
The optimizer is responsible for analyzing each AllocaInst and figure out whether it should be put on the stack or the frame.
In most cases, for data that we are unable to accurately analyze lifetime, we can just conservatively put them on the heap.
Unfortunately, there exists a few cases where certain data MUST be put on the stack, not on the heap. Without lifetime intrinsics, we are unable to correctly analyze those data's lifetime.
To dig into more details, there exists cases where at certain code points, the current coroutine frame may have already been destroyed. Hence no frame access would be allowed beyond that point.
The following is a common code pattern called "Symmetric Transfer" in coroutine:
```
auto tmp = await_suspend();
__builtin_coro_resume(tmp.address());
return;
```
In the above code example, `await_suspend()` returns a new coroutine handle, which we will obtain the address and then resume that coroutine. This essentially "transfered" from the current coroutine to a different coroutine.
During the call to `await_suspend()`, the current coroutine may be destroyed, which should be fine because we are not accessing any data afterwards.
However when LLVM is emitting IR for the above code, it needs to emit an AllocaInst for `tmp`. It will then call the `address` function on tmp. `address` function is a member function of coroutine, and there is no way for the LLVM optimizer to know that it does not capture the `tmp` pointer. So when the optimizer looks at it, it has to conservatively assume that `tmp` may escape and hence put it on the heap. Furthermore, in some cases `address` call would be inlined, which will generate a bunch of store/load instructions that move the `tmp` pointer around. Those stores will also make the compiler to think that `tmp` might escape.
To summarize, it's really difficult for the mid-end to figure out that the `tmp` data is short-lived.
I made some attempt in D98638, but it appears to be way too complex and is basically doing the same thing as inserting lifetime intrinsics in coroutines.
Also, for reference, we already force emitting lifetime intrinsics in O0 for AlwaysInliner: https://github.com/llvm/llvm-project/blob/main/llvm/lib/Passes/PassBuilder.cpp#L1893
Differential Revision: https://reviews.llvm.org/D99227
Summary:
This patch addresses https://bugs.llvm.org/show_bug.cgi?id=46256
The spec of coroutine requires that the expression co_await promise.final_suspend() shall not be potentially-throwing.
To check this, we recursively look at every call (including Call, MemberCall, OperatorCall and Constructor) in all code
generated by the final suspend, and ensure that the callees are declared with noexcept. We also look at any returned data
type that requires explicit destruction, and check their destructors for noexcept.
This patch does not check declarations with dependent types yet, which will be done in future patches.
Updated all tests to add noexcept to the required functions, and added a dedicated test for this patch.
This patch might start to cause existing codebase fail to compile because most people may not have been strict in tagging
all the related functions noexcept.
Reviewers: lewissbaker, modocache, junparser
Reviewed By: modocache
Subscribers: arphaman, junparser, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D82029
If we're going to assume references are dereferenceable, we should also
assume they're aligned: otherwise, we can't actually dereference them.
See also D80072.
Differential Revision: https://reviews.llvm.org/D80166
Summary:
Right now we annotate C++'s `operator new` with `noalias` attribute,
which very much is healthy for optimizations.
However as per [[ http://eel.is/c++draft/basic.stc.dynamic.allocation | `[basic.stc.dynamic.allocation]` ]],
there are more promises on global `operator new`, namely:
* non-`std::nothrow_t` `operator new` *never* returns `nullptr`
* If `std::align_val_t align` parameter is taken, the pointer will also be `align`-aligned
* ~~global `operator new`-returned pointer is `__STDCPP_DEFAULT_NEW_ALIGNMENT__`-aligned ~~ It's more caveated than that.
Supplying this information may not cause immediate landslide effects
on any specific benchmarks, but it for sure will be healthy for optimizer
in the sense that the IR will better reflect the guarantees provided in the source code.
The caveat is `-fno-assume-sane-operator-new`, which currently prevents emitting `noalias`
attribute, and is automatically passed by Sanitizers ([[ https://bugs.llvm.org/show_bug.cgi?id=16386 | PR16386 ]]) - should it also cover these attributes?
The problem is that the flag is back-end-specific, as seen in `test/Modules/explicit-build-flags.cpp`.
But while it is okay to add `noalias` metadata in backend, we really should be adding at least
the alignment metadata to the AST, since that allows us to perform sema checks on it.
Reviewers: erichkeane, rjmccall, jdoerfert, eugenis, rsmith
Reviewed By: rsmith
Subscribers: xbolva00, jrtc27, atanasyan, nlopes, cfe-commits
Tags: #llvm, #clang
Differential Revision: https://reviews.llvm.org/D73380
Summary:
https://reviews.llvm.org/rL325291 implemented Coroutines TS N4723
section [dcl.fct.def.coroutine]/7, but it performed lookup of allocator
functions within both the global and class scope, whereas the specified
behavior is to perform lookup for custom allocators within just the
class scope.
To fix, add parameters to the `Sema::FindAllocationFunctions` function
such that it can be used to lookup allocators in global scope,
class scope, or both (instead of just being able to look up in just global
scope or in both global and class scope). Then, use those parameters
from within the coroutine Sema.
This incorrect behavior had the unfortunate side-effect of causing the
bug https://bugs.llvm.org/show_bug.cgi?id=36578 (or at least the reports
of that bug in C++ programs). That bug would occur for any C++ user with
a coroutine frame that took a single pointer argument, since it would
then find the global placement form `operator new`, described in the
C++ standard 18.6.1.3.1. This patch prevents Clang from generating code
that triggers the LLVM assert described in that bug report.
Test Plan: `check-clang`
Reviewers: GorNishanov, eric_niebler, lewissbaker
Reviewed By: GorNishanov
Subscribers: EricWF, cfe-commits
Differential Revision: https://reviews.llvm.org/D44552
llvm-svn: 328949
Summary:
Depends on https://reviews.llvm.org/D42605.
An implementation of the behavior described in `[dcl.fct.def.coroutine]/7`:
when a promise type overloads `operator new` using a "placement new"
that takes the same argument types as the coroutine function, that
overload is used when allocating the coroutine frame.
Simply passing references to the coroutine function parameters directly
to `operator new` results in invariant violations in LLVM's coroutine
splitting pass, so this implementation modifies Clang codegen to
produce allocator-specific alloc/store/loads for each parameter being
forwarded to the allocator.
Test Plan: `check-clang`
Reviewers: rsmith, GorNishanov, eric_niebler
Reviewed By: GorNishanov
Subscribers: lewissbaker, EricWF, cfe-commits
Differential Revision: https://reviews.llvm.org/D42606
llvm-svn: 325291
Summary:
Fix NRVO for Gro variable.
Previously, we only marked the GRO declaration as an NRVO variable
when its QualType and the function return's QualType matched exactly
(using operator==). However, this was incorrect for two reasons:
1. We were marking non-class types, such as ints, as being NRVO variables.
2. We failed to handle cases where the canonical types were the same, but the actual `QualType` objects were different. For example, if one was represented by a typedef. (Example: https://godbolt.org/g/3UFgsL)
This patch fixes these bugs by marking the Gro variable as supporting NRVO only
when `BuildReturnStmt` marks the Gro variable as a coroutine candidate.
Reviewers: rsmith, GorNishanov, nicholas
Reviewed By: GorNishanov
Subscribers: majnemer, cfe-commits
Differential Revision: https://reviews.llvm.org/D42343
llvm-svn: 324037
Summary:
Fix NRVO for Gro variable.
Previously, we only marked the GRO declaration as an NRVO variable
when its QualType and the function return's QualType matched exactly
(using operator==). However, this was incorrect for two reasons:
1. We were marking non-class types, such as ints, as being NRVO variables.
2. We failed to handle cases where the canonical types were the same, but the actual `QualType` objects were different. For example, if one was represented by a typedef. (Example: https://godbolt.org/g/3UFgsL)
This patch fixes these bugs by marking the Gro variable as supporting NRVO only
when `BuildReturnStmt` marks the Gro variable as a coroutine candidate.
Reviewers: rsmith, GorNishanov, nicholas
Reviewed By: GorNishanov
Subscribers: majnemer, cfe-commits
Differential Revision: https://reviews.llvm.org/D42343
llvm-svn: 323712
Wrap deallocation code with:
if (auto *mem = coro.free()) Deallocate
When backend decides to elide allocations it will replace coro.free with nullptr to suppress deallocation code.
llvm-svn: 303599
SemaCoroutine forms expressions referring to the coroutine frame of the enclosing coroutine using coro.frame builtin.
During codegen, we emit llvm.coro.begin intrinsic that returns the address of the coroutine frame.
When coro.frame is emitted, we replace it with SSA value of coro.begin.
llvm-svn: 303598
Summary:
1. build declaration of the gro local variable that keeps the result of get_return_object.
2. build return statement returning the gro variable
3. emit them during CodeGen
4. sema and CodeGen tests updated
Reviewers: EricWF, rsmith
Reviewed By: rsmith
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D31646
llvm-svn: 303573
Summary:
This patch implements [dcl.fct.def.coroutine]p8:
> The unqualified-id get_return_object_on_allocation_failure is looked up in the scope of
> class P by class member access lookup (3.4.5). If a declaration is found, ..., and if a
> global allocation function is selected, the ::operator new(size_t, nothrow_t) form shall be used.
> [...]
> The allocation function used in this case must have a non-throwing noexcept-specification.
Reviewers: GorNishanov, rsmith, majnemer, aaron.ballman
Reviewed By: GorNishanov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D31562
llvm-svn: 300524
Summary:
Certain implicitly generated coroutine statements, such as the calls to 'return_value()' or `return_void()` or `get_return_object_on_allocation_failure()`, cannot be built until the promise type is no longer dependent. This means they are not built until after the coroutine body statement has been transformed.
This patch fixes an issue where these statements would never be built for coroutine templates.
It also fixes a small issue where diagnostics about `get_return_object_on_allocation_failure()` were incorrectly suppressed.
Reviewers: rsmith, majnemer, GorNishanov, aaron.ballman
Reviewed By: GorNishanov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D31487
llvm-svn: 300504
Summary:
This patch implements [dcl.fct.def.coroutine]p8:
> The unqualified-id get_return_object_on_allocation_failure is looked up in the scope of
> class P by class member access lookup (3.4.5). If a declaration is found, ..., and if a
> global allocation function is selected, the ::operator new(size_t, nothrow_t) form shall be used.
> [...]
> The allocation function used in this case must have a non-throwing noexcept-specification.
Reviewers: GorNishanov, rsmith, majnemer, aaron.ballman
Reviewed By: GorNishanov
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D31562
llvm-svn: 300420
Summary:
If promise_type has get_return_object_on_allocation_failure defined,
check if an allocation function returns nullptr, and if so,
return the result of get_return_object_on_allocation_failure().
Reviewers: rsmith, EricWF
Reviewed By: EricWF
Subscribers: mehdi_amini, cfe-commits
Differential Revision: https://reviews.llvm.org/D31399
llvm-svn: 298891
Summary:
This patch adds passing a coroutine_handle object to await_suspend calls.
It builds the coroutine_handle using coroutine_handle<PromiseType>::from_address(__builtin_coro_frame()).
(a revision of https://reviews.llvm.org/D26316 that for some reason refuses to apply via arc patch)
Reviewers: GorNishanov
Subscribers: mehdi_amini, cfe-commits, EricWF
Differential Revision: https://reviews.llvm.org/D30769
llvm-svn: 297356
Summary:
Added co_return statement emission.
Tweaked coro-alloc.cpp test to use co_return to trigger coroutine processing instead of co_await, since this change starts emitting the body of the coroutine and await expression handling has not been upstreamed yet.
Reviewers: rsmith, majnemer, EricWF, aaron.ballman
Reviewed By: rsmith
Subscribers: majnemer, llvm-commits, mehdi_amini
Differential Revision: https://reviews.llvm.org/D29979
llvm-svn: 297076
Xun Li