2018-12-01 03:38:22 +08:00
/ * *
2022-10-18 23:19:24 +08:00
* Copyright ( c ) Meta Platforms , Inc . and affiliates .
2018-12-01 03:38:22 +08:00
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree .
*
* @ flow
* /
import type { Writable } from 'stream' ;
2022-04-12 00:13:44 +08:00
import { TextEncoder } from 'util' ;
2022-10-23 13:06:58 +08:00
import { AsyncLocalStorage } from 'async_hooks' ;
2018-12-01 03:38:22 +08:00
2022-10-04 23:04:47 +08:00
interface MightBeFlushable {
flush ? : ( ) => void ;
}
2018-12-01 03:38:22 +08:00
export type Destination = Writable & MightBeFlushable ;
2021-03-16 01:36:23 +08:00
export type PrecomputedChunk = Uint8Array ;
2022-10-23 00:56:40 +08:00
export opaque type Chunk = string ;
2021-03-16 01:36:23 +08:00
2018-12-01 03:38:22 +08:00
export function scheduleWork ( callback : ( ) => void ) {
setImmediate ( callback ) ;
}
export function flushBuffered ( destination : Destination ) {
// If we don't have any more data to send right now.
// Flush whatever is in the buffer to the wire.
if ( typeof destination . flush === 'function' ) {
2021-06-05 03:17:57 +08:00
// By convention the Zlib streams provide a flush function for this purpose.
// For Express, compression middleware adds this method.
destination . flush ( ) ;
2018-12-01 03:38:22 +08:00
}
}
2022-10-23 13:06:58 +08:00
export const supportsRequestStorage = true ;
2023-01-31 21:25:05 +08:00
export const requestStorage : AsyncLocalStorage < Map < Function , mixed >> =
new AsyncLocalStorage ( ) ;
2022-10-23 13:06:58 +08:00
2022-04-12 00:13:44 +08:00
const VIEW _SIZE = 2048 ;
let currentView = null ;
let writtenBytes = 0 ;
let destinationHasCapacity = true ;
2018-12-01 03:38:22 +08:00
export function beginWriting ( destination : Destination ) {
2022-04-12 00:13:44 +08:00
currentView = new Uint8Array ( VIEW _SIZE ) ;
writtenBytes = 0 ;
destinationHasCapacity = true ;
}
function writeStringChunk ( destination : Destination , stringChunk : string ) {
if ( stringChunk . length === 0 ) {
return ;
}
// maximum possible view needed to encode entire string
if ( stringChunk . length * 3 > VIEW _SIZE ) {
if ( writtenBytes > 0 ) {
writeToDestination (
destination ,
( ( currentView : any ) : Uint8Array ) . subarray ( 0 , writtenBytes ) ,
) ;
currentView = new Uint8Array ( VIEW _SIZE ) ;
writtenBytes = 0 ;
}
writeToDestination ( destination , textEncoder . encode ( stringChunk ) ) ;
return ;
}
let target : Uint8Array = ( currentView : any ) ;
if ( writtenBytes > 0 ) {
target = ( ( currentView : any ) : Uint8Array ) . subarray ( writtenBytes ) ;
}
const { read , written } = textEncoder . encodeInto ( stringChunk , target ) ;
writtenBytes += written ;
if ( read < stringChunk . length ) {
writeToDestination ( destination , ( currentView : any ) ) ;
currentView = new Uint8Array ( VIEW _SIZE ) ;
2023-01-31 21:25:05 +08:00
writtenBytes = textEncoder . encodeInto (
stringChunk . slice ( read ) ,
// $FlowFixMe[incompatible-call] found when upgrading Flow
currentView ,
) . written ;
2022-04-12 00:13:44 +08:00
}
if ( writtenBytes === VIEW _SIZE ) {
writeToDestination ( destination , ( currentView : any ) ) ;
currentView = new Uint8Array ( VIEW _SIZE ) ;
writtenBytes = 0 ;
}
}
function writeViewChunk ( destination : Destination , chunk : PrecomputedChunk ) {
if ( chunk . byteLength === 0 ) {
return ;
}
if ( chunk . byteLength > VIEW _SIZE ) {
Fizz Browser: fix precomputed chunk being cleared on Node 18 (#25645)
## Edit
Went for another approach after talking with @gnoff. The approach is
now:
- add a dev-only error when a precomputed chunk is too big to be written
- suggest to copy it before passing it to `writeChunk`
This PR also includes porting the React Float tests to use the browser
build of Fizz so that we can test it out on that environment (which is
the one used by next).
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn debug-test --watch TestName`, open
`chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
Someone reported [a bug](https://github.com/vercel/next.js/issues/42466)
in Next.js that pointed to an issue with Node 18 in the streaming
renderer when using importing a CSS module where it only returned a
malformed bootstraping script only after loading the page once.
After investigating a bit, here's what I found:
- when using a CSS module in Next, we go into this code path, which
writes the aforementioned bootstrapping script
https://github.com/facebook/react/blob/5f7ef8c4cbe824ef126a947b7ae0e1c07b143357/packages/react-dom-bindings/src/server/ReactDOMServerFormatConfig.js#L2443-L2447
- the reason for the malformed script is that
`completeBoundaryWithStylesScript1FullBoth` is emptied after the call to
`writeChunk`
- it gets emptied in `writeChunk` because we stream the chunk directly
without copying it in this codepath
https://github.com/facebook/react/blob/a438590144d2ad40865b58e0c0e69595fc1aa377/packages/react-server/src/ReactServerStreamConfigBrowser.js#L63
- the reason why it only happens from Node 18 is because the Webstreams
APIs are available natively from that version and in their
implementation, [`enqueue` transfers the array buffer
ownership](https://github.com/nodejs/node/blob/9454ba6138d11e8a4d18b073de25781cad4bd2c8/lib/internal/webstreams/readablestream.js#L2641),
thus making it unavailable/empty for subsequent calls. In older Node
versions, we don't encounter the bug because we are using a polyfill in
Next.js, [which does not implement properly the array buffer transfer
behaviour](https://cs.github.com/MattiasBuelens/web-streams-polyfill/blob/d354a7457ca8a24030dbd0a135ee40baed7c774d/src/lib/abstract-ops/ecmascript.ts#L16).
I think the proper fix for this is to clone the array buffer before
enqueuing it. (we do this in the other code paths in the function later
on, see ```((currentView: any): Uint8Array).set(bytesToWrite,
writtenBytes);```
## How did you test this change?
Manually tested by applying the change in the compiled Next.js version.
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>
2022-11-22 08:33:41 +08:00
if ( _ _DEV _ _ ) {
if ( precomputedChunkSet && precomputedChunkSet . has ( chunk ) ) {
console . error (
'A large precomputed chunk was passed to writeChunk without being copied.' +
' Large chunks get enqueued directly and are not copied. This is incompatible with precomputed chunks because you cannot enqueue the same precomputed chunk twice.' +
' Use "cloneChunk" to make a copy of this large precomputed chunk before writing it. This is a bug in React.' ,
) ;
}
}
2022-04-12 00:13:44 +08:00
// this chunk may overflow a single view which implies it was not
// one that is cached by the streaming renderer. We will enqueu
// it directly and expect it is not re-used
if ( writtenBytes > 0 ) {
writeToDestination (
destination ,
( ( currentView : any ) : Uint8Array ) . subarray ( 0 , writtenBytes ) ,
) ;
currentView = new Uint8Array ( VIEW _SIZE ) ;
writtenBytes = 0 ;
}
writeToDestination ( destination , chunk ) ;
return ;
}
let bytesToWrite = chunk ;
const allowableBytes = ( ( currentView : any ) : Uint8Array ) . length - writtenBytes ;
if ( allowableBytes < bytesToWrite . byteLength ) {
// this chunk would overflow the current view. We enqueue a full view
// and start a new view with the remaining chunk
if ( allowableBytes === 0 ) {
// the current view is already full, send it
writeToDestination ( destination , ( currentView : any ) ) ;
} else {
// fill up the current view and apply the remaining chunk bytes
// to a new view.
( ( currentView : any ) : Uint8Array ) . set (
bytesToWrite . subarray ( 0 , allowableBytes ) ,
writtenBytes ,
) ;
writtenBytes += allowableBytes ;
writeToDestination ( destination , ( currentView : any ) ) ;
bytesToWrite = bytesToWrite . subarray ( allowableBytes ) ;
}
currentView = new Uint8Array ( VIEW _SIZE ) ;
writtenBytes = 0 ;
}
( ( currentView : any ) : Uint8Array ) . set ( bytesToWrite , writtenBytes ) ;
writtenBytes += bytesToWrite . byteLength ;
if ( writtenBytes === VIEW _SIZE ) {
writeToDestination ( destination , ( currentView : any ) ) ;
currentView = new Uint8Array ( VIEW _SIZE ) ;
writtenBytes = 0 ;
2019-11-07 01:10:26 +08:00
}
2018-12-01 03:38:22 +08:00
}
2019-11-07 01:48:34 +08:00
export function writeChunk (
destination : Destination ,
2022-04-12 00:13:44 +08:00
chunk : PrecomputedChunk | Chunk ,
2022-02-24 00:35:21 +08:00
) : void {
2022-04-12 00:13:44 +08:00
if ( typeof chunk === 'string' ) {
writeStringChunk ( destination , chunk ) ;
} else {
writeViewChunk ( destination , ( ( chunk : any ) : PrecomputedChunk ) ) ;
}
}
function writeToDestination ( destination : Destination , view : Uint8Array ) {
const currentHasCapacity = destination . write ( view ) ;
destinationHasCapacity = destinationHasCapacity && currentHasCapacity ;
2022-02-24 00:35:21 +08:00
}
export function writeChunkAndReturn (
destination : Destination ,
2022-04-12 00:13:44 +08:00
chunk : PrecomputedChunk | Chunk ,
2019-11-07 01:48:34 +08:00
) : boolean {
2022-04-12 00:13:44 +08:00
writeChunk ( destination , chunk ) ;
return destinationHasCapacity ;
2018-12-01 03:38:22 +08:00
}
export function completeWriting ( destination : Destination ) {
2022-04-12 00:13:44 +08:00
if ( currentView && writtenBytes > 0 ) {
destination . write ( currentView . subarray ( 0 , writtenBytes ) ) ;
2019-11-07 01:10:26 +08:00
}
2022-04-12 00:13:44 +08:00
currentView = null ;
writtenBytes = 0 ;
destinationHasCapacity = true ;
2018-12-01 03:38:22 +08:00
}
export function close ( destination : Destination ) {
destination . end ( ) ;
}
2022-04-12 00:13:44 +08:00
const textEncoder = new TextEncoder ( ) ;
2021-03-16 01:36:23 +08:00
export function stringToChunk ( content : string ) : Chunk {
return content ;
}
Fizz Browser: fix precomputed chunk being cleared on Node 18 (#25645)
## Edit
Went for another approach after talking with @gnoff. The approach is
now:
- add a dev-only error when a precomputed chunk is too big to be written
- suggest to copy it before passing it to `writeChunk`
This PR also includes porting the React Float tests to use the browser
build of Fizz so that we can test it out on that environment (which is
the one used by next).
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn debug-test --watch TestName`, open
`chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
Someone reported [a bug](https://github.com/vercel/next.js/issues/42466)
in Next.js that pointed to an issue with Node 18 in the streaming
renderer when using importing a CSS module where it only returned a
malformed bootstraping script only after loading the page once.
After investigating a bit, here's what I found:
- when using a CSS module in Next, we go into this code path, which
writes the aforementioned bootstrapping script
https://github.com/facebook/react/blob/5f7ef8c4cbe824ef126a947b7ae0e1c07b143357/packages/react-dom-bindings/src/server/ReactDOMServerFormatConfig.js#L2443-L2447
- the reason for the malformed script is that
`completeBoundaryWithStylesScript1FullBoth` is emptied after the call to
`writeChunk`
- it gets emptied in `writeChunk` because we stream the chunk directly
without copying it in this codepath
https://github.com/facebook/react/blob/a438590144d2ad40865b58e0c0e69595fc1aa377/packages/react-server/src/ReactServerStreamConfigBrowser.js#L63
- the reason why it only happens from Node 18 is because the Webstreams
APIs are available natively from that version and in their
implementation, [`enqueue` transfers the array buffer
ownership](https://github.com/nodejs/node/blob/9454ba6138d11e8a4d18b073de25781cad4bd2c8/lib/internal/webstreams/readablestream.js#L2641),
thus making it unavailable/empty for subsequent calls. In older Node
versions, we don't encounter the bug because we are using a polyfill in
Next.js, [which does not implement properly the array buffer transfer
behaviour](https://cs.github.com/MattiasBuelens/web-streams-polyfill/blob/d354a7457ca8a24030dbd0a135ee40baed7c774d/src/lib/abstract-ops/ecmascript.ts#L16).
I think the proper fix for this is to clone the array buffer before
enqueuing it. (we do this in the other code paths in the function later
on, see ```((currentView: any): Uint8Array).set(bytesToWrite,
writtenBytes);```
## How did you test this change?
Manually tested by applying the change in the compiled Next.js version.
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>
2022-11-22 08:33:41 +08:00
const precomputedChunkSet = _ _DEV _ _ ? new Set ( ) : null ;
2021-03-16 01:36:23 +08:00
export function stringToPrecomputedChunk ( content : string ) : PrecomputedChunk {
Fizz Browser: fix precomputed chunk being cleared on Node 18 (#25645)
## Edit
Went for another approach after talking with @gnoff. The approach is
now:
- add a dev-only error when a precomputed chunk is too big to be written
- suggest to copy it before passing it to `writeChunk`
This PR also includes porting the React Float tests to use the browser
build of Fizz so that we can test it out on that environment (which is
the one used by next).
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes. Please
provide enough information so that others can review your pull request.
The three fields below are mandatory.
Before submitting a pull request, please make sure the following is
done:
1. Fork [the repository](https://github.com/facebook/react) and create
your branch from `main`.
2. Run `yarn` in the repository root.
3. If you've fixed a bug or added code that should be tested, add tests!
4. Ensure the test suite passes (`yarn test`). Tip: `yarn test --watch
TestName` is helpful in development.
5. Run `yarn test --prod` to test in the production environment. It
supports the same options as `yarn test`.
6. If you need a debugger, run `yarn debug-test --watch TestName`, open
`chrome://inspect`, and press "Inspect".
7. Format your code with
[prettier](https://github.com/prettier/prettier) (`yarn prettier`).
8. Make sure your code lints (`yarn lint`). Tip: `yarn linc` to only
check changed files.
9. Run the [Flow](https://flowtype.org/) type checks (`yarn flow`).
10. If you haven't already, complete the CLA.
Learn more about contributing:
https://reactjs.org/docs/how-to-contribute.html
-->
## Summary
Someone reported [a bug](https://github.com/vercel/next.js/issues/42466)
in Next.js that pointed to an issue with Node 18 in the streaming
renderer when using importing a CSS module where it only returned a
malformed bootstraping script only after loading the page once.
After investigating a bit, here's what I found:
- when using a CSS module in Next, we go into this code path, which
writes the aforementioned bootstrapping script
https://github.com/facebook/react/blob/5f7ef8c4cbe824ef126a947b7ae0e1c07b143357/packages/react-dom-bindings/src/server/ReactDOMServerFormatConfig.js#L2443-L2447
- the reason for the malformed script is that
`completeBoundaryWithStylesScript1FullBoth` is emptied after the call to
`writeChunk`
- it gets emptied in `writeChunk` because we stream the chunk directly
without copying it in this codepath
https://github.com/facebook/react/blob/a438590144d2ad40865b58e0c0e69595fc1aa377/packages/react-server/src/ReactServerStreamConfigBrowser.js#L63
- the reason why it only happens from Node 18 is because the Webstreams
APIs are available natively from that version and in their
implementation, [`enqueue` transfers the array buffer
ownership](https://github.com/nodejs/node/blob/9454ba6138d11e8a4d18b073de25781cad4bd2c8/lib/internal/webstreams/readablestream.js#L2641),
thus making it unavailable/empty for subsequent calls. In older Node
versions, we don't encounter the bug because we are using a polyfill in
Next.js, [which does not implement properly the array buffer transfer
behaviour](https://cs.github.com/MattiasBuelens/web-streams-polyfill/blob/d354a7457ca8a24030dbd0a135ee40baed7c774d/src/lib/abstract-ops/ecmascript.ts#L16).
I think the proper fix for this is to clone the array buffer before
enqueuing it. (we do this in the other code paths in the function later
on, see ```((currentView: any): Uint8Array).set(bytesToWrite,
writtenBytes);```
## How did you test this change?
Manually tested by applying the change in the compiled Next.js version.
<!--
Demonstrate the code is solid. Example: The exact commands you ran and
their output, screenshots / videos if the pull request changes the user
interface.
How exactly did you verify that your PR solves the issue you wanted to
solve?
If you leave this empty, your PR will very likely be closed.
-->
Co-authored-by: Sebastian Markbage <sebastian@calyptus.eu>
2022-11-22 08:33:41 +08:00
const precomputedChunk = textEncoder . encode ( content ) ;
if ( _ _DEV _ _ ) {
if ( precomputedChunkSet ) {
precomputedChunkSet . add ( precomputedChunk ) ;
}
}
return precomputedChunk ;
}
export function clonePrecomputedChunk (
precomputedChunk : PrecomputedChunk ,
) : PrecomputedChunk {
return precomputedChunk . length > VIEW _SIZE
? precomputedChunk . slice ( )
: precomputedChunk ;
2018-12-01 03:38:22 +08:00
}
2021-03-13 07:21:02 +08:00
export function closeWithError ( destination : Destination , error : mixed ) : void {
// $FlowFixMe: This is an Error object or the destination accepts other types.
destination . destroy ( error ) ;
}