diff --git a/design/src/SUMMARY.md b/design/src/SUMMARY.md index 4d8d89822d..2266e66ed5 100644 --- a/design/src/SUMMARY.md +++ b/design/src/SUMMARY.md @@ -23,3 +23,4 @@ - [RFC-0008: Paginators](./rfcs/rfc0008_paginators.md) - [RFC-0009: Example Consolidation](./rfcs/rfc0009_example_consolidation.md) - [RFC-0010: Waiters](./rfcs/rfc0010_waiters.md) + - [RFC-0011: Publishing Alpha to Crates.io](./rfcs/rfc0011_crates_io_alpha_publishing.md) diff --git a/design/src/rfcs/overview.md b/design/src/rfcs/overview.md index 8f144a0bfc..dc2c4a8fe2 100644 --- a/design/src/rfcs/overview.md +++ b/design/src/rfcs/overview.md @@ -10,3 +10,4 @@ - [RFC-0008: Paginators](./rfc0008_paginators.md) - [RFC-0009: Example Consolidation](./rfc0009_example_consolidation.md) - [RFC-0010: Waiters](./rfc0010_waiters.md) +- [RFC-0011: Publishing Alpha to Crates.io](./rfc0011_crates_io_alpha_publishing.md) diff --git a/design/src/rfcs/rfc0011_crates_io_alpha_publishing.md b/design/src/rfcs/rfc0011_crates_io_alpha_publishing.md new file mode 100644 index 0000000000..967d19f1e3 --- /dev/null +++ b/design/src/rfcs/rfc0011_crates_io_alpha_publishing.md @@ -0,0 +1,121 @@ +RFC: Publishing the Alpha SDK to Crates.io +========================================== + +> Status: Implemented + +The AWS SDK for Rust and its supporting Smithy crates need to be published to [crates.io](https://crates.io/) +so that customers can include them in their projects and also publish crates of their own that depend on them. + +This doc proposes a short-term solution for publishing to crates.io. This approach is intended to be executed +manually by a developer using scripts and an SOP no more than once per week, and should require less than a +dev week to implement. + +Terminology +----------- + +- **AWS SDK Crate**: A crate that provides a client for calling a given AWS service, such as `aws-sdk-s3` for calling S3. +- **AWS Runtime Crate**: Any runtime crate that the AWS SDK generated code relies on, such as `aws-types`. +- **Smithy Runtime Crate**: Any runtime crate that the smithy-rs generated code relies on, such as `smithy-types`. + +Requirements +------------ + +### Versioning + +Cargo uses [semver](https://github.com/dtolnay/semver#requirements) for versioning, +with a `major.minor.patch-pre` format: +- `major`: Incompatible API changes +- `minor`: Added functionality in backwards compatible manner +- `patch`: Backwards compatible bug fixes +- `pre`: Pre-release version tag (omitted for normal releases) + +For now, AWS SDK crates (including `aws-config`) will maintain a consistent `major` and `minor` version number +across all services. The latest version of `aws-sdk-s3` will always have the same `major.minor` version as the +latest `aws-sdk-dynamodb`, for example. The `patch` version is allowed to be different between service crates, +but it is unlikely that we will make use of `patch` versions throughout alpha and dev preview. +Smithy runtime crates will have different version numbers from the AWS SDK crates, but will also maintain +a consistent `major.minor`. + +The `pre` version tag will be `alpha` during the Rust SDK alpha, and will be removed once the SDK is in +dev preview. + +During alpha, the `major` version will always be 0, and the `minor` will be bumped for all published +crates for every release. A later RFC may change the process during dev preview. + +### Yanking + +Mistakes will inevitably be made, and a mechanism is needed to yank packages while keeping the latest version +of the SDK successfully consumable from crates.io. To keep this simple, the entire published batch of crates +will be yanked if any crate in that batch needs to be yanked. For example, if 260 crates were published in a batch, +and it turns out there's a problem that requires yanking one of them, then all 260 will be yanked. Attempting to do +partial yanking will require a lot of effort and be difficult to get right. Yanking should be a last resort. + +Concrete Scenarios +------------------ + +The following changes will be bundled together as a `minor` version bump during weekly releases: + +- AWS model updates +- New features +- Bug fixes in runtime crates or codegen + +In exceptional circumstances, a `patch` version will be issued if the fix doesn't require API breaking changes: + +- CVE discovered in a runtime crate +- Buggy update to a runtime crate + +In the event of a CVE being discovered in an external dependency, if the external dependency is +internal to a crate, then a `patch` revision can be issued for that crate to correct it. Otherwise if the CVE +is in a dependency that is part of the public API, a `minor` revision will be issued with an expedited release. + +For a CVE in generated code, a `minor` revision will be issued with an expedited release. + +Proposal +-------- + +The short-term approach builds off our pre-crates.io weekly release process. That process was the following: + +1. Run script to update AWS models +2. Manually update AWS SDK version in `aws/sdk/gradle.properties` in smithy-rs +3. Tag smithy-rs +4. Wait for GitHub actions to generate AWS SDK using newly released smithy-rs +5. Check out aws-sdk-rust, delete existing SDK code, unzip generated SDK in place, and update readme +6. Tag aws-sdk-rust + +To keep things simple: +- The Smithy runtime crates will have the same smithy-rs version +- All AWS crates will have the same AWS SDK version +- `patch` revisions are exceptional and will be one-off manually published by a developer + +All runtime crate version numbers in smithy-rs will be locked at `0.0.0-smithy-rs-head`. This is a fake +version number that gets replaced when generating the SDK. + +The SDK generator script in smithy-rs will be updated to: +- Replace Smithy runtime crate versions with the smithy-rs version from `aws/sdk/gradle.properties` +- Replace AWS runtime crate versions with AWS SDK version from `aws/sdk/gradle.properties` +- Add correct version numbers to all path dependencies in all the final crates that end up in the build artifacts + +This will result in all the crates having the correct version and manifests when imported into aws-sdk-rust. +From there, a script needs to be written to determine crate dependency order, and publish crates (preferably +with throttling and retry) in the correct order. This script needs to be able to recover from an interruption +part way through publishing all the crates, and it also needs to output a list of all crate versions published +together. This crate list will be commented on the release issue so that yanking the batch can be done if +necessary. + +The new release process would be: + +1. Run script to update AWS models +2. Manually update _both_ the AWS SDK version _and_ the smithy-rs version in `aws/sdk/gradle.properties` in smithy-rs +3. Tag smithy-rs +4. Wait for automation to sync changes to `aws-sdk-rust/next` +5. Cut a PR to merge `aws-sdk-rust/next` into `aws-sdk-rust/main` +6. Tag aws-sdk-rust +7. Run publish script + +### Short-term Changes Checklist + +- [x] Prepare runtime crate manifests for publication to crates.io (https://github.com/awslabs/smithy-rs/pull/755) +- [x] Update SDK generator to set correct crate versions (https://github.com/awslabs/smithy-rs/pull/755) +- [x] Write bulk publish script +- [x] Write bulk yank script +- [x] Write automation to sync smithy-rs to aws-sdk-rust