## Motivation and Context
Restores `SmokeTestsDecoratorTest` that was temporarily removed in
https://github.com/smithy-lang/smithy-rs/pull/3808.
## Description
The said test was temporarily removed because `SmokeTestsDecorator` is
included in the [predefined
decorators](b38ccb969e/aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/AwsCodegenDecorator.kt (L35))
and pulls-in the `aws-config` crate for code generation. The issue was
conflicts between the runtime crates used by `aws-config` (located in
the `aws/sdk/build` directory) and those brought-in by that predefined
decorators used by `awsSdkIntegrationTest` (located in the
`rust-runtime` and `aws/rust-runtime` directories). This PR addresses
these conflicts and restores the test functionality.
Given the challenges, the code changes are centered around the following
ideas:
- Focus solely on testing the core class, `SmokeTestsInstantiator`. By
doing this, we avoid running `SmokeTestsDecorator`, which would
otherwise pull in the `aws-config` crate.
- Initializing a config builder in smoke tests needs to be parameterized
depending on the environment; in production we use
`aws_config::load_defaults` and in testing we avoid including
`aws-config` by using a default-constructed config builder, which is
sufficient for compile-only tests.
- The generated smoke tests in `SmokeTestsDecoratorTest` require minimal
runtime crates for compilation. We need the `CodegenVisitor` but with a
different set of codegen decorators. Prior to this PR,
`awsSdkIntegrationTest` used
[RustClientCodegenPlugin](b38ccb969e/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/RustClientCodegenPlugin.kt (L46))
that in turn loaded [predefined
decorators](b38ccb969e/aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/AwsCodegenDecorator.kt (L35))
on the classpath, which included `SmokeTestsDecorator`. In this PR, we
work around this by defining a minimal set of codegen decorators and
making them pluggable through `awsSdkIntegrationTest`.
## Testing
Tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR addresses an issue where, if an operation's input includes an
event streaming member, the builder for the operation input or output
may raise a `ConstraintViolation` when `build_enforcing_all_constraints`
is called and the event streaming member field is not set. This occurs
because the member shape is required.
The standard error message that is shown when `ValidationException` is
not attached to an operation is also displayed in this case:
*Operation test#TestOperation takes in input that is constrained
(https://awslabs.github.io/smithy/2.0/spec/constraint-traits.html), and
as such can fail with a validation exception. You must model this
behavior in the operation shape in your model file.*
```smithy
use smithy.framework#ValidationException
operation TestOperation {
...
errors: [..., ValidationException] // <-- Add this.
}
```
Closes: [3813](https://github.com/smithy-lang/smithy-rs/issues/3813)
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
The Server SDK requires the model to include
`aws.smithy.framework#ValidationException` in each operation that can
access a constrained member shape. This becomes problematic when
generating the server SDK for a model not owned by the team creating the
SDK, as they cannot easily modify the model.
This PR introduces a codegen flag,
`addValidationExceptionToConstrainedOperations`. When set in
`smithy-build-template.json`, this flag will automatically add
`ValidationException` to operations that require it but do not already
list it among their errors.
Closes Issue:
[3802](https://github.com/smithy-lang/smithy-rs/issues/3802)
Sample `smithy-build-template.json`
```
"plugins": {
"rust-server-codegen": {
"service": "ServiceToGenerateSDKFor",
"module": "amzn-sample-server-sdk",
"codegen": {
"addValidationExceptionToConstrainedOperations": true,
}
}
}
```
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
## Motivation and Context
Upgrades MSRV to 1.78.0 since the [latest
Rust](https://www.rust-lang.org/) at the time of writing is 1.80.1 (we
try keeping [two releases
behind](https://github.com/awslabs/aws-sdk-rust?tab=readme-ov-file#supported-rust-versions-msrv)).
## Description
In Rust 1.78.0, `clippy` enforces stricter checks in two areas:
- `dead_code`
It seems to be able to uncover more unused code patterns, especially
when different combinations of cargo features are enabled
- `empty_docs`
Prior to 1.78.0, the codegen used to render the following empty docs
`///`, regardless whether the codegen is for the server or for the
client. This is now flagged as `empty_docs` by `clippy`:
```
///
pub(crate) mod XXX {
...
}
```
Other than bumping MSRV to 1.78.0, the remaining code changes address
issues related to the fixes mentioned above.
**EDIT:**
`cargo-semver-checks` required an upgrade due to incompatibility issues
following the MSRV upgrade ([example
error](https://github.com/smithy-lang/smithy-rs/actions/runs/10534205294/job/29261105195#step:4:2068)).
Specifically, after updating the MSRV to 1.78.0, the rustdoc version was
upgraded to v28. However, the version of `cargo-semver-checks` we were
using (0.24.1) only supported rustdoc [up to
v27](4bce03d4a6/Cargo.toml (L18)).
To resolve this, we have updated `cargo-semver-checks` to the first
version that supports rustdoc
[v28](94a491f085/Cargo.toml (L18)).
## Testing
Existing tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
https://github.com/awslabs/aws-sdk-rust/issues/820
## Description
This PR re-exports `ByteStream`'s `Length` and `FsBuilder`. By making
these types available in a client crate, customers can use
`ByteStream::read_from` without needing to import them separately from
the `aws-smithy-types` crate.
## Testing
- Updated an existing re-export test to include `Length` and `FsBuilder`
as well.
- Existing tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Version `User-Agent` header string and begin tracking business metrics
in that header
## Description
This PR versions `User-Agent` string and the version is set to `2.1`.
Furthermore, we track business metrics for SDK features in `User-Agent`
header. Specifically, we now track the following metrics in the
User-Agent header:
- RPC V2 CBOR (M)
- Gzip request compression (L)
- paginator (C)
- waiter (B)
Each letter corresponds to a metric value defined [in the
specification](3f3c874c9f/aws/rust-runtime/aws-runtime/src/user_agent/metrics.rs (L207-L226))).
#### Overall implementation strategy ####
Since business metrics is an AWS-specific concept, the
`aws-smithy-runtime` crate cannot directly reference it or call
`AwsUserAgent::add_business_metric`. Instead, the crate tracks Smithy
SDK features using the config bag with the `StoreAppend` mode. During
the execution of `UserAgentInterceptor::modify_before_signing`, this
method retrieves the SDK features from the config bag and converts them
into business metrics. This implies that any SDK features—whether
specific to Smithy or AWS—that we intend to track must be added to the
config bag prior to the invocation of the `modify_before_signing`
method.
## Testing
- Added a test-only utility function,
`assert_ua_contains_metric_values`, in the `aws-runtime` crate to verify
the presence of metric values in the `User-Agent` string. Since the
position of metric values in the `business-metrics` string may change as
new metrics are introduced (e.g., previously `m/A` but now `m/C,A,B`),
it is essential that this function accounts for potential variations and
does not rely solely on substring matching.
- Added unit and integration tests to verify tracking of the business
metrics introduced in this PR: RPC V2 CBOR, Gzip request compression,
paginator, and waiter.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Customers want to be able to use `serde` with Smithy models. For
details, see the
[RFC](https://github.com/smithy-lang/smithy-rs/blob/configurable-serde/design/src/rfcs/rfc0045_configurable_serde.md)
## Description
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Implementation of `serde::Serialize` for smithy-rs code generators. This
takes the approach of a trait, `SerializeConfigured` which _returns_ a
type that implements `Serialize`. This allows customers to control serde
behavior for their use case.
```rust
/// Trait that allows configuring serialization
/// **This trait should not be implemented directly!** Instead, `impl Serialize for ConfigurableSerdeRef<T>`**
pub trait SerializeConfigured {
/// Return a `Serialize` implementation for this object that owns the object. This is what you want
/// If you need to pass something that `impl`s serialize elsewhere.
fn serialize_owned(self, settings: SerializationSettings) -> impl Serialize;
/// Return a `Serialize` implementation for this object that borrows from the given object
fn serialize_ref<'a>(&'a self, settings: &'a SerializationSettings) -> impl Serialize + 'a;
}
```
This can be used as follows:
```rust
serde_json::to_string(&my_struct.serialize_ref(&SerializationSettings::redact_sensitive_fields());
```
Currently, this codegen plugin is not used by anything. It can be
enabled by bringing it in scope during code generation & adding the
`@serde` trait to the model for the fields where serialization is
desired. This will allow the SDK (if they choose to) roll it out only
for specific types or services that have customer demand.
There are a number of follow on items required:
- [x] Ensure the generated impls work for `Server` codegen
- [ ] Generate `Deserialize` implementations
- [x] Test the implementation all Smithy protocol test models (ensure it
compiles)
## Testing
Unit test testing serialization with a kitchen-sink model
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: david-perez <d@vidp.dev>
## Motivation and Context
Follow-up on https://github.com/smithy-lang/smithy-rs/pull/2544 to add
client-side support for the protocol
## Description
The client implementation mainly focuses on a sub-section
[Requests](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#requests)
in the spec. To that end, this PR addresses `TODO` for the client to
fill in the blanks and includes additional adjustments/refactoring to
pass client protocol tests.
## Testing
- Existing tests in CI
- Upstream protocol test `rpcv2Cbor`
- Our handwritten protocol test `rpcv2Cbor-extras.smithy`
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Lists and maps with sensitive members were not being properly redacted.
Closes https://github.com/smithy-lang/smithy-rs/issues/3757
## Description
<!--- Describe your changes in detail -->
Updated `Shape.shouldRedact` method to properly check for list and map
shapes with sensitive members. Also updated the test definition so it
would actually run since previously the test code was generated in a
nested function inside a no-op function and never run.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added test cases for lists with sensitive members, maps with sensitive
keys, and maps with sensitive values.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
The `struct` created for a Builder should not have attributes because
these structs only derive from `Debug`, `PartialEq`, `Clone` and
`Default`. None of these support attributes.
This becomes a problem if a plugin tries to add attributes to the
`metadata` field to configure the generated code for the `struct`. In
this case, the attributes will also be added to the `Builder` struct;
which is wrong.
## Motivation and Context
I'm customizing client code generation by overriding the method:
```kotlin
override fun symbolProvider(base: RustSymbolProvider): RustSymbolProvider
```
I override the `symbol.expectRustMetadata().additionalAttributes` value
and add extra values there. The generated code adds the correct
attributes for the `struct`, however, it also adds these attributes to
the `Builder`.
The `Builder` is restricted to deriving from `Debug`, `PartialEq`,
`Clone` and `Default` (see
[code](a76dc184c2/codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/BuilderGenerator.kt (L188))).
I believe this change was added by mistake in
https://github.com/smithy-lang/smithy-rs/pull/2222/.
## Description
To solve the issue, I simply remove the offending line.
<!--- Describe your changes in detail -->
## Testing
I don't expect any existing code to break, given that this is a bug. It
only manifests when someone is customizing the generated code.
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Co-authored-by: Gustavo Muenz <muenze@amazon.com>
This is technically a breaking change because we stop implementing
`FromIterator<(String, S)>`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR upgrades Smithy to 1.50.0. The majority of the changes follow
`TODO` added in https://github.com/smithy-lang/smithy-rs/pull/3690.
Other than that, a few adjustments needed to be made:
- for the client
- added two failing tests `RestJsonClientPopulatesDefaultValuesInInput`
and `RestJsonClientUsesExplicitlyProvidedMemberValuesOverDefaults` to
known failing tests for the same reason
[here](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/protocol/ClientProtocolTestGenerator.kt#L72)
- added one broken test (i.e. the upstream test definition is incorrect
but our implementation is correct) to known broken tests per
([smithy#2341](https://github.com/smithy-lang/smithy/pull/2341),
[smithy-rs#3726](https://github.com/smithy-lang/smithy-rs/pull/3726#issuecomment-2199833659))
- for the server
- removed `rest-xml-extras.smithy` since
`RestXmlMustSupportParametersInContentType` is now available upstream
Smithy 1.50.0
- added the following to known failing tests (since the `awsJson1_0`
counterparts are already in the list, but we need the server team to
verify this assumption & provide additional `TODO` comments if
necessary)
- `RestJsonServerPopulatesDefaultsWhenMissingInRequestBody`
- `RestJsonServerPopulatesDefaultsInResponseWhenMissingInParams`,
-
`RestJsonServerPopulatesNestedDefaultValuesWhenMissingInInResponseParams`
## Testing
Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
At some point in the near future (after this customization is applied to
all existing SDKs) the S3 model will change the type of `Expires`
members to `String` from the current `Timestamp`. This change would
break backwards compatibility for us.
## Description
<!--- Describe your changes in detail -->
Add customization to ensure S3 `Expires` field is always a `Timestamp`
and ass a new synthetic member `ExpiresString` that persists the
un-parsed data from the `Expires` header.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added tests to ensure that the model is pre-processed correctly. Added
integration tests for S3. Considered making this more generic codegen
tests, but since this customization will almost certainly only ever
apply to S3 and I wanted to ensure that it was properly applied to the
generated S3 SDK I opted for this route.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
We currently "hotfix" a broken protocol test in-memory, but there's no
mechanism that alerts us when the broken protocol test has been fixed
upstream when updating our Smithy version. This commit introduces such a
mechanism by generating both the original and the fixed test, with a
`#[should_panic]` attribute on the former, so that the test fails when
all its assertions succeed.
With this change, in general this approach of fixing tests in-memory
should now be used over adding the broken test to `expectFail` and
adding the fixed test to a `<protocol>-extras.smithy` Smithy model,
which is substantially more effort.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Fixes: #3471
## Motivation and Context
An issue was raised about a mobile client that appends "; charset=utf-8"
to the Content-Type when using restJson1. The [latest
RFC](https://www.rfc-editor.org/rfc/rfc8259) for "application/json" does
not register a charset parameter, but indicates it is reasonable to
accept it.
## Description
This change loosens the validation of the expected content type to allow
all parameters.
## Testing
* Tests for each protocol were added to
[smithy](https://github.com/smithy-lang/smithy/pull/2296)
* ran the runtime and codegen tests
* Added test for rest-xml, as smithy-rs does not currently run the
smithy tests.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Protocol test generation code was one of the earlier parts of
smithy-rs's codebase and it has accrued a fair amount of tech debt as we
have evolved the code generation primitives. Its code was also forked
when the server code generator was introduced, introducing a lot of
duplicated code that has deviated over time.
This commit refactors the code to modern standards and aims to reconcile
commonalities in `ProtocolTestGenerator`, so that both client and server
can reap centralized improvements over time.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
to get the path of the top level of the repo
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Fixing issue with test workspaces that would break if the repo is not
named `smithy-rs`
## Description
<!--- Describe your changes in detail -->
Moved to using `git rev-parse --show-toplevel` to get the project
directory instead of climbing the file tree.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
All of our CI tests this since it is part of generating test workspaces
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Update lockfiles based on [the codegen
release](https://github.com/awslabs/aws-sdk-rust/releases/tag/release-2024-06-18)
(a release including code changes in aws runtime crates, smithy runtime
crates, or codegen code).
## Testing
- [ ] Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This fixes two bugs:
1. `Content-Type` header checking was succeeding when no `Content-Type`
header was present but one was expected.
2. When a shape was `@httpPayload`-bound, `Content-Type` header checking
occurred even when no payload was being sent. In this case it is not
necessary to check the header, since there is no content.
Code has been refactored and cleaned up. The crux of the logic is now
easier to understand, and contained in `content_type_header_classifier`.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
We have had numerous recent issues where updated dependencies have
broken our build/release process. This seeks to fix that in a brute
force way by manually generating Cargo.lock files for our main
workspaces.
## Description
<!--- Describe your changes in detail -->
Introduce a new gradle task `generateAllLockfiles` (and some child tasks
that it invokes) that generates lock files for
* `aws-config` crate
* `aws/rust-runtime` workspace
* `rust-runtime` workspace
* the superset workspace generated by the `assemble` task
* NOTE: This Cargo.lock is actually generated from the workspace in the
`aws-sdk-rust` repo, saved in this repo, and copied into place at build
time. Generating this one requires a local copy of the `aws-sdk-rust`
repo whose absolute path is indicated by the `-Paws-sdk-rust-path`
parameter.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I ran our E2E tests against 3 commits to confirm that the Cargo.lock
files operate as we expect:
* `9d84980ee2e37deadb7e45450d65c5e6bc69e07c` The head commit from this
PR which has our Cargo.lock files
* ✅ Passes
* `d9d9fcec2de0e898bcc804ee0bb31393d7098e72` with lockfiles and with the
`httparse` and `crc323c` crates unpinned
* ✅ Passes
* `df75beb160890b2a9c382c8a2f7da8d7d0ddd3c8` with no lockfiles and has
the `httparse` and `crc32c` crates unpinned
* ❌ Fails (as expected)
This series of tests confirms that that Cargo.lock files will protect us
against bad dependency updates going forward.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This commit gets rid of some Clippy and Rust doc warnings we produce in
generated code, which are surfaced in our codegen integration tests. The
aim is that eventually we will be able to deny Clippy and Rust doc
warnings in #3678 (despite us baking in `RUSTDOCFLAGS="-D warnings"` and
`RUSTFLAGS="-D warnings"` in our CI Docker image, we curently clear
these in codegen integration tests). Note that denying compiler warnings
is separately tracked in #3194.
The commit contains fixes for the following:
- Unconditionally referring to the `crate::types` module in Rust docs
when
the Smithy model is such that it is not generated.
- Broken intra-crate doc links for client documentation.
- Incorrectly escaping Rust reserved keywords when referring to
operation names in Rust docs.
- An unnecessary semi-colon when rendering additional client
plugins.
- Not escaping brackets in text when rendering Rust docs containing
HTML, that are interpreted as broken links.
- A broken link to unit variants of unions.
- Using `TryFrom` instead of `From` when infallibly converting from
`&str` to `String` when deserializing an `@httpPayload` in the server.
- Using a redundant closure with `unwrap_or_else` when falling back to a
`@default` boolean or number value for a `document` shape, instead of
`unwrap_or`.
- Not opting into `clippy::enum_variant_names` when rendering the
`Operation` enum in the server (since we render the operation names
that were modeled as-is).
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Reverts e4260f5ab6
## Description
We recently had a CI failure where some of the SDK codegen integration
tests encountered a cargo dependency graph resolution error
([example](https://github.com/smithy-lang/smithy-rs/actions/runs/9351952926/job/25740638098?pr=3677#step:3:639)).
The way the issue manifested is as follows:
1. A preceding test `SdkCodegenIntegrationTest.smokeTestSdkCodegen`
generated a `Cargo.lock` at the top-level directory of a shared test
workspace whose `tokio` dependency did NOT include the `macros` feature:
```
[[package]]
name = "tokio"
version = "1.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba4f4a02a7a80d6f274636f0aa95c7e383b912d41fe721a31f29e29698585a4a"
dependencies = [
"backtrace",
"bytes",
"libc",
"mio",
"pin-project-lite",
"socket2",
"windows-sys 0.48.0",
]
```
2. During its turn, `CredentialProviderConfigTest`, which is one of the
failed tests and depends on the `macros` feature of `tokio`, generated
its own test crate and made it as a member of the workspace, for
instance:
```
[workspace]
resolver = "2"
members = ["smithy-test1475522104838765591"]
```
3. Docker containers used by GitHub action have the `CARGO_HOME` set to
`/opt/cargo` where it stores crates io index, cached crates, and so on.
The registry cache did not contain the sought crate `tokio-macros-2.3.0`
at the end of previous step:
```
bash-5.2# ls /opt/cargo/registry/cache/index.crates.io-6f17d22bba15001f/tokio-
tokio-1.37.0.crate tokio-macros-2.2.0.crate tokio-rustls-0.24.1.crate tokio-util-0.7.11.crate
tokio-1.38.0.crate tokio-native-tls-0.3.1.crate tokio-util-0.7.10.crate
```
(`bash-5.2#` indicates that we shelled into a container)
4. Running `cargo test` from the workspace root (or from within the
crate generated at step 2) then triggered the error in question:
```
bash-5.2# cargo t --all-features
error: failed to select a version for the requirement `tokio-macros = "~2.3.0"`
candidate versions found which didn't match: 2.2.0, 2.1.0, 2.0.0, ...
location searched: crates.io index
required by package `tokio v1.38.0`
```
Whether the issue occurs or not for a given test depends upon the
contents of the lockfile of a workspace generated by prior tests. To
ensure we avoid this during codegen integration tests, this PR updates
crates io index before executing `cargo test`.
## Testing
- [x] Passed CI
- [x] Passed internal release pipeline E2E tests
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
https://github.com/smithy-lang/smithy-rs/issues/3676
## Description
Internals one of our tests was leveraging has changed in latest release
of `tracing-test`, updates version and test to be compatible.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Upgrades Smithy to 1.49.0
## Description
As part of the upgrade, it updates the serializer for EC2 query protocol
to handle empty lists in response to
[this](https://github.com/smithy-lang/smithy/pull/2269) (otherwise [a
protocol
test](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-client-test/build.gradle.kts#L75)
`ec2_empty_query_lists_request` would fail).
## Testing
Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
PR 3 of 3. Most of the compression tests happen here since we don't have
a request-compression-supporting model in our set of "smoke test"
service models.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: Aaron Todd <aajtodd@users.noreply.github.com>
Event stream operations with struct shaped messages were using the wrong
`:content-type` message header value, which I think wasn't caught before
since the supported AWS S3/Transcribe event stream operations don't
serialize struct messages. This PR fixes the message content type
serialization.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR sets the `rust-version` property on generated Cargo.toml files
for the AWS SDK crates. It doesn't attempt to place the property on the
runtime crates for now since that will either require manual updating,
or more machinery to automate.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR implements waiters according to the [Smithy waiters
spec](https://smithy.io/2.0/additional-specs/waiters.html).
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Running `cargo build --target wasm32-unknown-unknown
--no-default-features` on an S3 crate has stopped working since
https://github.com/smithy-lang/smithy-rs/pull/3465 with the following
error:
```
error: the wasm*-unknown-unknown targets are not supported by default, you may need to enable the "js" feature. For more information see: https://docs.rs/getrandom/#webassembly-support
--> /Users/[REDACTED]/.cargo/registry/src/index.crates.io-6f17d22bba15001f/getrandom-0.2.14/src/lib.rs:352:9
|
352 | / compile_error!("the wasm*-unknown-unknown targets are not supported by \
353 | | default, you may need to enable the \"js\" feature. \
354 | | For more information see: \
355 | | https://docs.rs/getrandom/#webassembly-support");
| |________________________________________________________________________^
```
To address the issue, this PR updates an S3's dependency on `ahash` in a
way that disables default features.
## Testing
Updated the existing test `integration-tests/webassembly` so that
`check-aws-sdk-standalone-integration-tests` will run `cargo check`
`aws-sdk-s3` against both `wasm32-wasi` and `wasm32-unknown-unknown`
(the updated check would break if we removed `default-features = false`
from the `ahash` dependency).
## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
The latest documentation is Smithy v2. In most cases it's identical. As
far as I can tell, there are no links to items with breaking changes
between v1 and v2.
---------
Co-authored-by: Harry Barber <hlbarber@amazon.co.uk>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
In #3526, I forgot about the existence of input/output matchers in
Smithy waiters, so it only supported binding against a single global
output shape. This PR revises the generator so that it supports multiple
bindings so that it can generate an input/output matcher path traversal.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Unions that have a [unit target member
shape](https://smithy.io/2.0/spec/model.html#unit-type) do not have an
associated data in the generated Rust enum.
Closes Issue:
[2546](https://github.com/smithy-lang/smithy-rs/issues/2546)
## Description
On the **server** side, when the union has constrained members, the code
generated for the conversion from the `Unconstrained` type to the
`Constrained` type incorrectly assumed that each Rust enum would have
associated data.
```
rust-server-codegen/src/unconstrained.rs:31:129
|
| crate::unconstrained::some_union_with_unit_unconstrained::SomeUnionWithUnitUnconstrained::Option1(unconstrained) => Self::Option1(
| -^^^^^^^^^^^^- help: consider using a semicolon here to finish the statement: `;`
| _________________________________________________________________________________________________________________________________|
| |
| | unconstrained
| | ),
| |_____________- call expression requires function
|
::: rust-server-codegen/src/model.rs:152:5
|
| Option1,
| ------- `SomeUnionWithUnit::Option1` defined here
```
The marshaling code for event streams with unit target types incorrectly
assumed that the variant would have associated data.
```
rust-server-codegen/src/event_stream_serde.rs
impl ::aws_smithy_eventstream::frame::MarshallMessage for TestEventMarshaller {
fn marshal() {
let payload = match input {
Self::Input::KeepAlive(inner) => {
```
On the **client** side, the `event_stream_serde` code incorrectly
assumes that a union member, which has the `@streaming` trait applied to
it, takes a `model::Unit` type.
```
rust-client-codegen/src/event_stream_serde.rs:
|
| crate::types::TestEvent::KeepAlive(crate::types::Unit::builder().build()),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^---------------------------------------
| |
| call expression requires function
|
::: rust-client-codegen/src/types/_test_event.rs
|
| KeepAlive,
| --------- `TestEvent::KeepAlive` defined here
```
## Testing
A unit test has been added that tests the following model:
```
$version: "2"
namespace com.example
use aws.protocols#restJson1
use smithy.framework#ValidationException
@restJson1 @title("Test Service")
service TestService {
version: "0.1",
operations: [
TestOperation
TestSimpleUnionWithUnit
]
}
@http(uri: "/testunit", method: "POST")
operation TestSimpleUnionWithUnit {
input := {
@required
request: SomeUnionWithUnit
}
output := {
result : SomeUnionWithUnit
}
errors: [
ValidationException
]
}
@length(min: 13)
string StringRestricted
union SomeUnionWithUnit {
Option1: Unit
Option2: StringRestricted
}
@http(uri: "/test", method: "POST")
operation TestOperation {
input := { payload: String }
output := {
@httpPayload
events: TestEvent
},
errors: [ValidationException]
}
@streaming
union TestEvent {
KeepAlive: Unit,
Response: TestResponseEvent,
}
structure TestResponseEvent {
data: String
}
```
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Fixes#290.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR adds a `RustJmespathShapeTraversalGenerator` class that will
generate code to traverse Smithy generated shapes based on a Jmespath
expression. This is a necessary prerequisite for implementing Smithy
waiters support.
The generator only implements a subset of the full Jmespath spec. This
subset was determined by examining all the waiter expressions in the
existing SDK models.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR upgrades Smithy to 1.47.0. The new S3 tests needed to be
excluded in the server tests since they fail to compile for the same
reason as the other ones that are excluded.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
tracked in issue #3518
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Fixes an S3-related bug introduced when s3 Express was added
## Description
This PR fixes a bug where s3 Express auth flow was incorrectly used when
no auth schemes were available for an endpoint. This use case was
observed in test settings and manifested itself as a confusing behavior.
To address this issue, the fix ensures that an auth scheme for S3
express will be registered after that for sigv4 has been registered.
## Testing
Added an integration test
`s3_express_auth_flow_should_not_be_reached_with_no_auth_schemes`
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Resolves [issue
2523](https://github.com/smithy-lang/smithy-rs/issues/2523) to migrate
to Smithy IDL v2
## Description
I followed the [Smithy Migration
Guide](https://smithy.io/2.0/guides/model-translations/migrating-idl-1-to-2.html)
to convert the models. One change was that by moving the inputs/outputs
into the operation, it did result in a renaming of the CapturePokemon
structures. They went from CapturePokemonEvents(Input|Output) to
CapturePokemon(Input|Output). From a brief look at the commit history,
it seems the API used to be named different and the inputs/outputs
weren't renamed at that time. So I decided to have them match the normal
naming convention (it didn't seem to break any tests).
## Testing
As this was my first commit, I ran almost every target I could discover,
but I think the key ones were:
* codegen-server-test
* codegen-client-test
* examples: `make test`
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
- https://github.com/awslabs/aws-sdk-rust/issues/1095
## Description
Update the JSON parser generator to allow for `null` to be set in
unions. Servers can send unions like this:
```json
{
"AmazonElasticsearchParameters": null,
"AmazonOpenSearchParameters": null,
"AppFlowParameters": null,
"AthenaParameters": null,
"AuroraParameters": null,
"AuroraPostgreSqlParameters": null,
"AwsIotAnalyticsParameters": null,
"BigQueryParameters": null,
"DatabricksParameters": null,
"Db2Parameters": null,
"DenodoParameters": null,
"DocumentDBParameters": null,
"DremioParameters": null,
"ExasolParameters": null,
"GoogleAnalyticsParameters": null,
"JiraParameters": null,
"MariaDbParameters": null,
"MongoAtlasParameters": null,
"MongoDBParameters": null,
"MySqlParameters": null,
"OracleParameters": null,
"PostgreSqlParameters": null,
"PrestoParameters": null,
"RdsParameters": null,
"RedshiftParameters": null,
"S3Parameters": {
"IsUploaded": false,
"ManifestFileLocation": {
"Bucket": "deided-bucket.prod.us-east-1",
"Key": "sales/manifest.json"
},
"RoleArn": null
},
"SalesforceParameters": null,
"SapHanaParameters": null,
"ServiceNowParameters": null,
"SnowflakeParameters": null,
"SparkParameters": null,
"SqlServerParameters": null,
"StarburstParameters": null,
"TeradataParameters": null,
"TrinoParameters": null,
"TwitterParameters": null
}
```
This caused our parser to fail.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
- [x] New unit test
- [x] Dry run against new [smithy protocol
test](https://github.com/smithy-lang/smithy/pull/2180)
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Zelda Hessler