## Description
<!--- Describe your changes in detail -->
This PR includes several fixes for request compression:
- `aws_smithy_compression::body::compress::CompressedBody` will no
longer incorrectly return the inner body's `SizeHint`, returning
`SizeHint::default()` instead.
- Fixed a bug where compressed payloads would have an incorrect content
length, causing those requests to hang.
- Compress in-memory request payloads instead of the previous lazy
approach.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Tests are included. Additionally, I ran a reproducer for a user issues
to ensure that their use case works:
```
#[tokio::test]
async fn use_case_reproducer() {
tracing_subscriber::fmt::init();
let shared_config = aws_config::from_env()
.region(aws_sdk_cloudwatch::config::Region::new("us-west-2"))
.load()
.await;
let service_config = aws_sdk_cloudwatch::config::Config::from(&shared_config)
.to_builder()
.request_min_compression_size_bytes(1)
.build();
let client = Client::from_conf(service_config);
tracing::info!("sending metrics...");
client
.put_metric_data()
.namespace("CloudWatchTestMetricsBrivinc")
.metric_data(
aws_sdk_cloudwatch::types::MetricDatum::builder()
.metric_name(format!("MyMetricNameIsALittleLong"))
.value(0.0)
.build(),
)
.send()
.await
.unwrap();
}
```
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Description
This PR updates lockfiles by running:
```
./gradlew aws:sdk:generateAllLockfiles -Paws-sdk-rust-path=/Users/awsaito/src/aws-sdk-rust
```
However, due to [the minicbor
issue](https://github.com/smithy-lang/smithy-rs/pull/3818), we still pin
it to 0.24.2 by running the following on `rust-runtime/Cargo.lock` and
`aws/sdk/Cargo.lock`:
```
RUSTFLAGS="--cfg aws_sdk_unstable" cargo update -p minicbor --precise 0.24.2
```
The rest of the changes handle miscellaneous scenarios:
- **Updated nightly version**: Upgraded to `nightly-2024-03-15` to
address a [compatibility
issue](https://github.com/serde-rs/serde/issues/2770) introduced by the
updated serde library.
- **Cleaned up use statements**: Removed redundant use statements, which
were flagged as warnings by the new nightly version.
## Testing
Tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
- #3805
## Description
Recover from floating point values extremely close to whole numbers by
incrementing the epoch seconds value.
## Testing
- proptest
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Upgrades MSRV to 1.78.0 since the [latest
Rust](https://www.rust-lang.org/) at the time of writing is 1.80.1 (we
try keeping [two releases
behind](https://github.com/awslabs/aws-sdk-rust?tab=readme-ov-file#supported-rust-versions-msrv)).
## Description
In Rust 1.78.0, `clippy` enforces stricter checks in two areas:
- `dead_code`
It seems to be able to uncover more unused code patterns, especially
when different combinations of cargo features are enabled
- `empty_docs`
Prior to 1.78.0, the codegen used to render the following empty docs
`///`, regardless whether the codegen is for the server or for the
client. This is now flagged as `empty_docs` by `clippy`:
```
///
pub(crate) mod XXX {
...
}
```
Other than bumping MSRV to 1.78.0, the remaining code changes address
issues related to the fixes mentioned above.
**EDIT:**
`cargo-semver-checks` required an upgrade due to incompatibility issues
following the MSRV upgrade ([example
error](https://github.com/smithy-lang/smithy-rs/actions/runs/10534205294/job/29261105195#step:4:2068)).
Specifically, after updating the MSRV to 1.78.0, the rustdoc version was
upgraded to v28. However, the version of `cargo-semver-checks` we were
using (0.24.1) only supported rustdoc [up to
v27](4bce03d4a6/Cargo.toml (L18)).
To resolve this, we have updated `cargo-semver-checks` to the first
version that supports rustdoc
[v28](94a491f085/Cargo.toml (L18)).
## Testing
Existing tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Fixes the execution order of
[modify_before_serialization](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/interceptors/trait.Intercept.html#method.modify_before_serialization)
and
[read_before_serialization](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/interceptors/trait.Intercept.html#method.read_before_serialization)
in the orchestrator.
## Description
The fix ensures that the execution order of these interceptor methods
aligns with our orchestrator design. While customers may see a behavior
change as a result, we don't treat this as a new `BehaviorVersion` since
it is intended as a bug fix.
## Testing
Existing tests in CI
## Checklist
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
https://github.com/awslabs/aws-sdk-rust/issues/821
## Description
This PR addresses an issue with the
[Length::UpTo](https://docs.rs/aws-smithy-types/1.2.2/aws_smithy_types/byte_stream/enum.Length.html)
usage in `FsBuilder`. Previously, if a value specified for `UpTo`
exceeded the remaining file length, `FsBuilder` would incorrectly accept
the value. This discrepancy led to failures in subsequent request
dispatches, as the actual body size did not match the advertised
`Content-Length`, as explained
[here](https://github.com/awslabs/aws-sdk-rust/issues/821#issuecomment-1937354372)
(thank you @pablosichert for a self-contained reproducer and problem
analysis!).
## Testing
- Added a unit test for `FsBuilder` verifying the `Length::UpTo` usage
- Ran successfully a customer provided
[reproducer](https://github.com/awslabs/aws-sdk-rust/issues/821#issuecomment-1937354372)
with the code changes in this PR (with an added a call to
`complete_multipart_upload` at the end, it also succeeded in uploading
the object):
```
upload_id: cTDSngbubD25cOoFCNgjpG55o0hAMQNjO16dNFyNTKjg9PEtkcrKG5rTGzBns7CXoO8T.Qm9GpNj6jgwJTKcXDpsca95wSMWMDfPF0DBhmbk3OAGHuuGM1E70spk2suW
File created
part_number: 1, e_tag: "5648ddf58c7c90a788d7f16717a61b08"
part_number: 2, e_tag: "a6bdad6d65d18d842ef1d57ca4673bc3"
part_number: 3, e_tag: "f518f6b19b255ec49b61d511288554fc"
part_number: 4, e_tag: "1496524801eb1d0a7cfbe608eb037d9c"
part_number: 5, e_tag: "21340de04927ce1bed58ad0375c03e01"
```
## Checklist
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
https://github.com/smithy-lang/smithy-rs/issues/1925
## Description
Backports connection poisoning that hyper 0.14 HTTP client has to the
hyper 1.x client.
See also:
* upstream support: https://github.com/hyperium/hyper-util/pull/121
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [X] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Version `User-Agent` header string and begin tracking business metrics
in that header
## Description
This PR versions `User-Agent` string and the version is set to `2.1`.
Furthermore, we track business metrics for SDK features in `User-Agent`
header. Specifically, we now track the following metrics in the
User-Agent header:
- RPC V2 CBOR (M)
- Gzip request compression (L)
- paginator (C)
- waiter (B)
Each letter corresponds to a metric value defined [in the
specification](3f3c874c9f/aws/rust-runtime/aws-runtime/src/user_agent/metrics.rs (L207-L226))).
#### Overall implementation strategy ####
Since business metrics is an AWS-specific concept, the
`aws-smithy-runtime` crate cannot directly reference it or call
`AwsUserAgent::add_business_metric`. Instead, the crate tracks Smithy
SDK features using the config bag with the `StoreAppend` mode. During
the execution of `UserAgentInterceptor::modify_before_signing`, this
method retrieves the SDK features from the config bag and converts them
into business metrics. This implies that any SDK features—whether
specific to Smithy or AWS—that we intend to track must be added to the
config bag prior to the invocation of the `modify_before_signing`
method.
## Testing
- Added a test-only utility function,
`assert_ua_contains_metric_values`, in the `aws-runtime` crate to verify
the presence of metric values in the `User-Agent` string. Since the
position of metric values in the `business-metrics` string may change as
new metrics are introduced (e.g., previously `m/A` but now `m/C,A,B`),
it is essential that this function accounts for potential variations and
does not rely solely on substring matching.
- Added unit and integration tests to verify tracking of the business
metrics introduced in this PR: RPC V2 CBOR, Gzip request compression,
paginator, and waiter.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
A preparatory PR that relaxes test verification of the connection
recording tests.
## Description
This PR is a preparatory step for upcoming changes to the `UserAgent`,
which will introduce new header values in `x-amz-user-agent`, such as
`ua/2.0` (user agent metadata) and `m/A` (business metrics).
However, the introduction of new header values will cause the following
pain points:
- we have to update many connection recording tests to make them pass
again (i.e. the very places updated in this PR)
- check for semver hazards [fail to
pass](https://github.com/smithy-lang/smithy-rs/actions/runs/10209305234/job/28247956895).
This is much the same as we encountered in [content length enforcement
tests](https://github.com/smithy-lang/smithy-rs/issues/3523). This
creates a chicken-and-egg problem: tests need to be updated for the PRs
to pass CI, but the "released SDKs" in the `aws-sdk-rust` repository
won't implement the new `UserAgent` header values until the PRs are
merged and released.
To prevent recurring issues with headers affecting connection recording
tests (hence semver checks), this PR preemptively updates the connection
recording tests. Specifically, it adjusts them to ignore certain
headers, ensuring that updates to the `x-amz-user-agent` header do not
trigger semver hazards in subsequent PRs.
**Questions**:
- This PR modifies the connection recording tests to skip verification
of the `x-amz-user-agent` and `authorization` headers. Consequently, we
no longer test the SigV4 signature match in `aws/sdk/integration-tests`.
Although we continue to run canary tests in CI, it would be beneficial
to maintain at least one integration test for verifying the correctness
of the SigV4 signature. This helps in detecting potential bugs affecting
SigV4 signature correctness early on.
To address this, I’ve added [an
awsSdkIntegrationTest](f513b924dc/aws/sdk-codegen/src/test/kotlin/software/amazon/smithy/rustsdk/SigV4AuthDecoratorTest.kt (L72))
that excludes the `UserAgentInterceptor` and checks the `Signature`
value in the `authorization` header. The question is, do we want to keep
this test? If future header updates cause semver hazards to fail, this
test would also be affected. We would then need to repeat the process we
are going through with this PR: update the test, release the change to
aws-sdk-rust, and only then can we make subsequent changes without
breaing semver hazards.
- I've removed the commented-out tests and their associated connection
recording files from `request_information_headers.rs` as part of
cleanup, since there were no explanatory comments. Let me know if we
want to restore these tests, and I will do so along with a comment
explaining their purpose.
## Testing
- Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Customers want to be able to use `serde` with Smithy models. For
details, see the
[RFC](https://github.com/smithy-lang/smithy-rs/blob/configurable-serde/design/src/rfcs/rfc0045_configurable_serde.md)
## Description
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Implementation of `serde::Serialize` for smithy-rs code generators. This
takes the approach of a trait, `SerializeConfigured` which _returns_ a
type that implements `Serialize`. This allows customers to control serde
behavior for their use case.
```rust
/// Trait that allows configuring serialization
/// **This trait should not be implemented directly!** Instead, `impl Serialize for ConfigurableSerdeRef<T>`**
pub trait SerializeConfigured {
/// Return a `Serialize` implementation for this object that owns the object. This is what you want
/// If you need to pass something that `impl`s serialize elsewhere.
fn serialize_owned(self, settings: SerializationSettings) -> impl Serialize;
/// Return a `Serialize` implementation for this object that borrows from the given object
fn serialize_ref<'a>(&'a self, settings: &'a SerializationSettings) -> impl Serialize + 'a;
}
```
This can be used as follows:
```rust
serde_json::to_string(&my_struct.serialize_ref(&SerializationSettings::redact_sensitive_fields());
```
Currently, this codegen plugin is not used by anything. It can be
enabled by bringing it in scope during code generation & adding the
`@serde` trait to the model for the fields where serialization is
desired. This will allow the SDK (if they choose to) roll it out only
for specific types or services that have customer demand.
There are a number of follow on items required:
- [x] Ensure the generated impls work for `Server` codegen
- [ ] Generate `Deserialize` implementations
- [x] Test the implementation all Smithy protocol test models (ensure it
compiles)
## Testing
Unit test testing serialization with a kitchen-sink model
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: david-perez <d@vidp.dev>
This improves error messaging for HTTP header related errors.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
The nonprintable characters in this source file make tools misidentify
it as a binary file, rather than source code.
## Description
**Before:**
```console
$ file rust-runtime/aws-smithy-checksums/src/http.rs
rust-runtime/aws-smithy-checksums/src/http.rs: data
```
**After:**
```console
$ file rust-runtime/aws-smithy-checksums/src/http.rs
rust-runtime/aws-smithy-checksums/src/http.rs: ASCII text
```
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Follow-up on https://github.com/smithy-lang/smithy-rs/pull/2544 to add
client-side support for the protocol
## Description
The client implementation mainly focuses on a sub-section
[Requests](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#requests)
in the spec. To that end, this PR addresses `TODO` for the client to
fill in the blanks and includes additional adjustments/refactoring to
pass client protocol tests.
## Testing
- Existing tests in CI
- Upstream protocol test `rpcv2Cbor`
- Our handwritten protocol test `rpcv2Cbor-extras.smithy`
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
The Python server relies on types defined in `aws-smithy-http-server`,
which has a new version published. A new version of the Python server
should be released to ensure there is only one `aws-smithy-http-server`
in the dependency closure of the generated crates.
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
this fixes an issue submitted by an internal user.
## Description
<!--- Describe your changes in detail -->
If the first event pushed into the log underlying stalled stream
protection, then the log manager would panic. This is because it was
expecting to merge the new event in but there was nothing to merge it
with.
To fix this, I updated the code to `push` the event when the log manager
is empty, instead of merging it.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
ran existing tests and wrote a new test exercising the
previous-panicking code.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This is technically a breaking change because we stop implementing
`FromIterator<(String, S)>`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
When a user defines a handler function with additional parameters, these
parameters are constructed using the `FromRequest` trait implementation
specific to their types. If `FromRequest` fails to construct the
parameter – for instance, if `Extension<State>` is expected by the user
but `Extension<Arc<State>>` is actually added by the extension layer –
the server currently does not log a message indicating this construction
failure. Instead, it simply returns a 500 internal server error.
## Description
`tracing::{trace, error}` calls have been added.
## Breaking change
`FromParts<P>::Rejection` **must** implement `std::fmt::Display`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
I was speaking to several customers that found our connection poisoning
terminology confusing. To address this, I've changed the messaging and
I've also created a new section in the docs that explains some errors in
greater detail. To direct users to these explanations, I've added a link
to the `tracing::info!` call.
I propose that we consider doing the same any time we identify error
messaging that
- must be kept succinct to avoid logging too much info
- must be more understandable to our users.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Fixes: #3471
## Motivation and Context
An issue was raised about a mobile client that appends "; charset=utf-8"
to the Content-Type when using restJson1. The [latest
RFC](https://www.rfc-editor.org/rfc/rfc8259) for "application/json" does
not register a charset parameter, but indicates it is reasonable to
accept it.
## Description
This change loosens the validation of the expected content type to allow
all parameters.
## Testing
* Tests for each protocol were added to
[smithy](https://github.com/smithy-lang/smithy/pull/2296)
* ran the runtime and codegen tests
* Added test for rest-xml, as smithy-rs does not currently run the
smithy tests.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Also fixes IntelliJ lints
This PR is not intended to change any behavior. If you see me doing that
somewhere, then it's a mistake.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Due to numerous dependency issues over the past few weeks several of our
dependencies were pinned to specific minor versions. Now that we have
lockfiles for all of our workspaces those pinned versions can be undone.
## Description
<!--- Describe your changes in detail -->
I unpinned those dependencies and generated new lockfiles after that
change.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I ran the gradle `assemble` and `sdkTest` tasks locally and they both
passed. I ran a full set of preview/release e2e tests against this
change in my local stack and those all passed. And since we copy the
Cargo.lock into the test workspaces the CI run for this PR will also
test the changes.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
No functional changes, only dependency changes.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Update lockfiles based on [the codegen
release](https://github.com/awslabs/aws-sdk-rust/releases/tag/release-2024-06-18)
(a release including code changes in aws runtime crates, smithy runtime
crates, or codegen code).
## Testing
- [ ] Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Our endpoint partition parsing was dropping the `implicitGlobalRegion`
value
## Description
<!--- Describe your changes in detail -->
Updated our Partition structs to include that value and update the
parser to include it.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added check for `implicitGlobalRegion` to existing partition unit tests
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This fixes two bugs:
1. `Content-Type` header checking was succeeding when no `Content-Type`
header was present but one was expected.
2. When a shape was `@httpPayload`-bound, `Content-Type` header checking
occurred even when no payload was being sent. In this case it is not
necessary to check the header, since there is no content.
Code has been refactored and cleaned up. The crux of the logic is now
easier to understand, and contained in `content_type_header_classifier`.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
We have had numerous recent issues where updated dependencies have
broken our build/release process. This seeks to fix that in a brute
force way by manually generating Cargo.lock files for our main
workspaces.
## Description
<!--- Describe your changes in detail -->
Introduce a new gradle task `generateAllLockfiles` (and some child tasks
that it invokes) that generates lock files for
* `aws-config` crate
* `aws/rust-runtime` workspace
* `rust-runtime` workspace
* the superset workspace generated by the `assemble` task
* NOTE: This Cargo.lock is actually generated from the workspace in the
`aws-sdk-rust` repo, saved in this repo, and copied into place at build
time. Generating this one requires a local copy of the `aws-sdk-rust`
repo whose absolute path is indicated by the `-Paws-sdk-rust-path`
parameter.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I ran our E2E tests against 3 commits to confirm that the Cargo.lock
files operate as we expect:
* `9d84980ee2e37deadb7e45450d65c5e6bc69e07c` The head commit from this
PR which has our Cargo.lock files
* ✅ Passes
* `d9d9fcec2de0e898bcc804ee0bb31393d7098e72` with lockfiles and with the
`httparse` and `crc323c` crates unpinned
* ✅ Passes
* `df75beb160890b2a9c382c8a2f7da8d7d0ddd3c8` with no lockfiles and has
the `httparse` and `crc32c` crates unpinned
* ❌ Fails (as expected)
This series of tests confirms that that Cargo.lock files will protect us
against bad dependency updates going forward.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
https://github.com/smithy-lang/smithy-rs/issues/3672
## Description
Pin to a newer working version of `crc32c` (will still pin to a
particular version to avoid a breaking change, until the lockfile
protection is available)
## Testing
- Verified locally build of `aws-smithy-checksums` with the updated
`crc32c`
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
part of the `hyper` 1.0 upgrade
## Description
<!--- Describe your changes in detail -->
This PR adds information about the connector used to send a request to
the metadata included in the user agent.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I added an integration test.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
To fix current CI and release pipeline failures coming from
`aws-smithy-runtime` and `aws-sigv4` now depending on `httparse` version
1.9.0 or greater.
## Testing
Tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
Making this conversion bidirectional is helpful.
<!--- If it fixes an open issue, please link to the issue here -->
## Description
<!--- Describe your changes in detail -->
Add `impl From<StatusCode> for httpN::StatusCode` that does the obvious
thing.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I'm using these conversions in my own project. As all three crates have
identical constraints on the status code value the changes are trivially
correct.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
Not clear to me what if anything to do here.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Rustc 1.74.0 removed the private-in-public check, which raises a
compile-time error if a member of a struct has a type which is less
visible than the parent struct. Fixing this private-in-public issue
enables aws-smithy-runtime to build in projects using rustc older than
1.74.0.
## Description
Make UploadThroughputCheckFuture pub(crate) because it is needed in
MaybeUploadThroughputCheckFuture which is pub(crate).
## Testing
Verified that crate now compiles using rustc 1.72.1.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
#1925
## Description
<!--- Describe your changes in detail -->
Implements the v1 `http_body::Body` trait for `PathBody`. Part of the
ongoing hyper v1 upgrade.
This also moves a pre-1.0 impl into its own module.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I ported the tests too
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Our logs had several entries that were extremely verbose and made them
harder to sort through. This change aims to reduce the verbosity of
those logs to something more manageable.
## Description
<!--- Describe your changes in detail -->
- Removed the logging of the full IMDS Client struct, replaced with a
message that it was truncated
- Removed logging the full `Configbag` in a couple places in
`RuntimeComponentsBuilder`
- Removed logging of full `ProvideCredentials` objects in the
CredentialsProviderChain` and replaced with just their names
There are some verbose logs I did not remove because I was not sure of
their usefulness. Most notably the `PartitionResolver` struct logs
several hundred lines of region information each time it appears. Happy
to truncate that as well if those logs aren't too helpful.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
This was tested locally by running the SDK [logging
example](https://docs.aws.amazon.com/sdk-for-rust/latest/dg/logging.html)
code with `RUST_LOG=trace` prepended to the `cargo run` command. For
comparison, when saved as a .txt file, the old logs take up `2.8MB` and
the new logs take `273KB` for the same operation.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
The `invoke` debug span now includes invocation IDs for operation
invocations. It is a random unique ID per client and shared across
retries for the same operation.
Example debug trace output with the changes in this PR:
```
2024-05-29T17:50:22.230526Z DEBUG invoke{service=s3 operation=ListObjectsV2 sdk_invocation_id=4652364}:try_op:try_attempt: aws_smithy_runtime_api::client::interceptors::context: entering 'before deserialization' phase
```
## Description
Note that this invocation ID is currently for internal use only to
improve debuggability when looking at logs (usually with customers).
Note also that the invocation ID is not related to [that used in
amz-sdk-invocation-id](https://github.com/smithy-lang/smithy-rs/pull/2626)
(we [attempted to make it
related](https://github.com/smithy-lang/smithy-rs/pull/3661) but decided
to go for this two-way door approach).
In sum, this PR is meant to be simple, and if we are to augment the
functionality (e.g. make the ID configurable or make it related to that
in `amz-sdk-invocation-id`), we can do so without breaking anything.
## Testing
Relies on the existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Works around https://github.com/zowens/crc32c/issues/67
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
* https://github.com/awslabs/aws-sdk-rust/issues/1141
* https://github.com/awslabs/aws-sdk-rust/issues/1146
* https://github.com/awslabs/aws-sdk-rust/issues/1148
## Description
* Disables stalled stream upload protection for requests with an
empty/zero length body.
* Disables stalled stream upload throughput checking once the request
body has been read and handed off to the HTTP layer.
## Testing
Additional integration tests added covering empty bodies and completed
uploads.
Tested SQS issue against latest runtime and can see it works now. The S3
`CopyObject` issue is related to downloads and will need a different
solution.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: ysaito1001 <awsaito@amazon.com>
PR 3 of 3. Most of the compression tests happen here since we don't have
a request-compression-supporting model in our set of "smoke test"
service models.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: Aaron Todd <aajtodd@users.noreply.github.com>
## Motivation and Context
Avoids panic when [Duration for exponential
backoff](https://github.com/smithy-lang/smithy-rs/blob/main/rust-runtime/aws-smithy-runtime/src/client/retries/strategy/standard.rs#L150)
could not be created from a large float.
## Description
[Duration::from_secs_f64](https://doc.rust-lang.org/core/time/struct.Duration.html#method.from_secs_f64)
may panic. This PR switches to use a fallible sibling
`Duration::try_from_secs_f64` to avoid panics. If
`Duration::try_from_secs_f64` returns an `Err` we fallback to
`max_backoff` for subsequent retries. Furthermore, we learned from
internal discussion that jitter also needs to be applied to
`max_backoff`. This PR updates `calculate_exponential_backoff` to handle
all the said business logic in one place.
## Testing
- Added a unit test
`should_not_panic_when_exponential_backoff_duration_could_not_be_created`
- Manually verified reproduction steps provided [in the original
PR](https://github.com/awslabs/aws-sdk-rust/issues/1133)
<details>
<summary>More details</summary>
```
#[tokio::test]
async fn repro_1133() {
let config = aws_config::from_env()
.region(Region::new("non-existing-region")) // forces retries
.retry_config(
RetryConfig::standard()
.with_initial_backoff(Duration::from_millis(1))
.with_max_attempts(100),
)
.timeout_config(
TimeoutConfig::builder()
.operation_attempt_timeout(Duration::from_secs(180))
.operation_timeout(Duration::from_secs(600))
.build(),
)
.load()
.await;
let client: Client = Client::new(&config);
let res = client
.list_objects_v2()
.bucket("bucket-name-does-not-matter")
.send()
.await;
dbg!(res);
}
```
Without changes in this PR:
```
---- repro_1133 stdout ----
thread 'repro_1133' panicked at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/time.rs:741:23:
can not convert float seconds to Duration: value is either too big or NaN
stack backtrace:
...
failures:
repro_1133
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 338.18s
```
With changes in this PR:
```
// no panic
---- repro_1133 stdout ----
res = Err(
TimeoutError(
TimeoutError {
source: MaybeTimeoutError {
kind: Operation,
duration: 600s,
},
},
),
)
```
</details>
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
## Appendix
<details>
<summary>runtime-versioner bug fix</summary>
This PR also fixes a limitation in `runtime-versioner audit`. I included
the fix in the PR because the issue occurred with special conditions,
and we don't get to reproduce it every day. The way the issue manifests
is this.
1. We have a branch from the main whose latest release tag at the time
was `release-2024-04-30`
2. The main has moved ahead and a new `smithy-rs` release has been made
`release-2024-05-08`
3. We perform `git merge main`, pre-commit hooks run, and we then get
`audit` failures from `runtime-versioner`:
```
2024-05-10T16:54:36.434968Z WARN runtime_versioner::command::audit: there are newer releases since 'release-2024-04-30'
aws-config was changed and version bumped, but the new version number (1.4.0) has already been published to crates.io. Choose a new version number.
aws-runtime was changed and version bumped, but the new version number (1.2.2) has already been published to crates.io. Choose a new version number.
aws-smithy-runtime-api was changed and version bumped, but the new version number (1.6.0) has already been published to crates.io. Choose a new version number.
aws-smithy-types was changed and version bumped, but the new version number (1.1.9) has already been published to crates.io. Choose a new version number.
aws-types was changed and version bumped, but the new version number (1.2.1) has already been published to crates.io. Choose a new version number.
Error: there are audit failures in the runtime crates
```
This happens because when the latest `main` is being merged to our
branch, `runtime-versioner audit` should use `previous_release_tag`
`release-2024-05-08` to perform audit but [pre-commit hooks run the
tool](https://github.com/smithy-lang/smithy-rs/blob/main/.pre-commit-hooks/runtime-versioner.sh#L8)
using the latest previous release tag that can be traced back from
`HEAD` of our branch, which is `release-2024-04-30`. Hence the error.
The fix adds an environment variable
`SMITHY_RS_RUNTIME_VERSIONER_AUDIT_PREVIOUS_RELEASE_TAG` to specify a
previous release tag override, in addition to a `--previous-release-tag`
command-line argument.
Furthermore, the fix has relaxed certain checks in `audit`. Taking our
example for instance, when `HEAD` is now behind `release-2024-05-08`,
it's OK to fail even if `release-2024-05-08` is not the ancestor of
`HEAD` (as stated above, `git merge-base --is-ancestor` does not know
that while main is being merged) as long as `release-2024-04-28` (the
latest release seen from `HEAD`) is the ancestor of
`release-2024-05-08`.
```
release-2024-04-28 release-2024-05-08
│ │
────────────┼───────────────┬────────────────┼───────x─────► main
│ │HEAD │ x
│ of x
│branch x
│ x
│ x
│ xxxxxxxxxxx
│ x
│ x git merge main
│ x
│ ▼
│
└──────────────────► feature branch
```
To use the fix, set the environment variable with the new release tag
and perform `git merge main`:
```
➜ smithy-rs git:(ysaito/fix-panic-in-exp-backoff) ✗ export SMITHY_RS_RUNTIME_VERSIONER_AUDIT_PREVIOUS_RELEASE_TAG=release-2024-05-08
➜ smithy-rs git:(ysaito/fix-panic-in-exp-backoff) ✗ git merge main
...
Running `/Users/awsaito/src/smithy-rs/tools/target/debug/runtime-versioner audit`
2024-05-10T19:32:26.665578Z WARN runtime_versioner::tag: expected previous release to be 'release-2024-04-30', but 'release-2024-05-08' was specified. Proceeding with 'release-2024-05-08'.
SUCCESS
```
</summary>
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This adds a new runtime crate. The new crate contains code related to
compressing requests. Not included in this PR is everything needed to
actually make use of the new crate.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR is the combination of two previously reviewed PRs that both
enable new behaviors behind a new Behavior Major Version (BMV):
* https://github.com/smithy-lang/smithy-rs/pull/3527
* https://github.com/smithy-lang/smithy-rs/pull/3578
## Description
* Enables stalled stream protection by default on latest behavior
version.
* Enables creation of a default identity cache.
## Testing
All testing done on prior PRs. See for details.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
According to #3161 , the Debug output was in a marginally legible
format.
<!--- If it fixes an open issue, please link to the issue here -->
#3161
## Description
<!--- Describe your changes in detail -->
Since there's no expected difference between `Debug` and `Display` for
date_time, the `fmt::Debug` for `date_time` calls the already
implemented `Display`.
## Testing
<!--- Please describe in detail how you tested your changes -->
I added unit tests, and ran `cargo test` within the `rust-runtime`
directory.
<!--- Include details of your testing environment, and the tests you ran
to -->
Aarch64 mac terminal - `$ cd rust-runtime && cargo test`
<!--- see how your change affects other areas of the code, etc. -->
A single test failed:
`client::waiters::backoff::tests::backoff_with_seeded_jitter` at first.
After rebasing on upstream, the failure went away. Thanks for the fix.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [X] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Mike Nissenbaum <mnissenb@amazon.com>
## Motivation and Context
Pins major and minor versions for `fastrand` during testing in
`aws-smithy-runtime`
## Description
`aws-smithy-runtime` uses the `fastrand` crate in a waiters' [unit
test](eac52eb69c/rust-runtime/aws-smithy-runtime/src/client/waiters/backoff.rs (L137)).
Two days ago, `fastrand` of version 2.1.0 got released (previously
2.0.2). According to [this
PR](https://github.com/smol-rs/fastrand/issues/20), a minor version bump
can cause seed-value-breaking changes.
This PR will only allow patch-level bumps for `fastrand` in
`[dev-dependencies]`.
## Testing
Relies on testing in CI (since CI in the main branch currently fails, if
CI passes in this PR we're good).
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR implements waiters according to the [Smithy waiters
spec](https://smithy.io/2.0/additional-specs/waiters.html).
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Zelda Hessler