The GitHub workflow for `exotic platform` downloads the `openssl.sh`
script, which tries to download
`https://www.openssl.org/source/openssl-1.0.2t.tar.gz`, but the file has
been permanently moved. This PR updates the `curl` command to include
the `-L` option.
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Add tests the were skipped in
https://github.com/smithy-lang/smithy-rs/pull/3836 in favor of merging
and unblocking customer.
## Description
<!--- Describe your changes in detail -->
Testing our smoketest generator with the `UseDualStack` and `UseFips`
endpoint built-ins. Previously the smoketest generator assumed that
these were always available, but they are only available if those
built-ins are included in the endpoints rule set of the model. We now
test the smoke test with both of these built-ins, with each by itself,
and with neither.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
Just a test change, no changelog
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Description
This PR introduces an additional step to the
`generate-smithy-rs-release` script to synchronize the SDK lockfile with
the runtime lockfiles and the Cargo dependencies specified in
[CargoDependency.kt](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/rustlang/CargoDependency.kt).
For more details on why we synchronize the SDK lockfile, please refer to
the sdk-lockfiles
[README](https://github.com/smithy-lang/smithy-rs/tree/main/tools/ci-build/sdk-lockfiles).
We've decided to synchronize the SDK lockfile during release for the
following reasons:
- Synchronization occurs behind the scenes, avoiding disruptions to the
development workflow, unlike enforcing this check during pre-commit
- It is a single-sourced place for synchronizing the SDK lockfile
## Testing
Tested with the following scenario:
1. Added a dummy dependency to `aws-smithy-runtime` and updated
`rust-runtime/Cargo.lock`
```
--- a/rust-runtime/aws-smithy-runtime/Cargo.toml
+++ b/rust-runtime/aws-smithy-runtime/Cargo.toml
@@ -1,6 +1,6 @@
[package]
name = "aws-smithy-runtime"
-version = "1.7.1"
+version = "1.7.2"
authors = ["AWS Rust SDK Team <aws-sdk-rust@amazon.com>", "Zelda Hessler <zhessler@amazon.com>"]
description = "The new smithy runtime crate"
edition = "2021"
@@ -41,6 +41,7 @@ hyper-rustls = { version = "0.24", features = ["rustls-native-certs", "http2"],
once_cell = "1.18.0"
pin-project-lite = "0.2.7"
pin-utils = "0.1.0"
+predicates = "3.1.2"
```
2. `sdk-lockfiles audit` failed as expected
```
➜ smithy-rs git:(ysaito/sync-sdk-lockfile-during-release) sdk-lockfiles audit
2024-09-17T21:48:12.346747Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-17T21:48:12.381919Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-17T21:48:12.382360Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
`difflib` (0.4.0), used by `rust-runtime/Cargo.lock`, is not contained in the SDK lockfile!
`float-cmp` (0.9.0), used by `rust-runtime/Cargo.lock`, is not contained in the SDK lockfile!
`normalize-line-endings` (0.3.0), used by `rust-runtime/Cargo.lock`, is not contained in the SDK lockfile!
`predicates` (3.1.2), used by `rust-runtime/Cargo.lock`, is not contained in the SDK lockfile!
`predicates-core` (1.0.8), used by `rust-runtime/Cargo.lock`, is not contained in the SDK lockfile!
Error: there are lockfile audit failures
```
3. Ran [a dry-run
release](https://github.com/smithy-lang/smithy-rs/actions/runs/10914801466)
4. Confirmed that [the SDK lockfile was synchronized during Generate
release
artifacts](https://github.com/smithy-lang/smithy-rs/actions/runs/10914801466/job/30294275147#step:4:390)
5. Confirmed that [the diffs in the SDK
lockfile](d57a7896d3)
only contained updating the `aws-smithy-runtime` crate version and
adding dummy dependencies.
6. `sdk-lockfiles audit` succeeded in the release artifacts
```
➜ smithy-rs git:(ysaito/sync-sdk-lockfile-during-release) ✗ sdk-lockfiles audit --smithy-rs-path ~/Downloads/smithy-rs-release/smithy-rs
2024-09-17T21:54:15.491070Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-17T21:54:15.521851Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-17T21:54:15.522278Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS
```
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Description
<!--- Describe your changes in detail -->
This PR includes several fixes for request compression:
- `aws_smithy_compression::body::compress::CompressedBody` will no
longer incorrectly return the inner body's `SizeHint`, returning
`SizeHint::default()` instead.
- Fixed a bug where compressed payloads would have an incorrect content
length, causing those requests to hang.
- Compress in-memory request payloads instead of the previous lazy
approach.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Tests are included. Additionally, I ran a reproducer for a user issues
to ensure that their use case works:
```
#[tokio::test]
async fn use_case_reproducer() {
tracing_subscriber::fmt::init();
let shared_config = aws_config::from_env()
.region(aws_sdk_cloudwatch::config::Region::new("us-west-2"))
.load()
.await;
let service_config = aws_sdk_cloudwatch::config::Config::from(&shared_config)
.to_builder()
.request_min_compression_size_bytes(1)
.build();
let client = Client::from_conf(service_config);
tracing::info!("sending metrics...");
client
.put_metric_data()
.namespace("CloudWatchTestMetricsBrivinc")
.metric_data(
aws_sdk_cloudwatch::types::MetricDatum::builder()
.metric_name(format!("MyMetricNameIsALittleLong"))
.value(0.0)
.build(),
)
.send()
.await
.unwrap();
}
```
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Motivation and Context
Primarily for @drganjoo providing feedback on smithy-rs#3827 (thanks),
but anyone is welcome to review the changes.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Smoketests for service were failing because the smoke test assumed that
`use_dual_stack` would be present. But the service's endpoint rules did
not use that bulit-in so it was not.
## Description
<!--- Describe your changes in detail -->
Check if the service actually uses the fips or dual stack built-ins
before adding them to the test.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Did not add tests for this because the current `SmokeTestsDecoratorTest`
isn't actually working and fixing it is going to be a bit of work. Want
to get this in to unblock customer ASAP and will go back and update the
tests once they are unblocked.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
https://github.com/awslabs/aws-sdk-rust/issues/1181
## Checklist
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
See
[README](b9f3633526/tools/ci-build/sdk-lockfiles/README.md)
A later PR will use this tool to enforce the check (likely during the
execution of pre-commit-hooks).
## Testing
- Added unit tests for the tool
- Ran the tool against currently checked-in lockflies in the `main`
branch and ensured the SDK lockfile contained dependencies listed in the
other runtime lockfiles.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Description
This PR introduces and updates gradle tasks for managing lockfiles. Here
are the highlights:
- [The SDK
lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/sdk/Cargo.lock)
can now be generated directly within the `smithy-rs` repository without
the `aws-sdk-rust` repository.
- The SDK lockfile can be synchronized with runtime lockfiles, updating
only new dependencies while preserving the versions of existing ones.
- To prevent updating broken dependencies to the latest versions, we
track the last known good versions and downgrade them to those versions.
New/updated gradle tasks are intended for automation:
- This existing task no longer requires `-Paws-sdk-rust-path`. We plan
to incorporate it into a weekly GitHub Action to automate lockfile
updates:
```
./gradlew aws:sdk:cargoUpdateAllLockfiles
```
- This new task synchronizes the SDK lockfile with runtime lockfiles. We
plan to integrate it into pre-commit hooks:
```
./gradlew aws:sdk:syncAwsSdkLockfile
```
In addition, this PR has updated the SDK lockfile by executing
`./gradlew aws:sdk:syncAwsSdkLockfile`. The updated lockfile no longer
includes many SDK crates that are unused in CI/CD processes. The new SDK
lockfile is in sync with the runtime lockfiles:
```
➜ smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) sdk-lockfiles audit
2024-09-12T16:02:25.193765Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T16:02:25.224862Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T16:02:25.225389Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS
```
## Testing
I have verified the change against basic use cases:
#### When running `cargoUpdateAllLockfiles`, dependencies will be
updated to their latest versions, while broken crates will be pinned to
the last known good versions.
<details>
<summary> Expand for more details...</summary>
When we execute
```
smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ ./gradlew aws:sdk:cargoUpdateAllLockfiles
...
BUILD SUCCESSFUL in 1m 7s
```
all lockfiles include the latest versions of dependencies, except for
those that are pinned due to being broken. Currently, minicbor is
[pinned to
0.24.2](7f1d992214/aws/sdk/build.gradle.kts (L503-L504)):
```
➜ smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ git status
On branch ysaito/enhance-gradle-tasks-for-lockfile
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: aws/rust-runtime/Cargo.lock
modified: aws/rust-runtime/aws-config/Cargo.lock
modified: aws/sdk/Cargo.lock
modified: rust-runtime/Cargo.lock
no changes added to commit (use "git add" and/or "git commit -a")
```
```
➜ smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ cat aws/sdk/Cargo.lock | rg -C1 minicbor
...
---
[[package]]
name = "minicbor"
version = "0.24.2"
---
...
➜ smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ cat rust-runtime/Cargo.lock | rg -C1 minicbor
...
---
[[package]]
name = "minicbor"
version = "0.24.2"
---
...
```
Finally, the `sdk-lockfiles audit` command should run successfully after
updating all lockfiles:
```
➜ smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ sdk-lockfiles audit
2024-09-12T15:35:47.890530Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:35:47.922468Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:35:47.922898Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS
```
I also specified multiple broken dependencies and verified they were all
downgraded to the specified versions.
</details>
#### When a new dependency is added to a runtime crate, running
`syncAwsSdkLockfile` will ensure that this new dependency is included in
the SDK lockfile.
<details>
<summary> Expand for more details...</summary>
For instance, with [this hypothetical new
dependency](https://github.com/smithy-lang/smithy-rs/pull/3826/files#diff-1ff3734bb74b7c43e3bd74b410f7058c6d40dbe9380458f642201035f9217457):
```
smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ sdk-lockfiles audit
2024-09-12T15:40:52.795951Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:40:52.827407Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:40:52.827835Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
`jiff` (0.1.13), used by `rust-runtime/Cargo.lock`, is not contained in SDK lockfile!
Error: there are lockfile audit failures
```
If we then execute
```
smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ ./gradlew aws:sdk:syncAwsSdkLockfile
...
BUILD SUCCESSFUL in 1m 17s
```
the SDK lockfile will be updated to reflect only the change from
`rust-runtime/Cargo.lock`:
```
smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ git diff aws/sdk/Cargo.lock
diff --git a/aws/sdk/Cargo.lock b/aws/sdk/Cargo.lock
index bc3870e20..c52040432 100644
--- a/aws/sdk/Cargo.lock
+++ b/aws/sdk/Cargo.lock
@@ -1627,6 +1627,7 @@ dependencies = [
"aws-smithy-types 1.2.6",
"chrono",
"futures-core",
+ "jiff",
"time",
]
@@ -2895,6 +2896,12 @@ version = "1.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49f1f14873335454500d59611f1cf4a4b0f786f9ac11f4312a78e4cf2566695b"
+[[package]]
+name = "jiff"
+version = "0.1.13"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8a45489186a6123c128fdf6016183fcfab7113e1820eb813127e036e287233fb"
+
[[package]]
name = "jobserver"
version = "0.1.32"
(END)
```
The updated SDK lockfile should now be in sync with runtime crates:
```
➜ smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ sdk-lockfiles audit
2024-09-12T15:41:28.004702Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:41:28.034118Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:41:28.034555Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS
```
</details>
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
* https://github.com/smithy-lang/smithy-rs/issues/1925
* https://github.com/awslabs/aws-sdk-rust/issues/977
## Description
Deprecate http-02x APIs from inlineable `PresignedRequest` API. These
should have been feature gated originally but they weren't. For now
we'll mark them deprecated and encourage people to move to the 1.x
equivalents.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x ] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Description
This PR updates lockfiles by running:
```
./gradlew aws:sdk:generateAllLockfiles -Paws-sdk-rust-path=/Users/awsaito/src/aws-sdk-rust
```
However, due to [the minicbor
issue](https://github.com/smithy-lang/smithy-rs/pull/3818), we still pin
it to 0.24.2 by running the following on `rust-runtime/Cargo.lock` and
`aws/sdk/Cargo.lock`:
```
RUSTFLAGS="--cfg aws_sdk_unstable" cargo update -p minicbor --precise 0.24.2
```
The rest of the changes handle miscellaneous scenarios:
- **Updated nightly version**: Upgraded to `nightly-2024-03-15` to
address a [compatibility
issue](https://github.com/serde-rs/serde/issues/2770) introduced by the
updated serde library.
- **Cleaned up use statements**: Removed redundant use statements, which
were flagged as warnings by the new nightly version.
## Testing
Tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Restores `SmokeTestsDecoratorTest` that was temporarily removed in
https://github.com/smithy-lang/smithy-rs/pull/3808.
## Description
The said test was temporarily removed because `SmokeTestsDecorator` is
included in the [predefined
decorators](b38ccb969e/aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/AwsCodegenDecorator.kt (L35))
and pulls-in the `aws-config` crate for code generation. The issue was
conflicts between the runtime crates used by `aws-config` (located in
the `aws/sdk/build` directory) and those brought-in by that predefined
decorators used by `awsSdkIntegrationTest` (located in the
`rust-runtime` and `aws/rust-runtime` directories). This PR addresses
these conflicts and restores the test functionality.
Given the challenges, the code changes are centered around the following
ideas:
- Focus solely on testing the core class, `SmokeTestsInstantiator`. By
doing this, we avoid running `SmokeTestsDecorator`, which would
otherwise pull in the `aws-config` crate.
- Initializing a config builder in smoke tests needs to be parameterized
depending on the environment; in production we use
`aws_config::load_defaults` and in testing we avoid including
`aws-config` by using a default-constructed config builder, which is
sufficient for compile-only tests.
- The generated smoke tests in `SmokeTestsDecoratorTest` require minimal
runtime crates for compilation. We need the `CodegenVisitor` but with a
different set of codegen decorators. Prior to this PR,
`awsSdkIntegrationTest` used
[RustClientCodegenPlugin](b38ccb969e/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/RustClientCodegenPlugin.kt (L46))
that in turn loaded [predefined
decorators](b38ccb969e/aws/sdk-codegen/src/main/kotlin/software/amazon/smithy/rustsdk/AwsCodegenDecorator.kt (L35))
on the classpath, which included `SmokeTestsDecorator`. In this PR, we
work around this by defining a minimal set of codegen decorators and
making them pluggable through `awsSdkIntegrationTest`.
## Testing
Tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR addresses an issue where, if an operation's input includes an
event streaming member, the builder for the operation input or output
may raise a `ConstraintViolation` when `build_enforcing_all_constraints`
is called and the event streaming member field is not set. This occurs
because the member shape is required.
The standard error message that is shown when `ValidationException` is
not attached to an operation is also displayed in this case:
*Operation test#TestOperation takes in input that is constrained
(https://awslabs.github.io/smithy/2.0/spec/constraint-traits.html), and
as such can fail with a validation exception. You must model this
behavior in the operation shape in your model file.*
```smithy
use smithy.framework#ValidationException
operation TestOperation {
...
errors: [..., ValidationException] // <-- Add this.
}
```
Closes: [3813](https://github.com/smithy-lang/smithy-rs/issues/3813)
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
**Will merge to the release branch**
## Motivation and Context
Fixes a failure observed in our release pipeline
## Description
A lockfile located at `aws/sdk/Cargo.lock` did not include the
`minicbor` crate that was introduced as part of RPC V2 CBOR. This has
caused a build failure in our release pipeline due to a new version of
`minicbor` 0.24.3 uploaded to crates.io.
```
error: unsupported output in build script of `minicbor v0.24.3`: `cargo::rustc-check-cfg=cfg(atomic64, atomic32)`
```
To address this issue, this PR pins `minicbor` to 0.24.2 in
`aws/sdk/Cargo.lock` (`rust-runtime/Cargo.lock` already pins it to
0.24.2)
The change in `aws/sdk/Cargo.lock` was obtained by
1. running `git pull` in `/Users/awsaito/src/aws-sdk-rust`
2. running `./gradlew aws:sdk:generateAllLockfiles
-Paws-sdk-rust-path=/Users/awsaito/src/aws-sdk-rust`
3. retaining the portion only relevant to `minicbor`
In addition, since the release of aws-sdk-rust the other day, smoke
tests have started getting rendered and compiled in cargo-semver-checks.
This has caused cargo-semver-checks to exceed the previous timeout of 20
minutes. To address it, we have increased the timeout to 30 minutes.
## Testing
- [ ] Tests in CI
- [x] End-to-end tests in our release pipeline
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
- #3805
## Description
Recover from floating point values extremely close to whole numbers by
incrementing the epoch seconds value.
## Testing
- proptest
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
The Server SDK requires the model to include
`aws.smithy.framework#ValidationException` in each operation that can
access a constrained member shape. This becomes problematic when
generating the server SDK for a model not owned by the team creating the
SDK, as they cannot easily modify the model.
This PR introduces a codegen flag,
`addValidationExceptionToConstrainedOperations`. When set in
`smithy-build-template.json`, this flag will automatically add
`ValidationException` to operations that require it but do not already
list it among their errors.
Closes Issue:
[3802](https://github.com/smithy-lang/smithy-rs/issues/3802)
Sample `smithy-build-template.json`
```
"plugins": {
"rust-server-codegen": {
"service": "ServiceToGenerateSDKFor",
"module": "amzn-sample-server-sdk",
"codegen": {
"addValidationExceptionToConstrainedOperations": true,
}
}
}
```
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
## Motivation and Context
Attempts to fix [a release
blocker](https://github.com/smithy-lang/smithy-rs/actions/runs/10601508703/job/29381493978)
## Description
This PR ensures that `SmokeTestsDecorator` renders test functions into
`smoketests` in a consistent order, e.g. sorted first by operation name
and then within that operation, sorted by test case ID.
## Testing
- [x] Existing tests in CI (specifically `check-deterministic-codegen`)
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Follow-up on https://github.com/smithy-lang/smithy-rs/pull/3799
## Description
This PR updates the smoke test rendering process for services whose
models specify the `smokeTests` trait
([example](c349073b34/aws-models/sns.json (L3392-L3404))).
Previously, `SmokeTestsDecorator` skipped rendering when
`useAccountIdRouting` was set to true in the vendor parameters, which is
the default for `AwsVendorParams`. Even though the Rust SDK does not
currently support the account ID routing, it is safe to render these
tests. This is because existing smoke tests do not rely on this feature,
and when account ID routing is used, the Rust SDK will fall back to
other means to sourcing the identity.
This PR includes a minor fix: it uses `aws_config` to load default
configurations. `config::Builder::new()` would have no credentials
provider chain available, which would then result in test failures at
runtime.
```
---- test_list_certificates_success stdout ----
thread 'test_list_certificates_success' panicked at sdk/acm/tests/smoketests.rs:22:9:
request should succeed: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source: NoMatchingAuthSchemeError(ExploredList { items: [ExploredAuthOption { scheme_id: AuthSchemeId { scheme_id: "sigv4" }, result: NoIdentityResolver }], truncated: false }), connection: Unknown } })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
By bringing `aws-config`, the `SmokeTestsDecoratorTest` no longer works,
since the AWS runtime crates required by `aws-config` conflict with
those generated by `awsSdkIntegrationTest` (a known limitation). For
example:
```
error: package collision in the lockfile: packages aws-credential-types v1.2.0 (smithy-rs/aws/rust-runtime/aws-credential-types) and aws-credential-types v1.2.0 (smithy-rs/aws/sdk/build/aws-sdk/sdk/aws-credential-types) are different, but only one can be written to lockfile unambiguously
```
Therefore, we removed `SmokeTestsDecoratorTest.kt` _only_ in this PR to
ship the feature, but are planning to restore it in the next PR by
directly testing `SmokeTestsInstantiator` without using
`awsSdkIntegrationTest`.
## Testing
- [x] Existing tests in CI
- [x] Executed and passed smoke tests for services that support them in
our release pipeline
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Upgrades MSRV to 1.78.0 since the [latest
Rust](https://www.rust-lang.org/) at the time of writing is 1.80.1 (we
try keeping [two releases
behind](https://github.com/awslabs/aws-sdk-rust?tab=readme-ov-file#supported-rust-versions-msrv)).
## Description
In Rust 1.78.0, `clippy` enforces stricter checks in two areas:
- `dead_code`
It seems to be able to uncover more unused code patterns, especially
when different combinations of cargo features are enabled
- `empty_docs`
Prior to 1.78.0, the codegen used to render the following empty docs
`///`, regardless whether the codegen is for the server or for the
client. This is now flagged as `empty_docs` by `clippy`:
```
///
pub(crate) mod XXX {
...
}
```
Other than bumping MSRV to 1.78.0, the remaining code changes address
issues related to the fixes mentioned above.
**EDIT:**
`cargo-semver-checks` required an upgrade due to incompatibility issues
following the MSRV upgrade ([example
error](https://github.com/smithy-lang/smithy-rs/actions/runs/10534205294/job/29261105195#step:4:2068)).
Specifically, after updating the MSRV to 1.78.0, the rustdoc version was
upgraded to v28. However, the version of `cargo-semver-checks` we were
using (0.24.1) only supported rustdoc [up to
v27](4bce03d4a6/Cargo.toml (L18)).
To resolve this, we have updated `cargo-semver-checks` to the first
version that supports rustdoc
[v28](94a491f085/Cargo.toml (L18)).
## Testing
Existing tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR restores [an environment
variable](62ff67ed89 (diff-903f493806a39cd49c26728eee0410545eb911162b1d889dd7154eae81b7c7c7))
we attempted to add in
https://github.com/smithy-lang/smithy-rs/pull/3758.
## Description
We removed that environment variable since we ran into an error
`ENABLE_SMOKETESTS: unbound variable` in our release pipeline. This PR
will have a default value `false` when `ENABLE_SMOKETESTS` is not
present.
We expect the value to be `false` in smithy-rs CI and `true` in our
release pipeline (can set it to `true` in our release pipeline
asynchronously).
## Testing
- Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Fixes the execution order of
[modify_before_serialization](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/interceptors/trait.Intercept.html#method.modify_before_serialization)
and
[read_before_serialization](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/interceptors/trait.Intercept.html#method.read_before_serialization)
in the orchestrator.
## Description
The fix ensures that the execution order of these interceptor methods
aligns with our orchestrator design. While customers may see a behavior
change as a result, we don't treat this as a new `BehaviorVersion` since
it is intended as a bug fix.
## Testing
Existing tests in CI
## Checklist
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
https://github.com/awslabs/aws-sdk-rust/issues/821
## Description
This PR addresses an issue with the
[Length::UpTo](https://docs.rs/aws-smithy-types/1.2.2/aws_smithy_types/byte_stream/enum.Length.html)
usage in `FsBuilder`. Previously, if a value specified for `UpTo`
exceeded the remaining file length, `FsBuilder` would incorrectly accept
the value. This discrepancy led to failures in subsequent request
dispatches, as the actual body size did not match the advertised
`Content-Length`, as explained
[here](https://github.com/awslabs/aws-sdk-rust/issues/821#issuecomment-1937354372)
(thank you @pablosichert for a self-contained reproducer and problem
analysis!).
## Testing
- Added a unit test for `FsBuilder` verifying the `Length::UpTo` usage
- Ran successfully a customer provided
[reproducer](https://github.com/awslabs/aws-sdk-rust/issues/821#issuecomment-1937354372)
with the code changes in this PR (with an added a call to
`complete_multipart_upload` at the end, it also succeeded in uploading
the object):
```
upload_id: cTDSngbubD25cOoFCNgjpG55o0hAMQNjO16dNFyNTKjg9PEtkcrKG5rTGzBns7CXoO8T.Qm9GpNj6jgwJTKcXDpsca95wSMWMDfPF0DBhmbk3OAGHuuGM1E70spk2suW
File created
part_number: 1, e_tag: "5648ddf58c7c90a788d7f16717a61b08"
part_number: 2, e_tag: "a6bdad6d65d18d842ef1d57ca4673bc3"
part_number: 3, e_tag: "f518f6b19b255ec49b61d511288554fc"
part_number: 4, e_tag: "1496524801eb1d0a7cfbe608eb037d9c"
part_number: 5, e_tag: "21340de04927ce1bed58ad0375c03e01"
```
## Checklist
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
https://github.com/smithy-lang/smithy-rs/issues/1925
## Description
Backports connection poisoning that hyper 0.14 HTTP client has to the
hyper 1.x client.
See also:
* upstream support: https://github.com/hyperium/hyper-util/pull/121
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [X] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
https://github.com/awslabs/aws-sdk-rust/issues/820
## Description
This PR re-exports `ByteStream`'s `Length` and `FsBuilder`. By making
these types available in a client crate, customers can use
`ByteStream::read_from` without needing to import them separately from
the `aws-smithy-types` crate.
## Testing
- Updated an existing re-export test to include `Length` and `FsBuilder`
as well.
- Existing tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Version `User-Agent` header string and begin tracking business metrics
in that header
## Description
This PR versions `User-Agent` string and the version is set to `2.1`.
Furthermore, we track business metrics for SDK features in `User-Agent`
header. Specifically, we now track the following metrics in the
User-Agent header:
- RPC V2 CBOR (M)
- Gzip request compression (L)
- paginator (C)
- waiter (B)
Each letter corresponds to a metric value defined [in the
specification](3f3c874c9f/aws/rust-runtime/aws-runtime/src/user_agent/metrics.rs (L207-L226))).
#### Overall implementation strategy ####
Since business metrics is an AWS-specific concept, the
`aws-smithy-runtime` crate cannot directly reference it or call
`AwsUserAgent::add_business_metric`. Instead, the crate tracks Smithy
SDK features using the config bag with the `StoreAppend` mode. During
the execution of `UserAgentInterceptor::modify_before_signing`, this
method retrieves the SDK features from the config bag and converts them
into business metrics. This implies that any SDK features—whether
specific to Smithy or AWS—that we intend to track must be added to the
config bag prior to the invocation of the `modify_before_signing`
method.
## Testing
- Added a test-only utility function,
`assert_ua_contains_metric_values`, in the `aws-runtime` crate to verify
the presence of metric values in the `User-Agent` string. Since the
position of metric values in the `business-metrics` string may change as
new metrics are introduced (e.g., previously `m/A` but now `m/C,A,B`),
it is essential that this function accounts for potential variations and
does not rely solely on substring matching.
- Added unit and integration tests to verify tracking of the business
metrics introduced in this PR: RPC V2 CBOR, Gzip request compression,
paginator, and waiter.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
This change adds minimal support for the `AWS::Auth::AccountId` and
`AWS::Auth::AccountIdEndpointMode` endpoint built-ins. Endpoint rules
containing these builtins will generate correct code, but we do not
support user setting of these values (or any other features defined in
the spec for account id based endpoint resolution).
## Description
<!--- Describe your changes in detail -->
* Add option to endpoint builtins decorator to skip the `SdkConfig`
related codegen
* Add `AccountId` and `AccountIdEndpointMode` built-ins to the list of
generated ones
* Update the test input generator to add a `us-east-1` region if the
test provides built-ins but does not specify a region
* Add the documentation string from the test definition to the generated
test code
* Update tests for `EndpointBulitInsDecorator` to include the two new
bulit-ins
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
* Update tests for `EndpointBulitInsDecorator` to include the two new
bulit-ins
* Ran our CI against this change with the updated DDB endpoints rules
and all tests passed.
https://github.com/smithy-lang/smithy-rs/actions/runs/10423338690
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
## Motivation and Context
A preparatory PR that relaxes test verification of the connection
recording tests.
## Description
This PR is a preparatory step for upcoming changes to the `UserAgent`,
which will introduce new header values in `x-amz-user-agent`, such as
`ua/2.0` (user agent metadata) and `m/A` (business metrics).
However, the introduction of new header values will cause the following
pain points:
- we have to update many connection recording tests to make them pass
again (i.e. the very places updated in this PR)
- check for semver hazards [fail to
pass](https://github.com/smithy-lang/smithy-rs/actions/runs/10209305234/job/28247956895).
This is much the same as we encountered in [content length enforcement
tests](https://github.com/smithy-lang/smithy-rs/issues/3523). This
creates a chicken-and-egg problem: tests need to be updated for the PRs
to pass CI, but the "released SDKs" in the `aws-sdk-rust` repository
won't implement the new `UserAgent` header values until the PRs are
merged and released.
To prevent recurring issues with headers affecting connection recording
tests (hence semver checks), this PR preemptively updates the connection
recording tests. Specifically, it adjusts them to ignore certain
headers, ensuring that updates to the `x-amz-user-agent` header do not
trigger semver hazards in subsequent PRs.
**Questions**:
- This PR modifies the connection recording tests to skip verification
of the `x-amz-user-agent` and `authorization` headers. Consequently, we
no longer test the SigV4 signature match in `aws/sdk/integration-tests`.
Although we continue to run canary tests in CI, it would be beneficial
to maintain at least one integration test for verifying the correctness
of the SigV4 signature. This helps in detecting potential bugs affecting
SigV4 signature correctness early on.
To address this, I’ve added [an
awsSdkIntegrationTest](f513b924dc/aws/sdk-codegen/src/test/kotlin/software/amazon/smithy/rustsdk/SigV4AuthDecoratorTest.kt (L72))
that excludes the `UserAgentInterceptor` and checks the `Signature`
value in the `authorization` header. The question is, do we want to keep
this test? If future header updates cause semver hazards to fail, this
test would also be affected. We would then need to repeat the process we
are going through with this PR: update the test, release the change to
aws-sdk-rust, and only then can we make subsequent changes without
breaing semver hazards.
- I've removed the commented-out tests and their associated connection
recording files from `request_information_headers.rs` as part of
cleanup, since there were no explanatory comments. Let me know if we
want to restore these tests, and I will do so along with a comment
explaining their purpose.
## Testing
- Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Constrained members are extracted into standalone shapes with
constraints on them, and the original shape's targets are replaced by
the newly created shapes in the model.
This PR adds support for including error shapes in this transformation.
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
## Motivation and Context
Customers want to be able to use `serde` with Smithy models. For
details, see the
[RFC](https://github.com/smithy-lang/smithy-rs/blob/configurable-serde/design/src/rfcs/rfc0045_configurable_serde.md)
## Description
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Implementation of `serde::Serialize` for smithy-rs code generators. This
takes the approach of a trait, `SerializeConfigured` which _returns_ a
type that implements `Serialize`. This allows customers to control serde
behavior for their use case.
```rust
/// Trait that allows configuring serialization
/// **This trait should not be implemented directly!** Instead, `impl Serialize for ConfigurableSerdeRef<T>`**
pub trait SerializeConfigured {
/// Return a `Serialize` implementation for this object that owns the object. This is what you want
/// If you need to pass something that `impl`s serialize elsewhere.
fn serialize_owned(self, settings: SerializationSettings) -> impl Serialize;
/// Return a `Serialize` implementation for this object that borrows from the given object
fn serialize_ref<'a>(&'a self, settings: &'a SerializationSettings) -> impl Serialize + 'a;
}
```
This can be used as follows:
```rust
serde_json::to_string(&my_struct.serialize_ref(&SerializationSettings::redact_sensitive_fields());
```
Currently, this codegen plugin is not used by anything. It can be
enabled by bringing it in scope during code generation & adding the
`@serde` trait to the model for the fields where serialization is
desired. This will allow the SDK (if they choose to) roll it out only
for specific types or services that have customer demand.
There are a number of follow on items required:
- [x] Ensure the generated impls work for `Server` codegen
- [ ] Generate `Deserialize` implementations
- [x] Test the implementation all Smithy protocol test models (ensure it
compiles)
## Testing
Unit test testing serialization with a kitchen-sink model
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: david-perez <d@vidp.dev>
This improves error messaging for HTTP header related errors.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
The nonprintable characters in this source file make tools misidentify
it as a binary file, rather than source code.
## Description
**Before:**
```console
$ file rust-runtime/aws-smithy-checksums/src/http.rs
rust-runtime/aws-smithy-checksums/src/http.rs: data
```
**After:**
```console
$ file rust-runtime/aws-smithy-checksums/src/http.rs
rust-runtime/aws-smithy-checksums/src/http.rs: ASCII text
```
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Fahad Zubair