Commit Graph

2590 Commits

Author SHA1 Message Date
ysaito1001 8ccd2da8fe
Re-export `ByteStream`'s `Length` and `FsBuilder` in client crates (#3796)
## Motivation and Context
https://github.com/awslabs/aws-sdk-rust/issues/820

## Description
This PR re-exports `ByteStream`'s `Length` and `FsBuilder`. By making
these types available in a client crate, customers can use
`ByteStream::read_from` without needing to import them separately from
the `aws-smithy-types` crate.

## Testing
- Updated an existing re-export test to include `Length` and `FsBuilder`
as well.
- Existing tests in CI

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-20 21:54:41 +00:00
ysaito1001 5a19a6c504
Provide business metrics for RPC V2 CBOR, Gzip request compression, paginator, and waiter (#3793)
## Motivation and Context
Version `User-Agent` header string and begin tracking business metrics
in that header

## Description
This PR versions `User-Agent` string and the version is set to `2.1`.
Furthermore, we track business metrics for SDK features in `User-Agent`
header. Specifically, we now track the following metrics in the
User-Agent header:
- RPC V2 CBOR (M)
- Gzip request compression (L)
- paginator (C)
- waiter (B)
 
Each letter corresponds to a metric value defined [in the
specification](3f3c874c9f/aws/rust-runtime/aws-runtime/src/user_agent/metrics.rs (L207-L226))).

#### Overall implementation strategy ####
Since business metrics is an AWS-specific concept, the
`aws-smithy-runtime` crate cannot directly reference it or call
`AwsUserAgent::add_business_metric`. Instead, the crate tracks Smithy
SDK features using the config bag with the `StoreAppend` mode. During
the execution of `UserAgentInterceptor::modify_before_signing`, this
method retrieves the SDK features from the config bag and converts them
into business metrics. This implies that any SDK features—whether
specific to Smithy or AWS—that we intend to track must be added to the
config bag prior to the invocation of the `modify_before_signing`
method.

## Testing
- Added a test-only utility function,
`assert_ua_contains_metric_values`, in the `aws-runtime` crate to verify
the presence of metric values in the `User-Agent` string. Since the
position of metric values in the `business-metrics` string may change as
new metrics are introduced (e.g., previously `m/A` but now `m/C,A,B`),
it is essential that this function accounts for potential variations and
does not rely solely on substring matching.
- Added unit and integration tests to verify tracking of the business
metrics introduced in this PR: RPC V2 CBOR, Gzip request compression,
paginator, and waiter.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-20 21:20:27 +00:00
Landon James 3ee5dcbdc0
Adding minimal support for accountId related endpoint rules built-ins (#3792)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
This change adds minimal support for the `AWS::Auth::AccountId` and
`AWS::Auth::AccountIdEndpointMode` endpoint built-ins. Endpoint rules
containing these builtins will generate correct code, but we do not
support user setting of these values (or any other features defined in
the spec for account id based endpoint resolution).

## Description
<!--- Describe your changes in detail -->
* Add option to endpoint builtins decorator to skip the `SdkConfig`
related codegen
* Add `AccountId` and `AccountIdEndpointMode` built-ins to the list of
generated ones
* Update the test input generator to add a `us-east-1` region if the
test provides built-ins but does not specify a region
* Add the documentation string from the test definition to the generated
test code
* Update tests for `EndpointBulitInsDecorator` to include the two new
bulit-ins

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
* Update tests for `EndpointBulitInsDecorator` to include the two new
bulit-ins
* Ran our CI against this change with the updated DDB endpoints rules
and all tests passed.
https://github.com/smithy-lang/smithy-rs/actions/runs/10423338690

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] For changes to the smithy-rs codegen or runtime crates, I have
created a changelog entry Markdown file in the `.changelog` directory,
specifying "client," "server," or both in the `applies_to` key.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
2024-08-19 18:56:12 +00:00
AWS SDK Rust Bot 3f3c874c9f
Merge smithy-rs-release-1.x.y into main (#3791) 2024-08-15 18:38:41 -07:00
AWS SDK Rust Bot 2082a6e528 Merge remote-tracking branch "origin/smithy-rs-release-1.x.y" into "merge-smithy-rs-release-1.x.y-to-main" 2024-08-16 01:03:29 +00:00
AWS SDK Rust Bot a3406b69f1 Update changelog 2024-08-16 00:59:52 +00:00
Landon James 83ef515296
Fix behavior of datetime formatting (#3790) 2024-08-15 16:57:29 -07:00
Russell Cohen 95774795e8 Fix behavior of datetime formatting 2024-08-15 19:26:39 -04:00
AWS SDK Rust Bot e294fe3d14
Merge smithy-rs-release-1.x.y into main (#3789) 2024-08-14 15:46:59 -07:00
AWS SDK Rust Bot f4e3b40a02 Update changelog 2024-08-14 22:08:16 +00:00
ysaito1001 8a78e6e243
Make connection recording tests less senstive to semver hazards (#3786)
## Motivation and Context
A preparatory PR that relaxes test verification of the connection
recording tests.

## Description
This PR is a preparatory step for upcoming changes to the `UserAgent`,
which will introduce new header values in `x-amz-user-agent`, such as
`ua/2.0` (user agent metadata) and `m/A` (business metrics).

However, the introduction of new header values will cause the following
pain points:
- we have to update many connection recording tests to make them pass
again (i.e. the very places updated in this PR)
- check for semver hazards [fail to
pass](https://github.com/smithy-lang/smithy-rs/actions/runs/10209305234/job/28247956895).
This is much the same as we encountered in [content length enforcement
tests](https://github.com/smithy-lang/smithy-rs/issues/3523). This
creates a chicken-and-egg problem: tests need to be updated for the PRs
to pass CI, but the "released SDKs" in the `aws-sdk-rust` repository
won't implement the new `UserAgent` header values until the PRs are
merged and released.

To prevent recurring issues with headers affecting connection recording
tests (hence semver checks), this PR preemptively updates the connection
recording tests. Specifically, it adjusts them to ignore certain
headers, ensuring that updates to the `x-amz-user-agent` header do not
trigger semver hazards in subsequent PRs.

**Questions**:
- This PR modifies the connection recording tests to skip verification
of the `x-amz-user-agent` and `authorization` headers. Consequently, we
no longer test the SigV4 signature match in `aws/sdk/integration-tests`.
Although we continue to run canary tests in CI, it would be beneficial
to maintain at least one integration test for verifying the correctness
of the SigV4 signature. This helps in detecting potential bugs affecting
SigV4 signature correctness early on.
To address this, I’ve added [an
awsSdkIntegrationTest](f513b924dc/aws/sdk-codegen/src/test/kotlin/software/amazon/smithy/rustsdk/SigV4AuthDecoratorTest.kt (L72))
that excludes the `UserAgentInterceptor` and checks the `Signature`
value in the `authorization` header. The question is, do we want to keep
this test? If future header updates cause semver hazards to fail, this
test would also be affected. We would then need to repeat the process we
are going through with this PR: update the test, release the change to
aws-sdk-rust, and only then can we make subsequent changes without
breaing semver hazards.

- I've removed the commented-out tests and their associated connection
recording files from `request_information_headers.rs` as part of
cleanup, since there were no explanatory comments. Let me know if we
want to restore these tests, and I will do so along with a comment
explaining their purpose.

## Testing
- Existing tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-14 19:12:44 +00:00
Fahad Zubair 8733038a96
Include error shapes in member constraint transformation (#3787)
Constrained members are extracted into standalone shapes with
constraints on them, and the original shape's targets are replaced by
the newly created shapes in the model.

This PR adds support for including error shapes in this transformation.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-08-14 10:22:11 +00:00
Russell Cohen a107c0154a
Add initial implementation of a Serde Decorator (#3753)
## Motivation and Context
Customers want to be able to use `serde` with Smithy models. For
details, see the
[RFC](https://github.com/smithy-lang/smithy-rs/blob/configurable-serde/design/src/rfcs/rfc0045_configurable_serde.md)

## Description
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Implementation of `serde::Serialize` for smithy-rs code generators. This
takes the approach of a trait, `SerializeConfigured` which _returns_ a
type that implements `Serialize`. This allows customers to control serde
behavior for their use case.

```rust
/// Trait that allows configuring serialization
/// **This trait should not be implemented directly!** Instead, `impl Serialize for ConfigurableSerdeRef<T>`**
pub trait SerializeConfigured {
    /// Return a `Serialize` implementation for this object that owns the object. This is what you want
    /// If you need to pass something that `impl`s serialize elsewhere.
    fn serialize_owned(self, settings: SerializationSettings) -> impl Serialize;

    /// Return a `Serialize` implementation for this object that borrows from the given object
    fn serialize_ref<'a>(&'a self, settings: &'a SerializationSettings) -> impl Serialize + 'a;
}
```

This can be used as follows:
```rust
serde_json::to_string(&my_struct.serialize_ref(&SerializationSettings::redact_sensitive_fields());
```


Currently, this codegen plugin is not used by anything. It can be
enabled by bringing it in scope during code generation & adding the
`@serde` trait to the model for the fields where serialization is
desired. This will allow the SDK (if they choose to) roll it out only
for specific types or services that have customer demand.

There are a number of follow on items required:
- [x] Ensure the generated impls work for `Server` codegen
- [ ] Generate `Deserialize` implementations
- [x] Test the implementation all Smithy protocol test models (ensure it
compiles)


## Testing
Unit test testing serialization with a kitchen-sink model

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: david-perez <d@vidp.dev>
2024-08-12 16:44:32 +00:00
AWS SDK Rust Bot 40d6fb9e91
Merge smithy-rs-release-1.x.y into main (#3784) 2024-08-08 11:46:43 -05:00
AWS SDK Rust Bot c1eeb4eacd Update changelog 2024-08-08 15:46:17 +00:00
Zelda Hessler 433e1a00e2
Improve HTTP header errors (#3779)
This improves error messaging for HTTP header related errors.
----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-07 22:20:08 +00:00
Fahad Zubair f796170188
Add missing imports for `kotlin.streams.toList` (#3773)
Kotlin `toList` is required for the `stream.toList` extension functions
that are used in this file.

The
[PR](https://github.com/smithy-lang/smithy-rs/pull/2544/files#diff-4444943dae2fce81f218aa052d99a5db6a584f25207d742575ff042c69f7f60a)
accidentally removed the import statement from `OperationNormalizer.kt`.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2024-08-07 15:57:59 +00:00
David Tolnay 191103c524
Delete nonprintable characters from aws-smithy-checksums test (#3778)
## Motivation and Context

The nonprintable characters in this source file make tools misidentify
it as a binary file, rather than source code.

## Description

**Before:**

```console
$ file rust-runtime/aws-smithy-checksums/src/http.rs
rust-runtime/aws-smithy-checksums/src/http.rs: data
```

**After:**

```console
$ file rust-runtime/aws-smithy-checksums/src/http.rs
rust-runtime/aws-smithy-checksums/src/http.rs: ASCII text
```

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-07 15:39:14 +00:00
David Tolnay 1ee932f7a2
Destroy history of github-pages branch (#3783)
## Motivation and Context

Keeping the history of `github-pages` is not useful, and it hogs the
output of `git log --graph --all` and other such views of the repository
history. In the event that someone would need to look at old contents of
the website, it can be regenerated using `mdbook` from any commit on
`main`.

<img
src="https://github.com/user-attachments/assets/f37630fa-92ef-4e49-a9e5-a1357a785e46"
width="550">

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-07 02:49:18 +00:00
ysaito1001 c0f817330c
Allow User Agent to accept business metrics (#3781)
## Motivation and Context
This PR lays the groundwork for emitting and tracking business metrics
related to features like waiters, request compression, and S3 express
within the Rust SDKs. It introduces `BusinessMetric` to the `user_agent`
module and updates `AwsUserAgent` to support its integration.

## Description
While establishing the groundwork for the said motivation, this PR does
not send metrics from features yet.
The PR deprecates existing feature and config metadata, which are now
superseded by the new business metrics.

Related tasks include
- implementing versioning for the user agent string format, e.g.
`ua/2.0` (note that including a user agent version string to
`x-amz-user-agent` could cause many connection recording tests to fail
and may also break semver check in CI just like the [content-length
enforcement test](https://github.com/smithy-lang/smithy-rs/pull/3620)
needed to be disabled to work around it)
- sending metrics from recently implement features, maybe starting with
request compression and RPC V2 Cbor (this will also modify
`x-amz-user-agent` in connection recording tests, so the argument above
applies)

## Testing
- Added unit tests for `BusinessMetric` and `AwsUserAgent`
- Existing tests in CI

## Checklist
- [x] For changes to the AWS SDK, generated SDK code, or SDK runtime
crates, I have created a changelog entry Markdown file in the
`.changelog` directory, specifying "aws-sdk-rust" in the `applies_to`
key.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-06 03:02:05 +00:00
david-perez bb1971a1ba
Publish "Detailed error explanations" design documentation page (#3780)
The page was added back in #3734, but was not added to the index, so

https://smithy-lang.github.io/smithy-rs/design/client/detailed_error_explanations.html,
which is exposed here:


10a92052b1/rust-runtime/aws-smithy-runtime-api/src/client/connection.rs (L24-L27)

currently points to a 404.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-02 15:18:53 +00:00
ysaito1001 6a602d434b
Update `CODEOWNERS` for the changelog directory (#3777)
## Motivation and Context
Removes `CHANGELOG.next.toml` from `CODEOWNERS` since it no longer
exists. The file now lists the `.changelog` directory instead under
`Shared ownership`.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-02 14:44:10 +00:00
ysaito1001 10a92052b1
Switch to file-per-change changelog (#3774)
## Motivation and Context
Implements [RFC: File-per-change
changelog](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#rfc-file-per-change-changelog)

## Description
When this PR is merged to `main`, it will impact developer workflow for
creating a changelog entry that goes with a PR.
Prior to this, one edited `CHANGELOG.next.toml` to enter a changelog
entry, but it often caused merge conflicts among us. With this PR, one
creates a `.md` file (any filename stem will do) under the
`smithy-rs/.changelog` directory. This Markdown file has the YAML front
matter, and it can be created manually or via the `ChangeLogger` CLI
(see [this
commit](83cda76608)
for more details).

Here are the highlights of the code changes:
- [The release
script](https://github.com/smithy-lang/smithy-rs/pull/3774/files#diff-162dc2b82eb3d72b105291d2fc5fc8d10c4fbc1e6f35862bc08ac0d6a7c53f76)
has been pointed to `smithy-rs/.changelog`. This is pretty much what it
takes to let the `smithy-rs` release infra switch to the new changelog
mode.
-
[sdk-lints](efcc772b37)
now disallows `CHANGELOG.next.toml`. It will complain if the file is
present.
- The `--source-to-truncate` option for the `changelogger`'s `render` is
now optional because our internal release infra code no longer needs to
specify that option (i.e. there is nothing to truncate there).
- Tests in `smithy-rs-tool-common` and `changelogger` have been updated
so TOML changelog entries used in tests have been re-written to Markdown
changelog entries. We want to avoid giving the false impression that
TOML changelog entries are still well-supported.

## Testing
- Existing tests & lints in CI
- Ran `smithy-rs` [dry-run
release](https://github.com/smithy-lang/smithy-rs/actions/runs/10100442953)
and confirmed the correctness of `CHANGELOG.md` and
`SDK_CHANGELOG.next.json` in the release artifacts
- Ran the internal release and confirmed `CHANGELOG.md` was rendered
correctly in `aws-sdk-rust`

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-01 04:31:19 +00:00
ysaito1001 63753f3fc7
Add new subcommands to `changelogger` for supporting file-per-change changelog (#3771)
## Motivation and Context
Adds new subcommands `--new` and `--ls` to `changelogger`

## Description
This is part 2 in a series of PRs supporting [file-per-change
changelog](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html).
This PR just adds utilities for humans and does NOT impact the current
dev workflow, smithy-rs CI, or our release pipeline.

#### A subcommand `--new`
We can use this subcommand when creating a new changelog entry markdown
file. An example usage:
```
$ changelogger new -t client -t aws-sdk-rust -r smithy-rs#1234 -a someone --bug-fix -m "Some changelog" \
# for long flags, replace -t with --applies-to, -r with --references, -a with --authors, and -m with --message \
# also remember to escape with \ when including backticks in the message at command line, e.g. \`pub\`

The following changelog entry has been written to "/Users/ysaito1001/src/smithy-rs/.changelog/5745197.md":
---
applies_to:
- aws-sdk-rust
- client
authors:
- someone
references:
- smithy-rs#1234
breaking: false
new_feature: false
bug_fix: true
---
Some changelog
```
The following CLI arguments are "logically" required
- `--applies-to`
- `--ref`
- `--author`
- `--message`

If any of the above is not passed a value at command line, then an
editor is opened for further edit (which editor to open can be
configured per the [edit crate](https://docs.rs/edit/0.1.5/edit/)).

Note that the YAML syntax above is different from the single line array
syntax [proposed in the
RFC](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#the-proposed-developer-experience).
This is due to [a
limitation](https://github.com/dtolnay/serde-yaml/issues/355) in
`serde-yaml` (which has been archived unfortunately), but the multi-line
values are still a valid YAML syntax and, most importantly, rendering
changelong entries will continue working. For now, I won't post-process
from the multi-line values syntax to the single line array syntax. One
can work around this by handwriting a changelog entry Markdown file in
`smithy-rs/.changelog` without using this subcommand.

#### A subcommand `--ls`
This subcommand will render a preview of changelog entries stored in
`smithy-rs/.changelog` and print it to the standard output. Example
usages (using the entry created at 5745197.md above):
```
$ changelogger ls -c smithy-rs \
# for long flag, replace -c with --change-set

Next changelog preview for smithy-rs
=====================================
**New this release:**
- 🐛 (client, [smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234), @someone) Some changelog

**Contributors**
Thank you for your contributions! ❤
- @someone ([smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234))
```
```
$ changelogger ls -c aws-sdk \
# for long flag, replace -c with --change-set

Next changelog preview for aws-sdk-rust
========================================
**New this release:**
- 🐛 ([smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234), @someone) Some changelog

**Contributors**
Thank you for your contributions! ❤
- @someone ([smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234))
```


## Testing
- Existing tests in CI
- Basic unit tests for subcommands

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-31 21:31:37 +00:00
ysaito1001 36b50b3dac
Add client-support for RPC v2 CBOR (#3767)
## Motivation and Context
Follow-up on https://github.com/smithy-lang/smithy-rs/pull/2544 to add
client-side support for the protocol

## Description
The client implementation mainly focuses on a sub-section
[Requests](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#requests)
in the spec. To that end, this PR addresses `TODO` for the client to
fill in the blanks and includes additional adjustments/refactoring to
pass client protocol tests.

## Testing
- Existing tests in CI
- Upstream protocol test `rpcv2Cbor`
- Our handwritten protocol test `rpcv2Cbor-extras.smithy`

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-31 19:22:03 +00:00
Fahad Zubair 50148e6983
Use `serverIntegrationTest` instead of local server invocation function (#3775)
Member constraint shape tests were using a server codegen invocation
function which was very similar to `serverIntegrationTest`. This PR
removes that and uses `serverIntegrationTest` instead.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-31 11:05:10 +00:00
Zelda Hessler 507ebe8ac6
v2 Smoketest codegen (#3758)
This PR adds codegen for service-defined smoketests.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-30 16:07:50 +00:00
Jack Kleeman 1223f61e10
Remove unused hyper dependency in aws-config (#3770)
This allows end-users to keep hyper 0.x out of their tree

## Motivation and Context
This dependency appears unused and means that I have a 0.14 import in my
tree

## Description
Removes the dependency

## Testing
Builds locally, if CI passes I think its fair to say this is a no-op. I
can't find any usages of this lib.

## Checklist
I'm not sure this change warrants a changelog as there is no user facing
change

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-19 17:58:48 +00:00
Landon James 3881938099
Reverting some accidentally ressurected CHANGELOG entries (#3768)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

## Description
<!--- Describe your changes in detail -->

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 19:52:44 +00:00
Landon James c39792b3d8
Fix maps/lists with sensitive members not redacted (#3765)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Lists and maps with sensitive members were not being properly redacted.
Closes https://github.com/smithy-lang/smithy-rs/issues/3757

## Description
<!--- Describe your changes in detail -->
Updated `Shape.shouldRedact` method to properly check for list and map
shapes with sensitive members. Also updated the test definition so it
would actually run since previously the test code was generated in a
nested function inside a no-op function and never run.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added test cases for lists with sensitive members, maps with sensitive
keys, and maps with sensitive values.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 18:01:04 +00:00
david-perez c63e792454
Add server RPC v2 CBOR support (#2544)
RPC v2 CBOR is a new protocol that ~is being added~ has [recently been
added](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html)
to the Smithy
specification.

_(I'll add more details here as the patchset evolves)_

Credit goes to @jjant for initial implementation of the router, which I
built on top of from his
[`jjant/smithy-rpc-v2-exploration`](https://github.com/awslabs/smithy-rs/tree/jjant/smithy-rpc-v2-exploration)
branch.

Tracking issue: https://github.com/smithy-lang/smithy-rs/issues/3573.

## Caveats

`TODO`s are currently exhaustively sprinkled throughout the patch
documenting what remains to be done. Most of these need to be addressed
before this can be merged in; some can be punted on to not make this PR
bigger.

However, I'd like to call out the major caveats and blockers here. I'll
keep updating this list as the patchset evolves.

- [x] RPC v2 has still not been added to the Smithy specification. It is
currently being worked on over in the
[`smithy-rpc-v2`](https://github.com/awslabs/smithy/tree/smithy-rpc-v2)
branch. The following are prerrequisites for this PR to be merged;
**until they are done CI on this PR will fail**:
    - [x] Smithy merges in RPC v2 support.
    - [x] Smithy releases a new version incorporating RPC v2 support.
- Released in [Smithy
v1.47](https://github.com/smithy-lang/smithy/releases/tag/1.47.0)
    - [x] smithy-rs updates to the new version.
        - Updated in https://github.com/smithy-lang/smithy-rs/pull/3552
- [x] ~Protocol tests for the protocol do not currently exist in Smithy.
Until those get written~, this PR resorts to Rust unit tests and
integration tests that use `serde` to round-trip messages and compare
`serde`'s encoders and decoders with ours for correctness.
- Protocol tests are under the
[`smithy-protocol-tests`](https://github.com/smithy-lang/smithy/tree/main/smithy-protocol-tests/model/rpcv2Cbor)
directory in Smithy.
- We're keeping the `serde_cbor` round-trip tests for defense in depth.
- [ ] https://github.com/smithy-lang/smithy-rs/issues/3709 - Currently
only server-side support has been implemented, because that's what I'm
most familiar. However, we're almost all the way there to add
client-side support.
- ~[ ] [Smithy `document`
shapes](https://smithy.io/2.0/spec/simple-types.html#document) are not
supported. RPC v2's specification currently doesn't indicate how to
implement them.~
- [The
spec](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#shape-serialization)
ended up leaving them as unsupported: "Document types are not currently
supported in this protocol."

## Prerequisite PRs

This section lists prerequisite PRs and issues that would make the diff
for this one lighter or easier to understand. It's preferable that these
PRs be merged prior to this one; some are hard prerequisites. They
mostly relate to parts of the codebase I've had to touch or ~pilfer~
inspect in this PR where I've made necessary changes, refactors and
"drive-by improvements" that are mostly unrelated, although some
directly unlock things I've needed in this patchset. It makes sense to
pull them out to ease reviewability and make this patch more
semantically self-contained.

- https://github.com/awslabs/smithy-rs/pull/2516
- https://github.com/awslabs/smithy-rs/pull/2517
- https://github.com/awslabs/smithy-rs/pull/2522
- https://github.com/awslabs/smithy-rs/pull/2524
- https://github.com/awslabs/smithy-rs/pull/2528
- https://github.com/awslabs/smithy-rs/pull/2536
- https://github.com/awslabs/smithy-rs/pull/2537
- https://github.com/awslabs/smithy-rs/pull/2531
- https://github.com/awslabs/smithy-rs/pull/2538
- https://github.com/awslabs/smithy-rs/pull/2539
- https://github.com/awslabs/smithy-rs/pull/2542
- https://github.com/smithy-lang/smithy-rs/pull/3684
- https://github.com/smithy-lang/smithy-rs/pull/3678
- https://github.com/smithy-lang/smithy-rs/pull/3690
- https://github.com/smithy-lang/smithy-rs/pull/3713
- https://github.com/smithy-lang/smithy-rs/pull/3726
- https://github.com/smithy-lang/smithy-rs/pull/3752

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

~RPC v2 has still not been added to the Smithy specification. It is
currently being worked on over in the
[`smithy-rpc-v2`](https://github.com/awslabs/smithy/tree/smithy-rpc-v2)
branch.~

This can only be tested _locally_ following these steps:

~1. Clone [the Smithy
repository](https://github.com/smithy-lang/smithy/tree/smithy-rpc-v2)
and checkout the `smithy-rpc-v2` branch.
2. Inside your local checkout of smithy-rs pointing to this PR's branch,
make sure you've added `mavenLocal()` as a repository in the
`build.gradle.kts` files.
[Example](8df82fd3fc).
4. Inside your local checkout of Smithy's `smithy-rpc-v2` branch:
1. Set `VERSION` to the current Smithy version used in smithy-rs (1.28.1
as of writing, but [check
here](https://github.com/awslabs/smithy-rs/blob/main/gradle.properties#L21)).
    2. Run `./gradlew clean build pTML`.~
~6.~ 1. In your local checkout of the smithy-rs's `smithy-rpc-v2`
branch, run `./gradlew codegen-server-test:build -P
modules='rpcv2Cbor'`.

~You can troubleshoot whether you have Smithy correctly set up locally
by inspecting
`~/.m2/repository/software/amazon/smithy/smithy-protocols-traits`.~

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 09:50:52 +00:00
AWS SDK Rust Bot e4ced33954
Merge smithy-rs-release-1.x.y into main (#3766) 2024-07-16 20:29:43 -05:00
AWS SDK Rust Bot de982bfd51 Merge remote-tracking branch "origin/smithy-rs-release-1.x.y" into "merge-smithy-rs-release-1.x.y-to-main" 2024-07-16 23:18:30 +00:00
AWS SDK Rust Bot 73df1e285e Update changelog 2024-07-16 23:14:53 +00:00
ysaito1001 0b059fb5e5
Add support for changelog entry markdown files (#3763)
## Motivation and Context
[RFC: File-per-change
changelog](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#rfc-file-per-change-changelog)

## Description
This PR is the first of the two implementing the proposal outlined in
the RFC, and it focuses on the initial 4 bullets in the [Changes
checklist](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#changes-checklist).
Crucially, this update does not modify the workflow for developers
regarding changelog entries (editing `CHANGELOG.next.toml` remains
necessary) or impact the `smithy-rs` CI process and our release
pipeline.

The PR introduces support for deserializing the following Markdown front
matter in the YAML format described in the RFC, e.g.
```
---
# Adding `aws-sdk-rust` here duplicates this entry into the SDK changelog.
applies_to: ["client", "server", "aws-sdk-rust"]
authors: ["author1", "author2"]
references: ["smithy-rs#1234", "aws-sdk-rust#1234"]
# The previous `meta` section is broken up into its constituents:
breaking: false
# This replaces "tada":
new_feature: false
bug_fix: false
---

Some message for the change.
```
These changelog entry Markdown files will be saved in the
`smithy-rs/.changelog` directory:
```
┌───────────┐                  
│ smithy-rs │                  
└─────┬─────┘                  
      │                        
      │    ┌───────────┐       
      ├────┤.changelog │       
           └─────┬─────┘       
                 │             
                 ├─────test1.md  
                 │ 
                 └─────test2.md 
```
For a concrete example, see a test
`end_to_end_changelog_entry_markdown_files` that directly converts from
`end_to_end_changelog` and uses Markdown files (the expected test
results remain the same).

The next PR is expected to
- add new subcommands to `changelogger` to a) create a Markdown file and
b) preview changelog entries pending to be released
- port existing `CHANGELOG.next.toml` at that time to individual
Markdown files
- update `sdk-lints` to disallow the existence of `CHANGELOG.next.toml`
- pass a directory path for `smithy-rs/.changelog` to `--source` (one
for
[split](dc970b3738/tools/ci-scripts/generate-smithy-rs-release (L22))
and one for
[render](dc970b3738/tools/ci-scripts/generate-smithy-rs-release (L22)))
and to
[--source-to-truncate](dc970b3738/tools/ci-scripts/generate-smithy-rs-release (L23C1-L23C47))

(the 4th bullet effectively does the cutover to the new changelog mode)

## Testing
- Added `parse_markdown` and `end_to_end_changelog_entry_markdown_files`
for testing new changelog format (the rest of the tests showing up in
the diff are moved from different places)
- Confirmed via [a release
dry-run](https://github.com/smithy-lang/smithy-rs/actions/runs/9947165056)
that this PR does not break the existing `smithy-rs` release process
- Confirmed that this PR does not break the use of `changelogger` in our
internal release process

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-16 16:14:16 +00:00
Fahad Zubair 8d23be1392
Bump `aws-smithy-http-server-python` version number (#3764)
The Python server relies on types defined in `aws-smithy-http-server`,
which has a new version published. A new version of the Python server
should be released to ensure there is only one `aws-smithy-http-server`
in the dependency closure of the generated crates.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-16 13:32:52 +00:00
ysaito1001 dc970b3738
Re-export `HttpRequest` and `HttpResponse` in client crates (#3762)
## Motivation and Context
https://github.com/smithy-lang/smithy-rs/issues/3591

## Description
In early days of smithy-rs, we used to re-export
[Request](https://docs.rs/aws-sdk-s3/0.25.0/aws_sdk_s3/client/customize/struct.Request.html)
and
[Response](https://docs.rs/aws-sdk-s3/0.25.0/aws_sdk_s3/client/customize/struct.Request.html)
types. When we overhauled the underlying smithy runtime from middleware
to orchestrator, we did not re-export the corresponding types
[HttpRequest](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/orchestrator/type.HttpRequest.html)
and
[HttpResponse](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/orchestrator/type.HttpResponse.html)
to client crates.

This PR will re-export them in `crate::config::http`.

## Testing
Added a new test file for `ClientRuntimeTypesReExportGenerator.kt` that
verifies re-exports.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-15 22:23:12 +00:00
Landon James e4a58c3608
Add support for `operationContextParams` Endpoints trait (#3755)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
We have to support the new [`operationContextParams`
trait](https://smithy.io/2.0/additional-specs/rules-engine/parameters.html#smithy-rules-operationcontextparams-trait)
for endpoint resolution. This trait specifies JMESPath expressions for
selecting parameter data from the operation's input type.

## Description
<!--- Describe your changes in detail -->
* Add codegen support for the [JMESPath
`keys`](https://jmespath.org/specification.html#keys) function (required
by the trait
[spec](https://smithy.io/2.0/additional-specs/rules-engine/parameters.html#smithy-rules-operationcontextparams-trait))
* Add codegen support for the trait itself. This is achieved by
generating `get_param_name` functions for each param specified in
`operationContextParams`. These functions pull the data out of the input
object and it is added to the endpoint params in the
`${operationName}EndpointParamsInterceptor`

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Updated the existing test suite for JMESPath codegen to test the `keys`
function. Updated the existing EndpointsDecoratorTest with an
`operationContextParams` trait specifying one param of each supported
type to test the codegen.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-13 00:31:21 +00:00
Fahad Zubair 2313eb9e75
Bump `serial_test` dependency to `3.1.1` (#3740)
`serial_test = "1.0.0"` has a bug where it sometimes does not honor the
`#[ignore]` attribute on a test.

This PR bumps the dependency to version `3.1.1`.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-10 16:12:44 +00:00
david-perez 4c30f00f64
Refactor determining server error type when deserializing an `@httpPayload` (#3752)
Determining the error type when deserializing an `@httpPayload` is a
protocol-specific concern, and as such should not live in
`ServerHttpBoundProtocolGenerator`, which should remain
protocol-agnostic. This commits makes that determination part of the
`ServerProtocol` interface.

As a drive-by improvement, the companion object in
`ServerHttpBoundProtocolGenerator` has also been removed, since its
members have been unused for a long time.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-10 14:31:03 +00:00
AWS SDK Rust Bot dc66ae4b67
Merge smithy-rs-release-1.x.y into main (#3751) 2024-07-09 20:04:54 +01:00
AWS SDK Rust Bot 78c50eea17 Update changelog 2024-07-09 16:34:58 +00:00
Fahad Zubair 56f4c8e947
CHANGELOG entry for PR:3378 (#3747)
Adds an entry to CHANGELOG.next.toml for the
[PR:3378](https://github.com/smithy-lang/smithy-rs/pull/3378)

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: david-perez <d@vidp.dev>
2024-07-09 13:11:10 +00:00
Gustavo Muenz dcf16ac5de
Remove attributes annotation of Builder `struct` (#3674)
The `struct` created for a Builder should not have attributes because
these structs only derive from `Debug`, `PartialEq`, `Clone` and
`Default`. None of these support attributes.

This becomes a problem if a plugin tries to add attributes to the
`metadata` field to configure the generated code for the `struct`. In
this case, the attributes will also be added to the `Builder` struct;
which is wrong.

## Motivation and Context

I'm customizing client code generation by overriding the method:

```kotlin
override fun symbolProvider(base: RustSymbolProvider): RustSymbolProvider
```

I override the `symbol.expectRustMetadata().additionalAttributes` value
and add extra values there. The generated code adds the correct
attributes for the `struct`, however, it also adds these attributes to
the `Builder`.

The `Builder` is restricted to deriving from `Debug`, `PartialEq`,
`Clone` and `Default` (see
[code](a76dc184c2/codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/BuilderGenerator.kt (L188))).

I believe this change was added by mistake in
https://github.com/smithy-lang/smithy-rs/pull/2222/.


## Description

To solve the issue, I simply remove the offending line.

<!--- Describe your changes in detail -->

## Testing

I don't expect any existing code to break, given that this is a bug. It
only manifests when someone is customizing the generated code.


<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

Co-authored-by: Gustavo Muenz <muenze@amazon.com>
2024-07-09 09:38:53 +00:00
Zelda Hessler 0c7271664b
Fix panic when merging a stalled stream log to an empty LogBuffer (#3744)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
this fixes an issue submitted by an internal user.

## Description
<!--- Describe your changes in detail -->
If the first event pushed into the log underlying stalled stream
protection, then the log manager would panic. This is because it was
expecting to merge the new event in but there was nothing to merge it
with.

To fix this, I updated the code to `push` the event when the log manager
is empty, instead of merging it.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
ran existing tests and wrote a new test exercising the
previous-panicking code.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-08 21:20:58 +00:00
Landon James 2295d5b6a8
Support `stringArray` type in endpoints params (#3742)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

## Description
<!--- Describe your changes in detail -->
Updated our endpoint rule generation to support the new `stringArray`
parameter type

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Updated the existing `EndpointsDecoratorTest` with tests for
`stringArray`

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2024-07-08 20:23:26 +00:00
david-perez 20ad88862f
AWS JSON 1.x server request specs can be `&'static str`s (#3741)
This is technically a breaking change because we stop implementing
`FromIterator<(String, S)>`.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-08 16:33:51 +00:00
Fahad Zubair 1588945cd9
Add tracing calls when an extension parameter cannot be constructed (#3378)
## Motivation and Context

When a user defines a handler function with additional parameters, these
parameters are constructed using the `FromRequest` trait implementation
specific to their types. If `FromRequest` fails to construct the
parameter – for instance, if `Extension<State>` is expected by the user
but `Extension<Arc<State>>` is actually added by the extension layer –
the server currently does not log a message indicating this construction
failure. Instead, it simply returns a 500 internal server error.

## Description

`tracing::{trace, error}` calls have been added.

## Breaking change

`FromParts<P>::Rejection` **must** implement `std::fmt::Display`.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-08 13:56:21 +00:00
ysaito1001 17545f6fc1
Upgrade Smithy to 1.50.0 (#3728)
## Motivation and Context
This PR upgrades Smithy to 1.50.0. The majority of the changes follow
`TODO` added in https://github.com/smithy-lang/smithy-rs/pull/3690.
Other than that, a few adjustments needed to be made:
- for the client
- added two failing tests `RestJsonClientPopulatesDefaultValuesInInput`
and `RestJsonClientUsesExplicitlyProvidedMemberValuesOverDefaults` to
known failing tests for the same reason
[here](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/protocol/ClientProtocolTestGenerator.kt#L72)
- added one broken test (i.e. the upstream test definition is incorrect
but our implementation is correct) to known broken tests per
([smithy#2341](https://github.com/smithy-lang/smithy/pull/2341),
[smithy-rs#3726](https://github.com/smithy-lang/smithy-rs/pull/3726#issuecomment-2199833659))
- for the server
- removed `rest-xml-extras.smithy` since
`RestXmlMustSupportParametersInContentType` is now available upstream
Smithy 1.50.0
- added the following to known failing tests (since the `awsJson1_0`
counterparts are already in the list, but we need the server team to
verify this assumption & provide additional `TODO` comments if
necessary)
    - `RestJsonServerPopulatesDefaultsWhenMissingInRequestBody`
    - `RestJsonServerPopulatesDefaultsInResponseWhenMissingInParams`,
-
`RestJsonServerPopulatesNestedDefaultValuesWhenMissingInInResponseParams`

## Testing
Existing tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2024-07-05 14:58:17 +00:00
david-perez 2bf03857f0
Adjust error message when bodies don't match in `aws-smithy-protocol-test` (#3736)
The left parameter is the expected one, see

<dc1ffb8a0d/rust-runtime/aws-smithy-protocol-test/src/lib.rs (L389-L394)>.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-04 10:45:48 +00:00