Commit Graph

2568 Commits

Author SHA1 Message Date
ysaito1001 10a92052b1
Switch to file-per-change changelog (#3774)
## Motivation and Context
Implements [RFC: File-per-change
changelog](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#rfc-file-per-change-changelog)

## Description
When this PR is merged to `main`, it will impact developer workflow for
creating a changelog entry that goes with a PR.
Prior to this, one edited `CHANGELOG.next.toml` to enter a changelog
entry, but it often caused merge conflicts among us. With this PR, one
creates a `.md` file (any filename stem will do) under the
`smithy-rs/.changelog` directory. This Markdown file has the YAML front
matter, and it can be created manually or via the `ChangeLogger` CLI
(see [this
commit](83cda76608)
for more details).

Here are the highlights of the code changes:
- [The release
script](https://github.com/smithy-lang/smithy-rs/pull/3774/files#diff-162dc2b82eb3d72b105291d2fc5fc8d10c4fbc1e6f35862bc08ac0d6a7c53f76)
has been pointed to `smithy-rs/.changelog`. This is pretty much what it
takes to let the `smithy-rs` release infra switch to the new changelog
mode.
-
[sdk-lints](efcc772b37)
now disallows `CHANGELOG.next.toml`. It will complain if the file is
present.
- The `--source-to-truncate` option for the `changelogger`'s `render` is
now optional because our internal release infra code no longer needs to
specify that option (i.e. there is nothing to truncate there).
- Tests in `smithy-rs-tool-common` and `changelogger` have been updated
so TOML changelog entries used in tests have been re-written to Markdown
changelog entries. We want to avoid giving the false impression that
TOML changelog entries are still well-supported.

## Testing
- Existing tests & lints in CI
- Ran `smithy-rs` [dry-run
release](https://github.com/smithy-lang/smithy-rs/actions/runs/10100442953)
and confirmed the correctness of `CHANGELOG.md` and
`SDK_CHANGELOG.next.json` in the release artifacts
- Ran the internal release and confirmed `CHANGELOG.md` was rendered
correctly in `aws-sdk-rust`

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-08-01 04:31:19 +00:00
ysaito1001 63753f3fc7
Add new subcommands to `changelogger` for supporting file-per-change changelog (#3771)
## Motivation and Context
Adds new subcommands `--new` and `--ls` to `changelogger`

## Description
This is part 2 in a series of PRs supporting [file-per-change
changelog](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html).
This PR just adds utilities for humans and does NOT impact the current
dev workflow, smithy-rs CI, or our release pipeline.

#### A subcommand `--new`
We can use this subcommand when creating a new changelog entry markdown
file. An example usage:
```
$ changelogger new -t client -t aws-sdk-rust -r smithy-rs#1234 -a someone --bug-fix -m "Some changelog" \
# for long flags, replace -t with --applies-to, -r with --references, -a with --authors, and -m with --message \
# also remember to escape with \ when including backticks in the message at command line, e.g. \`pub\`

The following changelog entry has been written to "/Users/ysaito1001/src/smithy-rs/.changelog/5745197.md":
---
applies_to:
- aws-sdk-rust
- client
authors:
- someone
references:
- smithy-rs#1234
breaking: false
new_feature: false
bug_fix: true
---
Some changelog
```
The following CLI arguments are "logically" required
- `--applies-to`
- `--ref`
- `--author`
- `--message`

If any of the above is not passed a value at command line, then an
editor is opened for further edit (which editor to open can be
configured per the [edit crate](https://docs.rs/edit/0.1.5/edit/)).

Note that the YAML syntax above is different from the single line array
syntax [proposed in the
RFC](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#the-proposed-developer-experience).
This is due to [a
limitation](https://github.com/dtolnay/serde-yaml/issues/355) in
`serde-yaml` (which has been archived unfortunately), but the multi-line
values are still a valid YAML syntax and, most importantly, rendering
changelong entries will continue working. For now, I won't post-process
from the multi-line values syntax to the single line array syntax. One
can work around this by handwriting a changelog entry Markdown file in
`smithy-rs/.changelog` without using this subcommand.

#### A subcommand `--ls`
This subcommand will render a preview of changelog entries stored in
`smithy-rs/.changelog` and print it to the standard output. Example
usages (using the entry created at 5745197.md above):
```
$ changelogger ls -c smithy-rs \
# for long flag, replace -c with --change-set

Next changelog preview for smithy-rs
=====================================
**New this release:**
- 🐛 (client, [smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234), @someone) Some changelog

**Contributors**
Thank you for your contributions! ❤
- @someone ([smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234))
```
```
$ changelogger ls -c aws-sdk \
# for long flag, replace -c with --change-set

Next changelog preview for aws-sdk-rust
========================================
**New this release:**
- 🐛 ([smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234), @someone) Some changelog

**Contributors**
Thank you for your contributions! ❤
- @someone ([smithy-rs#1234](https://github.com/smithy-lang/smithy-rs/issues/1234))
```


## Testing
- Existing tests in CI
- Basic unit tests for subcommands

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-31 21:31:37 +00:00
ysaito1001 36b50b3dac
Add client-support for RPC v2 CBOR (#3767)
## Motivation and Context
Follow-up on https://github.com/smithy-lang/smithy-rs/pull/2544 to add
client-side support for the protocol

## Description
The client implementation mainly focuses on a sub-section
[Requests](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#requests)
in the spec. To that end, this PR addresses `TODO` for the client to
fill in the blanks and includes additional adjustments/refactoring to
pass client protocol tests.

## Testing
- Existing tests in CI
- Upstream protocol test `rpcv2Cbor`
- Our handwritten protocol test `rpcv2Cbor-extras.smithy`

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-31 19:22:03 +00:00
Fahad Zubair 50148e6983
Use `serverIntegrationTest` instead of local server invocation function (#3775)
Member constraint shape tests were using a server codegen invocation
function which was very similar to `serverIntegrationTest`. This PR
removes that and uses `serverIntegrationTest` instead.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-31 11:05:10 +00:00
Zelda Hessler 507ebe8ac6
v2 Smoketest codegen (#3758)
This PR adds codegen for service-defined smoketests.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-30 16:07:50 +00:00
Jack Kleeman 1223f61e10
Remove unused hyper dependency in aws-config (#3770)
This allows end-users to keep hyper 0.x out of their tree

## Motivation and Context
This dependency appears unused and means that I have a 0.14 import in my
tree

## Description
Removes the dependency

## Testing
Builds locally, if CI passes I think its fair to say this is a no-op. I
can't find any usages of this lib.

## Checklist
I'm not sure this change warrants a changelog as there is no user facing
change

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-19 17:58:48 +00:00
Landon James 3881938099
Reverting some accidentally ressurected CHANGELOG entries (#3768)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

## Description
<!--- Describe your changes in detail -->

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 19:52:44 +00:00
Landon James c39792b3d8
Fix maps/lists with sensitive members not redacted (#3765)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Lists and maps with sensitive members were not being properly redacted.
Closes https://github.com/smithy-lang/smithy-rs/issues/3757

## Description
<!--- Describe your changes in detail -->
Updated `Shape.shouldRedact` method to properly check for list and map
shapes with sensitive members. Also updated the test definition so it
would actually run since previously the test code was generated in a
nested function inside a no-op function and never run.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added test cases for lists with sensitive members, maps with sensitive
keys, and maps with sensitive values.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 18:01:04 +00:00
david-perez c63e792454
Add server RPC v2 CBOR support (#2544)
RPC v2 CBOR is a new protocol that ~is being added~ has [recently been
added](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html)
to the Smithy
specification.

_(I'll add more details here as the patchset evolves)_

Credit goes to @jjant for initial implementation of the router, which I
built on top of from his
[`jjant/smithy-rpc-v2-exploration`](https://github.com/awslabs/smithy-rs/tree/jjant/smithy-rpc-v2-exploration)
branch.

Tracking issue: https://github.com/smithy-lang/smithy-rs/issues/3573.

## Caveats

`TODO`s are currently exhaustively sprinkled throughout the patch
documenting what remains to be done. Most of these need to be addressed
before this can be merged in; some can be punted on to not make this PR
bigger.

However, I'd like to call out the major caveats and blockers here. I'll
keep updating this list as the patchset evolves.

- [x] RPC v2 has still not been added to the Smithy specification. It is
currently being worked on over in the
[`smithy-rpc-v2`](https://github.com/awslabs/smithy/tree/smithy-rpc-v2)
branch. The following are prerrequisites for this PR to be merged;
**until they are done CI on this PR will fail**:
    - [x] Smithy merges in RPC v2 support.
    - [x] Smithy releases a new version incorporating RPC v2 support.
- Released in [Smithy
v1.47](https://github.com/smithy-lang/smithy/releases/tag/1.47.0)
    - [x] smithy-rs updates to the new version.
        - Updated in https://github.com/smithy-lang/smithy-rs/pull/3552
- [x] ~Protocol tests for the protocol do not currently exist in Smithy.
Until those get written~, this PR resorts to Rust unit tests and
integration tests that use `serde` to round-trip messages and compare
`serde`'s encoders and decoders with ours for correctness.
- Protocol tests are under the
[`smithy-protocol-tests`](https://github.com/smithy-lang/smithy/tree/main/smithy-protocol-tests/model/rpcv2Cbor)
directory in Smithy.
- We're keeping the `serde_cbor` round-trip tests for defense in depth.
- [ ] https://github.com/smithy-lang/smithy-rs/issues/3709 - Currently
only server-side support has been implemented, because that's what I'm
most familiar. However, we're almost all the way there to add
client-side support.
- ~[ ] [Smithy `document`
shapes](https://smithy.io/2.0/spec/simple-types.html#document) are not
supported. RPC v2's specification currently doesn't indicate how to
implement them.~
- [The
spec](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#shape-serialization)
ended up leaving them as unsupported: "Document types are not currently
supported in this protocol."

## Prerequisite PRs

This section lists prerequisite PRs and issues that would make the diff
for this one lighter or easier to understand. It's preferable that these
PRs be merged prior to this one; some are hard prerequisites. They
mostly relate to parts of the codebase I've had to touch or ~pilfer~
inspect in this PR where I've made necessary changes, refactors and
"drive-by improvements" that are mostly unrelated, although some
directly unlock things I've needed in this patchset. It makes sense to
pull them out to ease reviewability and make this patch more
semantically self-contained.

- https://github.com/awslabs/smithy-rs/pull/2516
- https://github.com/awslabs/smithy-rs/pull/2517
- https://github.com/awslabs/smithy-rs/pull/2522
- https://github.com/awslabs/smithy-rs/pull/2524
- https://github.com/awslabs/smithy-rs/pull/2528
- https://github.com/awslabs/smithy-rs/pull/2536
- https://github.com/awslabs/smithy-rs/pull/2537
- https://github.com/awslabs/smithy-rs/pull/2531
- https://github.com/awslabs/smithy-rs/pull/2538
- https://github.com/awslabs/smithy-rs/pull/2539
- https://github.com/awslabs/smithy-rs/pull/2542
- https://github.com/smithy-lang/smithy-rs/pull/3684
- https://github.com/smithy-lang/smithy-rs/pull/3678
- https://github.com/smithy-lang/smithy-rs/pull/3690
- https://github.com/smithy-lang/smithy-rs/pull/3713
- https://github.com/smithy-lang/smithy-rs/pull/3726
- https://github.com/smithy-lang/smithy-rs/pull/3752

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

~RPC v2 has still not been added to the Smithy specification. It is
currently being worked on over in the
[`smithy-rpc-v2`](https://github.com/awslabs/smithy/tree/smithy-rpc-v2)
branch.~

This can only be tested _locally_ following these steps:

~1. Clone [the Smithy
repository](https://github.com/smithy-lang/smithy/tree/smithy-rpc-v2)
and checkout the `smithy-rpc-v2` branch.
2. Inside your local checkout of smithy-rs pointing to this PR's branch,
make sure you've added `mavenLocal()` as a repository in the
`build.gradle.kts` files.
[Example](8df82fd3fc).
4. Inside your local checkout of Smithy's `smithy-rpc-v2` branch:
1. Set `VERSION` to the current Smithy version used in smithy-rs (1.28.1
as of writing, but [check
here](https://github.com/awslabs/smithy-rs/blob/main/gradle.properties#L21)).
    2. Run `./gradlew clean build pTML`.~
~6.~ 1. In your local checkout of the smithy-rs's `smithy-rpc-v2`
branch, run `./gradlew codegen-server-test:build -P
modules='rpcv2Cbor'`.

~You can troubleshoot whether you have Smithy correctly set up locally
by inspecting
`~/.m2/repository/software/amazon/smithy/smithy-protocols-traits`.~

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 09:50:52 +00:00
AWS SDK Rust Bot e4ced33954
Merge smithy-rs-release-1.x.y into main (#3766) 2024-07-16 20:29:43 -05:00
AWS SDK Rust Bot de982bfd51 Merge remote-tracking branch "origin/smithy-rs-release-1.x.y" into "merge-smithy-rs-release-1.x.y-to-main" 2024-07-16 23:18:30 +00:00
AWS SDK Rust Bot 73df1e285e Update changelog 2024-07-16 23:14:53 +00:00
ysaito1001 0b059fb5e5
Add support for changelog entry markdown files (#3763)
## Motivation and Context
[RFC: File-per-change
changelog](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#rfc-file-per-change-changelog)

## Description
This PR is the first of the two implementing the proposal outlined in
the RFC, and it focuses on the initial 4 bullets in the [Changes
checklist](https://smithy-lang.github.io/smithy-rs/design/rfcs/rfc0042_file_per_change_changelog.html#changes-checklist).
Crucially, this update does not modify the workflow for developers
regarding changelog entries (editing `CHANGELOG.next.toml` remains
necessary) or impact the `smithy-rs` CI process and our release
pipeline.

The PR introduces support for deserializing the following Markdown front
matter in the YAML format described in the RFC, e.g.
```
---
# Adding `aws-sdk-rust` here duplicates this entry into the SDK changelog.
applies_to: ["client", "server", "aws-sdk-rust"]
authors: ["author1", "author2"]
references: ["smithy-rs#1234", "aws-sdk-rust#1234"]
# The previous `meta` section is broken up into its constituents:
breaking: false
# This replaces "tada":
new_feature: false
bug_fix: false
---

Some message for the change.
```
These changelog entry Markdown files will be saved in the
`smithy-rs/.changelog` directory:
```
┌───────────┐                  
│ smithy-rs │                  
└─────┬─────┘                  
      │                        
      │    ┌───────────┐       
      ├────┤.changelog │       
           └─────┬─────┘       
                 │             
                 ├─────test1.md  
                 │ 
                 └─────test2.md 
```
For a concrete example, see a test
`end_to_end_changelog_entry_markdown_files` that directly converts from
`end_to_end_changelog` and uses Markdown files (the expected test
results remain the same).

The next PR is expected to
- add new subcommands to `changelogger` to a) create a Markdown file and
b) preview changelog entries pending to be released
- port existing `CHANGELOG.next.toml` at that time to individual
Markdown files
- update `sdk-lints` to disallow the existence of `CHANGELOG.next.toml`
- pass a directory path for `smithy-rs/.changelog` to `--source` (one
for
[split](dc970b3738/tools/ci-scripts/generate-smithy-rs-release (L22))
and one for
[render](dc970b3738/tools/ci-scripts/generate-smithy-rs-release (L22)))
and to
[--source-to-truncate](dc970b3738/tools/ci-scripts/generate-smithy-rs-release (L23C1-L23C47))

(the 4th bullet effectively does the cutover to the new changelog mode)

## Testing
- Added `parse_markdown` and `end_to_end_changelog_entry_markdown_files`
for testing new changelog format (the rest of the tests showing up in
the diff are moved from different places)
- Confirmed via [a release
dry-run](https://github.com/smithy-lang/smithy-rs/actions/runs/9947165056)
that this PR does not break the existing `smithy-rs` release process
- Confirmed that this PR does not break the use of `changelogger` in our
internal release process

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-16 16:14:16 +00:00
Fahad Zubair 8d23be1392
Bump `aws-smithy-http-server-python` version number (#3764)
The Python server relies on types defined in `aws-smithy-http-server`,
which has a new version published. A new version of the Python server
should be released to ensure there is only one `aws-smithy-http-server`
in the dependency closure of the generated crates.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-16 13:32:52 +00:00
ysaito1001 dc970b3738
Re-export `HttpRequest` and `HttpResponse` in client crates (#3762)
## Motivation and Context
https://github.com/smithy-lang/smithy-rs/issues/3591

## Description
In early days of smithy-rs, we used to re-export
[Request](https://docs.rs/aws-sdk-s3/0.25.0/aws_sdk_s3/client/customize/struct.Request.html)
and
[Response](https://docs.rs/aws-sdk-s3/0.25.0/aws_sdk_s3/client/customize/struct.Request.html)
types. When we overhauled the underlying smithy runtime from middleware
to orchestrator, we did not re-export the corresponding types
[HttpRequest](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/orchestrator/type.HttpRequest.html)
and
[HttpResponse](https://docs.rs/aws-smithy-runtime-api/latest/aws_smithy_runtime_api/client/orchestrator/type.HttpResponse.html)
to client crates.

This PR will re-export them in `crate::config::http`.

## Testing
Added a new test file for `ClientRuntimeTypesReExportGenerator.kt` that
verifies re-exports.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-15 22:23:12 +00:00
Landon James e4a58c3608
Add support for `operationContextParams` Endpoints trait (#3755)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
We have to support the new [`operationContextParams`
trait](https://smithy.io/2.0/additional-specs/rules-engine/parameters.html#smithy-rules-operationcontextparams-trait)
for endpoint resolution. This trait specifies JMESPath expressions for
selecting parameter data from the operation's input type.

## Description
<!--- Describe your changes in detail -->
* Add codegen support for the [JMESPath
`keys`](https://jmespath.org/specification.html#keys) function (required
by the trait
[spec](https://smithy.io/2.0/additional-specs/rules-engine/parameters.html#smithy-rules-operationcontextparams-trait))
* Add codegen support for the trait itself. This is achieved by
generating `get_param_name` functions for each param specified in
`operationContextParams`. These functions pull the data out of the input
object and it is added to the endpoint params in the
`${operationName}EndpointParamsInterceptor`

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Updated the existing test suite for JMESPath codegen to test the `keys`
function. Updated the existing EndpointsDecoratorTest with an
`operationContextParams` trait specifying one param of each supported
type to test the codegen.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-13 00:31:21 +00:00
Fahad Zubair 2313eb9e75
Bump `serial_test` dependency to `3.1.1` (#3740)
`serial_test = "1.0.0"` has a bug where it sometimes does not honor the
`#[ignore]` attribute on a test.

This PR bumps the dependency to version `3.1.1`.

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-10 16:12:44 +00:00
david-perez 4c30f00f64
Refactor determining server error type when deserializing an `@httpPayload` (#3752)
Determining the error type when deserializing an `@httpPayload` is a
protocol-specific concern, and as such should not live in
`ServerHttpBoundProtocolGenerator`, which should remain
protocol-agnostic. This commits makes that determination part of the
`ServerProtocol` interface.

As a drive-by improvement, the companion object in
`ServerHttpBoundProtocolGenerator` has also been removed, since its
members have been unused for a long time.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-10 14:31:03 +00:00
AWS SDK Rust Bot dc66ae4b67
Merge smithy-rs-release-1.x.y into main (#3751) 2024-07-09 20:04:54 +01:00
AWS SDK Rust Bot 78c50eea17 Update changelog 2024-07-09 16:34:58 +00:00
Fahad Zubair 56f4c8e947
CHANGELOG entry for PR:3378 (#3747)
Adds an entry to CHANGELOG.next.toml for the
[PR:3378](https://github.com/smithy-lang/smithy-rs/pull/3378)

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: david-perez <d@vidp.dev>
2024-07-09 13:11:10 +00:00
Gustavo Muenz dcf16ac5de
Remove attributes annotation of Builder `struct` (#3674)
The `struct` created for a Builder should not have attributes because
these structs only derive from `Debug`, `PartialEq`, `Clone` and
`Default`. None of these support attributes.

This becomes a problem if a plugin tries to add attributes to the
`metadata` field to configure the generated code for the `struct`. In
this case, the attributes will also be added to the `Builder` struct;
which is wrong.

## Motivation and Context

I'm customizing client code generation by overriding the method:

```kotlin
override fun symbolProvider(base: RustSymbolProvider): RustSymbolProvider
```

I override the `symbol.expectRustMetadata().additionalAttributes` value
and add extra values there. The generated code adds the correct
attributes for the `struct`, however, it also adds these attributes to
the `Builder`.

The `Builder` is restricted to deriving from `Debug`, `PartialEq`,
`Clone` and `Default` (see
[code](a76dc184c2/codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/BuilderGenerator.kt (L188))).

I believe this change was added by mistake in
https://github.com/smithy-lang/smithy-rs/pull/2222/.


## Description

To solve the issue, I simply remove the offending line.

<!--- Describe your changes in detail -->

## Testing

I don't expect any existing code to break, given that this is a bug. It
only manifests when someone is customizing the generated code.


<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

Co-authored-by: Gustavo Muenz <muenze@amazon.com>
2024-07-09 09:38:53 +00:00
Zelda Hessler 0c7271664b
Fix panic when merging a stalled stream log to an empty LogBuffer (#3744)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
this fixes an issue submitted by an internal user.

## Description
<!--- Describe your changes in detail -->
If the first event pushed into the log underlying stalled stream
protection, then the log manager would panic. This is because it was
expecting to merge the new event in but there was nothing to merge it
with.

To fix this, I updated the code to `push` the event when the log manager
is empty, instead of merging it.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
ran existing tests and wrote a new test exercising the
previous-panicking code.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-08 21:20:58 +00:00
Landon James 2295d5b6a8
Support `stringArray` type in endpoints params (#3742)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

## Description
<!--- Describe your changes in detail -->
Updated our endpoint rule generation to support the new `stringArray`
parameter type

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Updated the existing `EndpointsDecoratorTest` with tests for
`stringArray`

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2024-07-08 20:23:26 +00:00
david-perez 20ad88862f
AWS JSON 1.x server request specs can be `&'static str`s (#3741)
This is technically a breaking change because we stop implementing
`FromIterator<(String, S)>`.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-08 16:33:51 +00:00
Fahad Zubair 1588945cd9
Add tracing calls when an extension parameter cannot be constructed (#3378)
## Motivation and Context

When a user defines a handler function with additional parameters, these
parameters are constructed using the `FromRequest` trait implementation
specific to their types. If `FromRequest` fails to construct the
parameter – for instance, if `Extension<State>` is expected by the user
but `Extension<Arc<State>>` is actually added by the extension layer –
the server currently does not log a message indicating this construction
failure. Instead, it simply returns a 500 internal server error.

## Description

`tracing::{trace, error}` calls have been added.

## Breaking change

`FromParts<P>::Rejection` **must** implement `std::fmt::Display`.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-08 13:56:21 +00:00
ysaito1001 17545f6fc1
Upgrade Smithy to 1.50.0 (#3728)
## Motivation and Context
This PR upgrades Smithy to 1.50.0. The majority of the changes follow
`TODO` added in https://github.com/smithy-lang/smithy-rs/pull/3690.
Other than that, a few adjustments needed to be made:
- for the client
- added two failing tests `RestJsonClientPopulatesDefaultValuesInInput`
and `RestJsonClientUsesExplicitlyProvidedMemberValuesOverDefaults` to
known failing tests for the same reason
[here](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/protocol/ClientProtocolTestGenerator.kt#L72)
- added one broken test (i.e. the upstream test definition is incorrect
but our implementation is correct) to known broken tests per
([smithy#2341](https://github.com/smithy-lang/smithy/pull/2341),
[smithy-rs#3726](https://github.com/smithy-lang/smithy-rs/pull/3726#issuecomment-2199833659))
- for the server
- removed `rest-xml-extras.smithy` since
`RestXmlMustSupportParametersInContentType` is now available upstream
Smithy 1.50.0
- added the following to known failing tests (since the `awsJson1_0`
counterparts are already in the list, but we need the server team to
verify this assumption & provide additional `TODO` comments if
necessary)
    - `RestJsonServerPopulatesDefaultsWhenMissingInRequestBody`
    - `RestJsonServerPopulatesDefaultsInResponseWhenMissingInParams`,
-
`RestJsonServerPopulatesNestedDefaultValuesWhenMissingInInResponseParams`

## Testing
Existing tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2024-07-05 14:58:17 +00:00
david-perez 2bf03857f0
Adjust error message when bodies don't match in `aws-smithy-protocol-test` (#3736)
The left parameter is the expected one, see

<dc1ffb8a0d/rust-runtime/aws-smithy-protocol-test/src/lib.rs (L389-L394)>.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-04 10:45:48 +00:00
AWS SDK Rust Bot e3646b118b
Merge smithy-rs-release-1.x.y into main (#3737) 2024-07-03 17:02:21 -05:00
AWS SDK Rust Bot 4552ba152d Update changelog 2024-07-03 20:51:04 +00:00
Zelda Hessler fe1b341e8f
add detailed error explanations page (#3734)
I was speaking to several customers that found our connection poisoning
terminology confusing. To address this, I've changed the messaging and
I've also created a new section in the docs that explains some errors in
greater detail. To direct users to these explanations, I've added a link
to the `tracing::info!` call.

I propose that we consider doing the same any time we identify error
messaging that
- must be kept succinct to avoid logging too much info
- must be more understandable to our users.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-03 17:13:30 +00:00
Fahad Zubair dc1ffb8a0d
Refactor `UnconstrainedUnionGenerator` to use `ValidationExceptionConversionGenerator` (#3733)
`UnconstrainedUnionGenerator` should use
`ValidationExceptionConversionGenerator` to generate the
`as_validation_exception` method.

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-07-02 15:58:18 +00:00
Landon James 9af72f5f2a
Add customization to ensure S3 `Expires` field is always a `DateTime` (#3730)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
At some point in the near future (after this customization is applied to
all existing SDKs) the S3 model will change the type of `Expires`
members to `String` from the current `Timestamp`. This change would
break backwards compatibility for us.

## Description
<!--- Describe your changes in detail -->
Add customization to ensure S3 `Expires` field is always a `Timestamp`
and ass a new synthetic member `ExpiresString` that persists the
un-parsed data from the `Expires` header.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added tests to ensure that the model is pre-processed correctly. Added
integration tests for S3. Considered making this more generic codegen
tests, but since this customization will almost certainly only ever
apply to S3 and I wanted to ensure that it was properly applied to the
generated S3 SDK I opted for this route.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-01 18:48:05 +00:00
Zelda Hessler 5c0baa7907
update smoketest models (#3732)
Updates the smithy-rs models to the latest versions.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-01 18:18:24 +00:00
Rbatistab 84fbadc857
Fix broken link on example's smithy model (#3725)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
There's a broken link on the Readme of the example:
https://github.com/smithy-lang/smithy-rs/blob/main/examples/README.md?plain=1#L34

## Description
Fix the link to point to the common-test models instead:
https://github.com/smithy-lang/smithy-rs/blob/main/codegen-core/common-test-models/pokemon.smithy

## Testing
Readme update, no testing

## Checklist
None
----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2024-07-01 16:02:52 +00:00
david-perez 7299cdd3cd
Improve broken protocol test generation (#3726)
We currently "hotfix" a broken protocol test in-memory, but there's no
mechanism that alerts us when the broken protocol test has been fixed
upstream when updating our Smithy version. This commit introduces such a
mechanism by generating both the original and the fixed test, with a
`#[should_panic]` attribute on the former, so that the test fails when
all its assertions succeed.

With this change, in general this approach of fixing tests in-memory
should now be used over adding the broken test to `expectFail` and
adding the fixed test to a `<protocol>-extras.smithy` Smithy model,
which is substantially more effort.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-01 09:23:16 +00:00
Christian Schwarz 24a011b9d6
fix(docs: client construction): wrong type name (#3633)
## Motivation and Context
Docs are wrong.

## Description
The docs are referring to the wrong type name (`Config`).
The example right below the change uses the right type name
(`$crateName::config::Builder`).

## Testing
n/a

## Checklist

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-28 16:07:59 +00:00
Dj 4beac5f19d
Allow expected content-type to ignore parameters (#3724)
Fixes: #3471

## Motivation and Context
An issue was raised about a mobile client that appends "; charset=utf-8"
to the Content-Type when using restJson1. The [latest
RFC](https://www.rfc-editor.org/rfc/rfc8259) for "application/json" does
not register a charset parameter, but indicates it is reasonable to
accept it.

## Description
This change loosens the validation of the expected content type to allow
all parameters.

## Testing
* Tests for each protocol were added to
[smithy](https://github.com/smithy-lang/smithy/pull/2296)
* ran the runtime and codegen tests
* Added test for rest-xml, as smithy-rs does not currently run the
smithy tests.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-28 15:25:23 +00:00
Fahad Zubair 5498a180cf
Define a function to return operation shapes that need a `ValidationException` (#3720)
Refactor and define a separate function that returns a set of operation
shapes that must have a supported validation exception shape in their
associated errors list.

This helps identify which type of `ValidationException` has been added
to the operation shape's errors list.

Closes: [3722](https://github.com/smithy-lang/smithy-rs/issues/3722)

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: david-perez <d@vidp.dev>
2024-06-28 11:32:19 +00:00
david-perez e56f0dd632
Refactor and DRY up protocol test generation (#3713)
Protocol test generation code was one of the earlier parts of
smithy-rs's codebase and it has accrued a fair amount of tech debt as we
have evolved the code generation primitives. Its code was also forked
when the server code generator was introduced, introducing a lot of
duplicated code that has deviated over time.

This commit refactors the code to modern standards and aims to reconcile
commonalities in `ProtocolTestGenerator`, so that both client and server
can reap centralized improvements over time.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-27 13:24:01 +00:00
Fahad Zubair cba3edfbe3
Refactor the `as_validation_exception_field` function definition in `ServerBuilderConstraintViolations`. (#3719)
This PR refactors the code to limit the visibility of
`crate::model::ValidationExceptionField` to only those decorators and
associated classes that are invoked when
`smithy.framework#ValidationException` is being used.

More information is in [this
issue](https://github.com/smithy-lang/smithy-rs/issues/3718).

Closes: [3178](https://github.com/smithy-lang/smithy-rs/issues/3718)

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: david-perez <d@vidp.dev>
2024-06-27 10:48:28 +00:00
Landon James b748878612
Remove assumption about repo name by using git (#3715)
to get the path of the top level of the repo

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Fixing issue with test workspaces that would break if the repo is not
named `smithy-rs`

## Description
<!--- Describe your changes in detail -->
Moved to using `git rev-parse --show-toplevel` to get the project
directory instead of climbing the file tree.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
All of our CI tests this since it is part of generating test workspaces

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-25 16:28:58 +00:00
Zelda Hessler 21c6cc7e10
Make `http` v0 usage more evident (#3711)
Also fixes IntelliJ lints

This PR is not intended to change any behavior. If you see me doing that
somewhere, then it's a mistake.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-25 14:09:00 +00:00
Landon James 31a4b8e648
Unpin pinned dependencies (#3712)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Due to numerous dependency issues over the past few weeks several of our
dependencies were pinned to specific minor versions. Now that we have
lockfiles for all of our workspaces those pinned versions can be undone.

## Description
<!--- Describe your changes in detail -->
I unpinned those dependencies and generated new lockfiles after that
change.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I ran the gradle `assemble` and `sdkTest` tasks locally and they both
passed. I ran a full set of preview/release e2e tests against this
change in my local stack and those all passed. And since we copy the
Cargo.lock into the test workspaces the CI run for this PR will also
test the changes.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
No functional changes, only dependency changes.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-21 18:31:05 +00:00
david-perez fbde9d8700
Do not set `RUSTFLAGS` environment variable when invoking Cargo commands via Gradle tasks (#3678)
Otherwise, if you generate a crate and compile it using Gradle, like for
example using the invocation:

```
./gradlew -P modules='simple' -P cargoCommands='test' codegen-server-test:build
```

And then manually run a `cargo` command within the generated crate
directory, or open the project using `rust-analyzer`, Cargo will
re-compile the project from scratch, with
`CARGO_LOG=cargo::core::compiler::fingerprint=trace` reporting that
flags have changed since the last compilation.

Instead, it's best if we persist these flags to `.cargo/config.toml`, so
all `cargo` invocations, either through Gradle, manually, or through
`rust-analyzer`, use the same set. This way, if no files were changed,
subsequent compilations since code generation will truly be no-ops, with
Cargo reusing all artifacts.

Note this commit fixes a regression that was introduced when `--cfg
aws_sdk_unstable` was introduced in #2614, since I fixed this the first
time back in #1422.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-21 14:26:22 +00:00
Fahad Zubair 99e1e208ef
Add `aws-lambda` as a feature to generated server SDK (#3648)
## Motivation and Context
The generated SDK conditionally re-exports `routing::LambdaHandler`
under the feature flag aws-lambda, but the `Cargo.toml` does not define
such a feature.

## Description
Closes: https://github.com/smithy-lang/smithy-rs/issues/3643

## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: david-perez <d@vidp.dev>
2024-06-20 10:27:28 +00:00
AWS SDK Rust Bot 6e0f418c21
Merge smithy-rs-release-1.x.y into main (#3708) 2024-06-19 12:21:48 -07:00
AWS SDK Rust Bot ae2882d247 Update changelog 2024-06-19 18:35:26 +00:00
Landon James 21b3995a2f
Update canary build tool (#3701)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

Our WASI canary was breaking with the newest Rust version due to the new
WASI build targets:
https://blog.rust-lang.org/2024/04/09/updates-to-rusts-wasi-targets.html

## Description
<!--- Describe your changes in detail -->
Updating the version of `cargo-component` used to build our WASI canary.
Made the same update in the `aws-sdk-rust` repo:
https://github.com/awslabs/aws-sdk-rust/pull/1168

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
This same change was made in the canary workflow in the `aws-sdk-rust`
repo and run successfully there:
* PR: https://github.com/awslabs/aws-sdk-rust/pull/1168
* Canary run:
https://github.com/awslabs/aws-sdk-rust/actions/runs/9575014647

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-19 16:36:57 +00:00
ysaito1001 6fff79a749
Update lockfiles based on release-2024-06-18 in aws-sdk-rust #3706 (#3707)
## Motivation and Context
Update lockfiles based on [the codegen
release](https://github.com/awslabs/aws-sdk-rust/releases/tag/release-2024-06-18)
(a release including code changes in aws runtime crates, smithy runtime
crates, or codegen code).

## Testing
- [ ] Existing tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-19 00:02:48 +00:00