Commit Graph

306 Commits

Author SHA1 Message Date
Landon James 9af72f5f2a
Add customization to ensure S3 `Expires` field is always a `DateTime` (#3730)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
At some point in the near future (after this customization is applied to
all existing SDKs) the S3 model will change the type of `Expires`
members to `String` from the current `Timestamp`. This change would
break backwards compatibility for us.

## Description
<!--- Describe your changes in detail -->
Add customization to ensure S3 `Expires` field is always a `Timestamp`
and ass a new synthetic member `ExpiresString` that persists the
un-parsed data from the `Expires` header.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Added tests to ensure that the model is pre-processed correctly. Added
integration tests for S3. Considered making this more generic codegen
tests, but since this customization will almost certainly only ever
apply to S3 and I wanted to ensure that it was properly applied to the
generated S3 SDK I opted for this route.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-01 18:48:05 +00:00
david-perez 7299cdd3cd
Improve broken protocol test generation (#3726)
We currently "hotfix" a broken protocol test in-memory, but there's no
mechanism that alerts us when the broken protocol test has been fixed
upstream when updating our Smithy version. This commit introduces such a
mechanism by generating both the original and the fixed test, with a
`#[should_panic]` attribute on the former, so that the test fails when
all its assertions succeed.

With this change, in general this approach of fixing tests in-memory
should now be used over adding the broken test to `expectFail` and
adding the fixed test to a `<protocol>-extras.smithy` Smithy model,
which is substantially more effort.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-01 09:23:16 +00:00
Dj 4beac5f19d
Allow expected content-type to ignore parameters (#3724)
Fixes: #3471

## Motivation and Context
An issue was raised about a mobile client that appends "; charset=utf-8"
to the Content-Type when using restJson1. The [latest
RFC](https://www.rfc-editor.org/rfc/rfc8259) for "application/json" does
not register a charset parameter, but indicates it is reasonable to
accept it.

## Description
This change loosens the validation of the expected content type to allow
all parameters.

## Testing
* Tests for each protocol were added to
[smithy](https://github.com/smithy-lang/smithy/pull/2296)
* ran the runtime and codegen tests
* Added test for rest-xml, as smithy-rs does not currently run the
smithy tests.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-28 15:25:23 +00:00
david-perez e56f0dd632
Refactor and DRY up protocol test generation (#3713)
Protocol test generation code was one of the earlier parts of
smithy-rs's codebase and it has accrued a fair amount of tech debt as we
have evolved the code generation primitives. Its code was also forked
when the server code generator was introduced, introducing a lot of
duplicated code that has deviated over time.

This commit refactors the code to modern standards and aims to reconcile
commonalities in `ProtocolTestGenerator`, so that both client and server
can reap centralized improvements over time.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-27 13:24:01 +00:00
Landon James b748878612
Remove assumption about repo name by using git (#3715)
to get the path of the top level of the repo

## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Fixing issue with test workspaces that would break if the repo is not
named `smithy-rs`

## Description
<!--- Describe your changes in detail -->
Moved to using `git rev-parse --show-toplevel` to get the project
directory instead of climbing the file tree.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
All of our CI tests this since it is part of generating test workspaces

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-25 16:28:58 +00:00
ysaito1001 6fff79a749
Update lockfiles based on release-2024-06-18 in aws-sdk-rust #3706 (#3707)
## Motivation and Context
Update lockfiles based on [the codegen
release](https://github.com/awslabs/aws-sdk-rust/releases/tag/release-2024-06-18)
(a release including code changes in aws runtime crates, smithy runtime
crates, or codegen code).

## Testing
- [ ] Existing tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-19 00:02:48 +00:00
david-perez b583a2f14a
Fix request `Content-Type` header checking in servers (#3690)
This fixes two bugs:

1. `Content-Type` header checking was succeeding when no `Content-Type`
   header was present but one was expected.
2. When a shape was `@httpPayload`-bound, `Content-Type` header checking
   occurred even when no payload was being sent. In this case it is not
   necessary to check the header, since there is no content.

Code has been refactored and cleaned up. The crux of the logic is now
easier to understand, and contained in `content_type_header_classifier`.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-18 16:10:33 +00:00
Zelda Hessler 9009d1b69e
Add support for converting PresignedRequest to http 1.x request (#3696)
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-17 15:48:18 +00:00
Landon James d7593493ab
Adding new gradle tasks for generating Cargo.lock (#3689)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
We have had numerous recent issues where updated dependencies have
broken our build/release process. This seeks to fix that in a brute
force way by manually generating Cargo.lock files for our main
workspaces.

## Description
<!--- Describe your changes in detail -->
Introduce a new gradle task `generateAllLockfiles` (and some child tasks
that it invokes) that generates lock files for
* `aws-config` crate
* `aws/rust-runtime` workspace
* `rust-runtime` workspace
* the superset workspace generated by the `assemble` task
* NOTE: This Cargo.lock is actually generated from the workspace in the
`aws-sdk-rust` repo, saved in this repo, and copied into place at build
time. Generating this one requires a local copy of the `aws-sdk-rust`
repo whose absolute path is indicated by the `-Paws-sdk-rust-path`
parameter.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

I ran our E2E tests against 3 commits to confirm that the Cargo.lock
files operate as we expect:
* `9d84980ee2e37deadb7e45450d65c5e6bc69e07c` The head commit from this
PR which has our Cargo.lock files
	*  Passes
* `d9d9fcec2de0e898bcc804ee0bb31393d7098e72` with lockfiles and with the
`httparse` and `crc323c` crates unpinned
	*  Passes
* `df75beb160890b2a9c382c8a2f7da8d7d0ddd3c8` with no lockfiles and has
the `httparse` and `crc32c` crates unpinned
	*  Fails (as expected)

This series of tests confirms that that Cargo.lock files will protect us
against bad dependency updates going forward.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-16 02:59:53 +00:00
david-perez 41b938dfe9
Squash assorted Clippy and Rust doc warnings in codegen integration tests (#3684)
This commit gets rid of some Clippy and Rust doc warnings we produce in
generated code, which are surfaced in our codegen integration tests. The
aim is that eventually we will be able to deny Clippy and Rust doc
warnings in #3678 (despite us baking in `RUSTDOCFLAGS="-D warnings"` and
`RUSTFLAGS="-D warnings"` in our CI Docker image, we curently clear
these in codegen integration tests). Note that denying compiler warnings
is separately tracked in #3194.

The commit contains fixes for the following:

- Unconditionally referring to the `crate::types` module in Rust docs
when
  the Smithy model is such that it is not generated.
- Broken intra-crate doc links for client documentation.
- Incorrectly escaping Rust reserved keywords when referring to
  operation names in Rust docs.
- An unnecessary semi-colon when rendering additional client
  plugins.
- Not escaping brackets in text when rendering Rust docs containing
  HTML, that are interpreted as broken links.
- A broken link to unit variants of unions.
- Using `TryFrom` instead of `From` when infallibly converting from
  `&str` to `String` when deserializing an `@httpPayload` in the server.
- Using a redundant closure with `unwrap_or_else` when falling back to a
  `@default` boolean or number value for a `document` shape, instead of
  `unwrap_or`.
- Not opting into `clippy::enum_variant_names` when rendering the
  `Operation` enum in the server (since we render the operation names
  that were modeled as-is).

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-11 14:52:22 +00:00
ysaito1001 0cbeef351b
Revert pinning tokio version in codegen integ tests (#3681)
## Motivation and Context
Reverts e4260f5ab6

## Description
We recently had a CI failure where some of the SDK codegen integration
tests encountered a cargo dependency graph resolution error
([example](https://github.com/smithy-lang/smithy-rs/actions/runs/9351952926/job/25740638098?pr=3677#step:3:639)).
The way the issue manifested is as follows:
1. A preceding test `SdkCodegenIntegrationTest.smokeTestSdkCodegen`
generated a `Cargo.lock` at the top-level directory of a shared test
workspace whose `tokio` dependency did NOT include the `macros` feature:
```
[[package]]
name = "tokio"
version = "1.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba4f4a02a7a80d6f274636f0aa95c7e383b912d41fe721a31f29e29698585a4a"
dependencies = [
 "backtrace",
 "bytes",
 "libc",
 "mio",
 "pin-project-lite",
 "socket2",
 "windows-sys 0.48.0",
]
```
2. During its turn, `CredentialProviderConfigTest`, which is one of the
failed tests and depends on the `macros` feature of `tokio`, generated
its own test crate and made it as a member of the workspace, for
instance:
```
[workspace]
resolver = "2"
members = ["smithy-test1475522104838765591"]
```
3. Docker containers used by GitHub action have the `CARGO_HOME` set to
`/opt/cargo` where it stores crates io index, cached crates, and so on.
The registry cache did not contain the sought crate `tokio-macros-2.3.0`
at the end of previous step:
```
bash-5.2# ls /opt/cargo/registry/cache/index.crates.io-6f17d22bba15001f/tokio-
tokio-1.37.0.crate            tokio-macros-2.2.0.crate      tokio-rustls-0.24.1.crate     tokio-util-0.7.11.crate       
tokio-1.38.0.crate            tokio-native-tls-0.3.1.crate  tokio-util-0.7.10.crate
```
(`bash-5.2#` indicates that we shelled into a container)

4. Running `cargo test` from the workspace root (or from within the
crate generated at step 2) then triggered the error in question:
```
bash-5.2# cargo t --all-features
error: failed to select a version for the requirement `tokio-macros = "~2.3.0"`
candidate versions found which didn't match: 2.2.0, 2.1.0, 2.0.0, ...
location searched: crates.io index
required by package `tokio v1.38.0`
```

Whether the issue occurs or not for a given test depends upon the
contents of the lockfile of a workspace generated by prior tests. To
ensure we avoid this during codegen integration tests, this PR updates
crates io index before executing `cargo test`.

## Testing
- [x] Passed CI
- [x] Passed internal release pipeline E2E tests

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-06 17:22:01 +00:00
Aaron Todd 610a8e2661
fix tracing-test 0.2.5 compat (#3677)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

https://github.com/smithy-lang/smithy-rs/issues/3676

## Description

Internals one of our tests was leveraging has changed in latest release
of `tracing-test`, updates version and test to be compatible.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-03 19:01:35 +00:00
ysaito1001 d5ea2cdd91
Upgrade Smithy to 1.49 (#3662)
## Motivation and Context
Upgrades Smithy to 1.49.0

## Description
As part of the upgrade, it updates the serializer for EC2 query protocol
to handle empty lists in response to
[this](https://github.com/smithy-lang/smithy/pull/2269) (otherwise [a
protocol
test](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-client-test/build.gradle.kts#L75)
`ec2_empty_query_lists_request` would fail).

## Testing
Existing tests in CI

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-05-24 18:07:55 +00:00
Zelda Hessler a6a3cf18b4
Merge branch 'main' into zhessler/request-compression 2024-05-20 16:59:33 -05:00
Landon James 68704b1062 Bumping MSRV to 1.76.0 2024-05-20 11:58:10 -07:00
Zelda Hessler a4301fa43d
Request Compression PR 3/3 - Codegen (#3638)
PR 3 of 3. Most of the compression tests happen here since we don't have
a request-compression-supporting model in our set of "smoke test"
service models.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: Aaron Todd <aajtodd@users.noreply.github.com>
2024-05-16 06:53:59 -07:00
Zelda Hessler 0baea0b81d fix one const that I missed. 2024-05-14 09:38:52 -05:00
Zelda Hessler afb8ef0194 fix const names in codegen with wrong casing 2024-05-10 14:40:59 -05:00
John DiSanti eac52eb69c
Fix event stream `:content-type` for struct messages (#3603)
Event stream operations with struct shaped messages were using the wrong
`:content-type` message header value, which I think wasn't caught before
since the supported AWS S3/Transcribe event stream operations don't
serialize struct messages. This PR fixes the message content type
serialization.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-26 23:37:50 +00:00
John DiSanti 386ec3f1f7
Add MSRV to generated SDK Cargo.toml files (#3601)
This PR sets the `rust-version` property on generated Cargo.toml files
for the AWS SDK crates. It doesn't attempt to place the property on the
runtime crates for now since that will either require manual updating,
or more machinery to automate.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-25 16:03:34 -07:00
John DiSanti cfb97edbc4
Implement waiters (#3595)
This PR implements waiters according to the [Smithy waiters
spec](https://smithy.io/2.0/additional-specs/waiters.html).

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-25 19:30:40 +00:00
ysaito1001 6b59ba26c4
Fix compiling S3 crate for wasm (#3590)
## Motivation and Context
Running `cargo build --target wasm32-unknown-unknown
--no-default-features` on an S3 crate has stopped working since
https://github.com/smithy-lang/smithy-rs/pull/3465 with the following
error:
```
error: the wasm*-unknown-unknown targets are not supported by default, you may need to enable the "js" feature. For more information see: https://docs.rs/getrandom/#webassembly-support
   --> /Users/[REDACTED]/.cargo/registry/src/index.crates.io-6f17d22bba15001f/getrandom-0.2.14/src/lib.rs:352:9
    |
352 | /         compile_error!("the wasm*-unknown-unknown targets are not supported by \
353 | |                         default, you may need to enable the \"js\" feature. \
354 | |                         For more information see: \
355 | |                         https://docs.rs/getrandom/#webassembly-support");
    | |________________________________________________________________________^
```

To address the issue, this PR updates an S3's dependency on `ahash` in a
way that disables default features.

## Testing
Updated the existing test `integration-tests/webassembly` so that
`check-aws-sdk-standalone-integration-tests` will run `cargo check`
`aws-sdk-s3` against both `wasm32-wasi` and `wasm32-unknown-unknown`
(the updated check would break if we removed `default-features = false`
from the `ahash` dependency).

## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-18 21:19:22 +00:00
Harry Barber 7629358584
Switch links to Smithy v2 documentation (#2037)
## Motivation and Context

The latest documentation is Smithy v2. In most cases it's identical. As
far as I can tell, there are no links to items with breaking changes
between v1 and v2.

---------

Co-authored-by: Harry Barber <hlbarber@amazon.co.uk>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2024-04-12 23:11:33 +00:00
John DiSanti b1cbce65a5
Upgrade MSRV to 1.75 (#3553)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-11 23:20:06 +00:00
John DiSanti 55faed73c0
Add multiple binding support to Jmespath traversal generator (#3569)
In #3526, I forgot about the existence of input/output matchers in
Smithy waiters, so it only supported binding against a single global
output shape. This PR revises the generator so that it supports multiple
bindings so that it can generate an input/output matcher path traversal.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-09 16:05:18 +00:00
Fahad Zubair 64b9b91faa
Fix errors for unions with unit target membershape (#3547)
## Motivation and Context

Unions that have a [unit target member
shape](https://smithy.io/2.0/spec/model.html#unit-type) do not have an
associated data in the generated Rust enum.

Closes Issue:
[2546](https://github.com/smithy-lang/smithy-rs/issues/2546)

## Description

On the **server** side, when the union has constrained members, the code
generated for the conversion from the `Unconstrained` type to the
`Constrained` type incorrectly assumed that each Rust enum would have
associated data.

```
rust-server-codegen/src/unconstrained.rs:31:129
  |
  |               crate::unconstrained::some_union_with_unit_unconstrained::SomeUnionWithUnitUnconstrained::Option1(unconstrained) => Self::Option1(
    |                                                                                                                                   -^^^^^^^^^^^^- help: consider using a semicolon here to finish the statement: `;`
    |  _________________________________________________________________________________________________________________________________|
  | |
  | |                 unconstrained
  | |             ),
  | |_____________- call expression requires function
    |
   ::: rust-server-codegen/src/model.rs:152:5
    |
    |       Option1,
    |       ------- `SomeUnionWithUnit::Option1` defined here
```

The marshaling code for event streams with unit target types incorrectly
assumed that the variant would have associated data.

```
rust-server-codegen/src/event_stream_serde.rs

impl ::aws_smithy_eventstream::frame::MarshallMessage for TestEventMarshaller {
        fn marshal() {
            let payload = match input {
                Self::Input::KeepAlive(inner) => {
```

On the **client** side, the `event_stream_serde` code incorrectly
assumes that a union member, which has the `@streaming` trait applied to
it, takes a `model::Unit` type.

```
rust-client-codegen/src/event_stream_serde.rs:
    |
    |                     crate::types::TestEvent::KeepAlive(crate::types::Unit::builder().build()),
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^---------------------------------------
    |                     |
    |                     call expression requires function
    |
   ::: rust-client-codegen/src/types/_test_event.rs
    |
    |     KeepAlive,
    |     --------- `TestEvent::KeepAlive` defined here
```

## Testing

A unit test has been added that tests the following model:

```
            $version: "2"
            namespace com.example
            use aws.protocols#restJson1
            use smithy.framework#ValidationException
            
            @restJson1 @title("Test Service") 
            service TestService { 
                version: "0.1", 
                operations: [ 
                    TestOperation
                    TestSimpleUnionWithUnit
                ] 
            }
            
            @http(uri: "/testunit", method: "POST")
            operation TestSimpleUnionWithUnit {
                input := {
                    @required
                    request: SomeUnionWithUnit
                }
                output := {
                    result : SomeUnionWithUnit
                }
                errors: [
                    ValidationException
                ]
            }
            
            @length(min: 13)
            string StringRestricted
            
            union SomeUnionWithUnit {
                Option1: Unit
                Option2: StringRestricted
            }

            @http(uri: "/test", method: "POST")
            operation TestOperation {
                input := { payload: String }
                output := {
                    @httpPayload
                    events: TestEvent
                },
                errors: [ValidationException]
            }
            
            @streaming
            union TestEvent {
                KeepAlive: Unit,
                Response: TestResponseEvent,
            }
            
            structure TestResponseEvent { 
                data: String 
            }            

```

---------

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2024-04-09 16:00:24 +00:00
John DiSanti 879a8e6a3c
Forbid unsafe in generated code (#3559)
Fixes #290.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-08 23:21:01 +00:00
John DiSanti 105d30d4cb
Implement Jmespath shape traversal codegen for waiters (#3526)
This PR adds a `RustJmespathShapeTraversalGenerator` class that will
generate code to traverse Smithy generated shapes based on a Jmespath
expression. This is a necessary prerequisite for implementing Smithy
waiters support.

The generator only implements a subset of the full Jmespath spec. This
subset was determined by examining all the waiter expressions in the
existing SDK models.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-08 23:11:46 +00:00
John DiSanti 1c454bcd3c
Upgrade Smithy to 1.47 (#3552)
This PR upgrades Smithy to 1.47.0. The new S3 tests needed to be
excluded in the server tests since they fail to compile for the same
reason as the other ones that are excluded.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-08 22:55:09 +00:00
Zelda Hessler f97e6f3b10
fix the `./gradlew :aws:sdk:assemble` command and aws-config tests on Windows (#3522)
tracked in issue #3518 

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-28 03:10:32 +00:00
ysaito1001 3e81645f1f
Fix `sigv4-s3express` position in `auth_scheme_options` (#3498)
## Motivation and Context
Fixes an S3-related bug introduced when s3 Express was added

## Description
This PR fixes a bug where s3 Express auth flow was incorrectly used when
no auth schemes were available for an endpoint. This use case was
observed in test settings and manifested itself as a confusing behavior.

To address this issue, the fix ensures that an auth scheme for S3
express will be registered after that for sigv4 has been registered.

## Testing
Added an integration test
`s3_express_auth_flow_should_not_be_reached_with_no_auth_schemes`

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-20 16:17:36 +00:00
Dj 87d6e978c1
Update Pokémon models to Smithy IDL v2 (#3494)
## Motivation and Context
Resolves [issue
2523](https://github.com/smithy-lang/smithy-rs/issues/2523) to migrate
to Smithy IDL v2

## Description

I followed the [Smithy Migration
Guide](https://smithy.io/2.0/guides/model-translations/migrating-idl-1-to-2.html)
to convert the models. One change was that by moving the inputs/outputs
into the operation, it did result in a renaming of the CapturePokemon
structures. They went from CapturePokemonEvents(Input|Output) to
CapturePokemon(Input|Output). From a brief look at the commit history,
it seems the API used to be named different and the inputs/outputs
weren't renamed at that time. So I decided to have them match the normal
naming convention (it didn't seem to break any tests).

## Testing
As this was my first commit, I ran almost every target I could discover,
but I think the key ones were:
* codegen-server-test
* codegen-client-test
* examples: `make test`

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-19 16:15:51 +00:00
Russell Cohen 23cdff1d8f
Allow 'null' variants in unions (#3481)
## Motivation and Context
- https://github.com/awslabs/aws-sdk-rust/issues/1095

## Description
Update the JSON parser generator to allow for `null` to be set in
unions. Servers can send unions like this:
```json
{
  "AmazonElasticsearchParameters": null,
  "AmazonOpenSearchParameters": null,
  "AppFlowParameters": null,
  "AthenaParameters": null,
  "AuroraParameters": null,
  "AuroraPostgreSqlParameters": null,
  "AwsIotAnalyticsParameters": null,
  "BigQueryParameters": null,
  "DatabricksParameters": null,
  "Db2Parameters": null,
  "DenodoParameters": null,
  "DocumentDBParameters": null,
  "DremioParameters": null,
  "ExasolParameters": null,
  "GoogleAnalyticsParameters": null,
  "JiraParameters": null,
  "MariaDbParameters": null,
  "MongoAtlasParameters": null,
  "MongoDBParameters": null,
  "MySqlParameters": null,
  "OracleParameters": null,
  "PostgreSqlParameters": null,
  "PrestoParameters": null,
  "RdsParameters": null,
  "RedshiftParameters": null,
  "S3Parameters": {
    "IsUploaded": false,
    "ManifestFileLocation": {
      "Bucket": "deided-bucket.prod.us-east-1",
      "Key": "sales/manifest.json"
    },
    "RoleArn": null
  },
  "SalesforceParameters": null,
  "SapHanaParameters": null,
  "ServiceNowParameters": null,
  "SnowflakeParameters": null,
  "SparkParameters": null,
  "SqlServerParameters": null,
  "StarburstParameters": null,
  "TeradataParameters": null,
  "TrinoParameters": null,
  "TwitterParameters": null
}
```

This caused our parser to fail.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
- [x] New unit test
- [x] Dry run against new [smithy protocol
test](https://github.com/smithy-lang/smithy/pull/2180)

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-15 22:36:42 +00:00
ysaito1001 f3b332da47
Add support for S3 Express One Zone (#3465)
## Motivation and Context
Allows the Rust SDK to use [S3 Express One
Zone](https://aws.amazon.com/s3/storage-classes/express-one-zone/)

## Description
The PR adds the said S3-specific functionality to the Rust SDK. The code
changes have already been reviewed by previous sub PRs, but it's worth
going through them again as a whole:
- https://github.com/smithy-lang/smithy-rs/pull/3386
- https://github.com/smithy-lang/smithy-rs/pull/3388
- https://github.com/smithy-lang/smithy-rs/pull/3390
- https://github.com/smithy-lang/smithy-rs/pull/3432
- https://github.com/smithy-lang/smithy-rs/pull/3433
- https://github.com/smithy-lang/smithy-rs/pull/3459
- https://github.com/smithy-lang/smithy-rs/pull/3457
- https://github.com/smithy-lang/smithy-rs/pull/3462

In addition to the PRs above, commit eebe8af increases the canary
lambda's memory size to 512MB from 128MB (also makes it configurable
through a command line arg for `canary-runner`). By default, lambda's
allowed memory size is 128MB but with the addition of `canary-wasm` in
main, canary lambda's memory usage will be 152MB, causing the lambda to
be killed by a signal during runtime. The commit addresses that issue.

## Testing
- Unit tests in
[aws/rust-runtime/aws-inlineable/src/s3_express.rs](7f8c28b703/aws/rust-runtime/aws-inlineable/src/s3_express.rs)
- Integration tests in
[aws/sdk/integration-tests/s3/tests/express.rs](7f8c28b703/aws/sdk/integration-tests/s3/tests/express.rs)
- Canary in smithy-rs#3462

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: AWS SDK Rust Bot <aws-sdk-rust-primary@amazon.com>
Co-authored-by: AWS SDK Rust Bot <97246200+aws-sdk-rust-ci@users.noreply.github.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: Russell Cohen <rcoh@amazon.com>
2024-03-11 16:01:48 +00:00
John DiSanti a413b9ba28
Upgrade Smithy to 1.45 (#3470)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-08 13:16:02 -08:00
Russell Cohen f68b5eeaa9
Add Initial Support for Hyper 1.0 (#3461)
## Motivation and Context
- #1925 

## Description
This adds a minimal Hyper client, focusing on not exposing any unstable
APIs. For this reason, the `Client::Builder` customization API is not
exposed anymore. We do this because at some point in the future, we will
likely move away from the hyper-util based Client.

The code for this was lifted directly from the Hyper 0.14 implementation
but updated for new traits.

However, this does come with some new valuable pieces:
1. Support for aws-lc (no FIPS yet)
2. Support for providing a custom DNS resolver

## Testing
- E2E test with Hyper. A Canary should also be added
(https://github.com/awslabs/aws-sdk-rust/issues/1089)

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-07 10:57:57 -08:00
Russell Cohen 40f44cee6c
Rust msrv 1.74 (#3410)
## Motivation and Context
Update MSRV to 1.74

## Description
Normal slate of things, mostly appeasing clippy.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2024-02-16 21:15:48 +00:00
John DiSanti 68ab569878
Upgrade Smithy to 1.44 (#3397)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-08 22:05:29 +00:00
John DiSanti ae52b69572
Upgrade Smithy Gradle Plugin to 0.9 (#3394)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-07 22:18:49 +00:00
John DiSanti e3f0de42db
Move `Metadata` config bag type into a stable crate (#3325)
This issue addresses a semver compatibility problem similar to the one
described in #3318, except for the
`aws_smithy_http::operation::Metadata` type.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-12 19:11:29 +00:00
John DiSanti ad133fd31a
Resolve runtime crate versions from manifests first (#3351)
This PR changes codegen-core's build-time runtime crate version
resolution to pull versions from the actual runtime crate manifest
files, only changing them to the version numbers in gradle.properties if
they use the special `0.0.0-smithy-rs-head` version number. This is a
prerequisite step in allowing independent crate versioning.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-11 22:39:00 +00:00
Russell Cohen 30205973b9
Bring back tests of floating point (#3350)
## Motivation and Context
The PR that had these tests was closed because the fixes were made
elsewhere. This brings back the added tests.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-09 16:23:55 +00:00
John DiSanti 1446cad47a
Upgrade to Smithy 1.43 (#3349)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-09 14:22:30 +00:00
Russell Cohen ac01102fb8 Merge branch '2023-12-20-sdk-release-branch' into main-clone 2023-12-21 10:20:54 -05:00
Russell Cohen e0d5258207 Upgrade Smithy to 1.42 2023-12-21 10:17:15 -05:00
ysaito1001 fe9f492c44 Merge remote-tracking branch 'origin/smithy-rs-release-1.x.y' into merge-smithy-rs-release-1.x.y-to-main 2023-12-15 14:10:23 -06:00
Miles Ziemer 345624eb6c
Ignore actual default value instead of 0/false (#3252)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Previously, there was logic used to ignore the default 0/false values
for numbers/booleans when serializing members if they weren't boxed.
This was to fix issues that occured when upstream models didn't properly
box shapes that were meant to be optional, and for the most part worked
because services would just fill in the default if it wasn't passed.
However, with Smithy 2.0, models may have defaults != 0/false, but the
codegenerator still ignores the zero value.

## Description
<!--- Describe your changes in detail -->
This commit makes it so that the actual default value for the member is
ignored for booleans and numbers, instead of just 0/false. It also
updates serialization for http bindings so that headers and query
parameters with 0/false values aren't ignored if they are optional
parameters.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
- [x] Generated AWS SDK and inspected diff. Only changes are not
ignoring default value inside `if let Some() = ...` blocks, and ignoring
default value instead of just 0 (only seems to effect nimble).
- [x] Added protocol tests for serializing 0/false in query params for
restXml and restJson

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <john@vinylsquid.com>
2023-12-15 16:37:36 +00:00
John DiSanti 952b7d8943
Fix the build against JDK 20 (#3323)
When using JDK 20, the following error would occur when attempting to
build or run tests:

```
Execution failed for task ':codegen-client:compileKotlin'.
> Inconsistent JVM-target compatibility detected for tasks 'compileJava' (20) and 'compileKotlin' (11).
```

This PR explicitly sets the Java source and target versions to fix this
error.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-12-15 16:18:36 +00:00
John DiSanti 24029ab256
Fix warning for unwrap_or_else in generated builders (#3307)
While attempting to upgrade to Smithy 1.42, I noticed a warning in the
codegen-client integration tests:
```
warning: unnecessary closure used to substitute value for `Option::None`
   --> json_rpc10/rust-client-codegen/src/operation/operation_with_defaults/_operation_with_defaults_output.rs:477:31
    |
477 |             default_int_enum: self.default_int_enum.unwrap_or_else(|| 1),
    |                               ^^^^^^^^^^^^^^^^^^^^^^--------------------
    |                                                     |
    |                                                     help: use `unwrap_or(..)` instead: `unwrap_or(1)`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_lazy_evaluations
```

This change fixes these warnings by switching between unwrap_or_else and
unwrap_or based on the shape.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-12-14 21:45:12 +00:00
John DiSanti fe36727499
Upgrade Kotlin to 1.9.20 and Ktlint to 1.0.1 (#3320)
This PR upgrades Kotlin to 1.9.20, and Ktlint to 1.0.1.

I had to abandon the pretty-format-kotlin pre-commit hook since it was
ignoring our `.editorconfig`, which made it impossible to suppress some
of the newer lints that would be difficult and undesirable to fix. Now
all Ktlint runs go through gradle, so there isn't any way for
inconsistencies to creep in.

I recommend reviewing this PR commit by commit as the commit that runs
Ktlint against the entire repo has a very large diff. When reviewing the
Ktlint formatting changes, be sure to hide whitespace changes to reduce
how much needs to be examined.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-12-14 12:35:18 -08:00