Commit Graph

36 Commits

Author SHA1 Message Date
Fahad Zubair 755dd28c8d Simplify the match statements 2024-10-02 08:48:14 +01:00
Fahad Zubair fce2d6d696 Use fully qualified ciborium namespaces to satisfy the cargo minimal-versions check command. 2024-10-02 01:36:35 +01:00
Fahad Zubair e4fe8fb1fb Fix comment 2024-10-01 21:33:58 +01:00
Fahad Zubair 0460c875f7 Bump `aws-smithy-protocol-test` crate version to `0.63` 2024-10-01 21:26:20 +01:00
Fahad Zubair 7ae4517d74 Use HashMap instead of BTreeMap 2024-10-01 20:43:12 +01:00
Fahad Zubair a2679acf30 Specifically, use float.is_nan() when comparing floats that may have NaN values. 2024-10-01 20:19:19 +01:00
Fahad Zubair 275f997db2 Fix map comparison: implement order-insensitive equality for `ciborium::Value::Map` 2024-10-01 19:24:23 +01:00
Fahad Zubair 20ff1c7bbc Replace `serde_cbor` with `ciborium` due to security vulnerability
- `serde_cbor` is unmaintained and has known [security issues](https://rustsec.org/advisories/RUSTSEC-2021-0127.html)
- `ciborium` is an actively maintained CBOR library
2024-10-01 16:33:43 +01:00
ysaito1001 36b50b3dac
Add client-support for RPC v2 CBOR (#3767)
## Motivation and Context
Follow-up on https://github.com/smithy-lang/smithy-rs/pull/2544 to add
client-side support for the protocol

## Description
The client implementation mainly focuses on a sub-section
[Requests](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#requests)
in the spec. To that end, this PR addresses `TODO` for the client to
fill in the blanks and includes additional adjustments/refactoring to
pass client protocol tests.

## Testing
- Existing tests in CI
- Upstream protocol test `rpcv2Cbor`
- Our handwritten protocol test `rpcv2Cbor-extras.smithy`

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-31 19:22:03 +00:00
david-perez c63e792454
Add server RPC v2 CBOR support (#2544)
RPC v2 CBOR is a new protocol that ~is being added~ has [recently been
added](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html)
to the Smithy
specification.

_(I'll add more details here as the patchset evolves)_

Credit goes to @jjant for initial implementation of the router, which I
built on top of from his
[`jjant/smithy-rpc-v2-exploration`](https://github.com/awslabs/smithy-rs/tree/jjant/smithy-rpc-v2-exploration)
branch.

Tracking issue: https://github.com/smithy-lang/smithy-rs/issues/3573.

## Caveats

`TODO`s are currently exhaustively sprinkled throughout the patch
documenting what remains to be done. Most of these need to be addressed
before this can be merged in; some can be punted on to not make this PR
bigger.

However, I'd like to call out the major caveats and blockers here. I'll
keep updating this list as the patchset evolves.

- [x] RPC v2 has still not been added to the Smithy specification. It is
currently being worked on over in the
[`smithy-rpc-v2`](https://github.com/awslabs/smithy/tree/smithy-rpc-v2)
branch. The following are prerrequisites for this PR to be merged;
**until they are done CI on this PR will fail**:
    - [x] Smithy merges in RPC v2 support.
    - [x] Smithy releases a new version incorporating RPC v2 support.
- Released in [Smithy
v1.47](https://github.com/smithy-lang/smithy/releases/tag/1.47.0)
    - [x] smithy-rs updates to the new version.
        - Updated in https://github.com/smithy-lang/smithy-rs/pull/3552
- [x] ~Protocol tests for the protocol do not currently exist in Smithy.
Until those get written~, this PR resorts to Rust unit tests and
integration tests that use `serde` to round-trip messages and compare
`serde`'s encoders and decoders with ours for correctness.
- Protocol tests are under the
[`smithy-protocol-tests`](https://github.com/smithy-lang/smithy/tree/main/smithy-protocol-tests/model/rpcv2Cbor)
directory in Smithy.
- We're keeping the `serde_cbor` round-trip tests for defense in depth.
- [ ] https://github.com/smithy-lang/smithy-rs/issues/3709 - Currently
only server-side support has been implemented, because that's what I'm
most familiar. However, we're almost all the way there to add
client-side support.
- ~[ ] [Smithy `document`
shapes](https://smithy.io/2.0/spec/simple-types.html#document) are not
supported. RPC v2's specification currently doesn't indicate how to
implement them.~
- [The
spec](https://smithy.io/2.0/additional-specs/protocols/smithy-rpc-v2.html#shape-serialization)
ended up leaving them as unsupported: "Document types are not currently
supported in this protocol."

## Prerequisite PRs

This section lists prerequisite PRs and issues that would make the diff
for this one lighter or easier to understand. It's preferable that these
PRs be merged prior to this one; some are hard prerequisites. They
mostly relate to parts of the codebase I've had to touch or ~pilfer~
inspect in this PR where I've made necessary changes, refactors and
"drive-by improvements" that are mostly unrelated, although some
directly unlock things I've needed in this patchset. It makes sense to
pull them out to ease reviewability and make this patch more
semantically self-contained.

- https://github.com/awslabs/smithy-rs/pull/2516
- https://github.com/awslabs/smithy-rs/pull/2517
- https://github.com/awslabs/smithy-rs/pull/2522
- https://github.com/awslabs/smithy-rs/pull/2524
- https://github.com/awslabs/smithy-rs/pull/2528
- https://github.com/awslabs/smithy-rs/pull/2536
- https://github.com/awslabs/smithy-rs/pull/2537
- https://github.com/awslabs/smithy-rs/pull/2531
- https://github.com/awslabs/smithy-rs/pull/2538
- https://github.com/awslabs/smithy-rs/pull/2539
- https://github.com/awslabs/smithy-rs/pull/2542
- https://github.com/smithy-lang/smithy-rs/pull/3684
- https://github.com/smithy-lang/smithy-rs/pull/3678
- https://github.com/smithy-lang/smithy-rs/pull/3690
- https://github.com/smithy-lang/smithy-rs/pull/3713
- https://github.com/smithy-lang/smithy-rs/pull/3726
- https://github.com/smithy-lang/smithy-rs/pull/3752

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

~RPC v2 has still not been added to the Smithy specification. It is
currently being worked on over in the
[`smithy-rpc-v2`](https://github.com/awslabs/smithy/tree/smithy-rpc-v2)
branch.~

This can only be tested _locally_ following these steps:

~1. Clone [the Smithy
repository](https://github.com/smithy-lang/smithy/tree/smithy-rpc-v2)
and checkout the `smithy-rpc-v2` branch.
2. Inside your local checkout of smithy-rs pointing to this PR's branch,
make sure you've added `mavenLocal()` as a repository in the
`build.gradle.kts` files.
[Example](8df82fd3fc).
4. Inside your local checkout of Smithy's `smithy-rpc-v2` branch:
1. Set `VERSION` to the current Smithy version used in smithy-rs (1.28.1
as of writing, but [check
here](https://github.com/awslabs/smithy-rs/blob/main/gradle.properties#L21)).
    2. Run `./gradlew clean build pTML`.~
~6.~ 1. In your local checkout of the smithy-rs's `smithy-rpc-v2`
branch, run `./gradlew codegen-server-test:build -P
modules='rpcv2Cbor'`.

~You can troubleshoot whether you have Smithy correctly set up locally
by inspecting
`~/.m2/repository/software/amazon/smithy/smithy-protocols-traits`.~

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-17 09:50:52 +00:00
david-perez 2bf03857f0
Adjust error message when bodies don't match in `aws-smithy-protocol-test` (#3736)
The left parameter is the expected one, see

<dc1ffb8a0d/rust-runtime/aws-smithy-protocol-test/src/lib.rs (L389-L394)>.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-07-04 10:45:48 +00:00
John DiSanti b2f15ce961
Independently version all runtime crates (#3448)
This will make all the runtime crates independently versioned, which
will later remove the need for version number properties in the
gradle.properties file. A full release cycle should be done before
cleaning up the old logic just to be safe though.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-28 22:35:28 +00:00
Russell Cohen 06e66d191c
Scrape doc examples (#3334)
## Motivation and Context
Extracted from #3308, this adds doc example scraping. This won't make
much of a difference yet but it will help if we're able to move example
repos directly under the SDKs themselves when appropriate.

## Description
https://doc.rust-lang.org/rustdoc/scraped-examples.html

## Testing
verified that examples appear in docs when expected

## Checklist
no changelog

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-12-20 18:11:43 +00:00
Russell Cohen 0bcc193628 Add the `docsrs` autocfg attr to all crates
This also adds it to generated crates, and adds an autofix+lint to manage these attributes.
2023-12-08 10:24:19 -05:00
John DiSanti 5b93fd2f4a
Improve client init time by switching to regex-lite (#3269)
Each client initialization was taking between 1 and 2 milliseconds,
regardless if the client had been constructed before or not. For
example, if a customer wants five clients with different credentials
providers, that could be 10 milliseconds of time spent in
`Client::from_conf`. Approximately 98% of this time was spent compiling
regular expressions for the endpoint partition resolver.

This change switches everything over to the regex-lite crate, which has
faster regex compile times, and shouldn't have much of an impact on
performance for our specific use-cases (small strings, only evaluated at
client initialization).

The use of regex was entirely removed in aws-sigv4 since it was overkill
for what it was being used for.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-11-30 19:28:07 +00:00
John DiSanti 6438a09bef
Fix repo org move issues (#3166)
This PR fixes issues introduced by moving the repository from
awslabs/smithy-rs to smithy-lang/smithy-rs.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-11-10 18:51:04 +00:00
John DiSanti 8abeb04f5e
Move http types out of client and split headers out of request (#3138)
This PR moves the HTTP types into the root of aws-smithy-runtime-api
since they're not client-specific, and the serializers/deserializers
will need to rely on them. It also refactors the headers and errors out
of the request module.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-11-02 17:48:31 +00:00
Russell Cohen 684605ae4e
Http updates (#3059)
## Motivation and Context
- [ ] Upgrade guidance
- [x] changelog

To allow seamless adoption of the incoming `http` crate, we're updating
our client crates to use an intermediate type.

## Description

Update _client_ code to use `HttpRequest`. Some internal code still uses
`http::Request` but those usages are limited.

## Testing
CI

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-10-14 00:57:14 +00:00
John DiSanti 5401e0d364
Make order of expected/actual consistent in protocol tests (#2976)
Debugging tests is easier when the order of arguments in assertions is
consistent across the board. With consistent order, you don't need to
look at the line of code to know which value is the correct value. Most
of the rest of the project uses expected/actual order, so this PR makes
the protocol tests also use that order.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-09-08 16:09:45 +00:00
Zelda Hessler 8c4d186487
Update dependencies flagged by cargo audit (#2753)
Also, sort `Cargo.toml` dependencies if they were disordered.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-06-12 17:09:29 +00:00
Russell Cohen a85cef6d1e
Add configurable runtime plugin to send_v2 so that it is usable (#2635)
## Motivation and Context
- `send_v2()` doesn't work for a very small number of reasons—allow it
one last final runtime_plugin so that we can use it to make an e2e
request

## Description
- update a few decorators to be deterministic
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2023-04-27 00:09:53 +00:00
Zelda Hessler 92952cbe51
Update MSRV to 1.66.1 (#2468)
* update: MSRV to 1.66.1

* add: CHANGELOG.next.toml entry

* update: Dockerfile too

* fix: clippy lints

* fix: sigv4 event_stream.rs clippy lint
update: codegen clippy lint allowlist

* fix: doc lint that's nightly only
add: docs for aws-smithy-http
add: docs for aws-smithy-client
add: docs for aws-smithy-async

* add: more missing docs.

* fix: add more missing docs

* remove: unnecessary & in HttpBindingGenerator
fix: lint name

* fix: another doc import

* fix: test broken by changing compiler error

* fix: even more lints\nadd: lint warns to all smithy runtime crates

* fix: broken doc links

* fix: clippy bug

* fix: don't use modules named lib in codegen tests

* fix: the paginator_canary.rs unnecessary cast
2023-03-16 21:58:14 +00:00
Zelda Hessler bae93805de
Update pretty_assertions (#2332)
* update: pretty_assertions

* update: just use v1.3
2023-02-09 11:02:19 -06:00
Zelda Hessler f3e05625e1
Fix clippy lints and run formatter (#2080)
* fix: clippy lints in integration tests
format: integration tests

* fix: clippy lints in runtime crates
format: runtime crates

* fix: clippy lints in tools
format: tools

* fix: undo clippy lint change that doesn't compile with tracing v0.1
2022-12-09 19:00:02 +00:00
Luca Palmieri 18d969e160
Upgrade to Rust 1.62.0 (#1825)
* Run clippy --fix for Rust 1.62.0 and format the resulting code

* Run clippy --fix on all targets

* Run clippy --fix with all features enabled

* Avoid extra allocation

* Use more idiomatic assert

* Ignore noisy lint

* Update pyo3 and pyo3-asyncio to 0.17.0

* Implement Eq on aws-smithy-checksums::Error

* Implement Eq on Protocol

* Replace conditionals with range-containment

* Implement Eq on types in aws-smithy-types

* Implement Eq on types in aws-smithy-http-server-python

* Implement Eq on types in aws-smithy-eventstream

* Implement Eq on types in aws-smithy-xml

* Implement Eq on aws-sigv4

* Update CI to use Rust 1.62.0

* Add Eq for generated types that implement PartialEq

* Allow clippy::needless_return in generated code

* Remove unnecessary reborrow in http_serde

* Remove unnecessary borrow in operation_deser

* Add CHANGELOG entries

* Revert "Add Eq for generated types that implement PartialEq"

This reverts commit 5169bd95aa.

* Update pyo3 and pyo3-asyncio in generated code for python server

* Allow clippy::derive_partial_eq_without_eq on structs and builders

* Run clippy on tools

* Fix accidental move in generated code

* Revert "Allow clippy::derive_partial_eq_without_eq on structs and builders"

This reverts commit 068c63ca20.

* Fix another accidental move in generated code

* Undo unwanted change to model

* Re-add reborrow in HttpBindingGenerator

* Fix clippy::format-push-string in changelogger

* Fix more uses of str.push_str(&format!(...))

* Remove unnecessary parenthesis

* Run ktlint

* Update aws/rust-runtime/aws-http/src/content_encoding.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-http/src/content_encoding.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Ignore doctest for non-exported macro

Rust 1.62 introduced a breaking change where doctests for non-exported
macros are now run by default. These don't compile because the macro
can't be imported in it.
See https://github.com/rust-lang/rust/issues/97030 for more info.

* Run cargo fmt

* Use $crate instead of crate in macro

* Revert "Implement Eq on types in aws-smithy-types"

This reverts commit c45a6b5a56.

* Revert "Implement Eq on types in aws-smithy-eventstream"

This reverts commit 78f4b07344.

* Revert "Implement Eq on types in aws-smithy-xml"

This reverts commit 590f01af73.

* Revert "Implement Eq on aws-sigv4"

This reverts commit d78bb62124.

* Revert "Implement Eq on types in aws-smithy-http-server-python"

This reverts commit f2cd901884.

* Revert "Implement Eq on aws-smithy-checksums::Error"

This reverts commit 5da170405e.

Co-authored-by: Julian Antonielli <julianantonielli@gmail.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-10-17 09:32:12 +01:00
Russell Cohen c8fe6e2adf
Fix several minimal version bugs and add it to CI (#1365)
* Fix several minimal version bugs and add it to CI

* Fix docker image

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-05-09 17:14:14 +00:00
John DiSanti a0539e20b0
Fix `SPDX-License-Identifier` in header comments (#1377)
* Fix misc whitespace with pre-commit

* Revise copyright check in `sdk-lints`

* Fix `SPDX-License-Identifier` in header comments
2022-05-09 09:50:46 -05:00
Zelda Hessler b2c989630f
Update to Rust Edition 2021 (#1268)
* update: set rust edition to 2021
update: strict param of intoCrate to set clippy to set #![deny(clippy::all)]

* add: changelog entry
2022-03-22 21:29:04 +00:00
david-perez 8676219e43
Set server response headers (#1086)
This commit adds support for the `httpHeaders` and `httpPrefixHeaders`
when applied to error shapes and operation output shapes to the server
implementation.

The presence of response headers is now asserted against in server
protocol tests.

Functions that set headers in requests that were used by the client have
been refactored into `HttpBindingGenerator.kt`, since they are useful
for the server to set headers in responses.

This commit also makes error shapes be serialized in JSON responses with
a `__type` field in the body, in favor of setting the `X-Amzn-Errortype`
header, as recommended by the specs of all AWS protocols.

This commit also removes the generation of operation structs for the
server in `ProtocolGenerator.kt`, since they are not useful for the
server implementation.

Closes #1071 #1075.
2022-01-27 19:22:13 +01:00
Russell Cohen 00bc624a87
Add Support for SSO (#1051)
* Add Support for SSO

This commit adds support for the SSO credential provider, which enables the aws-config to support using SSO when specified in `~/.aws/config`.

* Rename & add test of configuration failure

* Add SSO to the smoke test list

* CR improvements

- Improve error messages
- zeroize token
- add track_caller to improve test failure error messages

* Apply suggestions from code review

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update changelogs

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-01-11 13:42:11 -05:00
Russell Cohen a77c5ca2a9
Add support for Paginators (#1006)
* Add Support for Paginators

This commit adds pagination support for both clients. It does this via the `FnStream` abstraction which combines a rendezvous-channel with a closure to produce a stream. When the stream is polled, the underlying closure is polled to advance the computation which then pushes data into the channel.

Two paginators are generated:
1. `.paginate()` which produces a stream of `Result<Page, Err>`
2. `.paginate().items()` which produces a stream of `Result<Item, Err>` where items are flattened from the individual pages. This uses `TryFlatMap` to support conveniently generating a flat page iterator.

In addition, docs are generated to point customers towards the paginators.

* Add RFC, more tests

* backout unrelated changes

* Fix paginators for glacier

* Fix clippy error

* Add test for paginators that point to maps & fix apigateway paginator

* Fix docs

* remove extraeneous println

* Fix DynamoDB examples

* Disallow paginating operations with the idempotency token trait

* Apply suggestions from code review

misc typos / cleanups

Co-authored-by: Zelda Hessler <zhessler@amazon.com>

* CR feedback

* Fix typo

* Update changelog

* Apply suggestions from code review

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* CR feedback round 1

* Rename paginate to into_paginator()

* update pr bot comment

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-01-05 17:46:57 +00:00
Russell Cohen cbd61ab1ff
Add recursion detection middleware to the default middleware stack (#1003)
* Add recursion detection middleware to the default middleware stack

Side refactorings:
- move AWS retry logic into its own module
- small update to protocol test to make it a little easier to use

* fixups

* Fix clippy

* Add more tests and comments
2021-12-27 18:04:10 +00:00
Russell Cohen 1f71c35713
Docs licenses (#909)
* CR feedback

* Correct warning
2021-12-01 13:01:53 -05:00
Russell Cohen cc82edf173
docs.rs metadata & license linter & fixes (#904)
* Lint & fix docs.rs + LICENSEs

* Clippy fixups

* Update changelogs

* Fix reverse condition & make repo_root lazy_static

* Update copyright
2021-12-01 12:26:47 +01:00
Russell Cohen 35de48a214
Add a few READMEs + linter script (#899)
* Add a few READMEs + linter script

* Add initial linter script

* Add CI job

* Apply suggestions from code review

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Add README fix command

* Add newline in footer

* Add footer support to generator

* Also run fixes in CI

* Add note about smithy-types

* cleanups to scripts and READMEs

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2021-11-30 14:37:42 -05:00
John DiSanti 1aa59693ee
Prefix Smithy runtime crates with `aws-` (#788)
* Prefix Smithy runtime crates with `aws-`

* Update changelogs
2021-10-20 10:21:15 -07:00