Commit Graph

66 Commits

Author SHA1 Message Date
Zelda Hessler 5b9d0c0fff
fix clippy lints from the future (#3438)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Because the build pipeline is checking for these.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-26 15:30:51 +00:00
Russell Cohen 06e66d191c
Scrape doc examples (#3334)
## Motivation and Context
Extracted from #3308, this adds doc example scraping. This won't make
much of a difference yet but it will help if we're able to move example
repos directly under the SDKs themselves when appropriate.

## Description
https://doc.rust-lang.org/rustdoc/scraped-examples.html

## Testing
verified that examples appear in docs when expected

## Checklist
no changelog

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-12-20 18:11:43 +00:00
John DiSanti aa1f556ae0
Upgrade nightly used in CI (#3073)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
2023-12-14 11:30:23 -08:00
Russell Cohen 0bcc193628 Add the `docsrs` autocfg attr to all crates
This also adds it to generated crates, and adds an autofix+lint to manage these attributes.
2023-12-08 10:24:19 -05:00
John DiSanti 6438a09bef
Fix repo org move issues (#3166)
This PR fixes issues introduced by moving the repository from
awslabs/smithy-rs to smithy-lang/smithy-rs.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-11-10 18:51:04 +00:00
ysaito1001 882ebaf7e0
Avoid making `FsBuilder` aware of http-body version at interface level (#3101)
## Motivation and Context
Follow up on #3088

## Description
This PR reverts `ByteStream::read_with_body_0_4_from`,
`ByteStream::from_path_body_0_4`, and `ByteStream::from_file_body_0_4`
to the old names since from a customers' point of view, `FsBuilder`
should not mention anything about an `http-body` version at the API
level.

`FsBuilder` is currently an opt-in feature (with `rt-tokio` enabled) and
we make it tied to `http-body-0-4-x` by requiring `rt-tokio` including a
dependency on `http-body` version 0.4.x.

## Testing
Relied on the existing tests in CI

## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-10-27 21:04:39 +00:00
ysaito1001 2a51e0bceb
Feature-gate public use of `http-body` and `hyper` within `aws-smithy-types` (#3088)
## Motivation and Context
Implements https://github.com/awslabs/smithy-rs/issues/3033

## Description
This PR hides behind cargo features the third-party types from
`http-body` and `hyper` crates that are used in`aws-smithy-types`'
public API. Customers need to opt-in by enabling a cargo feature
`http-body-0-4-x` in `aws-smithy-types` to create an `SdkBody` or
`ByteStream` using those third-party types. For more details, please see
[the upgrade
guide](https://github.com/awslabs/smithy-rs/discussions/3089).

As can been seen from code changes, to reduce the surface area where we
need to feature-gate things, we have fused the
`aws_smithy_types::body::Inner::Streaming` enum variant into
`aws_smithy_types::body::Inner::Dyn` variant, thereby removing
`SdkBody::from_dyn`.

## Testing
Relied on existing tests in CI

## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2023-10-25 16:45:22 +00:00
ysaito1001 ed8763e5df
Re-point those using `ByteStream` and `SdkBody` to smithy-types (#3076)
# Motivation and Context
A follow-up on #3026

## Description
#3026 moved
- `aws_smithy_http::body::{BoxBody, Error, SdkBody}` to
`aws_smithy_types::body::{BoxBody, Error, SdkBody}`
- `aws_smithy_http::byte_stream::{AggregatedBytes, ByteStream,
error::Error}` to `aws_smithy_types::byte_stream::{AggregatedBytes,
ByteStream, error::Error}`

and also left "breadcrumbs", so that customers could still consume
updated types from `aws_smithy_http` after the move.

This PR turns breadcrumbs into deprecation messages (via
`#[deprecated(...)]`) and updates existing places that used to use moved
types from `aws_smithy_http` to directly depend on `aws_smithy_types`.

## Testing
Relied on tests in CI.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-10-19 20:15:32 +00:00
ysaito1001 33cd698f7f
Remove `futures_core::stream::Stream` from `aws_smithy_http::byte_stream::ByteStream` (#2983)
## Motivation and Context
Removes `futures_core::stream::Stream` from
`aws_smithy_http::byte_stream::ByteStream`

## Description
This PR is part of our ongoing effort,
https://github.com/awslabs/smithy-rs/issues/2413. We remove the
`futures_core::stream::Stream` trait from
`aws_smithy_http::byte_stream::ByteStream`. To continue support existing
places that relied upon `ByteStream` implementing the `Stream` trait, we
provide explicit `.next`, `.try_next`, and `.size_hints` methods on that
type. As part of it, a doc-hidden type `FuturesStreamCompatByteStream`,
which implements `Stream`, has been added to `aws_smithy_http` to cater
to those who need a type implementing `Stream` (see
[discussion](https://github.com/awslabs/smithy-rs/pull/2910#discussion_r1317112233)
why doc-hidden).

Another place we need to update is codegen responsible for rendering
stream payload serializer, and this affects the server. The regular
server and the python server have slightly different rendering
requirements, since the former uses
`aws_smithy_http::byte_stream::ByteStream` (that does _not_ implement
the `Stream` trait) and latter uses its own
[ByteStream](cb79a68e3c/rust-runtime/aws-smithy-http-server-python/src/types.rs (L343))
(that does implement `Stream`). We use
`ServerHttpBoundProtocolCustomization` to handle the difference:
`StreamPayloadSerializerCustomization` and
`PythonServerStreamPayloadSerializerCustomization`.

## Testing
No new behavior has been added, relied on the existing tests in CI.

## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-09-28 19:47:05 +00:00
Fahad Zubair 3229089fba
Allow python middleware to set the URI of a request (#3002)
## Motivation and Context
Python middleware were unable to change the URI which makes it
impossible to change the route of an incoming request in Python.

## Description
URI has a #[setter] that Python side can use to change it.

## Testing
- Test has been added that ensures URI can be changed from python.
- If the URI is modified by a middleware in a local smithy-rs Python
server, a different route will be invoked.

## Checklist
- I've updated the changelog.next.toml

Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
2023-09-28 15:05:08 +00:00
Burak 735b635190
Python: Allow configuring logging formatter (#2829)
## Motivation and Context
Allow Python users to configure their logging formatter to either
`json`, `pretty` or `compact` (default).

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-07-10 09:31:01 +00:00
Harry Barber 934129cf4c
Consolidate `proto` and `protocols` modules into single `protocol` module (#2780)
## Motivation and Context

We have two modules `proto` and `protocols` in `aws_smithy_http_server`,
these can be consolidated.
2023-06-20 12:27:25 +00:00
Zelda Hessler 8c4d186487
Update dependencies flagged by cargo audit (#2753)
Also, sort `Cargo.toml` dependencies if they were disordered.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-06-12 17:09:29 +00:00
david-perez cf292d5817
Bump `lambda_http` dependency of `aws-smithy-http-server` to 0.8.0 (#2685) (#2691)
This is https://github.com/awslabs/smithy-rs/pull/2685 again, which was
merged
prematurely and reverted in
https://github.com/awslabs/smithy-rs/pull/2690.

This time we won't merge it until it's due time.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-05-30 14:47:20 +00:00
david-perez a78ac591fe
Revert "Bump `lambda_http` dependency of `aws-smithy-http-server` to 0.8.0 (#2685)" (#2690)
This reverts commit 04db5e3572.

This is a breaking change that needs to wait to be merged in until the
next breaking release.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-05-10 10:41:28 +00:00
david-perez 04db5e3572
Bump `lambda_http` dependency of `aws-smithy-http-server` to 0.8.0 (#2685)
This patch also removes the unneeded dependency on `lambda_runtime` by
`aws-smithy-http-server-python`.

This patch also refactors `LambdaHandler`'s `convert_event` to avoid
cloning the URI path when not needed.

This is a breaking change. See #2676 why.

This patch also bumps `aws-smithy-http-server` dependency on `mime` to
0.3.4.

`cargo +nightly-2022-11-16 minimal-versions check --all-features`
otherwise
fails when using 0.3.0, because we require `impl fmt::Display for
mime::FromStrError`, which was first introduced in 0.3.4.

As to why `minimal-versions` is now picking `mime` 0.3.0 with this
patch, it's
because in `lambda_http` 0.7.3, they had [`mime =

"0.3.16"`](99dba64472/lambda-http/Cargo.toml (L35-L35)),
and in `lambda_http` 0.8.0, they've now relaxed that to [`mime =

"0.3"`](393d6447be/lambda-http/Cargo.toml (L36)).


## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-05-10 09:39:15 +00:00
Matteo Bigoi 9c585a223b
[Python] Improve Python stubs generation (#2606)
## Motivation and Context
This PR improves the Python stubs generation.

## Description
The main change is about avoiding to setup a placeholder for the Python
module and use the real module name, which allows to generate correct
docstrings during codegeneration.

We also change the stubs layout on disk, with the main stub entrypoint
called `__init__.pyi` instead of `$module_name.pyi`.

The README from the Rust runtime crate has been moved completely to the
example folder and I run autoformatting and style checks on the Python
example code.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak <unexge@gmail.com>
2023-04-20 14:18:15 +00:00
Matteo Bigoi a2d37ad261
[Python] Automatically generate stubs (#2576)
## Motivation and Context
We want to automatically generate stubs in the codegen diff to ensure
they can be reviewed and have a simple way to generate and include the
stubs inside the Maturin wheel.

## Description
The Python example has been moved to the `examples` folder and
refactored. The refactoring
ensures the script `stubgen.py` is included in the codegeneration of the
SDK crate. The script is later used to generate stubs automatically
during testing and can be used by customers to add their own stubs
before the Maturin build

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak <unexge@gmail.com>
2023-04-17 16:19:12 +00:00
Burak 1a7a495df5
Python: Event Streaming (#2482)
* Add `RustType.Application` and `PythonType.Application` variants

* Use `RustType.Application` for event streaming symbols

* Call symbol provider to get `Receiver` type instead of hardcoding

* Add Python specific event streaming symbol provider

* Add event streaming wrapper generator

* Generate correct type information for event streaming types

* Add `CapturePokemon` operation to Python Pokemon service

* Add `InternalServerError` variant to all event streaming errors

* Use `PythonServerCargoDependency` for PyO3Asyncio imports

* Return an attribute error instead of panicking in `IntoPy` impls of wrappers

* Add `Sync` bound to `new` methods of wrappers

* Revert "Add `InternalServerError` variant to all event streaming errors"

This reverts commit b610cb27c7532102e3a3ec15a59da6c56b60c318.

* Add `PythonServerEventStreamErrorGenerator` to generate Python specific error types for unions

* Try to extract error type or inner type from incoming streaming value and ignore the value otherwise for now

* Allow missing type-stubs for `aiohttp`

* Propogate modelled errors through stream

* Raise modelled exceptions rather than sending through stream

* Allow senders from Python side to raise modelled exceptions

* Update `EventStreamSymbolProviderTest` to use `Application` type instead of `Opaque` type

* Fix `ktlint` issues

* Group up common variables in `codegenScope`

* Document `RustType.Application`

* Format `codegen-server-test/python/model/pokemon.smithy`

* Use `tokio-stream` crate instead of `futures` crate

* Use a better error message if user tries to reuse a stream

* Add some details about event streams to example Pokemon service

---------

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2023-04-11 11:21:16 +00:00
John DiSanti 87a62ae30f
Eliminate dependency on `futures-macros` (#2460)
The `futures-macros` crate is pulled in by default by `futures-util`,
and depends on `syn`. None of this functionality is used by smithy-rs.
2023-03-17 21:28:54 +00:00
Matteo Bigoi 48eda40437
Implement Python unions (#2427)
* Add initial implementation of unions with very broken symbol provider

* Add support for creating new unions in Python

* Generate getters and static methods for unions

* Allow to compile misc model

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>

* Doesn't work

* Now it works

* Simplify code a little

* Remove leftover from the many tries I did

* Finally fixed model generation with unions

* Fix wrong import

* Update to reflect changes in decorators

* Remove debugging output

* Simplify symbol provider

* Follow PR suggestions

* Remove union operation from python example

* Return `PyUnionMarker` for wrapped type in `IntoPy` impl

---------

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak Varlı <burakvar@amazon.co.uk>
2023-03-17 15:02:31 +00:00
Zelda Hessler 92952cbe51
Update MSRV to 1.66.1 (#2468)
* update: MSRV to 1.66.1

* add: CHANGELOG.next.toml entry

* update: Dockerfile too

* fix: clippy lints

* fix: sigv4 event_stream.rs clippy lint
update: codegen clippy lint allowlist

* fix: doc lint that's nightly only
add: docs for aws-smithy-http
add: docs for aws-smithy-client
add: docs for aws-smithy-async

* add: more missing docs.

* fix: add more missing docs

* remove: unnecessary & in HttpBindingGenerator
fix: lint name

* fix: another doc import

* fix: test broken by changing compiler error

* fix: even more lints\nadd: lint warns to all smithy runtime crates

* fix: broken doc links

* fix: clippy bug

* fix: don't use modules named lib in codegen tests

* fix: the paginator_canary.rs unnecessary cast
2023-03-16 21:58:14 +00:00
John DiSanti 4835af9d6b
Make `RustReservedWordsSymbolProvider` configurable (#2382)
* Remove `toEnumVariantName` from `RustSymbolProvider`

The `toEnumVariantName` function existed on symbol provider to work
around enum definitions not being shapes. In the future when we refactor
to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s
for each enum member. This change incrementally moves us to that future
by creating fake `MemberShape`s in the enum generator from the enum
definition.

* Fix escaping of `Self` in symbol providers

* Clean up an old hack

* Make `RustReservedWordsSymbolProvider` configurable

* Update changelog

* Incorporate feedback
2023-03-13 19:15:55 +00:00
Burak 26cb37afb0
Python: Add tests and fix the issues with `Timestamp` and `ByteStream` (#2431)
* Add `timestamp` type test and fix the conversion error

* Add some tests for `ByteStream` and fix async issues

* Use `__anext__` method instead of `anext`

---------

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2023-03-07 10:04:56 +00:00
John DiSanti 85d2ace0d6
Upgrade Kotlin and Ktlint (#2392)
* Upgrade Kotlin to 1.7.21

* Upgrade Ktlint to 0.48.2

* Run `pre-commit run --all-files` to fix broken Ktlints

* Fix string comparison broken by code formatting
2023-02-22 18:21:06 +00:00
Nugine de186678ee
Bump Rust MSRV to 1.63.0 (#2222)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2023-02-10 12:59:31 -08:00
Burak e84ef6c7e2
Python: Type-stub generation for SSDKs (#2149)
* Initial Python stub generation

* Handle default values correctly

* Only generate `__init__` for classes that have constructor signatures

* Preserve doc comments

* Make context class generic

* Put type hint into a string to fix runtime error

* Run `mypy` on CI

* Use `make` to build Python SSDKs while generating diffs

* Escape Python types in Rust comments

* Only mark class methods with

* Sort imports to minimize diffs

* Add type annotations for `PySocket`

* Dont extend classes from `object` as every class already implicitly extended from `object`

* Use `vars` instead of `inspect.getmembers` to skip inherited members of a class

* Fix linting issues

* Add some tests for stubgen and refactor it

* Add type annotations to `PyMiddlewareException`

* Fix tests on Python 3.7

Python 3.7 doesn't support reading signatures from `__text_signature__`
for non-builtin functions (i.e. C/Rust functions). For testing we're using
regular Python syntax for defining signature.

* Provide default values for `typing.Optional[T]` types in type-stubs

* Update `is_fn_like` to cover more cases

* Remove `tools/smithy-rs-tool-common/`

* Make `DECORATORS` an array instead of a list

* Ignore missing type stub errors for `aiohttp`
2023-02-10 14:00:53 +00:00
Burak 861d1d89db
Python: Support `document` type (#2188)
* Add Python wrapper for `aws_smithy_types::Document`

* Remove unused type

* Make sure Python SSDKs uses Python specific `Document` type

* Allow Python specific `Document` type to be used in serialization and deserialization

* Make `pyo3/extension-module` a default feature

* Add `PythonServerTypesTest`

* Fix linting issues
2023-01-30 12:21:13 +00:00
Zelda Hessler a559048ad8
formatting: run pre-commit on all files (#2236)
* formatting: run pre-commit on all files

* fix: test broken by string indent
2023-01-20 16:02:26 +00:00
david-perez adf30a8eee
Derive `Eq` and `Hash` wherever possible (#2223)
In server SDKs, these traits can be implemented by any shape _except_ if
the shape's closure contains:

1. A `float`, `double`, or `document` shape: floating point types in
   Rust do not implement `Eq`. Similarly, [`document` shapes] may
   contain arbitrary JSON-like data containing floating point values.
2. A [@streaming] shape: all the streaming data would need to be
   buffered first to compare it.

Additionally, the `Hash` trait cannot be implemented by shapes whose
closure contains:

1. A `map` shape: we render `map` shapes as `std::collections::HashMap`,
   which _do not_ implement `Hash`. See
   https://github.com/awslabs/smithy/issues/1567.

In **client SDKs, these traits cannot be derived on any code-generated
Rust types corresponding to Smithy shapes**, since e.g. adding new
optional members to a structure [is a backwards-compatible change], and
doing so alters the semantics of these traits.

However, this commit does implement these traits for the
`aws_smithy_types::date_time::DateTime` and `aws_smithy_types::Blob`
runtime types.

This change is necessary to efficiently implement the `@uniqueItems`
constraint trait in server SDKs.

[`Eq`]: https://doc.rust-lang.org/std/cmp/trait.Eq.html
[`Hash`]: https://doc.rust-lang.org/std/hash/trait.Hash.html
[`document` shapes]: https://smithy.io/2.0/spec/simple-types.html#document
[@streaming]: https://smithy.io/2.0/spec/streaming.html
[is a backwards-compatible change]: https://smithy.io/2.0/guides/evolving-models.html#updating-structures
2023-01-19 13:25:00 +01:00
82marbag 5363c41c6a
Support @default (#1879)
* Default trait in server

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: david-perez <d@vidp.dev>
2022-12-23 09:20:41 +00:00
John DiSanti 5970ef7725
Break up `RustCodegenDecorator` (#2099)
This change creates `ClientCodegenDecorator` and
`ServerCodegenDecorator` in `codegen-client` and `codegen-server`
respectively to replace `RustCodegenDecorator`. Client/server
equivalents are created to replace `CombinedCodegenDecorator` as well.

This eliminates the need for the `supportsCodegenContext` method since
the decorator interface is no longer generic, so the `ServiceLoader` is
now powerful enough to differentiate.

The largest benefit, however, is that now clients and servers can have
separate customizations.
2022-12-14 13:26:55 -08:00
Burak 2cc7c24be4
Python: Use Maturin to build SDKs (#2025)
* Remove `lib` prefix from generated module names

* Build Pokemon service with `Maturin`

* Fix Maturin build on CI

* Generate minimal `pyproject.toml` for generated SDKs to build from source using Maturin

* Fix `ktlint` issues

* Bring back type stubs for Pokemon service

* Update instructions for Lambda

* Make `build-wheel` and `build-wheel-release` to depend on `codegen`
2022-12-14 12:12:58 +00:00
Burak 3fb90968d7
Python: Allow injecting Lambda Context (#1985)
* Provide Python wrappers for Lambda related types

* Introduce `PyContext` to wrap raw context object

* Use new `PyContext` in handlers

* Expose `lambda` module to Python

* Use `LambdaContext` in example service

* Start Lambda handler in a different thread

* Print summary of Lambda context in Pokemon service

* Make sure to include Python `builtins` in tests

* Make `lambda_ctx` optional

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>

* Only inject types if they are type-hinted as `Optional[T]`

* Export Lambda module as `aws_lambda` instead of `lambda_`

* Comment why we need to run Hyper server in a background thread

* Move `is_optional_of` to `util` module

* Use `HeaderMap::from_iter` to build headers

* Support edge case of `(None, T)` in `util::is_optional_of`

* Make Lambda related types feature gated

* Remove feature gate for Lambda

* Make `xray_trace_id` an `Option`

* Remove `aws-lambda` feature from generated `Cargo.toml`s

* Fix linting issues

* Pin `lambda_runtime` to `0.7.1`

* Remove duplicate dependency in `Cargo.toml`

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2022-12-12 17:42:40 +00:00
Burak 9d2d08802e
Python: TLS Support for servers (#2002)
* Add `PyTlsConfig` struct

* Support TLS server

* Add TLS support to Pokemon service

* Make sure to create `tokio::net::TcpListener` in a Tokio context

* Fix doc link

* Add missing imports in tests

* Add `tls::Listener`

* Reload TLS config periodically

* Add context to `TODO`

* Return `&'static str` from `base_url()`

* Flatten `match` in `tls::Listener`

* Propogate listener errors but ignore handshake errors in `tls::Listener`

* Add tests to `tls::Listener`

* Add test to `tls::Listener` to make sure we are propogating listener errors

* Use `PathBuf` instead of plain `String`s for paths
2022-12-12 16:53:11 +00:00
Zelda Hessler f3e05625e1
Fix clippy lints and run formatter (#2080)
* fix: clippy lints in integration tests
format: integration tests

* fix: clippy lints in runtime crates
format: runtime crates

* fix: clippy lints in tools
format: tools

* fix: undo clippy lint change that doesn't compile with tracing v0.1
2022-12-09 19:00:02 +00:00
Luca Palmieri 3a9a42bb16
Introduce an `aws-lambda` feature. (#2035)
* Introduce an `aws-lambda` feature.

* Add CHANGELOG

* Use the new and shiny feature syntax to avoid creating an implicit `lambda_http` feature.

* Add `aws-lambda` feature to the Python server. Enable the `aws-lambda` feature in our example. Be explicit in providing an implementation of request rejection for Box<dyn Error>.

* Add an `aws-lambda` feature flag to the generated Python-specific crate. We enable it by default to make sure the Lambda method ends up visible in the final Python wrapper library.
2022-11-29 15:48:48 +00:00
John DiSanti fba88df3ee
Revamp errors in `aws-smithy-http` (#1884) 2022-11-15 19:18:26 +00:00
Burak 4f76e35f2a
Python: map Python middlewares to Tower layers (#1871)
* Python: map Python middlewares to Tower layers

* Make middleware layer infallible

* Use message and status code from `PyMiddlewareException`

* Introduce `FuncMetadata` to represent some information about a Python function

* Improve middleware errors

* Add missing copyright headers

* Allow accessing and changing request body

* Allow changing response

* Add some documentation about moving data back-and-forth between Rust and Python

* Add `mypy` to Pokemon service and update typings and comments for middlewares

* Add or update comments on the important types

* Add Rust equivalent of `collections.abc.MutableMapping`

* Add `PyHeaderMap` to make `HeaderMap` accessible from Python

* Apply suggestions from code review

Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>

* Improve logging

* Add `RichPyErr` to have a better output for `PyErr`s

* Better error messages for `PyMiddlewareError` variants

* Factor out repeating patterns in tests

* Preserve `__builtins__` in `globals` to fix tests in Python 3.7.10 (our CI version)

* Export `RichPyErr` to fix `cargo doc` error

* Apply suggestions from code review

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>

* Add missing SPDX headers

* Document that `keys`, `values` and `items` on `PyMutableMapping` causes clones

Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2022-11-10 16:22:43 +00:00
ysaito1001 f0b7f55bf5
Fix cargo audit issue on chrono (#1907)
* Avoid the chrono crate depending on the time crate

This commit is in response to RUSTSEC-2020-0071 where there is a
potential segfault in the time crate. The aws-smithy-types-convert
crate now disables the default features of the chrono crate so that
it will not depend on the time crate.

* Depend on lambda_http without RUSTSEC-2020-0071

This commit updates the version of lambda_http used by aws-smithy-http-server
and aws-smithy-http-server-python to 0.7.0. The prior version 0.6.0 of
lambda_http used the chrono crate in a way that exposed a security issue
described in RUSTSEC-2020-0071. By switching to 0.7.0 of lambda_http, those
two crates do not exhibit vulnerabilities as reported by cargo audit.

* Bump minor version of lambda_http in pokemon-service

This commit updates the version of `lambda_http` used by `pokemon-service`
from 0.6.0 to 0.7.0. This is in sync with the fact that both
`aws-smithy-http-server` and `aws-smithy-http-server-python` now depend
on 0.7.0 of `lambda_http`. Failing to do so would cause `pokemon-service`
to fail to compile due to an error at `lambda_http::run(handler)` in the
main function of the `pokemon-service-lambda` binary:
the trait `Service<lambda_http::http::Request<lambda_http::Body>>` is not
implemented for `LambdaHandler<aws_smithy_http_server::routing::Router>`

* Depend on lambda-http 0.7.1

This commit updates the version of `lambda_http` from 0.7.0 to 0.7.1 in
the crates within the top-level `rust-runtime` workspace. These updates
are needed to solve the issue described in awslabs/aws-lambda-rust-runtime#556

* Update CHANGELOG.next.toml

* Address https://github.com/awslabs/smithy-rs/pull/1907\#pullrequestreview-1161609833

Co-authored-by: Saito <awsaito@c889f3b5ddc4.ant.amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-10-31 20:06:09 +00:00
Burak c5dae879a5
Python: Migrate to new service builder (#1846)
* Python: Migrate to new service builder

* Fix orderings of imports
2022-10-17 15:52:37 +01:00
Harry Barber 55d16b1131
Make protocol ZSTs match shape ID names (#1858) 2022-10-17 14:00:43 +00:00
Luca Palmieri 18d969e160
Upgrade to Rust 1.62.0 (#1825)
* Run clippy --fix for Rust 1.62.0 and format the resulting code

* Run clippy --fix on all targets

* Run clippy --fix with all features enabled

* Avoid extra allocation

* Use more idiomatic assert

* Ignore noisy lint

* Update pyo3 and pyo3-asyncio to 0.17.0

* Implement Eq on aws-smithy-checksums::Error

* Implement Eq on Protocol

* Replace conditionals with range-containment

* Implement Eq on types in aws-smithy-types

* Implement Eq on types in aws-smithy-http-server-python

* Implement Eq on types in aws-smithy-eventstream

* Implement Eq on types in aws-smithy-xml

* Implement Eq on aws-sigv4

* Update CI to use Rust 1.62.0

* Add Eq for generated types that implement PartialEq

* Allow clippy::needless_return in generated code

* Remove unnecessary reborrow in http_serde

* Remove unnecessary borrow in operation_deser

* Add CHANGELOG entries

* Revert "Add Eq for generated types that implement PartialEq"

This reverts commit 5169bd95aa.

* Update pyo3 and pyo3-asyncio in generated code for python server

* Allow clippy::derive_partial_eq_without_eq on structs and builders

* Run clippy on tools

* Fix accidental move in generated code

* Revert "Allow clippy::derive_partial_eq_without_eq on structs and builders"

This reverts commit 068c63ca20.

* Fix another accidental move in generated code

* Undo unwanted change to model

* Re-add reborrow in HttpBindingGenerator

* Fix clippy::format-push-string in changelogger

* Fix more uses of str.push_str(&format!(...))

* Remove unnecessary parenthesis

* Run ktlint

* Update aws/rust-runtime/aws-http/src/content_encoding.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-http/src/content_encoding.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Ignore doctest for non-exported macro

Rust 1.62 introduced a breaking change where doctests for non-exported
macros are now run by default. These don't compile because the macro
can't be imported in it.
See https://github.com/rust-lang/rust/issues/97030 for more info.

* Run cargo fmt

* Use $crate instead of crate in macro

* Revert "Implement Eq on types in aws-smithy-types"

This reverts commit c45a6b5a56.

* Revert "Implement Eq on types in aws-smithy-eventstream"

This reverts commit 78f4b07344.

* Revert "Implement Eq on types in aws-smithy-xml"

This reverts commit 590f01af73.

* Revert "Implement Eq on aws-sigv4"

This reverts commit d78bb62124.

* Revert "Implement Eq on types in aws-smithy-http-server-python"

This reverts commit f2cd901884.

* Revert "Implement Eq on aws-smithy-checksums::Error"

This reverts commit 5da170405e.

Co-authored-by: Julian Antonielli <julianantonielli@gmail.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-10-17 09:32:12 +01:00
Burak eb6ffe3520
Lambda support for Python servers (#1807)
* Allow running Python servers with Lambda handler

* Fix doctests

* Add `Running servers on AWS Lambda` section to docs

* Remove empty spaces from `README` to make linter happy

* Rename `run_lambda_server` to `run_lambda_handler`

* Add more comments to example `Dockerfile`

* Fix capitalization of logs

* Preserve original formatting of `PythonApplicationGenerator.kt`
2022-10-11 12:03:44 +01:00
Harry Barber 78022d69ef
Remove Protocol enum (#1829)
* Remove Protocol enum

* Update Python implementation
2022-10-10 14:29:37 +01:00
Luca Palmieri 1b2b42d6ae
Support 32bits targets (#1811)
* Replace AtomicU64 with AtomicUsize to prevent compilation issues on 32 bits platforms.

* Make sure that Rust tests compile on MacOS.

* Add CHANGELOG next entry.
2022-10-06 10:17:47 +01:00
Burak 7fb7e177d0
Simplify Python Pokemon service (#1773)
* Simplify Python Pokemon service

* Use `threading.Lock` instead of `multiprocessing.Lock`

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>

* Explain why we need to pass `force=True`

* Add comment about synchronization of `Context` class

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2022-10-03 15:03:17 +01:00
John DiSanti 8e0b440437
Improve timeout config ergonomics and add SDK default timeouts (#1740)
* Overhaul timeout configuration
* Improve resiliency re-exports
* Refactor Smithy client retry config handling in builder
* Remove non-standard timeout env vars and profile keys
* Break operation customization codegen into its own file
* Move operation customization into a submodule
* Improve operation customization re-exports
* Add integration tests for read and connect timeouts
* Fix connector timeout config in the Smithy `Client` builder
* Rename `HttpSettings` to `ConnectorSettings`
2022-09-27 01:15:23 +00:00
Harry Barber 9405f00014
Cleanup pokemon.smithy (#1763)
* Remove Operation suffix

* Remove reuse of @output

* Add full-stops

* Rename to {Verb}{Qualifier}{Noun}

* Symlink pokemon operations

Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2022-09-23 17:37:32 +00:00
Matteo Bigoi e5c8cf3061
[Python] Allow to run pure Python request middlewares inside a Tower service (#1734)
## Motivation and Context
* Customers want to be able to implement simple middleware directly in Python. This PR aims to add the initial support for it.
* Improve the idiomatic experience of logging by exposing a handler compatible with Python's standard library `logging` module.

## Description
### Middleware
A middleware is defined as a sync or async Python function that can return multiple values, following these rules:

* Middleware not returning will let the execution continue without changing the original request.
* Middleware returning a modified Request will update the original request before continuing the execution.
* Middleware returning a Response will immediately terminate the request handling and return the response constructed from Python.
* Middleware raising MiddlewareException will immediately terminate the request handling and return a protocol specific error, with the option of setting the HTTP return code.
* Middleware raising any other exception will immediately terminate the request handling and return a protocol specific error, with HTTP status code 500.

Middlewares are registered into the Python application and executed in order of registration.

Example:

from sdk import App
from sdk.middleware import Request, MiddlewareException

app = App()

@app.request_middleware
def inject_header(request: Request):
    request.set_header("x-amzn-answer", "42")
    return request

@app.request_middleare
def check_header(request: Request):
    if request.get_header("x-amzn-answer") != "42":
        raise MiddlewareException("Wrong answer", 404)

@app.request_middleware
def dump_headers(request: Request):
    logging.debug(f"Request headers after middlewares: {request.headers()}")

**NOTE: this PR only adds support for request middlewares, which are executed before the operation handler. Response middlewares, executed after the operation are tracked here: https://github.com/awslabs/smithy-rs/issues/1754.**

### Logging
To improve the idiomatic experience, now logging need to be configured from the Python side by using the standard `logging` module. This allows customers to opt-out of our `tracing` based logging implementation and use their own and logging level is now driven by Python.

import logging
from sdk.logging import TracingHandler

logging.basicConfig(level=logging.INFO, handlers=[TracingHandler.handle()])

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak <burakvar@amazon.co.uk>
2022-09-22 18:01:48 +00:00