* Upgrade to Smithy 1.27.1
* Fix bugs exposed by 1.27.1
* Fix rules engine customization
* update rfc3339 parsing test
* refactor timestamp format & remove upstreamed test
* Update changelog
* rename test
* fix datetime parsing on server and clients
* fix up a few more failing tests
* precommit lints
* Fix bad merge
* Add Connection Poisoning to aws-smithy-client
* Fix doc links
* Remove required tokio dependency from aws-smithy-client
* Remove external type exposed
* Rename, re-add tokio dependency
* Change IP to 127.0.0.1 to attempt to fix windows
* Add dns::Name to external types
* Remove non_exhaustive not needed
* Add client target to changelog
* Remove `toEnumVariantName` from `RustSymbolProvider`
The `toEnumVariantName` function existed on symbol provider to work
around enum definitions not being shapes. In the future when we refactor
to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s
for each enum member. This change incrementally moves us to that future
by creating fake `MemberShape`s in the enum generator from the enum
definition.
* Fix escaping of `Self` in symbol providers
* Clean up an old hack
* Make `RustReservedWordsSymbolProvider` configurable
* Update changelog
* Incorporate feedback
* Add Plugin impl for &Plugin
* Add bespoke Either implementation
* Use Either in FilterByOperationName
* Add CHANGELOG.next.toml
* Rename A and B to Left and Right
* Re-export Either
* Add `timestamp` type test and fix the conversion error
* Add some tests for `ByteStream` and fix async issues
* Use `__anext__` method instead of `anext`
---------
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use `clientIntegrationTest` (WIP)
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` with `clientIntegrationTest`
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use generic test cases
* Start refactoring `ServerEventStreamUnmarshallerGeneratorTest`
* Make `ServerEventStreamUnmarshallerGeneratorTest` tests work
* Uncomment other test models
* Allow unused on `parse_generic_error`
* Rename `ServerEventStreamUnmarshallerGeneratorTest`
* Make `EventStreamUnmarshallTestCases` codegenTarget-agnostic
* Refactor `ClientEventStreamMarshallerGeneratorTest`: Tests run but fail
* Refactor `ServerEventStreamMarshallerGeneratorTest`
* Move `.into()` calls to `conditionalBuilderInput`
* Add "context" to TODO
* Fix client unmarshall tests
* Fix clippy lint
* Fix more clippy lints
* Add docs for `event_stream_serde` module
* Fix client tests
* Remove `#[allow(missing_docs)]` from event stream module
* Remove unused `EventStreamTestTools`
* Add `smithy-validation-model` test dep to `codegen-client`
* Temporarily add docs to make tests compile
* Undo change in model
* Make event stream unmarshaller tests a unit test
* Remove unused code
* Make `ServerEventStreamUnmarshallerGeneratorTest` a unit test
* Make `ServerEventStreamMarshallerGeneratorTest` a unit test
* Make `ServerEventStreamMarshallerGeneratorTest` pass
* Make remaining tests non-integration tests
* Make event stream serde module private again
* Remove unnecessary clippy allowances
* Remove clippy allowance
* Remove docs for `event_stream_serde` module
* Remove docs for `$unmarshallerTypeName::new`
* Remove more unnecessary docs
* Remove more superfluous docs
* Undo unnecessary diffs
* Uncomment last test
* Make `conditionalBuilderInput` internal
* Add builder symbol/module resolution to symbol providers
* Back out breaking changes
* Reduce codegen diff
* Fix server example clippy lints
* Associate #2396 with TODO comments
* Improve doc comment
* Revert doc hidden change
* Add support for the awsQueryCompatible trait
This commit adds support for the awsQueryCompatible trait. This allows
services already supporting custom error codes through the AWS Query
protocol with the awsQueryError trait to continue supporting them after
the services switch to the AWS JSON 1.0 protocol.
* Add copyright header
* Fix clippy warning for clippy::manual-map
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Remove unused variables from `errorScope`
This commit addresses https://github.com/awslabs/smithy-rs/pull/2398#discussion_r1114763528
* Reorder arguments for test verification
This commit addresses https://github.com/awslabs/smithy-rs/pull/2398#discussion_r1114766817
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
* Upgrade Kotlin to 1.7.21
* Upgrade Ktlint to 0.48.2
* Run `pre-commit run --all-files` to fix broken Ktlints
* Fix string comparison broken by code formatting
* feat(aws-types): add api key to configuration
* chore: set package version to 0.52.0
* feat(aws-smithy-types): create auth types
* chore: use auth from smithy types
* chore: fix return self type
* chore: create http auth definition type
* chore: add constructor for http auth definition
* chore: ensure properties are not moved
* chore: create convenience constructors
* chore: add some todo comments
* feat: move query writer to aws-smithy-http crate
* chore(codegen): expose smithy http tower dependency
* chore: remove setters for auth definition
* chore: fix logical error for scheme not allowed
* chore: add constructor for basic and digest auth
* chore: allow equality comparision for api key
* Revert "chore: set package version to 0.52.0"
This reverts commit da660fcf16.
* chore: fix additional references to querywriter
* chore: implement from string for api key struct
* chore: disallow none api key in sdk config
* chore: fix formatting
* chore: add unit tests for auth types
* chore: add auth api key to external types
* chore: make query writer doc hidden
* feat: create aws-smithy-http-auth crate
* chore: use zeroing for auth api key
* chore: use builder pattern for auth definition
* chore: restructure http auth package
* chore: define default lint warning
* chore: include http auth in runtime common list
* chore: define setter for optional scheme
* chore: should panic while building auth definition
* chore: return missing required field error
* chore: make few code simplications for api key
* Revert "chore: add auth api key to external types"
This reverts commit b2318b0230.
* chore: revert api key from sdk config
* chore: panic on missing required field
* Opt out of `clippy::derive_partial_eq_without_eq`
---------
Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* update: use `enforce_http = false` when creating native-tls hyper connector
refactor: move smithy conns module to its own file
add: sanity tests ensuring we can make HTTP and HTTPS requests with the rustls and native-tls connectors
remove: `use crate::*` imports in favor of explicit imports
* update: CHANGELOG.next.toml
* add: feature gate to conns tests
* Add `RequestId` trait
* Implement `RequestId` for generated AWS client errors
* Move `RustWriter.implBlock` out of `StructureGenerator`
* Create structure/builder customization hooks
* Customize `_request_id` into AWS outputs
* Set request ID on outputs
* Refactor SDK service decorators
* Refactor S3's extended request ID implementation
* Combine `Error` and `ErrorKind`
* Add test for service error conversion
* Move error generators into `codegen-client` and fix tests
* Re-export `ErrorMetadata`
* Add request IDs to trace logs
* Simplify some error trait handling
* Rename `ClientContextParamDecorator` to `ClientContextConfigCustomization`
* Add deprecated alias to guide customers through upgrading
* Rename the `ErrorMetadata` trait to `ProvideErrorMetadata`
* Rename `aws_smithy_types::Error` to `ErrorMetadata`
* Initial Python stub generation
* Handle default values correctly
* Only generate `__init__` for classes that have constructor signatures
* Preserve doc comments
* Make context class generic
* Put type hint into a string to fix runtime error
* Run `mypy` on CI
* Use `make` to build Python SSDKs while generating diffs
* Escape Python types in Rust comments
* Only mark class methods with
* Sort imports to minimize diffs
* Add type annotations for `PySocket`
* Dont extend classes from `object` as every class already implicitly extended from `object`
* Use `vars` instead of `inspect.getmembers` to skip inherited members of a class
* Fix linting issues
* Add some tests for stubgen and refactor it
* Add type annotations to `PyMiddlewareException`
* Fix tests on Python 3.7
Python 3.7 doesn't support reading signatures from `__text_signature__`
for non-builtin functions (i.e. C/Rust functions). For testing we're using
regular Python syntax for defining signature.
* Provide default values for `typing.Optional[T]` types in type-stubs
* Update `is_fn_like` to cover more cases
* Remove `tools/smithy-rs-tool-common/`
* Make `DECORATORS` an array instead of a list
* Ignore missing type stub errors for `aiohttp`
* Enforce the same minimum TLS version (1.2) for both TLS backends
* Add CHANGELOG entry
* Add documentation for both `https` and `native_tls`.
* Remove unnecessary mut
* Do not alter Operation shape ID
* Add OperationExtensionExt test
* Add CHANGELOG.next.toml entry
* Apply suggestions from code review
Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>
---------
Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>
* Move `acquire-build-image` into `.github`
* Move the `tools-hash` script into `.github`
* Upload to ECR from PRs as well
* Move build tools into `tools/ci-build/`
* Move CI scripts out of `ci-build`
* Split CI for forks/non-forks
* Remove `fetch-depth: 0` from PR workflows
* Add Python wrapper for `aws_smithy_types::Document`
* Remove unused type
* Make sure Python SSDKs uses Python specific `Document` type
* Allow Python specific `Document` type to be used in serialization and deserialization
* Make `pyo3/extension-module` a default feature
* Add `PythonServerTypesTest`
* Fix linting issues
In server SDKs, these traits can be implemented by any shape _except_ if
the shape's closure contains:
1. A `float`, `double`, or `document` shape: floating point types in
Rust do not implement `Eq`. Similarly, [`document` shapes] may
contain arbitrary JSON-like data containing floating point values.
2. A [@streaming] shape: all the streaming data would need to be
buffered first to compare it.
Additionally, the `Hash` trait cannot be implemented by shapes whose
closure contains:
1. A `map` shape: we render `map` shapes as `std::collections::HashMap`,
which _do not_ implement `Hash`. See
https://github.com/awslabs/smithy/issues/1567.
In **client SDKs, these traits cannot be derived on any code-generated
Rust types corresponding to Smithy shapes**, since e.g. adding new
optional members to a structure [is a backwards-compatible change], and
doing so alters the semantics of these traits.
However, this commit does implement these traits for the
`aws_smithy_types::date_time::DateTime` and `aws_smithy_types::Blob`
runtime types.
This change is necessary to efficiently implement the `@uniqueItems`
constraint trait in server SDKs.
[`Eq`]: https://doc.rust-lang.org/std/cmp/trait.Eq.html
[`Hash`]: https://doc.rust-lang.org/std/hash/trait.Hash.html
[`document` shapes]: https://smithy.io/2.0/spec/simple-types.html#document
[@streaming]: https://smithy.io/2.0/spec/streaming.html
[is a backwards-compatible change]: https://smithy.io/2.0/guides/evolving-models.html#updating-structures
* Add support for operationInput tests
* More unfication, fix tests, docs
* Set endpoint_url only when endpoint_url is used
* Fix test-util feature
* CR feedback
* fix missing path
* wip
* Fix region decorator
* Update S3 tests to succeed
* Create 'endpoint_url' setters
* Fix SDK adhoc tests
* Fix endpoint tests
* Fix protocol test generator to have a stub endpoint resolver
* Fix some more tests
* Fix aws rust runtime tests
* Update generator to appease clippy
* CR feedback
* Fix compilation
* Fix tests
* Fix doc links
* Fix SDK integration tests
* Update changelog
* Fix s3 control by adding transformer
* Throw a specific exception if the service doesn't have ep rules
* Add codecatalyst to the list of custom services
This change creates `ClientCodegenDecorator` and
`ServerCodegenDecorator` in `codegen-client` and `codegen-server`
respectively to replace `RustCodegenDecorator`. Client/server
equivalents are created to replace `CombinedCodegenDecorator` as well.
This eliminates the need for the `supportsCodegenContext` method since
the decorator interface is no longer generic, so the `ServiceLoader` is
now powerful enough to differentiate.
The largest benefit, however, is that now clients and servers can have
separate customizations.
* Remove `lib` prefix from generated module names
* Build Pokemon service with `Maturin`
* Fix Maturin build on CI
* Generate minimal `pyproject.toml` for generated SDKs to build from source using Maturin
* Fix `ktlint` issues
* Bring back type stubs for Pokemon service
* Update instructions for Lambda
* Make `build-wheel` and `build-wheel-release` to depend on `codegen`
* Provide Python wrappers for Lambda related types
* Introduce `PyContext` to wrap raw context object
* Use new `PyContext` in handlers
* Expose `lambda` module to Python
* Use `LambdaContext` in example service
* Start Lambda handler in a different thread
* Print summary of Lambda context in Pokemon service
* Make sure to include Python `builtins` in tests
* Make `lambda_ctx` optional
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Only inject types if they are type-hinted as `Optional[T]`
* Export Lambda module as `aws_lambda` instead of `lambda_`
* Comment why we need to run Hyper server in a background thread
* Move `is_optional_of` to `util` module
* Use `HeaderMap::from_iter` to build headers
* Support edge case of `(None, T)` in `util::is_optional_of`
* Make Lambda related types feature gated
* Remove feature gate for Lambda
* Make `xray_trace_id` an `Option`
* Remove `aws-lambda` feature from generated `Cargo.toml`s
* Fix linting issues
* Pin `lambda_runtime` to `0.7.1`
* Remove duplicate dependency in `Cargo.toml`
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Add `PyTlsConfig` struct
* Support TLS server
* Add TLS support to Pokemon service
* Make sure to create `tokio::net::TcpListener` in a Tokio context
* Fix doc link
* Add missing imports in tests
* Add `tls::Listener`
* Reload TLS config periodically
* Add context to `TODO`
* Return `&'static str` from `base_url()`
* Flatten `match` in `tls::Listener`
* Propogate listener errors but ignore handshake errors in `tls::Listener`
* Add tests to `tls::Listener`
* Add test to `tls::Listener` to make sure we are propogating listener errors
* Use `PathBuf` instead of plain `String`s for paths
* Emit spans for implementers of map request middleware traits
* Instrument dispatch with its own span
* Fix trace span hierarchy
* Partially flatten the middleware span hierarchy
* Make `MapRequest::name` required
* Add sub-spans to the `load_response` span
* Split endpoint resolution middleware into two parts & refactor endpoint generation
* Endpoints 2.0 Integration pre-work
This PR does a 3 bits of pre-work ahead of ep2 integration:
1. Split endpoint resolution into two separate middlewares:
1. A smithy native middleware that applies URI and headers
2. An AWS middleware that applies the auth schemes
2. Add vendorParams support to the ProtocolTestGenerator so that protocol tests can insert a region.
3. Simplify endpoint resolution logic by allowing `make_operation` to fail when an endpoint cannot be resolved.
* Back out previous change to insert endpoint directly into the bag
* backout changes to property bag
* Update changelog & add more docs
* Fix AWS test
* Fix test
* Link to super from Handler and OperationService
* Note that structure documentation is from model
* Don't refer to ZSTs in operation_shape.rs opener
* Re-export everything from service to root
* Add reference to root documentation on service
* Fix spelling of MissingOperationsError
* #[doc(hidden)] for opaque_future
* Rename from-parts.md to from_parts.md
* Fix Connected link
* Use S type parameter and service as variable name
* Remove MissingOperation dead code
* Remove Operation header from operation.rs
* Remove reference to closures
* Document ConnectInfo<T> .0
* Add BoxBody documentation
* Rephrase operation.rs documentation
* Reference PluginPipeline from PluginStack
* Move the BoxBody documentation
* Document Plugin associated types
* Add example implementation for Plugin
* Link to plugin module from Plugin
* Improve FromParts/FromRequest documentation
* Remove links to doc(hidden) RuntimeError
* Add link from Upgradable to operation module
* Add Endpoint Resolver Implementation
This commit adds `EndpointDecorator`, standard libary + tests that can be used to add endpoints 2.0 to the AWS SDK.
It _does not_ actually wire these things up. We'll follow up with a PR that actually integrates everything.
* CR Feedback
* CR feedback II
* Introduce an `aws-lambda` feature.
* Add CHANGELOG
* Use the new and shiny feature syntax to avoid creating an implicit `lambda_http` feature.
* Add `aws-lambda` feature to the Python server. Enable the `aws-lambda` feature in our example. Be explicit in providing an implementation of request rejection for Box<dyn Error>.
* Add an `aws-lambda` feature flag to the generated Python-specific crate. We enable it by default to make sure the Lambda method ends up visible in the final Python wrapper library.
- `Endpoint::set_endpoint` no longer panics when called on an endpoint
without a scheme
- `Endpoint::mutable` and `Endpoint::immutable` now both return a result
so that constructing an endpoint without a scheme is an error
- `Endpoint::mutable` and `Endpoint::immutable` both now take a string
instead of a `Uri` as a convenience
- `Endpoint::mutable_uri` and `Endpoint::immutable_uri` were added
to construct an endpoint directly from a `Uri`
* Implement RFC 23, with the exception of PluginBuilder
* Update documentation.
* Avoid star re-exports.
* Elide implementation details in `Upgradable`.
* Update wording in docs to avoid personal pronouns.
* Update `Upgradable`'s documentation.
* Template the service builder name.
* Template MissingOperationsError directly.
* Code-generate an array directly.
* Sketch out the implementation of `PluginPipeline`.
Remove `PluginExt`.
Add a public constructor for `FilterByOperationName`.
* Ask for a `PluginPipeline` as input for the generated builder.
* Rename `add` to `push`.
* Remove Pluggable.
Rename `composer` module to `pipeline`.
* Remove all mentions of `Pluggable` from docs and examples.
* Fix punctuation.
* Rename variable from `composer` to `pipeline` in doc examples.
* Add a free-standing function for filtering.
* Rename.
* Rename.
* Update design/src/server/anatomy.md
Co-authored-by: david-perez <d@vidp.dev>
* Use `rust` where we do not need templating.
* Remove unused variable.
* Add `expect` message to point users at our issue board in case a panic slips through.
* Typo.
* Update `expect` error message.
* Refactor the `for` loop in ``requestSpecMap` into an `associateWith` call.
* Remove unnecessary bound - since `new` is private, the condition is already enforced via `filter_by_operation_name`.
* Use `Self` in `new`.
* Rename `inner` to `into_inner`
* Rename `concat` to `append` to correctly mirror Vec's API terminology.
* Fix codegen to use renamed method.
* Cut down the public API surface to key methods for a sequence-like interface.
* Add a note about ordering.
* Add method docs.
* Add Default implementation.
* Fix new pokemon bin example.
* Fix new pokemon bin example.
* Fix code-generated builder.
* Fix unresolved symbolProvider.
* Assign the `expect` error message to a variable.
* Do not require a PluginPipeline as input to `builder_with_plugins`.
* Reverse plugin application order.
* Upgrade documentation.
* Add a test to verify that plugin layers are executed in registration order.
* Add license header.
* Update middleware.md
* Typo.
* Fix more builder() calls.
Co-authored-by: david-perez <d@vidp.dev>
* Implement RFC 23, with the exception of PluginBuilder
* Update documentation.
* Elide implementation details in `Upgradable`.
* Update wording in docs to avoid personal pronouns.
* Update `Upgradable`'s documentation.
* Template the service builder name.
* Template MissingOperationsError directly.
* Code-generate an array directly.
* Update design/src/server/anatomy.md
Co-authored-by: david-perez <d@vidp.dev>
* Use `rust` where we do not need templating.
* Remove unused variable.
* Add `expect` message to point users at our issue board in case a panic slips through.
* Typo.
* Update `expect` error message.
* Refactor the `for` loop in ``requestSpecMap` into an `associateWith` call.
* Fix new pokemon bin example.
* Fix new pokemon bin example.
* Fix unresolved symbolProvider.
* Assign the `expect` error message to a variable.
* Omit additional generic parameters in Upgradable when it's first introduced.
Co-authored-by: david-perez <d@vidp.dev>
This patchset, affectionately called "Builders of builders", lays the
groundwork for fully implementing [Constraint traits] in the server SDK
generator. [The RFC] illustrates what the end goal looks like, and is
recommended prerrequisite reading to understanding this cover letter.
This commit makes the sever deserializers work with _unconstrained_ types
during request parsing, and only after the entire request is parsed are
constraints enforced. Values for a constrained shape are stored in the
correspondingly unconstrained shape, and right before the operation input is
built, the values are constrained via a `TryFrom<UnconstrainedShape> for
ConstrainedShape` implementation that all unconstrained types enjoy. The
service owner only interacts with constrained types, the unconstrained ones are
`pub(crate)` and for use by the framework only.
In the case of structure shapes, the corresponding unconstrained shape is their
builders. This is what gives this commit its title: during request
deserialization, arbitrarily nested structures are parsed into _builders that
hold builders_. Builders keep track of whether their members are constrained or
not by storing its members in a `MaybeConstrained`
[Cow](https://doc.rust-lang.org/std/borrow/enum.Cow.html)-like `enum` type:
```rust
pub(crate) trait Constrained {
type Unconstrained;
}
#[derive(Debug, Clone)]
pub(crate) enum MaybeConstrained<T: Constrained> {
Constrained(T),
Unconstrained(T::Unconstrained),
}
```
Consult the documentation for the generator in `ServerBuilderGenerator.kt` for
more implementation details and for the differences with the builder types the
server has been using, generated by `BuilderGenerator.kt`, which after this
commit are exclusively used by clients.
Other shape types, when they are constrained, get generated with their
correspondingly unconstrained counterparts. Their Rust types are essentially
wrapper newtypes, and similarly enjoy `TryFrom` converters to constrain them.
See the documentation in `UnconstrainedShapeSymbolProvider.kt` for details and
an example.
When constraints are not met, the converters raise _constraint violations_.
These are currently `enum`s holding the _first_ encountered violation.
When a shape is _transitively but not directly_ constrained, newtype wrappers
are also generated to hold the nested constrained values. To illustrate their
need, consider for example a list of `@length` strings. Upon request parsing,
the server deserializers need a way to hold a vector of unconstrained regular
`String`s, and a vector of the constrained newtyped `LengthString`s. The former
requirement is already satisfied by the generated unconstrained types, but for
the latter we need to generate an intermediate constrained
`ListUnconstrained(Vec<LengthString>)` newtype that will eventually be
unwrapped into the `Vec<LengthString>` the user is handed. This is the purpose
of the `PubCrate*` generators: consult the documentation in
`PubCrateConstrainedShapeSymbolProvider.kt`,
`PubCrateConstrainedCollectionGenerator.kt`, and
`PubCrateConstrainedMapGenerator.kt` for more details. As their name implies,
all of these types are `pub(crate)`, and the user never interacts with them.
For users that would not like their application code to make use of constrained
newtypes for their modeled constrained shapes, a `codegenConfig` setting
`publicConstrainedTypes` has been added. They opt out of these by setting it to
`false`, and use the inner types directly: the framework will still enforce
constraints upon request deserialization, but once execution enters an
application handler, the user is on their own to honor (or not) the modeled
constraints. No user interest has been expressed for this feature, but I expect
we will see demand for it. Moreover, it's a good stepping stone for users that
want their services to honor constraints, but are not ready to migrate their
application code to constrained newtypes. As for how it's implemented, several
parts of the codebase inspect the setting and toggle or tweak generators based
on its value. Perhaps the only detail worth mentioning in this commit message
is that the structure shape builder types are generated by a much simpler and
entirely different generator, in
`ServerBuilderGeneratorWithoutPublicConstrainedTypes.kt`. Note that this
builder _does not_ enforce constraints, except for `required` and `enum`, which
are always (and already) baked into the type system. When
`publicConstrainedTypes` is disabled, this is the builder that end users
interact with, while the one that enforces all constraints,
`ServerBuilderGenerator`, is now generated as `pub(crate)` and left for
exclusive use by the deserializers. See the relevant documentation for the
details and differences among the builder types.
As proof that these foundations are sound, this commit also implements the
`length` constraint trait on Smithy map and string shapes. Likewise, the
`required` and `enum` traits, which were already baked in the generated types
as non-`Option`al and `enum` Rust types, respectively, are now also treated
like the rest of constraint traits upon request deserialization. See the
documentation in `ConstrainedMapGenerator.kt` and
`ConstrainedStringGenerator.kt` for details.
The rest of the constraint traits and target shapes are left as an exercise to
the reader, but hopefully the reader has been convinced that all of them can be
enforced within this framework, paving the way for straightforward
implementations. The diff is already large as it is. Any reamining work is
being tracked in #1401; this and other issues are referenced in the code as
TODOs.
So as to not give users the impression that the server SDK plugin _fully_
honors constraints as per the Smithy specification, a validator in
`ValidateUnsupportedConstraintsAreNotUsed.kt` has been added. This traverses
the model and detects yet-unsupported parts of the spec, aborting code
generation and printing informative warnings referencing the relevant tracking
issues. This is a regression in that models that used constraint traits
previously built fine (even though the constraint traits were silently not
being honored), and now they will break. To unblock generation of these models,
this commit adds another `codegenConfig` setting,
`ignoreUnsupportedConstraints`, that users can opt into.
Closes#1714.
Testing
-------
Several Kotlin unit test classes exercising the finer details of the added
generators and symbol providers have been added. However, the best way to test
is to generate server SDKs from models making use of constraint traits. The
biggest assurances come from the newly added `constraints.smithy` model, an
"academic" service that _heavily_ exercises constraint traits. It's a
`restJson1` service that also tests binding of constrained shapes to different
parts of the HTTP message. Deeply nested hierarchies and recursive shapes are
also featured.
```sh
./gradlew -P modules='constraints' codegen-server-test:build
```
This model is _additionally_ generated in CI with the `publicConstrainedTypes`
setting disabled:
```sh
./gradlew -P modules='constraints_without_public_constrained_types' codegen-server-test:build
``````
Similarly, models using currently unsupported constraints are now being
generated with the `ignoreUnsupportedConstraints` setting enabled.
See `codegen-server-test/build.gradle.kts` for more details.
[Constraint traits]: https://awslabs.github.io/smithy/2.0/spec/constraint-traits.html
[The RFC]: https://github.com/awslabs/smithy-rs/pull/1199
* feature: make HTTP connectors configurable
* add: test for HTTP connector configuration customization
add: impl<B> From<TestConnection<B>> for HttpConnector
add: impl From<CaptureRequestHandler> for HttpConnector
add: impl From<NeverConnector> for HttpConnector
add: impl From<ReplayingConnection> for HttpConnector
* add: to_vec method to AggregatedBytes
update: method param names of FluentClientGenerics.sendBounds to be more explicit
update: restructure s3/s3control tests to be uniform in structure
* update: CHANGELOG.next.toml
update: codegen `impl From<&SdkConfig> for Builder` to support HTTP connectors
* update: CHANGELOG entry references
* add: missing copyright header
* fix: clippy lint
* format: run cargo fmt
* format: run cargo fmt on aws_smithy_client::dvr modules
* format: run ktlintFormat
* refactor: use from_conf instead of from_conf_conn
remove: from_conf_conn
* update: impl From<SmithyConnector> for HttpConnector
remove: other From<T> for HttpConnector impls
update: HttpConnector config setter examples
* update: CHANGELOG.next.toml
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* update: CHANGELOG.next.toml
remove: obsolete test
update: `ConfigLoader::http_connector` setter method
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Apply suggestions from code review
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* update: aws_config::loader::ConfigLoader doc comments
update: CHANGELOG.next.toml examples
* fix: doc issues
add: reëxport aws_smithy_types::endpoint module to aws-config
* format: run rustfmt in the weird CI way to get it to actually format.
* fix: incorrect reëxport
* add: "aws_smithy_http::endpoint" to allowed external types for aws-config
* update: move `hyper-rustls` to deps so that it doesn't break exotic arch CI check
* remove: `hyper-rustls` dep because it's not actually needed
* fix: aws-types dep issue blocking exotic arch CI check
* fix: broken doc comment
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Do not use deprecated from_timestamp from chrono
CI is failing because
[from_timestamp](https://docs.rs/chrono/latest/chrono/naive/struct.NaiveDateTime.html#method.from_timestamp)
is now deprecated.
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Update changelog
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Fix error
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Use with_ymd_and_hms
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Python: map Python middlewares to Tower layers
* Make middleware layer infallible
* Use message and status code from `PyMiddlewareException`
* Introduce `FuncMetadata` to represent some information about a Python function
* Improve middleware errors
* Add missing copyright headers
* Allow accessing and changing request body
* Allow changing response
* Add some documentation about moving data back-and-forth between Rust and Python
* Add `mypy` to Pokemon service and update typings and comments for middlewares
* Add or update comments on the important types
* Add Rust equivalent of `collections.abc.MutableMapping`
* Add `PyHeaderMap` to make `HeaderMap` accessible from Python
* Apply suggestions from code review
Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>
* Improve logging
* Add `RichPyErr` to have a better output for `PyErr`s
* Better error messages for `PyMiddlewareError` variants
* Factor out repeating patterns in tests
* Preserve `__builtins__` in `globals` to fix tests in Python 3.7.10 (our CI version)
* Export `RichPyErr` to fix `cargo doc` error
* Apply suggestions from code review
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Add missing SPDX headers
* Document that `keys`, `values` and `items` on `PyMutableMapping` causes clones
Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Replace our base64 implementation with one based on `base64-simd`
* Bump Rust MSRV to 1.62.1
* Unmark base64-related tests as failing
* Use `usize` for `base64::encoded_length`
* Run Rust 1.62.1 on CI
* Don't use external `base64_simd::Error` type in public interface
* Add initial criterion bench setup for base64
* Compare `base64_simd::encode` vs Smithy's `encode`
* Use string length for `BenchmarkId`
* Compare `base64_simd::decode` vs Smithy's `decode`
* Compare `base64_simd::encoded_length` vs Smithy's `encoded_length`
* Avoid the chrono crate depending on the time crate
This commit is in response to RUSTSEC-2020-0071 where there is a
potential segfault in the time crate. The aws-smithy-types-convert
crate now disables the default features of the chrono crate so that
it will not depend on the time crate.
* Depend on lambda_http without RUSTSEC-2020-0071
This commit updates the version of lambda_http used by aws-smithy-http-server
and aws-smithy-http-server-python to 0.7.0. The prior version 0.6.0 of
lambda_http used the chrono crate in a way that exposed a security issue
described in RUSTSEC-2020-0071. By switching to 0.7.0 of lambda_http, those
two crates do not exhibit vulnerabilities as reported by cargo audit.
* Bump minor version of lambda_http in pokemon-service
This commit updates the version of `lambda_http` used by `pokemon-service`
from 0.6.0 to 0.7.0. This is in sync with the fact that both
`aws-smithy-http-server` and `aws-smithy-http-server-python` now depend
on 0.7.0 of `lambda_http`. Failing to do so would cause `pokemon-service`
to fail to compile due to an error at `lambda_http::run(handler)` in the
main function of the `pokemon-service-lambda` binary:
the trait `Service<lambda_http::http::Request<lambda_http::Body>>` is not
implemented for `LambdaHandler<aws_smithy_http_server::routing::Router>`
* Depend on lambda-http 0.7.1
This commit updates the version of `lambda_http` from 0.7.0 to 0.7.1 in
the crates within the top-level `rust-runtime` workspace. These updates
are needed to solve the issue described in awslabs/aws-lambda-rust-runtime#556
* Update CHANGELOG.next.toml
* Address https://github.com/awslabs/smithy-rs/pull/1907\#pullrequestreview-1161609833
Co-authored-by: Saito <awsaito@c889f3b5ddc4.ant.amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* update: add operation metadata to property bag during `make_operation`
add: test ensuring metadata is added to property bag
add: CHANGELOG.next.toml entry
* update: use new strategy for op metadata insertion
update: a new strategy requires a new test
update: CHANGELOG.next.toml
* format: run cargo fmt
* Route::new in upgrade
Move the creation of Routes in Upgradable to have a cleaner sequence of
constraints in ServiceBuilder::build
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Replace bool with enum in label::fmt_string
This commit addresses code smell where the said function takes a bool
to decide whether it percent-encodes the UTF-8 encoding of the given
string. For clarity, we define an enum EncodingStrategy so that we
can see the callers' intent at the call sites by reading the variant
names rather than true/false.
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Make test use updated function signature of fmt_string
This commit modifies a call site for the updated function signature of
`aws_smithy_http:🏷️:fmt_string`, otherwise the test would fail to
compile, leading to a failure in CI.
Co-authored-by: Saito <awsaito@c889f3b5ddc4.ant.amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Use hmac and sha2 instead of ring on powerpc
* Enable aws-sig-auth in CI
* Update CHANGELOG
* Run tests against exotic platforms
* Run tests only against aws rust runtime
* PowerPC 32 and 64 bit should be fully testable now
* Maybe this time build and test will work
* Add licence header to hmac.rs
* Properly use finalized_fixed
* Revert leftover
* Temporary disable crc32c test on powerpc
* Temporary disable system_time_conversion_test on 32bit CPUs
* Disable other 3 tests on 32bit
* Temporarily disable last test
* Update CHANGELOG and document TODOs with issues
* Run aws-smithy-client tests in CI with crosscompiled local openssl
* Simplify CI script
* Use correct curl options
* Use the right OS for i686
* Looks like I finally foung the right os type for i686
* Add `tcp` feature to `hyper` to get tests compiling.
* Enable verbose logging to debug CI failure in cross.
* Use pre-built openSSL on i686
* Fix empty spaces.
* Set environment variables based on matrix.target
* Remove all usages of `ring` from `aws-sigv4`. It ensures broader platform compatibility and higher performance.
* Update changelog entries.
* Remove redundant dev dependencies.
Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com>
* Run clippy --fix for Rust 1.62.0 and format the resulting code
* Run clippy --fix on all targets
* Run clippy --fix with all features enabled
* Avoid extra allocation
* Use more idiomatic assert
* Ignore noisy lint
* Update pyo3 and pyo3-asyncio to 0.17.0
* Implement Eq on aws-smithy-checksums::Error
* Implement Eq on Protocol
* Replace conditionals with range-containment
* Implement Eq on types in aws-smithy-types
* Implement Eq on types in aws-smithy-http-server-python
* Implement Eq on types in aws-smithy-eventstream
* Implement Eq on types in aws-smithy-xml
* Implement Eq on aws-sigv4
* Update CI to use Rust 1.62.0
* Add Eq for generated types that implement PartialEq
* Allow clippy::needless_return in generated code
* Remove unnecessary reborrow in http_serde
* Remove unnecessary borrow in operation_deser
* Add CHANGELOG entries
* Revert "Add Eq for generated types that implement PartialEq"
This reverts commit 5169bd95aa.
* Update pyo3 and pyo3-asyncio in generated code for python server
* Allow clippy::derive_partial_eq_without_eq on structs and builders
* Run clippy on tools
* Fix accidental move in generated code
* Revert "Allow clippy::derive_partial_eq_without_eq on structs and builders"
This reverts commit 068c63ca20.
* Fix another accidental move in generated code
* Undo unwanted change to model
* Re-add reborrow in HttpBindingGenerator
* Fix clippy::format-push-string in changelogger
* Fix more uses of str.push_str(&format!(...))
* Remove unnecessary parenthesis
* Run ktlint
* Update aws/rust-runtime/aws-http/src/content_encoding.rs
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-http/src/content_encoding.rs
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Ignore doctest for non-exported macro
Rust 1.62 introduced a breaking change where doctests for non-exported
macros are now run by default. These don't compile because the macro
can't be imported in it.
See https://github.com/rust-lang/rust/issues/97030 for more info.
* Run cargo fmt
* Use $crate instead of crate in macro
* Revert "Implement Eq on types in aws-smithy-types"
This reverts commit c45a6b5a56.
* Revert "Implement Eq on types in aws-smithy-eventstream"
This reverts commit 78f4b07344.
* Revert "Implement Eq on types in aws-smithy-xml"
This reverts commit 590f01af73.
* Revert "Implement Eq on aws-sigv4"
This reverts commit d78bb62124.
* Revert "Implement Eq on types in aws-smithy-http-server-python"
This reverts commit f2cd901884.
* Revert "Implement Eq on aws-smithy-checksums::Error"
This reverts commit 5da170405e.
Co-authored-by: Julian Antonielli <julianantonielli@gmail.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Allow running Python servers with Lambda handler
* Fix doctests
* Add `Running servers on AWS Lambda` section to docs
* Remove empty spaces from `README` to make linter happy
* Rename `run_lambda_server` to `run_lambda_handler`
* Add more comments to example `Dockerfile`
* Fix capitalization of logs
* Preserve original formatting of `PythonApplicationGenerator.kt`
* Replace AtomicU64 with AtomicUsize to prevent compilation issues on 32 bits platforms.
* Make sure that Rust tests compile on MacOS.
* Add CHANGELOG next entry.
* Add a new CI job to check that smithy-rs compiles on 32bit Linux-based targets.
* No trailing commas pretty please.
* Point cross at the manifest explicitly.
* Skip crates with a Python dependency. Exercise all features.
* Install required dependencies.
* Do not fail fast, we want to see the result on all platforms.
* Set paths for both commands.
* Openssl must be installed inside the cross Docker container, not on the host machine.
* Fix connector setup if `rustls` feature is not enabled.
* Restrict feature set on powerpc.
* Pass openssl env variables to the cross Docker container
* Split in two commands.
* Enable debug level logs.
* Remove openssl feature (temporarily).
* Raise verbosity.
* Trigger CI
* `native-tls`, here we go again.
* Clean up.
* Trigger CI
* Trigger CI
* Add pkg-config.
* Change include path to include arch
* Trigger CI
* Allow workflow_dispatch on ci-pr to enable triggering this CI workflow from the GitHub web interface.
* Trigger CI? Why are you doing this to me GitHub?
* Trigger CI? Are you alive GitHub?
* Fix env variables for openssl
* Use features only for rust-runtime crates.
* Check all feature combinations for aws-smithy-client
* Dry-up env variables.
* A rogue `echo` was missing
* Feature-gate doc tests based on the features they require.
* Put .github folder under shared ownership.
* Fix docs.
* Fix feature selection for doctest.
* We are using methods that are only available if rustls is enabled - adjust feature gates accordingly.
* Remove workflow dispatch trigger.
* Replace AtomicU64 with AtomicUsize to prevent compilation issues on 32 bits platforms.
* Make sure that Rust tests compile on MacOS.
* Add CHANGELOG next entry.
* Simplify Python Pokemon service
* Use `threading.Lock` instead of `multiprocessing.Lock`
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Explain why we need to pass `force=True`
* Add comment about synchronization of `Context` class
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Add documentation covering instrumentation approaches for Smithy Rust.
* Tweak the logging in the Pokemon service to better exemplify instrumentation.
* Remove `TraceLayer` which violates sensitivity contract.
* Switch to [Pretty](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/fmt/format/struct.Pretty.html) logs to better showcase the outputs.
* Update [Logging in the Presence of Sensitive Data](https://github.com/awslabs/smithy-rs/pull/1536) checklist.
* Rename `logging` module to `instrumentation` to improve coherence across struct names and documentation.
* Simplify sensitivity detection logic by assuming bindings only exist at top level input/output.
* Fix greedy label logic - they can now exist alongside of normal labels.
* Fix `httpPrefixHeader` false positive.
* Export `RequestFmt` and `ResponseFmt` types alongside the closures.
* More extensive use of `Writable` to improve readability.
## Motivation and Context
* Customers want to be able to implement simple middleware directly in Python. This PR aims to add the initial support for it.
* Improve the idiomatic experience of logging by exposing a handler compatible with Python's standard library `logging` module.
## Description
### Middleware
A middleware is defined as a sync or async Python function that can return multiple values, following these rules:
* Middleware not returning will let the execution continue without changing the original request.
* Middleware returning a modified Request will update the original request before continuing the execution.
* Middleware returning a Response will immediately terminate the request handling and return the response constructed from Python.
* Middleware raising MiddlewareException will immediately terminate the request handling and return a protocol specific error, with the option of setting the HTTP return code.
* Middleware raising any other exception will immediately terminate the request handling and return a protocol specific error, with HTTP status code 500.
Middlewares are registered into the Python application and executed in order of registration.
Example:
from sdk import App
from sdk.middleware import Request, MiddlewareException
app = App()
@app.request_middleware
def inject_header(request: Request):
request.set_header("x-amzn-answer", "42")
return request
@app.request_middleare
def check_header(request: Request):
if request.get_header("x-amzn-answer") != "42":
raise MiddlewareException("Wrong answer", 404)
@app.request_middleware
def dump_headers(request: Request):
logging.debug(f"Request headers after middlewares: {request.headers()}")
**NOTE: this PR only adds support for request middlewares, which are executed before the operation handler. Response middlewares, executed after the operation are tracked here: https://github.com/awslabs/smithy-rs/issues/1754.**
### Logging
To improve the idiomatic experience, now logging need to be configured from the Python side by using the standard `logging` module. This allows customers to opt-out of our `tracing` based logging implementation and use their own and logging level is now driven by Python.
import logging
from sdk.logging import TracingHandler
logging.basicConfig(level=logging.INFO, handlers=[TracingHandler.handle()])
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak <burakvar@amazon.co.uk>
* Allow custom layers to every operation
Third-party developers can add custom functions to a service builder,
through a `BuilderModifier`.
These functionalities are applied to every operation of that service.
An example of a builder modifier can be found in
rust-runtime/aws-smithy-http-server/examples/pokemon-service/lib.rs
Signed-off-by: Harry Barber <hlbarber@amazon.co.uk>
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: Harry Barber <hlbarber@amazon.co.uk>
* Fix retry for native Smithy clients
* Treat `SdkError::ResponseError` as a retryable transient failure
* Rename `ClassifyResponse` to `ClassifyRetry`
* Rename 'policy' to 'classifier' in AWS SDK public API
* Rename `AwsResponseClassifier` to `AwsResponseRetryClassifier`
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Add validation for the Content-Type header and pass (remove from the failing
list) the relevant protocol tests
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Add `ServerProtocol` interface to allow for server side protocol specific methods.
* Make public the structs merged in https://github.com/awslabs/smithy-rs/pull/1679.
* Add `ServerOperationGenerator`, which generates a ZST and implements `OperationShape` on it.
* Add `ServerServiceGeneratorV2`, which generates the service newtype around a router and a service builder.
* Add `hidden` argument to `RustModule` which allows modules to be marked with `#[doc(hidden)]`.
* Add `BuildModifier` trait to provide a common interface for extending service builders.
* Add `Upgradable` trait to simplifying bounds when upgrading from an `Operation` to a HTTP service.
* Add `FromRequest`, `FromParts`, and `IntoResponse` implementations.
* Make `RoutingService` accept general body types `B` for the inner services `http::Response<B>`.
* Use new service builder in protocol tests.
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
feature: customizable operations
update: CHANGELOG.next.toml
update: RFC0017
update: add IntelliJ idea folder to .gitignore
add: GenericsGenerator with tests and docs
add: rustTypeParameters helper fn with tests and docs
add: RetryPolicy optional arg to FluentClientGenerator
move: FluentClientGenerator into its own file
* Remove `Default` implementation from `RetryConfig`
* Add use case integration tests
* Panic when retries/timeouts are enabled without a `sleep_impl`
* Combine the sleep, retry, and timeout customizations
* Add `sleep_impl` validation to the Smithy client builder
* Upgrade minimum Tokio version to 1.8.4
This addresses RUSTSEC-2021-0072 and RUSTSEC-2021-0124.
The SDK, runtime crates, and server are set to the minimum secure
version of Tokio to allow a larger range of versions to be used by
consumers of those libraries. The tools and the Python server are
both set to the latest Tokio version since they are not intended
to be consumed as a libraries.
* Upgrade `tracing-subscriber` to 0.3.15
* Upgrade `criterion` to 0.3.6
This addresses RUSTSEC-2021-0093 by upgrading `criterion`, which
upgrades `rayon`, which upgrades `crossbeam-deque`.
* Add protocol specific `FromRequest` and `FromParts`.
* Add `OperationShape` trait to model Smithy operations.
* Add `Handler` and `OperationService` traits.
* Add `Upgrade` `Service` and `UpgradeLayer` `Layer`.
* Fix bug in coroutine scheduling. Add ByteStream implementation and simplify Python symbol visitor.
* Bug in coroutine scheduling:
In the previous iteration, the runtime crate aws-smithy-http-server-python
was exposing the python application implementation as a struct, which
was wrapped by the codegenerated App to allow to dynamically building
the router.
This caused scheduling of coroutines (handlers prefixed with async def)
to block becuse we were passing the Python eventloop of the parent
process that was stored pre-fork().
This commit changes the runtime PyApp to become a trait, allowing us to
dynamically build the router post-fork() and with the right event loop.
This change also allowed us to remove a bunch of unnecessary Arc(s).
* Add ByteStream implementation
Implementation of a ByteStream type for Python that can be roughly used like this:
let b = await ByteStream.from_path("file.txt")
async for chunk in b:
print(chunk)
Implement futures_core::stream::Stream for Python ByteStream wrapper.
The BytesStream implementation in python can now use a sync Mutex from
parking_lot because we are now using pyo3_asyncio::tokio::local_future_into_py()
to read a chunk, which supports !Send futures.
This allows to simply forward the implementation of Stream (poll_next()
and size_hint()) directly to our inner SDK ByteStream.
* Simplify Python symbol visitor
Inherit from Symbol visitor and override just what is needed to swap Python complex
types.
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Currently, conversions from `aws_smithy_types::Number` into numeric Rust
types (`{i,u}{8, 16, 32, 64}` and `f{32, 64}`) are always lossy, because
they use the `as` Rust keyword to cast into the target type. This means
that clients and servers are accepting lossy data: for example, if an
operation is modeled to take in a 32-bit integer as input, and a client
incorrectly sends an integer number that does not fit in 32 bits, the
server will silently accept the truncated input. There are malformed
request protocol tests that verify that servers must reject these
requests.
This commit removes the lossy `to_*` methods on `Number` and instead
implements `TryFrom<$typ> for Number` for the target numeric type
`$typ`. These converters will attempt their best to perform the
conversion safely, and fail if it is lossy.
The code-generated JSON parsers will now fail with
`aws_smithy_json::deserialize::ErrorReason::InvalidNumber` if the number
in the JSON document cannot be converted into the modeled integer type
without losing precision. For floating point target types, lossy
conversions are still performed, via `Number::to_f32_lossy` and
`Number::to_f64_lossy`.
* Add TLS support to Pokemon server
* Add missing copyright header to TLS module
* Handle connection errors
* Move TLS example to own binary
* Update comments according to reviews
* Move rewrite base url middleware to its own function
* Rename `cargo-api-linter` to `cargo-check-external-types`
* Run `cargo-check-external-types` on Smithy client/shared runtime crates
* Run `cargo-check-external-types` on SDK runtime crates
* Fix bug in determining crate name
* Run `cargo-check-external-types` on generated SDK crates
* Add `MakeFmt` trait which modifies `Display`/`Debug` implementations.
* Add `http` wrappers which modify the `Debug` and `Display` implementations based on closures marking sensitivity. Each wrapper has an associated `MakeFmt` impl.
* Add `InstrumentOperation` [tower::Service](https://docs.rs/tower/latest/tower/trait.Service.html) which logs requests and responses. Accepts a `MakeFmt` for each potentially sensitive component of the request/response.
* Add [ServerHttpSensitivityGenerator.kt](https://github.com/awslabs/smithy-rs/compare/harryb/add-logging?expand=1#diff-a7ad7f28d152bba6f2a63f3bde498582d91c8f7ab9c2de0c7f4494d6fd1c02ff) which generates marker closures from models.
* Apply `InstrumentationOperation` to the `OperationHandler` using `MakeFmt`s using the marker closures.
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
The `Content-Type` check of the incoming request in
`ServerHttpBoundProtocolGenerator` is performed by comparing against the
configured protocol directly within the class.
Delegating to the protocol enables the implementation of protocols in
decorators loaded from the classpath in other projects outside
smithy-rs.
This commit makes the `ServerHttpBoundProtocolGenerator`
protocol-agnostic by looking up the expected incoming request
`Content-Type` header from the protocol implementation directly. To this
end, the macro-generated `check_<protocol>_content_type` functions from
the `aws-smithy-http-server` runtime have been replaced with a single
general `check_content_type` function that takes in the expected mime
type.
Rust crate names are allowed to contain hyphens, while Rust identifiers
are not [0]. However, the Rust and Python server projects are currently
broken because we use the user-provided crate name verbatim
(`moduleName` in `settings`) in places where Rust expects valid
identifiers:
* The Rust server's generated operation registry Rust docs use the crate
name verbatim in import statements.
* The Python server uses the crate name verbatim to generate a shared
library, but library target names in `Cargo.toml` cannot contain
hyphens.
We never caught this bug because we don't generate any crates with
hyphens in their names.
This commit:
* fixes the above usages of verbatim crate names where identifiers are
expected by calling `toSnakeCase()`,
* modifies the unit test for the server's operation registry generator
to generate a crate with hyphens in its name,
* changes the Pokémon service server SDK crate names from
`pokemon_service_sdk` to `pokemon-service-server-sdk`, so that we
exercise generation of a crate with hyphens in its name, and to better
convey that the generated SDK is server-specific; and
* changes the Pokémon service client SDK crate name from
`pokemon_service_client` to `pokemon-service-client`, and changes the
Pokémon service crate name itself from `pokemon_service` to
`pokemon-service`, for symmetry.
[0]: https://doc.rust-lang.org/reference/identifiers.html
* Support server event streams
* Rename EventStreamInput to EventStreamSender
* Custom event stream errors
* EventStreamSender and Receiver are parametrized also on event stream error types
* Pokemon service model updated
* Pokemon server event handler
* Pokemon client to test event streams
* EventStreamDecorator to make optional using SigV4 signing
* Use forInlineFun for all errors
Closes: #1157
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Include aws-smithy-http-server and aws-smithy-http-server-python in the list of crates that are automatically published to crates.io on each release of smithy-rs.
* Update `hydrate-readmes` to take versions from `versions.toml`
* Add cleanup TODO comments to `changelogger`
* Update SDK readme template
* Split up `generate-version-manifest` subcommand
* Eliminate the `aws.sdk.version` property
* Fallback to model hash comparison if model metadata claims no changes
* Add `acquire-base-image` to `release.yml`
* Use empty model metadata for SDK generation in CI
* Fix the `aws-config` version number in SDK crate readmes
* add: checksum related body-wrappers
add: tests for checksum related body-wrappers
* add: aws-smithy-checksums to CrateSet.key
* add: missing base64::encoded_length fn
* update: use the checksum digest trait's output_size method to get checksum size
add: proptest for base64 encoded length
remove: old base64 encoded length test
remove: incorrect leftover comments and comment quotes
* remove: unnecessary pin
* remove: use of clone during checksum finalization
refactor: checksum fns are no longer fallible
refactor: HttpChecksum is now responsible for reporting size correctly
fix: don't consider trailers as part of ChecksumBody size hint
* remove: Taken error state
update: prefer if-let to match
fix: incorrect is_end_stream impl
refactor: use default trait impl for HttpChecksum::header_value
* fix: clippy lint
* add: header value tests
* undo: accidental retry changes
* add: http-checksum-related errors
* refactor: break up and modularize body wrappers
update: docs
update: ChecksumValidatedBody::new now takes a Box<dyn HttpChecksum>
* fix: broken test
* refactor: rename checksum body structs to be differentiated only by their module
`async fn` are currently not supported by Rust. To allow for such syntax today, the [async_trait](https://docs.rs/async-trait/latest/async_trait/) macro eagerly `Box::pin`s the `async` block. This incurs an unnecessary allocation.
One instance where `async_trait` is used is on the [FromRequest](https://docs.rs/axum/latest/axum/extract/trait.FromRequest.html) method (inherited from [axum](https://docs.rs/axum/latest/axum/)). The use case for `FromRequest` in `axum` does not exist in our case, it is therefore a candidate for removal.
While `IntoResponse` does not involve the same boxing as `FromRequest`, it also does not need to be a trait - and so we remove it for symmetry.
* Remove `FromRequest` and replace it with a static method
* Remove `IntoResponse` and replace it with a static method
This PR add several functionalities to the Python/Rust code-generation, making it usable for models that are not using Streaming and Union shapes.
* Export codegenerated shapes as shared library that can be imported by the Python interpreter.
* Add complete end-to-end testing running in CI.
* Register signals and handle workers processes lifecycle.
* Use a custom SymbolProvider to override non-primitive types that are exposed to Python.
* Implement support for DateTime.
* Cast Python exception into Rust error types.
* End to end testing suite with a Rust client calling a real Python server.
This commit contains non-functional (i.e. mostly stylistic, small
refactors, documentation and consistency) improvements that I've batched
and rolled up while working on #1342.
* Make `extraDependencies` from `InlineDependency` class private
* Miscellaneous improvements to `RustTypes.kt`
* Remove dead code from `RuntimeTypes.kt`
* Style change in `RuntimeTypes.kt`
* Miscellaneous improvements to `SymbolVisitor.kt`
* Miscellaneous improvements to `BuilderGenerator.kt`
- Make it consistent with the `ServerBuilderGenerator` from #1342.
* Miscellaneous improvements to `TestHelpers.kt`
* Add docs to `headers_for_prefix` function
- I keep rereading the implementation to see what this returns; it's
not intuitive from the function name.
We are incorrectly matching URIs to the spec when the URI has a prefix
that is not in the spec.
This is because when generating the regex describing the spec, we're not
using the `^` anchor to match from the beginning of the text.
Fixes#1483.
This commit splits each of:
* `CodegenContext`,
* `RustSettings`; and
* `CodegenConfig`
into two variants: one for the `rust-codegen` client plugin and another
for the `rust-server-codegen` plugin.
(`CodegenConfig` is a property of `RustSettings`, which is a property of
`CodegenContext`; they constitute pervasive read-only global data that
gets passed around to the code generators).
This allows each plugin to evolve separately, defining different
settings and passing around a tailor-made code-generation context.
Client-specific generators should now use:
* `ClientCodegenContext`,
* `ClientRustSettings`; and
* `ClientCodegenConfig`.
Likewise, server-specific generators should now use:
* `ServerCodegenContext`,
* `ServerRustSettings`; and
* `ServerCodegenConfig`.
This is a more robust and maintainable approach than the current one,
where both generator types have to read possibly null values (because
they don't make sense for a client/server target), and plugins have to
populate settings that are not relevant for the crate they're
generating. For example, the server plugin has to populate and pass
around the settings `renameExceptions`, `includeFluentClient` and
`addMessageToErrors` even though it never uses them and don't make sense
for it.
As of writing these client and server specific classes are very similar,
but in the future they will evolve in separate ways (for example, #1342
will add symbol providers for constrained shapes that only make sense in
a server code-generation context).
Common properties have not been duplicated however. Instead, utilizing
inheritance, code-generation context that is common to _all_ plugins can
be placed in:
* `CoreCodegenContext`,
* `CoreRustSettings`; and
* `CoreCodegenConfig`.
This way, generators that are shared by both client and server can take
in `CoreCodegenContext`. If they need to tweak the code they generate in
small and self-contained places, they can utilize the `CodegenTarget`
flag-like enum to disambiguate, like they have always been doing.
Note this change also makes `RustDecorator`s be targeted for clients or
servers, by making them generic over the code-generation context. When
loading the decorators from the classpath and combining them into one in
the executed smithy-rs plugin, all of them need to either target clients
or target servers. This makes sense: after all, all of the decorators
that conform the AWS Rust SDK are client-specific.
Closes#828.
When timeout support was added initially, the resulting error `SdkError::TimeoutError` was never added to the retry policy which prevented these errors from being retried. This was a bug—This commit rectifies the and adds two integration-level tests that ensure that timeouts are properly retried.
We are introducing code-generation for Python bindings of StructureShape, BlobShape, EnumShape, OperationShape.
This PR also add a runtime server implementation to support serving Python business logic directly and with an idiomatic experience.
Co-authored-by: david-perez <d@vidp.dev>
* Add `TinyMap` collection, which is a map backed by either `Vec` or `HashMap` depending on the number of entries.
* Replace `HashMap` with `TinyMap` for `Routes::AwsJson10` and `Routes::AwsJson11`.
* Use `form_urlencoding::parse` over `serde_urlencoded::from_str`, this removes our dependency on `serde_urlencoded`.
* Remove `percent_encoding::percent_decode_str` where possible.
* Contract `generateParsePercentEncodedStrFn` and its children into one function and make it optionally apply percent encoding.
* Add `impl From<std::convert::Infallible> for RequestRejection` to remove codegen branch.
The server must have the most up to date variants and the unknown enum
variant should not be used. Clients are generated with it because they
might not have the most recent model and the server might return
an unknown variant to them.
Closes#1187
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: david-perez <d@vidp.dev>
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
This change removes the dependency on axum-core and brings in the
relevant resources we are using from that crate.
Issue: #1170
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* add: offset setter to FsBuilder
remove: file_size setter from FsBuilder
add: length setter to FsBuilder
add: test for ByteStream w/ offset
add: test for ByteStream w/ length less than file size
update: path_based_bytestreams_with_builder test
* add: test ensuring offset and length work correctly together
add: test ensuring that no data is returned when offset is greater than file size
add: test ensuring that nothing breaks if length to read is larger than file size
* add: chunking test
add: CHANGELOG.next.toml entry
refactor: new code based on PR comments
* remove: unused use statement
* refactor: length API
* update: prefer unwrap() to returning error in tests
* update: prefer unwrap() to returning error in tests
add: test for Length::Exact behavior
* update: error if offset is greater than file length
* fix: test to work with new offset error
* update: error test to be more specific
* Add command line arguments for server and port. Default the bind address to localhost
* Update rust-runtime/aws-smithy-http-server/examples/pokemon_service/src/main.rs
* Remove tracing::instrument as we should create the spans inside the code generated code
Co-authored-by: david-perez <d@vidp.dev>
Zelda Hessler