* Remove `toEnumVariantName` from `RustSymbolProvider`
The `toEnumVariantName` function existed on symbol provider to work
around enum definitions not being shapes. In the future when we refactor
to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s
for each enum member. This change incrementally moves us to that future
by creating fake `MemberShape`s in the enum generator from the enum
definition.
* Fix escaping of `Self` in symbol providers
* Clean up an old hack
* Make `RustReservedWordsSymbolProvider` configurable
* Update changelog
* Incorporate feedback
* Add more client re-exports
This commit adds more client re-exports proposed in
https://github.com/awslabs/smithy-rs/issues/1759.
Specifically, it re-exports the following types:
- `aws_smithy_http::body::SdkBody`
- `aws_smithy_http::byte_stream::error::Error`
- `aws_smithy_http::operation::{Request, Response}`
* Update CHANGELOG.next.toml
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
* Split fluent client functions across multiple files
* Add (commented) test for large generated files to CI
* Split serialization/deserialization generated code across multiple files
* Remove extraneous newline from fluent client doc comments
* Add doc comments to `ProtocolFunctions`
* Simplify some doc comment generation
* Improve some function names
* Add hook for CodegenDecorators to insert a custom symbol provider
* rename {extraS=>s}ymbolProvider
* fix rebase conflict that led to invalid code
* Remove model parameter from CoreCodegenDecorator::symbolProvider
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Add `timestamp` type test and fix the conversion error
* Add some tests for `ByteStream` and fix async issues
* Use `__anext__` method instead of `anext`
---------
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use `clientIntegrationTest` (WIP)
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` with `clientIntegrationTest`
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use generic test cases
* Start refactoring `ServerEventStreamUnmarshallerGeneratorTest`
* Make `ServerEventStreamUnmarshallerGeneratorTest` tests work
* Uncomment other test models
* Allow unused on `parse_generic_error`
* Rename `ServerEventStreamUnmarshallerGeneratorTest`
* Make `EventStreamUnmarshallTestCases` codegenTarget-agnostic
* Refactor `ClientEventStreamMarshallerGeneratorTest`: Tests run but fail
* Refactor `ServerEventStreamMarshallerGeneratorTest`
* Move `.into()` calls to `conditionalBuilderInput`
* Add "context" to TODO
* Fix client unmarshall tests
* Fix clippy lint
* Fix more clippy lints
* Add docs for `event_stream_serde` module
* Fix client tests
* Remove `#[allow(missing_docs)]` from event stream module
* Remove unused `EventStreamTestTools`
* Add `smithy-validation-model` test dep to `codegen-client`
* Temporarily add docs to make tests compile
* Undo change in model
* Make event stream unmarshaller tests a unit test
* Remove unused code
* Make `ServerEventStreamUnmarshallerGeneratorTest` a unit test
* Make `ServerEventStreamMarshallerGeneratorTest` a unit test
* Make `ServerEventStreamMarshallerGeneratorTest` pass
* Make remaining tests non-integration tests
* Make event stream serde module private again
* Remove unnecessary clippy allowances
* Remove clippy allowance
* Remove docs for `event_stream_serde` module
* Remove docs for `$unmarshallerTypeName::new`
* Remove more unnecessary docs
* Remove more superfluous docs
* Undo unnecessary diffs
* Uncomment last test
* Make `conditionalBuilderInput` internal
* Add builder symbol/module resolution to symbol providers
* Back out breaking changes
* Reduce codegen diff
* Fix server example clippy lints
* Associate #2396 with TODO comments
* Improve doc comment
* Revert doc hidden change
* Conditionally re-export the Smithy Client Builder in clients
* Reorganize error type re-export
* Reorganize the `paginator` module
* Flatten the `presigning` module
* Hide the `http_body_checksum` module
The `toEnumVariantName` function existed on symbol provider to work
around enum definitions not being shapes. In the future when we refactor
to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s
for each enum member. This change incrementally moves us to that future
by creating fake `MemberShape`s in the enum generator from the enum
definition.
* Add support for the awsQueryCompatible trait
This commit adds support for the awsQueryCompatible trait. This allows
services already supporting custom error codes through the AWS Query
protocol with the awsQueryError trait to continue supporting them after
the services switch to the AWS JSON 1.0 protocol.
* Add copyright header
* Fix clippy warning for clippy::manual-map
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Remove unused variables from `errorScope`
This commit addresses https://github.com/awslabs/smithy-rs/pull/2398#discussion_r1114763528
* Reorder arguments for test verification
This commit addresses https://github.com/awslabs/smithy-rs/pull/2398#discussion_r1114766817
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
* Upgrade Kotlin to 1.7.21
* Upgrade Ktlint to 0.48.2
* Run `pre-commit run --all-files` to fix broken Ktlints
* Fix string comparison broken by code formatting
* Constraint member types are refactored as standalone shapes.
* ModelModule to ServerRustModule.model
* Constraints are written to the correct module
* Code generates for non-public constrained types.
* Removed a comment
* Using ConcurrentHashmap just to be on the safe side
* Clippy warnings removed on constraints, k.into() if gated
* Wordings for some of the checks changed
* Test need to call rustCrate.renderInlineMemoryModules
* ktlintFormat related changes
* RustCrate need to be passed for server builder
* Param renamed in getParentAndInlineModuleForConstrainedMember
* pubCrate to publicConstrainedType rename
* PythonServer symbol builder needed to pass publicConstrainedTypes
* @required still remains on the member shape after transformation
* ConcurrentLinkedQueue used for root RustWriters
* runTestCase does not run the tests but just sets them up, hence has been renamed
* CHANGELOG added
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
* feat(codegen): support for api key auth trait
* chore: update to new codegen decorator interface
* chore: include basic test
* chore: set api key into rest xml extras model
* chore: update test
* chore: refactor api key definition map
* feat(codegen): add api key decorator by default
* chore: add smithy-http-auth to runtime type
* chore: reference new smithy-http-auth crate
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Revert "chore: set api key into rest xml extras model"
This reverts commit 93b99c8703.
* chore: moved api key re-export to extras customization
* chore: include test for auth in query and header
* chore: fix linting
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/test/kotlin/software/amazon/smithy/rust/codegen/client/customizations/ApiKeyAuthDecoratorTest.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/test/kotlin/software/amazon/smithy/rust/codegen/client/customizations/ApiKeyAuthDecoratorTest.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* chore: add doc hidden to re-export
* chore: ensure extras are added only if it applies
* Revert "chore: add doc hidden to re-export"
This reverts commit 8a49e2b47b.
---------
Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Fix recursive constraint violations with paths over list and map shapes
There is a widespread assumption throughout the generation of constraint
violations that does not hold true all the time, namely, that a
recursive constraint violation graph has the same requirements with
regards to boxing as the regular shape graph.
Some types corresponding to recursive shapes are boxed to introduce
indirection and thus not generate an infinitely recursive type. The
algorithm however does not superfluously introduce boxes when the cycle
goes through a list shape or a map shape. Why list shapes and map
shapes? List shapes and map shapes get rendered in Rust as `Vec<T>` and
`HashMap<K, V>`, respectively, they're the only Smithy shapes that
"organically" introduce indirection (via a pointer to the heap) in the
recursive path. For other recursive paths, we thus have to introduce the
indirection artificially ourselves using `Box`. This is done in the
`RecursiveShapeBoxer` model transform.
However, the constraint violation graph needs to box types in recursive
paths more often. Since we don't collect constraint violations
(yet, see #2040), the constraint violation graph never holds
`Vec<T>`s or `HashMap<K, V>`s, only simple types. Indeed, the following simple
recursive model:
```smithy
union Recursive {
list: List
}
@length(min: 69)
list List {
member: Recursive
}
```
has a cycle that goes through a list shape, so no shapes in it need
boxing in the regular shape graph. However, the constraint violation
graph is infinitely recursive if we don't introduce boxing somewhere:
```rust
pub mod model {
pub mod list {
pub enum ConstraintViolation {
Length(usize),
Member(
usize,
crate::model::recursive::ConstraintViolation,
),
}
}
pub mod recursive {
pub enum ConstraintViolation {
List(crate::model::list::ConstraintViolation),
}
}
}
```
This commit fixes things by making the `RecursiveShapeBoxer` model
transform configurable so that the "cycles through lists and maps
introduce indirection" assumption can be lifted. This allows a server
model transform, `RecursiveConstraintViolationBoxer`, to tag member
shapes along recursive paths with a new trait,
`ConstraintViolationRustBoxTrait`, that the constraint violation type
generation then utilizes to ensure that no infinitely recursive
constraint violation types get generated.
For example, for the above model, the generated Rust code would now look
like:
```rust
pub mod model {
pub mod list {
pub enum ConstraintViolation {
Length(usize),
Member(
usize,
std::boxed::Box(crate::model::recursive::ConstraintViolation),
),
}
}
pub mod recursive {
pub enum ConstraintViolation {
List(crate::model::list::ConstraintViolation),
}
}
}
```
Likewise, places where constraint violations are handled (like where
unconstrained types are converted to constrained types) have been
updated to account for the scenario where they now are or need to be
boxed.
Parametrized tests have been added to exhaustively test combinations of
models exercising recursive paths going through (sparse and non-sparse)
list and map shapes, as well as union and structure shapes
(`RecursiveConstraintViolationsTest`). These tests even assert that the
specific member shapes along the cycles are tagged as expected
(`RecursiveConstraintViolationBoxerTest`).
* Address comments
* feat(aws-types): add api key to configuration
* chore: set package version to 0.52.0
* feat(aws-smithy-types): create auth types
* chore: use auth from smithy types
* chore: fix return self type
* chore: create http auth definition type
* chore: add constructor for http auth definition
* chore: ensure properties are not moved
* chore: create convenience constructors
* chore: add some todo comments
* feat: move query writer to aws-smithy-http crate
* chore(codegen): expose smithy http tower dependency
* chore: remove setters for auth definition
* chore: fix logical error for scheme not allowed
* chore: add constructor for basic and digest auth
* chore: allow equality comparision for api key
* Revert "chore: set package version to 0.52.0"
This reverts commit da660fcf16.
* chore: fix additional references to querywriter
* chore: implement from string for api key struct
* chore: disallow none api key in sdk config
* chore: fix formatting
* chore: add unit tests for auth types
* chore: add auth api key to external types
* chore: make query writer doc hidden
* feat: create aws-smithy-http-auth crate
* chore: use zeroing for auth api key
* chore: use builder pattern for auth definition
* chore: restructure http auth package
* chore: define default lint warning
* chore: include http auth in runtime common list
* chore: define setter for optional scheme
* chore: should panic while building auth definition
* chore: return missing required field error
* chore: make few code simplications for api key
* Revert "chore: add auth api key to external types"
This reverts commit b2318b0230.
* chore: revert api key from sdk config
* chore: panic on missing required field
* Opt out of `clippy::derive_partial_eq_without_eq`
---------
Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Make service name casing consistent
* Add naming-obstacle-course-casing.smithy
* Add missing import
* Relax obstacle course even further
* Better TODO
* Add CHANGELOG.next.toml
`simple.smithy` is currently not that simple. Besides, it serves no
effective purpose: everything that the model exercises is already
exercised in a clearer and more orderly manner in other integration
tests.
This PR makes the model minimal, so that it recovers its purpose as a
simple smoke test. Contributors can also use the model as a scratchpad
for features or bugfixs they're working on.
* Add `RequestId` trait
* Implement `RequestId` for generated AWS client errors
* Move `RustWriter.implBlock` out of `StructureGenerator`
* Create structure/builder customization hooks
* Customize `_request_id` into AWS outputs
* Set request ID on outputs
* Refactor SDK service decorators
* Refactor S3's extended request ID implementation
* Combine `Error` and `ErrorKind`
* Add test for service error conversion
* Move error generators into `codegen-client` and fix tests
* Re-export `ErrorMetadata`
* Add request IDs to trace logs
* Simplify some error trait handling
* Rename `ClientContextParamDecorator` to `ClientContextConfigCustomization`
* Add deprecated alias to guide customers through upgrading
* Rename the `ErrorMetadata` trait to `ProvideErrorMetadata`
* Rename `aws_smithy_types::Error` to `ErrorMetadata`
* Refactor modules to be configurable in `codegen-core`
* Remove panicking default test symbol provider
* Remove as many references to Error/Types as possible
* Rename module constants
* Initial Python stub generation
* Handle default values correctly
* Only generate `__init__` for classes that have constructor signatures
* Preserve doc comments
* Make context class generic
* Put type hint into a string to fix runtime error
* Run `mypy` on CI
* Use `make` to build Python SSDKs while generating diffs
* Escape Python types in Rust comments
* Only mark class methods with
* Sort imports to minimize diffs
* Add type annotations for `PySocket`
* Dont extend classes from `object` as every class already implicitly extended from `object`
* Use `vars` instead of `inspect.getmembers` to skip inherited members of a class
* Fix linting issues
* Add some tests for stubgen and refactor it
* Add type annotations to `PyMiddlewareException`
* Fix tests on Python 3.7
Python 3.7 doesn't support reading signatures from `__text_signature__`
for non-builtin functions (i.e. C/Rust functions). For testing we're using
regular Python syntax for defining signature.
* Provide default values for `typing.Optional[T]` types in type-stubs
* Update `is_fn_like` to cover more cases
* Remove `tools/smithy-rs-tool-common/`
* Make `DECORATORS` an array instead of a list
* Ignore missing type stub errors for `aiohttp`
This allows "easily" converting to other custom validation exception
shapes by implementing a decorator.
As a simple example, this PR adds a
`CustomValidationExceptionWithReasonDecorator` decorator that is able to
convert into a shape that is very similar to
`smithy.framework#ValidationException`, but that has an additional
`reason` field. The decorator can be enabled via the newly added
`experimentalCustomValidationExceptionWithReasonPleaseDoNotUse` codegen
config flag.
This effectively provides a way for users to use custom validation
exceptions without having to wait for the full implementation of #2053,
provided they're interested enough to write a decorator in a JVM
language. This mechanism is _experimental_ and will be removed once full
support for custom validation exceptions as described in #2053 lands,
hence why the configuration key is strongly worded in this respect.
This commit also ports the mechanism to run codegen integration tests
within Kotlin unit tests for client SDKs to the server. See #1956 for
details. The custom validation exception decorator is tested this way.
Rename `RustCodegenServerPlugin` to `RustServerCodegenPlugin`, for
consistency with `RustClientCodegenPlugin`. This is a better name, since
the plugin is named `rust-server-codegen`.
This commit also renames `PythonCodegenServerPlugin` to
`RustServerCodegenPythonPlugin` for the same reasons.
This commit also contains other drive-by improvements made while working
on #2302.
* Prevent test dependencies from leaking into production
* refactor & fix tests
* fix tests take two
* fix more tests
* Fix missed called to mergeDependencyFeatures
* Add test
* fix glacier compilation
* fix more tests
* fix one more test
This commit adds support for the `@uniqueItems` trait on `list` shapes
in server SDKs. Requests with duplicate values for `list` shapes
constrained with `@uniqueItems` will be rejected by servers.
In server SDKs, these traits can be implemented by any shape _except_ if
the shape's closure contains:
1. A `float`, `double`, or `document` shape: floating point types in
Rust do not implement `Eq`. Similarly, [`document` shapes] may
contain arbitrary JSON-like data containing floating point values.
2. A [@streaming] shape: all the streaming data would need to be
buffered first to compare it.
Additionally, the `Hash` trait cannot be implemented by shapes whose
closure contains:
1. A `map` shape: we render `map` shapes as `std::collections::HashMap`,
which _do not_ implement `Hash`. See
https://github.com/awslabs/smithy/issues/1567.
In **client SDKs, these traits cannot be derived on any code-generated
Rust types corresponding to Smithy shapes**, since e.g. adding new
optional members to a structure [is a backwards-compatible change], and
doing so alters the semantics of these traits.
However, this commit does implement these traits for the
`aws_smithy_types::date_time::DateTime` and `aws_smithy_types::Blob`
runtime types.
This change is necessary to efficiently implement the `@uniqueItems`
constraint trait in server SDKs.
[`Eq`]: https://doc.rust-lang.org/std/cmp/trait.Eq.html
[`Hash`]: https://doc.rust-lang.org/std/hash/trait.Hash.html
[`document` shapes]: https://smithy.io/2.0/spec/simple-types.html#document
[@streaming]: https://smithy.io/2.0/spec/streaming.html
[is a backwards-compatible change]: https://smithy.io/2.0/guides/evolving-models.html#updating-structures
* Add support for operationInput tests
* More unfication, fix tests, docs
* Set endpoint_url only when endpoint_url is used
* Fix test-util feature
* CR feedback
* fix missing path
Most usages are in the server constrained types generators, where
`#[allow(dead_code)]` suffices and makes the intent clearer.
See https://stackoverflow.com/a/64556868
Turns out we've never supported them, neither directly constrained nor
with constrained members, because of a lack of tests. Yet another data
point to prioritize working on code-generating `constraints.smithy` (see
https://github.com/awslabs/smithy-rs/issues/2101).
The implementation is simple: we just need to call the symbol provider
on the member symbols instead of on the target symbols so we get
`Option<T>` list members / map values if applicable, and handle the
wrapper when converting between unconstrained and constrained types with
help from `match` and `Option<T>::map`.
John DiSanti