## Motivation and Context
running codegen tests produces a diff you need to ignore
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Release build was broken because `smithy-build.json` was not being
generated prior to smithyBuild actually running.
## Description
Fix task names
## Testing
Manual patch of release tag with this fix.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Fix deprecation warnings in the gradle build scripts, and upgrade jsoup
and gson.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
This PR fixes issues introduced by moving the repository from
awslabs/smithy-rs to smithy-lang/smithy-rs.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Now that constraint traits are supported in server SDKs (with some
corner case caveats, see #1401), this flag will almost always be useless
for those early adopters of constraint traits. It is thus convenient to
inform the user that they should remove it.
See
https://github.com/awslabs/smithy-rs/pull/2516#issuecomment-1490393056.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Now we have the feature parity between Rust and Python servers (at least
for the Pokémon service's needs) we can use the same model in both.
Closes https://github.com/awslabs/smithy-rs/issues/1508
## Testing
```bash
$ cd smithy-rs/examples
$ make test # test Rust servers
$ cd python
$ make test # test Python servers
```
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
We want to automatically generate stubs in the codegen diff to ensure
they can be reviewed and have a simple way to generate and include the
stubs inside the Maturin wheel.
## Description
The Python example has been moved to the `examples` folder and
refactored. The refactoring
ensures the script `stubgen.py` is included in the codegeneration of the
SDK crate. The script is later used to generate stubs automatically
during testing and can be used by customers to add their own stubs
before the Maturin build
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak <unexge@gmail.com>
* Remove parameter from `Protocol`s `structuredDataParser`, `structuredDataSerializer`
No implementation of the `Protocol` interface makes use of the
`OperationShape` parameter in the `structuredDataParser` and
`structuredDataSerializer` methods.
* Remove the TypeConversionGenerator class in favor of using
customizations for JsonParserGenerator and ServerHttpBoundProtocolGenerator.
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Make the additionaParserCustomizations default to empty list
* Fix merge conflict
* Fix missing ;
* Use better defaults when checking for customizations
* Use better defaults when checking for customizations
* Add HttpBindingCustomization and relax the datetime symbol check
* Support recursive shapes and add a lot more models to the tests
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Support naming obstacle course
* Add support for constrained blobs conversions
* Support constraint traits
* Try to generate the full diff
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* A better way of checking if we need to go into the Timestamp branch
* Remove wheels folder
---------
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: david-perez <d@vidp.dev>
* Add `RustType.Application` and `PythonType.Application` variants
* Use `RustType.Application` for event streaming symbols
* Call symbol provider to get `Receiver` type instead of hardcoding
* Add Python specific event streaming symbol provider
* Add event streaming wrapper generator
* Generate correct type information for event streaming types
* Add `CapturePokemon` operation to Python Pokemon service
* Add `InternalServerError` variant to all event streaming errors
* Use `PythonServerCargoDependency` for PyO3Asyncio imports
* Return an attribute error instead of panicking in `IntoPy` impls of wrappers
* Add `Sync` bound to `new` methods of wrappers
* Revert "Add `InternalServerError` variant to all event streaming errors"
This reverts commit b610cb27c7532102e3a3ec15a59da6c56b60c318.
* Add `PythonServerEventStreamErrorGenerator` to generate Python specific error types for unions
* Try to extract error type or inner type from incoming streaming value and ignore the value otherwise for now
* Allow missing type-stubs for `aiohttp`
* Propogate modelled errors through stream
* Raise modelled exceptions rather than sending through stream
* Allow senders from Python side to raise modelled exceptions
* Update `EventStreamSymbolProviderTest` to use `Application` type instead of `Opaque` type
* Fix `ktlint` issues
* Group up common variables in `codegenScope`
* Document `RustType.Application`
* Format `codegen-server-test/python/model/pokemon.smithy`
* Use `tokio-stream` crate instead of `futures` crate
* Use a better error message if user tries to reuse a stream
* Add some details about event streams to example Pokemon service
---------
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* A barely working code generation in typescript
* Extract shared socket into feature inside aws-smithy-http-server
* Building a fully functional application, I think
* Add NAPI build.rs
* Refactor all names to use typescript instead of js
* Add (hopefully) the PR bot for TS
* Clippy fixes
* Fix documentation
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* set_reuse_port in socket not on windows
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Add example implementation
* Allow the new application to build
* Remove all occurrences of Python
* Simplify README
* Fix issue with the codegen-diff-revision script
* Try to prevent the ci-lint to bother us with TODO at this stage of development
* Remove codegen-client from typescript dependencies
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Add CODEOWNERS and fix some other linting issues
* Add license
* Prevent from running tests on typescript in Windows
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Initial work to remove error from this PR
* Update to call_async
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* types/node in package.json
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Generate app.ts
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Improve makefile
* Adapting code to the latest changes and removing runtime dependency (for now).
* Removing rust-runtime/aws-smithy-http-server-typescript.
* Making CI happy.
* Restoring ServerCodegenDecorator to be like main.
* Adding back the aws-smithy-http-server-typescript crate back.
* Removing index.js file.
---------
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: 82marbag <69267416+82marbag@users.noreply.github.com>
Co-authored-by: Alberto Pose <albepose@amazon.com>
* Remove `ignoreUnsupportedConstraintTraits` from integration test models
Now that constraint traits are supported in server SDKs (with some
corner case caveats, see
https://github.com/awslabs/smithy-rs/issues/1401), we can remove
`ignoreUnsupportedConstraintTraits` from the codegen config of the
integration test models.
* Bring back ignoreUnsupportedConstraintTraits in RestJsonValidation model
* Add initial implementation of unions with very broken symbol provider
* Add support for creating new unions in Python
* Generate getters and static methods for unions
* Allow to compile misc model
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Doesn't work
* Now it works
* Simplify code a little
* Remove leftover from the many tries I did
* Finally fixed model generation with unions
* Fix wrong import
* Update to reflect changes in decorators
* Remove debugging output
* Simplify symbol provider
* Follow PR suggestions
* Remove union operation from python example
* Return `PyUnionMarker` for wrapped type in `IntoPy` impl
---------
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak Varlı <burakvar@amazon.co.uk>
* Upgrade to Smithy 1.27.1
* Fix bugs exposed by 1.27.1
* Fix rules engine customization
* update rfc3339 parsing test
* refactor timestamp format & remove upstreamed test
* Update changelog
* rename test
* fix datetime parsing on server and clients
* fix up a few more failing tests
* precommit lints
* Fix bad merge
* Make service name casing consistent
* Add naming-obstacle-course-casing.smithy
* Add missing import
* Relax obstacle course even further
* Better TODO
* Add CHANGELOG.next.toml
This commit adds support for the `@uniqueItems` trait on `list` shapes
in server SDKs. Requests with duplicate values for `list` shapes
constrained with `@uniqueItems` will be rejected by servers.
* Draft of @range implementation for integer shapes
* Green tests.
* Fix serialization of constrained integers.
* Fix tagger to include Integer simple shapes.
Fix de/serialization codegen.
* Add range trait to the entry about constraint traits in our changelog.
* Bind a range-constrained integer to various HTTP parts to test our implementation.
* Rename ConstrainedIntGeneratorTest to ConstrainedIntegerGeneratorTest for consistency.
* Remove AsRef implementation.
* Fix constraints models.
* Fix conversion.
* Use ReturnSymbolToParse to dry up.
* Fix builder when constrained types should not be exposed.
* Add customisation to unwrap constrained integers.
* Getting closer - collections need to be handled differently to make everything fall into place.
* Refactor `renderHeaders`
* Fix constraints test.
* Fix ebs.
* Rename for clarity.
* Remove unnecessary From implementation.
* Rename `Size` variant to `Range`.
* Remove stray comments.
* Rename for clarity
* Rename for consistency
* Add test to guard against types for which we do not support range yet
* DRY up branches and the relevant tests.
* Fix header name.
* Fix serialization bug for default values in headers.
* Fix serialization issue for primitive headers.
* Remove SetOfRangeInteger
* Fix pubcrateconstrained unit test.
* Remove duplication
* Revert "Remove SetOfRangeInteger"
This reverts commit f37a560bd0.
* Reintroduce `SetOfRangeInteger`, but keep them commented out.
* Ignore leading whitespace.
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/http/HttpBindingGenerator.kt
Co-authored-by: david-perez <d@vidp.dev>
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/parse/JsonParserGenerator.kt
Co-authored-by: david-perez <d@vidp.dev>
* Update codegen-server/src/main/kotlin/software/amazon/smithy/rust/codegen/server/smithy/ValidateUnsupportedConstraints.kt
Co-authored-by: david-perez <d@vidp.dev>
* Update codegen-server/src/test/kotlin/software/amazon/smithy/rust/codegen/server/smithy/generators/ConstrainedIntegerGeneratorTest.kt
Co-authored-by: david-perez <d@vidp.dev>
* Unify with a single rustTemplate invocation.
* Rename `Type` to `NumberType`
* Update codegen-server/src/test/kotlin/software/amazon/smithy/rust/codegen/server/smithy/ConstrainedShapeSymbolProviderTest.kt
Co-authored-by: david-perez <d@vidp.dev>
* Formatting issue.
* Move and rename test helper.
* Dry up the logic for default serialization.
* Ooops, I swapped the two branches.
* Add a wrapper block
* Fix support detection.
* Fix CHANGELOG.
* Add (failing) protocol tests for @range on byte/integer/long.
* Fix validation message.
Fix test definitions.
* Mark some tests as expected to fail.
Rename service.
* Use ValueExpression everywhere for more granular control of the ownership component.
* Use the new enhanced module facilities
* Fixes.
* Fix formatting
* Remove unnecessary when.
* Update codegen-core/common-test-models/constraints.smithy
Co-authored-by: david-perez <d@vidp.dev>
* Remove unused shapes.
* Avoid mixing concerns in our test shapes for integer constraints.
* Reuse the trait info machinery
* Update stale comment
* Fix unused attribute.
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/protocols/serialize/JsonSerializerGenerator.kt
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Avoid unnecessary bindings by manipulating the serialization context directly in the customisations.
* Avoid unnecessary bindings in header serialization by introducing and manipulating the header value serialization context directly in the customisations.
* Add code examples.
* Move `safeName` call into the if branch.
Co-authored-by: david-perez <d@vidp.dev>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
This patchset, affectionately called "Builders of builders", lays the
groundwork for fully implementing [Constraint traits] in the server SDK
generator. [The RFC] illustrates what the end goal looks like, and is
recommended prerrequisite reading to understanding this cover letter.
This commit makes the sever deserializers work with _unconstrained_ types
during request parsing, and only after the entire request is parsed are
constraints enforced. Values for a constrained shape are stored in the
correspondingly unconstrained shape, and right before the operation input is
built, the values are constrained via a `TryFrom<UnconstrainedShape> for
ConstrainedShape` implementation that all unconstrained types enjoy. The
service owner only interacts with constrained types, the unconstrained ones are
`pub(crate)` and for use by the framework only.
In the case of structure shapes, the corresponding unconstrained shape is their
builders. This is what gives this commit its title: during request
deserialization, arbitrarily nested structures are parsed into _builders that
hold builders_. Builders keep track of whether their members are constrained or
not by storing its members in a `MaybeConstrained`
[Cow](https://doc.rust-lang.org/std/borrow/enum.Cow.html)-like `enum` type:
```rust
pub(crate) trait Constrained {
type Unconstrained;
}
#[derive(Debug, Clone)]
pub(crate) enum MaybeConstrained<T: Constrained> {
Constrained(T),
Unconstrained(T::Unconstrained),
}
```
Consult the documentation for the generator in `ServerBuilderGenerator.kt` for
more implementation details and for the differences with the builder types the
server has been using, generated by `BuilderGenerator.kt`, which after this
commit are exclusively used by clients.
Other shape types, when they are constrained, get generated with their
correspondingly unconstrained counterparts. Their Rust types are essentially
wrapper newtypes, and similarly enjoy `TryFrom` converters to constrain them.
See the documentation in `UnconstrainedShapeSymbolProvider.kt` for details and
an example.
When constraints are not met, the converters raise _constraint violations_.
These are currently `enum`s holding the _first_ encountered violation.
When a shape is _transitively but not directly_ constrained, newtype wrappers
are also generated to hold the nested constrained values. To illustrate their
need, consider for example a list of `@length` strings. Upon request parsing,
the server deserializers need a way to hold a vector of unconstrained regular
`String`s, and a vector of the constrained newtyped `LengthString`s. The former
requirement is already satisfied by the generated unconstrained types, but for
the latter we need to generate an intermediate constrained
`ListUnconstrained(Vec<LengthString>)` newtype that will eventually be
unwrapped into the `Vec<LengthString>` the user is handed. This is the purpose
of the `PubCrate*` generators: consult the documentation in
`PubCrateConstrainedShapeSymbolProvider.kt`,
`PubCrateConstrainedCollectionGenerator.kt`, and
`PubCrateConstrainedMapGenerator.kt` for more details. As their name implies,
all of these types are `pub(crate)`, and the user never interacts with them.
For users that would not like their application code to make use of constrained
newtypes for their modeled constrained shapes, a `codegenConfig` setting
`publicConstrainedTypes` has been added. They opt out of these by setting it to
`false`, and use the inner types directly: the framework will still enforce
constraints upon request deserialization, but once execution enters an
application handler, the user is on their own to honor (or not) the modeled
constraints. No user interest has been expressed for this feature, but I expect
we will see demand for it. Moreover, it's a good stepping stone for users that
want their services to honor constraints, but are not ready to migrate their
application code to constrained newtypes. As for how it's implemented, several
parts of the codebase inspect the setting and toggle or tweak generators based
on its value. Perhaps the only detail worth mentioning in this commit message
is that the structure shape builder types are generated by a much simpler and
entirely different generator, in
`ServerBuilderGeneratorWithoutPublicConstrainedTypes.kt`. Note that this
builder _does not_ enforce constraints, except for `required` and `enum`, which
are always (and already) baked into the type system. When
`publicConstrainedTypes` is disabled, this is the builder that end users
interact with, while the one that enforces all constraints,
`ServerBuilderGenerator`, is now generated as `pub(crate)` and left for
exclusive use by the deserializers. See the relevant documentation for the
details and differences among the builder types.
As proof that these foundations are sound, this commit also implements the
`length` constraint trait on Smithy map and string shapes. Likewise, the
`required` and `enum` traits, which were already baked in the generated types
as non-`Option`al and `enum` Rust types, respectively, are now also treated
like the rest of constraint traits upon request deserialization. See the
documentation in `ConstrainedMapGenerator.kt` and
`ConstrainedStringGenerator.kt` for details.
The rest of the constraint traits and target shapes are left as an exercise to
the reader, but hopefully the reader has been convinced that all of them can be
enforced within this framework, paving the way for straightforward
implementations. The diff is already large as it is. Any reamining work is
being tracked in #1401; this and other issues are referenced in the code as
TODOs.
So as to not give users the impression that the server SDK plugin _fully_
honors constraints as per the Smithy specification, a validator in
`ValidateUnsupportedConstraintsAreNotUsed.kt` has been added. This traverses
the model and detects yet-unsupported parts of the spec, aborting code
generation and printing informative warnings referencing the relevant tracking
issues. This is a regression in that models that used constraint traits
previously built fine (even though the constraint traits were silently not
being honored), and now they will break. To unblock generation of these models,
this commit adds another `codegenConfig` setting,
`ignoreUnsupportedConstraints`, that users can opt into.
Closes#1714.
Testing
-------
Several Kotlin unit test classes exercising the finer details of the added
generators and symbol providers have been added. However, the best way to test
is to generate server SDKs from models making use of constraint traits. The
biggest assurances come from the newly added `constraints.smithy` model, an
"academic" service that _heavily_ exercises constraint traits. It's a
`restJson1` service that also tests binding of constrained shapes to different
parts of the HTTP message. Deeply nested hierarchies and recursive shapes are
also featured.
```sh
./gradlew -P modules='constraints' codegen-server-test:build
```
This model is _additionally_ generated in CI with the `publicConstrainedTypes`
setting disabled:
```sh
./gradlew -P modules='constraints_without_public_constrained_types' codegen-server-test:build
``````
Similarly, models using currently unsupported constraints are now being
generated with the `ignoreUnsupportedConstraints` setting enabled.
See `codegen-server-test/build.gradle.kts` for more details.
[Constraint traits]: https://awslabs.github.io/smithy/2.0/spec/constraint-traits.html
[The RFC]: https://github.com/awslabs/smithy-rs/pull/1199
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
* Fix bug in coroutine scheduling. Add ByteStream implementation and simplify Python symbol visitor.
* Bug in coroutine scheduling:
In the previous iteration, the runtime crate aws-smithy-http-server-python
was exposing the python application implementation as a struct, which
was wrapped by the codegenerated App to allow to dynamically building
the router.
This caused scheduling of coroutines (handlers prefixed with async def)
to block becuse we were passing the Python eventloop of the parent
process that was stored pre-fork().
This commit changes the runtime PyApp to become a trait, allowing us to
dynamically build the router post-fork() and with the right event loop.
This change also allowed us to remove a bunch of unnecessary Arc(s).
* Add ByteStream implementation
Implementation of a ByteStream type for Python that can be roughly used like this:
let b = await ByteStream.from_path("file.txt")
async for chunk in b:
print(chunk)
Implement futures_core::stream::Stream for Python ByteStream wrapper.
The BytesStream implementation in python can now use a sync Mutex from
parking_lot because we are now using pyo3_asyncio::tokio::local_future_into_py()
to read a chunk, which supports !Send futures.
This allows to simply forward the implementation of Stream (poll_next()
and size_hint()) directly to our inner SDK ByteStream.
* Simplify Python symbol visitor
Inherit from Symbol visitor and override just what is needed to swap Python complex
types.
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
This commit fixes two bugs that were present in the server
implementation:
* We were incorrectly referring to the operation input shape name
instead of using the symbol provider in
`ServerOperationHandlerGenerator.kt` and
`ServerOperationRegistryGenerator.kt`.
* We were not escaping Rust reserved keywords in operation names in
`ServerOperationRegistryGenerator.kt`.
The second bug would have been caught had we been generating server
crates for the two naming obstacle course test suites
(`naming-obstacle-course-ops.smithy` and
`naming-obstacle-course-structs.smithy`) that the client is already
passing. This commit adds these suites to the `codegen-server-test`
subproject, and expands `naming-obstacle-course-ops.smithy` to exercise
the logic that was causing the first bug.
* Fix `aws:sdk-codegen-test` build
* Add config for SDK endpoint/readme generation
* Rename `sdk-codegen-test` to `sdk-adhoc-test` and update README
* Add `sdk-adhoc-tests` to CI
* Refactor CI scripts
The `check-rust-runtime-and-tools` script was getting beyond 20
minutes to run, so this commit splits it into `check-aws-config`,
`check-rust-runtimes`, and `check-tools`.
This also enables most of the checks to run before the smoke test SDK is
generated, which should deliver more immediate feedback on PRs.
* Give the canary its own CI step
* Add `MakeFmt` trait which modifies `Display`/`Debug` implementations.
* Add `http` wrappers which modify the `Debug` and `Display` implementations based on closures marking sensitivity. Each wrapper has an associated `MakeFmt` impl.
* Add `InstrumentOperation` [tower::Service](https://docs.rs/tower/latest/tower/trait.Service.html) which logs requests and responses. Accepts a `MakeFmt` for each potentially sensitive component of the request/response.
* Add [ServerHttpSensitivityGenerator.kt](https://github.com/awslabs/smithy-rs/compare/harryb/add-logging?expand=1#diff-a7ad7f28d152bba6f2a63f3bde498582d91c8f7ab9c2de0c7f4494d6fd1c02ff) which generates marker closures from models.
* Apply `InstrumentationOperation` to the `OperationHandler` using `MakeFmt`s using the marker closures.
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Exclude sigv4 test suite from pre-commit file endings hook
* Upgrade Ktlint
* Make Gradle ktlint target consistent with pre-commit
They were previously running two separate rulesets since pre-commit
pulls in the standard ruleset, and that wasn't pulled into the Gradle
dependencies.
* Fix Gradle Ktlint paths
* Establish editorconfig for Ktlint
* Fix Ktlint lints
* Enable `trailing-commas` rule
* Fix block quote indentation
Rust crate names are allowed to contain hyphens, while Rust identifiers
are not [0]. However, the Rust and Python server projects are currently
broken because we use the user-provided crate name verbatim
(`moduleName` in `settings`) in places where Rust expects valid
identifiers:
* The Rust server's generated operation registry Rust docs use the crate
name verbatim in import statements.
* The Python server uses the crate name verbatim to generate a shared
library, but library target names in `Cargo.toml` cannot contain
hyphens.
We never caught this bug because we don't generate any crates with
hyphens in their names.
This commit:
* fixes the above usages of verbatim crate names where identifiers are
expected by calling `toSnakeCase()`,
* modifies the unit test for the server's operation registry generator
to generate a crate with hyphens in its name,
* changes the Pokémon service server SDK crate names from
`pokemon_service_sdk` to `pokemon-service-server-sdk`, so that we
exercise generation of a crate with hyphens in its name, and to better
convey that the generated SDK is server-specific; and
* changes the Pokémon service client SDK crate name from
`pokemon_service_client` to `pokemon-service-client`, and changes the
Pokémon service crate name itself from `pokemon_service` to
`pokemon-service`, for symmetry.
[0]: https://doc.rust-lang.org/reference/identifiers.html
* Support server event streams
* Rename EventStreamInput to EventStreamSender
* Custom event stream errors
* EventStreamSender and Receiver are parametrized also on event stream error types
* Pokemon service model updated
* Pokemon server event handler
* Pokemon client to test event streams
* EventStreamDecorator to make optional using SigV4 signing
* Use forInlineFun for all errors
Closes: #1157
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
This PR add several functionalities to the Python/Rust code-generation, making it usable for models that are not using Streaming and Union shapes.
* Export codegenerated shapes as shared library that can be imported by the Python interpreter.
* Add complete end-to-end testing running in CI.
* Register signals and handle workers processes lifecycle.
* Use a custom SymbolProvider to override non-primitive types that are exposed to Python.
* Implement support for DateTime.
* Cast Python exception into Rust error types.
* End to end testing suite with a Rust client calling a real Python server.
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
We are introducing code-generation for Python bindings of StructureShape, BlobShape, EnumShape, OperationShape.
This PR also add a runtime server implementation to support serving Python business logic directly and with an idiomatic experience.
Co-authored-by: david-perez <d@vidp.dev>
To support these new protocols, we also changed the runtime Router definition as for both awsJson 1.0 and 1.1, every request MUST be sent to the root URL (/) using the HTTP "POST" method.
The runtime Router now supports explicitly all the available protocols.
There are still some protocol tests failing for awsJson 1.0 and 1.1. The failure are caused by the missing implementation of @endpoint trait and date parsing. Protocol tests for these 2 protocols are heavily biased towards Responses. Before announcing support for awsJson 1.0 and 1.1, we should increase the protocol tests coverage.
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: david-perez <d@vidp.dev>
This commit adds another suite of protocol tests to test miscellaneous
aspects of code generation where protocol selection should not be
relevant.
The first test is code-generating an operation with a nested required
shape, minimally reproducing the bug that was fixed in 75056c, so that
we don't regress on it. I've added it to both the `codegen-test` and
`codegen-server-test` subprojects, even though the bug only affected the
server, in part to also incentivize us into adding more tests to this
Smithy model for both clients and servers.
This PR introduce a benchmarking tool that is run as part of the GitHub actions to allow to spot performance regressions in the server implementation.
The "deviation" between the last and current benchmark is posted as a message in the pull request.
I want to let this run for a little so we can figure out if GitHub action capacity can give us consistent results, otherwise we will have to move this to some capacity we own.
Co-authored-by: david-perez <d@vidp.dev>
We are adding this example service in the aws-smithy-http-server folder to showcase the SSDK and to be able soon to run benchmarks against this example.
This change includes a new model for the Pokémon Service and all the necessary infrastructure to code generate the client and server SKDs and make them available to the runtime implementation.
A basic README has also been added with instructions on how to build, run and test the service.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Smithy defines one trait specifically for the behavior of server
protocol implementations.
From https://awslabs.github.io/smithy/1.0/spec/http-protocol-compliance-tests.html#httpmalformedrequesttests:
> The `httpMalformedRequestTests` trait is used to define how a malformed
> HTTP request is rejected given a specific protocol and HTTP message.
> Protocol implementations MUST assert that requests are rejected during
> request processing.
This commit implements code generation for these tests. The ones for the
restJson1 protocol have been added to the list of failing tests. These
should begin passing once we start implementing constraint traits
https://awslabs.github.io/smithy/1.0/spec/core/constraint-traits.html
This commit allows for the `modules` and `cargoCommands` properties to
be specified when running the `codegen-test` and `codegen-server-test`
Gradle modules. The `modules` property allows one to only generate a
subset of the integration test services, while the `cargoCommands`
property allows one to specify the Cargo commands to be run on the
generated Rust crates.
This functionality can be useful to reduce development iteration cycles. For
instance, to only run `cargo test` on the integration test
`simple.smithy`:
```sh
./gradlew codegen-test:build -P cargoCommands='test' -P modules='simple'
```
This commit also refactors the buildscripts of the `codegen-test` and
the `codegen-server-test` modules, extracting shared functionality to
`buildSrc/src/main/kotlin/CodegenTestCommon.kt`.
Russell Cohen