## Motivation and Context
This PR upgrades Smithy to 1.50.0. The majority of the changes follow
`TODO` added in https://github.com/smithy-lang/smithy-rs/pull/3690.
Other than that, a few adjustments needed to be made:
- for the client
- added two failing tests `RestJsonClientPopulatesDefaultValuesInInput`
and `RestJsonClientUsesExplicitlyProvidedMemberValuesOverDefaults` to
known failing tests for the same reason
[here](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/protocol/ClientProtocolTestGenerator.kt#L72)
- added one broken test (i.e. the upstream test definition is incorrect
but our implementation is correct) to known broken tests per
([smithy#2341](https://github.com/smithy-lang/smithy/pull/2341),
[smithy-rs#3726](https://github.com/smithy-lang/smithy-rs/pull/3726#issuecomment-2199833659))
- for the server
- removed `rest-xml-extras.smithy` since
`RestXmlMustSupportParametersInContentType` is now available upstream
Smithy 1.50.0
- added the following to known failing tests (since the `awsJson1_0`
counterparts are already in the list, but we need the server team to
verify this assumption & provide additional `TODO` comments if
necessary)
- `RestJsonServerPopulatesDefaultsWhenMissingInRequestBody`
- `RestJsonServerPopulatesDefaultsInResponseWhenMissingInParams`,
-
`RestJsonServerPopulatesNestedDefaultValuesWhenMissingInInResponseParams`
## Testing
Existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Otherwise, if you generate a crate and compile it using Gradle, like for
example using the invocation:
```
./gradlew -P modules='simple' -P cargoCommands='test' codegen-server-test:build
```
And then manually run a `cargo` command within the generated crate
directory, or open the project using `rust-analyzer`, Cargo will
re-compile the project from scratch, with
`CARGO_LOG=cargo::core::compiler::fingerprint=trace` reporting that
flags have changed since the last compilation.
Instead, it's best if we persist these flags to `.cargo/config.toml`, so
all `cargo` invocations, either through Gradle, manually, or through
`rust-analyzer`, use the same set. This way, if no files were changed,
subsequent compilations since code generation will truly be no-ops, with
Cargo reusing all artifacts.
Note this commit fixes a regression that was introduced when `--cfg
aws_sdk_unstable` was introduced in #2614, since I fixed this the first
time back in #1422.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This fixes two bugs:
1. `Content-Type` header checking was succeeding when no `Content-Type`
header was present but one was expected.
2. When a shape was `@httpPayload`-bound, `Content-Type` header checking
occurred even when no payload was being sent. In this case it is not
necessary to check the header, since there is no content.
Code has been refactored and cleaned up. The crux of the logic is now
easier to understand, and contained in `content_type_header_classifier`.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Release build was broken because `smithy-build.json` was not being
generated prior to smithyBuild actually running.
## Description
Fix task names
## Testing
Manual patch of release tag with this fix.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Fix deprecation warnings in the gradle build scripts, and upgrade jsoup
and gson.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
This PR fixes issues introduced by moving the repository from
awslabs/smithy-rs to smithy-lang/smithy-rs.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Now that constraint traits are supported in server SDKs (with some
corner case caveats, see #1401), this flag will almost always be useless
for those early adopters of constraint traits. It is thus convenient to
inform the user that they should remove it.
See
https://github.com/awslabs/smithy-rs/pull/2516#issuecomment-1490393056.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Now we have the feature parity between Rust and Python servers (at least
for the Pokémon service's needs) we can use the same model in both.
Closes https://github.com/awslabs/smithy-rs/issues/1508
## Testing
```bash
$ cd smithy-rs/examples
$ make test # test Rust servers
$ cd python
$ make test # test Python servers
```
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
We want to automatically generate stubs in the codegen diff to ensure
they can be reviewed and have a simple way to generate and include the
stubs inside the Maturin wheel.
## Description
The Python example has been moved to the `examples` folder and
refactored. The refactoring
ensures the script `stubgen.py` is included in the codegeneration of the
SDK crate. The script is later used to generate stubs automatically
during testing and can be used by customers to add their own stubs
before the Maturin build
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak <unexge@gmail.com>
* Remove parameter from `Protocol`s `structuredDataParser`, `structuredDataSerializer`
No implementation of the `Protocol` interface makes use of the
`OperationShape` parameter in the `structuredDataParser` and
`structuredDataSerializer` methods.
* Remove the TypeConversionGenerator class in favor of using
customizations for JsonParserGenerator and ServerHttpBoundProtocolGenerator.
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Make the additionaParserCustomizations default to empty list
* Fix merge conflict
* Fix missing ;
* Use better defaults when checking for customizations
* Use better defaults when checking for customizations
* Add HttpBindingCustomization and relax the datetime symbol check
* Support recursive shapes and add a lot more models to the tests
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Support naming obstacle course
* Add support for constrained blobs conversions
* Support constraint traits
* Try to generate the full diff
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* A better way of checking if we need to go into the Timestamp branch
* Remove wheels folder
---------
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: david-perez <d@vidp.dev>
* Add initial implementation of unions with very broken symbol provider
* Add support for creating new unions in Python
* Generate getters and static methods for unions
* Allow to compile misc model
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
* Doesn't work
* Now it works
* Simplify code a little
* Remove leftover from the many tries I did
* Finally fixed model generation with unions
* Fix wrong import
* Update to reflect changes in decorators
* Remove debugging output
* Simplify symbol provider
* Follow PR suggestions
* Remove union operation from python example
* Return `PyUnionMarker` for wrapped type in `IntoPy` impl
---------
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: Burak Varlı <burakvar@amazon.co.uk>
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
* Fix `aws:sdk-codegen-test` build
* Add config for SDK endpoint/readme generation
* Rename `sdk-codegen-test` to `sdk-adhoc-test` and update README
* Add `sdk-adhoc-tests` to CI
* Refactor CI scripts
The `check-rust-runtime-and-tools` script was getting beyond 20
minutes to run, so this commit splits it into `check-aws-config`,
`check-rust-runtimes`, and `check-tools`.
This also enables most of the checks to run before the smoke test SDK is
generated, which should deliver more immediate feedback on PRs.
* Give the canary its own CI step
* Exclude sigv4 test suite from pre-commit file endings hook
* Upgrade Ktlint
* Make Gradle ktlint target consistent with pre-commit
They were previously running two separate rulesets since pre-commit
pulls in the standard ruleset, and that wasn't pulled into the Gradle
dependencies.
* Fix Gradle Ktlint paths
* Establish editorconfig for Ktlint
* Fix Ktlint lints
* Enable `trailing-commas` rule
* Fix block quote indentation
Rust crate names are allowed to contain hyphens, while Rust identifiers
are not [0]. However, the Rust and Python server projects are currently
broken because we use the user-provided crate name verbatim
(`moduleName` in `settings`) in places where Rust expects valid
identifiers:
* The Rust server's generated operation registry Rust docs use the crate
name verbatim in import statements.
* The Python server uses the crate name verbatim to generate a shared
library, but library target names in `Cargo.toml` cannot contain
hyphens.
We never caught this bug because we don't generate any crates with
hyphens in their names.
This commit:
* fixes the above usages of verbatim crate names where identifiers are
expected by calling `toSnakeCase()`,
* modifies the unit test for the server's operation registry generator
to generate a crate with hyphens in its name,
* changes the Pokémon service server SDK crate names from
`pokemon_service_sdk` to `pokemon-service-server-sdk`, so that we
exercise generation of a crate with hyphens in its name, and to better
convey that the generated SDK is server-specific; and
* changes the Pokémon service client SDK crate name from
`pokemon_service_client` to `pokemon-service-client`, and changes the
Pokémon service crate name itself from `pokemon_service` to
`pokemon-service`, for symmetry.
[0]: https://doc.rust-lang.org/reference/identifiers.html
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
We are introducing code-generation for Python bindings of StructureShape, BlobShape, EnumShape, OperationShape.
This PR also add a runtime server implementation to support serving Python business logic directly and with an idiomatic experience.
Co-authored-by: david-perez <d@vidp.dev>
ysaito1001