* Refactor: Attribute.class used instead of helper methods in PythonServerAttributeUtils
Helper methods in PythonServerAttributeUtils have been deprecated in favor of using Attribute.class directly
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Smithy Python: PyException removed from enum
Smithy enums do not have ErrorTrait so there is no need to check and generate code for pyo3::exceptions::PyException
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
When `generateSmithyBuild` task is up to date, the hashes of the build
are not calculated, so `modifyMtime` fails, because it expects them to
be registered in Gradle's project properties. This can happen if you run
a command like `./gradlew codegen-server-test:test` twice consecutively.
In this case, we should skip the `modifyMtime` task altogether.
Everything is up to date, so no codegen will run, and no artifacts will
change.
* rename: hosted_zone_preprocessor.rs to route53_resource_id_preprocessor.rs
update: trim_hosted_zone to work for more resource ids
add: new Route53 protocol test for GetChange resource id trimming
update: InlineDependency.forRustFile to accept filenames with a ".rs" extension
add: CHANGELOG.next.toml entry
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
We are introducing code-generation for Python bindings of StructureShape, BlobShape, EnumShape, OperationShape.
This PR also add a runtime server implementation to support serving Python business logic directly and with an idiomatic experience.
Co-authored-by: david-perez <d@vidp.dev>
* Make `sdk-sync` Gradle heap/metaspace constraints configurable
* Dump process info on `sdk-sync` codegen failure
* Periodically log progress information in `sdk-sync`
* Enable verbose GC for codegen and use serial GC
- Fixes aws-sdk-rust#547
Previously, if no default profile was defined and no explicit profile was selected, the profile file provider would return an error. This relaxes that behavior to allow provider chains to move onto the next provider when that is the case.
* add: RFC for supporting flexible checksums
add: link to RFC0016 in SUMMARY.md
fix: too many zeroes in RFC filenames
* update: sigv4 update section language
* update: RFC overview
remove: arbitrary chunk size limit
* update: RFC code to be safe from num conversion panics
A small commit adding the ability to generate `pub(crate)` values.
Currently we don't need to, but this is used in #1342. It's also nicer
to use an enum instead of passing around a boolean.
We are currently generating a useless `message` getter that returns
`None` even when the `message` field is not defined in the model.
Granted, in the client this field is _almost always_ there, because the
`AddErrorMessage` model transformer is applied unless the user opts out
in their `smithy-build.json`.
* Add `TinyMap` collection, which is a map backed by either `Vec` or `HashMap` depending on the number of entries.
* Replace `HashMap` with `TinyMap` for `Routes::AwsJson10` and `Routes::AwsJson11`.
* Revert "remove parallelism from the sync tool (#1436)"
This reverts commit f0568e1f18.
* Reintroduce `sdk-sync` parallelism by disabling Smithy parallelism
* Make the Smithy parallelism configurable
Because some models have shapes that generate complex Rust types (e.g.
nested collection and map shapes).
I've also taken the opportunity to homogenize the comments in
`AllowLintsGenerator.kt`.
smithy-build actually utilizes `ForkJoinPool.common()` when executing build projections. Because each service is its own projection, the build internally is _already_ running in parallel. Because of this, running the sync in parallel produces uneccessary memory pressure.
This change:
- removes parallelism
- runs gradle `--no-daemon --info` for better log output
The `sdk-sync` tool is running into issues during release due to the
`sdk-versioner` refactor commit since it is using a Docker build image
with the tools precompiled into it. Either the `sdk-versioner` binary is
too new and can't code generate the old commits, or it's too old and
can't generate the new commits.
This reintroduces the old `sdk-versioner` behavior so that it can
succeed against both sets of commits to get a release out.
* add: contributing docs overview page
add: article on difficulties Zelda encountered while implementing streaming checksums
update: SUMMARY.md table of contents
add: link to contributor design docs in CONTRIBUTING.md
* Use `form_urlencoding::parse` over `serde_urlencoded::from_str`, this removes our dependency on `serde_urlencoded`.
* Remove `percent_encoding::percent_decode_str` where possible.
* Contract `generateParsePercentEncodedStrFn` and its children into one function and make it optionally apply percent encoding.
* Add `impl From<std::convert::Infallible> for RequestRejection` to remove codegen branch.
It's currently defaulting to 512 MiB. I often see this message in my
build logs:
> Daemon will be stopped at the end of the build after running out of JVM memory
I've done some (unscientific) profiling with this setting enabled, and
max heap size can peak at ~600 MiB, so I see no reason to increase it
more than 1024 MiB.
* Allow `@required` with `@httpPrefixHeaders`
The server generates members with `@required` and `@httpPrefixHeaders` as if
they were always optional. This commit allows the correct generation with `@required`.
Closes: #1394
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
* Correctly generate http response codes
http response codes will be:
* `@httpResponseCode` if set
* `@http`'s code, if `@httpResponseCode` is not set
* the default one, if neither of the above are set
@httpResponseCode honors @required and generates code that
doesn't use Option when applied.
Closes: #1229
The server must have the most up to date variants and the unknown enum
variant should not be used. Clients are generated with it because they
might not have the most recent model and the server might return
an unknown variant to them.
Closes#1187
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: Daniele Ahmed <ahmeddan@amazon.de>
Co-authored-by: david-perez <d@vidp.dev>
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
Fahad Zubair