* Remove smoketest/fullsdk model classifications
* Generate full SDK for CI using models in aws-sdk-rust
* Move the changelog renderer out of `sdk-lints` and add changelog splitter
* Save revision information into split off SDK changelog
* Replace common git impl with `sdk-sync`'s impl
* Filter SDK changelog entries on changelog render
* Add smithy-rs release workflow
* Add initial next SDK changelog
* Update `sdk-sync` to understand new models location
This commit contains non-functional (i.e. mostly stylistic, small
refactors, documentation and consistency) improvements that I've batched
and rolled up while working on #1342.
* Make `extraDependencies` from `InlineDependency` class private
* Miscellaneous improvements to `RustTypes.kt`
* Remove dead code from `RuntimeTypes.kt`
* Style change in `RuntimeTypes.kt`
* Miscellaneous improvements to `SymbolVisitor.kt`
* Miscellaneous improvements to `BuilderGenerator.kt`
- Make it consistent with the `ServerBuilderGenerator` from #1342.
* Miscellaneous improvements to `TestHelpers.kt`
* Add docs to `headers_for_prefix` function
- I keep rereading the implementation to see what this returns; it's
not intuitive from the function name.
This code was written a long time ago, and made heavy use of string
interpolation instead of the more robust Rust templating primitives from
`RustWriter.kt`.
We are incorrectly matching URIs to the spec when the URI has a prefix
that is not in the spec.
This is because when generating the regex describing the spec, we're not
using the `^` anchor to match from the beginning of the text.
Fixes#1483.
This commit splits each of:
* `CodegenContext`,
* `RustSettings`; and
* `CodegenConfig`
into two variants: one for the `rust-codegen` client plugin and another
for the `rust-server-codegen` plugin.
(`CodegenConfig` is a property of `RustSettings`, which is a property of
`CodegenContext`; they constitute pervasive read-only global data that
gets passed around to the code generators).
This allows each plugin to evolve separately, defining different
settings and passing around a tailor-made code-generation context.
Client-specific generators should now use:
* `ClientCodegenContext`,
* `ClientRustSettings`; and
* `ClientCodegenConfig`.
Likewise, server-specific generators should now use:
* `ServerCodegenContext`,
* `ServerRustSettings`; and
* `ServerCodegenConfig`.
This is a more robust and maintainable approach than the current one,
where both generator types have to read possibly null values (because
they don't make sense for a client/server target), and plugins have to
populate settings that are not relevant for the crate they're
generating. For example, the server plugin has to populate and pass
around the settings `renameExceptions`, `includeFluentClient` and
`addMessageToErrors` even though it never uses them and don't make sense
for it.
As of writing these client and server specific classes are very similar,
but in the future they will evolve in separate ways (for example, #1342
will add symbol providers for constrained shapes that only make sense in
a server code-generation context).
Common properties have not been duplicated however. Instead, utilizing
inheritance, code-generation context that is common to _all_ plugins can
be placed in:
* `CoreCodegenContext`,
* `CoreRustSettings`; and
* `CoreCodegenConfig`.
This way, generators that are shared by both client and server can take
in `CoreCodegenContext`. If they need to tweak the code they generate in
small and self-contained places, they can utilize the `CodegenTarget`
flag-like enum to disambiguate, like they have always been doing.
Note this change also makes `RustDecorator`s be targeted for clients or
servers, by making them generic over the code-generation context. When
loading the decorators from the classpath and combining them into one in
the executed smithy-rs plugin, all of them need to either target clients
or target servers. This makes sense: after all, all of the decorators
that conform the AWS Rust SDK are client-specific.
Closes#828.
When timeout support was added initially, the resulting error `SdkError::TimeoutError` was never added to the retry policy which prevented these errors from being retried. This was a bug—This commit rectifies the and adds two integration-level tests that ensure that timeouts are properly retried.
* Refactor: Attribute.class used instead of helper methods in PythonServerAttributeUtils
Helper methods in PythonServerAttributeUtils have been deprecated in favor of using Attribute.class directly
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Smithy Python: PyException removed from enum
Smithy enums do not have ErrorTrait so there is no need to check and generate code for pyo3::exceptions::PyException
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
When `generateSmithyBuild` task is up to date, the hashes of the build
are not calculated, so `modifyMtime` fails, because it expects them to
be registered in Gradle's project properties. This can happen if you run
a command like `./gradlew codegen-server-test:test` twice consecutively.
In this case, we should skip the `modifyMtime` task altogether.
Everything is up to date, so no codegen will run, and no artifacts will
change.
* rename: hosted_zone_preprocessor.rs to route53_resource_id_preprocessor.rs
update: trim_hosted_zone to work for more resource ids
add: new Route53 protocol test for GetChange resource id trimming
update: InlineDependency.forRustFile to accept filenames with a ".rs" extension
add: CHANGELOG.next.toml entry
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
We are introducing code-generation for Python bindings of StructureShape, BlobShape, EnumShape, OperationShape.
This PR also add a runtime server implementation to support serving Python business logic directly and with an idiomatic experience.
Co-authored-by: david-perez <d@vidp.dev>
* Make `sdk-sync` Gradle heap/metaspace constraints configurable
* Dump process info on `sdk-sync` codegen failure
* Periodically log progress information in `sdk-sync`
* Enable verbose GC for codegen and use serial GC
- Fixes aws-sdk-rust#547
Previously, if no default profile was defined and no explicit profile was selected, the profile file provider would return an error. This relaxes that behavior to allow provider chains to move onto the next provider when that is the case.
* add: RFC for supporting flexible checksums
add: link to RFC0016 in SUMMARY.md
fix: too many zeroes in RFC filenames
* update: sigv4 update section language
* update: RFC overview
remove: arbitrary chunk size limit
* update: RFC code to be safe from num conversion panics
A small commit adding the ability to generate `pub(crate)` values.
Currently we don't need to, but this is used in #1342. It's also nicer
to use an enum instead of passing around a boolean.
We are currently generating a useless `message` getter that returns
`None` even when the `message` field is not defined in the model.
Granted, in the client this field is _almost always_ there, because the
`AddErrorMessage` model transformer is applied unless the user opts out
in their `smithy-build.json`.
* Add `TinyMap` collection, which is a map backed by either `Vec` or `HashMap` depending on the number of entries.
* Replace `HashMap` with `TinyMap` for `Routes::AwsJson10` and `Routes::AwsJson11`.
* Revert "remove parallelism from the sync tool (#1436)"
This reverts commit f0568e1f18.
* Reintroduce `sdk-sync` parallelism by disabling Smithy parallelism
* Make the Smithy parallelism configurable
Because some models have shapes that generate complex Rust types (e.g.
nested collection and map shapes).
I've also taken the opportunity to homogenize the comments in
`AllowLintsGenerator.kt`.
smithy-build actually utilizes `ForkJoinPool.common()` when executing build projections. Because each service is its own projection, the build internally is _already_ running in parallel. Because of this, running the sync in parallel produces uneccessary memory pressure.
This change:
- removes parallelism
- runs gradle `--no-daemon --info` for better log output
The `sdk-sync` tool is running into issues during release due to the
`sdk-versioner` refactor commit since it is using a Docker build image
with the tools precompiled into it. Either the `sdk-versioner` binary is
too new and can't code generate the old commits, or it's too old and
can't generate the new commits.
This reintroduces the old `sdk-versioner` behavior so that it can
succeed against both sets of commits to get a release out.
* add: contributing docs overview page
add: article on difficulties Zelda encountered while implementing streaming checksums
update: SUMMARY.md table of contents
add: link to contributor design docs in CONTRIBUTING.md
* Use `form_urlencoding::parse` over `serde_urlencoded::from_str`, this removes our dependency on `serde_urlencoded`.
* Remove `percent_encoding::percent_decode_str` where possible.
* Contract `generateParsePercentEncodedStrFn` and its children into one function and make it optionally apply percent encoding.
* Add `impl From<std::convert::Infallible> for RequestRejection` to remove codegen branch.
It's currently defaulting to 512 MiB. I often see this message in my
build logs:
> Daemon will be stopped at the end of the build after running out of JVM memory
I've done some (unscientific) profiling with this setting enabled, and
max heap size can peak at ~600 MiB, so I see no reason to increase it
more than 1024 MiB.
John DiSanti