* update: CargoDependency companion fn names for smithy runtime crates
rename: CargoDependency.asType to CargoDependency.toType
fix: errors in InlineDependency doc comment
formatting: run import optimizer for all kotlin files
fix: gradle issue with incorrectly named tasks
* fix: server test broken by removal of rustName
This change provides compatibility with the latest Gradle 8.0 nightly.
Since the top-level code coverage stuff was already broken and
apparently no one noticed, I just deleted it. The coverage reports for
specific subprojects still work fine.
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
When registering the Cargo commands tasks. The `aws:sdk` is the only
Gradle subproject that depends on the Cargo commands tasks, but that
does not register the `modifyMtime` task.
Partially addresses #1596.
* Exclude sigv4 test suite from pre-commit file endings hook
* Upgrade Ktlint
* Make Gradle ktlint target consistent with pre-commit
They were previously running two separate rulesets since pre-commit
pulls in the standard ruleset, and that wasn't pulled into the Gradle
dependencies.
* Fix Gradle Ktlint paths
* Establish editorconfig for Ktlint
* Fix Ktlint lints
* Enable `trailing-commas` rule
* Fix block quote indentation
Include aws-smithy-http-server and aws-smithy-http-server-python in the list of crates that are automatically published to crates.io on each release of smithy-rs.
* Update `hydrate-readmes` to take versions from `versions.toml`
* Add cleanup TODO comments to `changelogger`
* Update SDK readme template
* Split up `generate-version-manifest` subcommand
* Eliminate the `aws.sdk.version` property
* Fallback to model hash comparison if model metadata claims no changes
* Add `acquire-base-image` to `release.yml`
* Use empty model metadata for SDK generation in CI
* Fix the `aws-config` version number in SDK crate readmes
* add: support for unsigned aws-chunked content encoding
* add: aws-smithy-checksums to CrateSet.key
* refactor: API of AwsChunkedBodyOptions
update: AwsChunkedBody to note we only support single chunks
remove: unnecessary `Option`s
remove: overly smart total_length_of_trailers_in_bytes in trailers_as_aws_chunked_bytes
update: use "where"-style declaration for `impl<Inner> Body for AwsChunkedBody<Inner>`
add: helpful data to trace logging
add: trailer len double check in AwsChunkedBody::poll_data
add: test for trailer len double check
add: assert to size_hint
fix: incorrect body emitted when body is empty
add: test for empty encoded body
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* update: CHANGELOG.next.toml message
* refactor: aws-chunked trailer encoding and size calculation
update: when aws-chunked formatting a `HeaderMap`, header names with multiple values will be written out one value per line
remove: unnecessary AwsChunkedBodyOptions::stream_length method
add: trace fields
refactor: make inserting the final body CRLF more explicit
add: test to ensure trailer encoding and trailer len calculation stay in sync
* udpate: emit error instead of panicking for incorrect trailer length
* add: test ensuring trailer serialization and length calculation stay in sync for empty HeaderMap
* update: avoid repeated allocations in trailers_as_aws_chunked_bytes
fix: don't convert to str before getting len of HeaderValue
* add: aws-chunked body test for inner bodies that emit intermittent chunks
add: double check that stream_length used to create an `AwsChunkedBody` is correct.
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Remove smoketest/fullsdk model classifications
* Generate full SDK for CI using models in aws-sdk-rust
* Move the changelog renderer out of `sdk-lints` and add changelog splitter
* Save revision information into split off SDK changelog
* Replace common git impl with `sdk-sync`'s impl
* Filter SDK changelog entries on changelog render
* Add smithy-rs release workflow
* Add initial next SDK changelog
* Update `sdk-sync` to understand new models location
When `generateSmithyBuild` task is up to date, the hashes of the build
are not calculated, so `modifyMtime` fails, because it expects them to
be registered in Gradle's project properties. This can happen if you run
a command like `./gradlew codegen-server-test:test` twice consecutively.
In this case, we should skip the `modifyMtime` task altogether.
Everything is up to date, so no codegen will run, and no artifacts will
change.
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
This commit upgrades:
* Kotlin to 1.6.20,
* Kotest to 5.2.3,
* Ktlint to 0.45.2; and
* Jacoco to 0.8.8.
Perhaps the only noteworthy change is that Ktlint now has a lint to
detect that the name of a class/object defined solely in a file has to
coincide with the filename.
* Upgrade to Smithy 1.16.2
* Remove extra test fixed in Smithy 1.16.2
* Upgrade to Smithy 1.16.3
* Remove extra test fixed in Smithy 1.16.3
* Remove failing expectation from server tests
* Upgrade to Smithy 1.17.0
* Upgrade to Smithy 1.18.1
* Upgrade to Smithy 1.19.0
* Upgrade to Smithy 1.21.0
* Update changelog
* Fix Gradle repositories
This is the result of running `ktlint` from the repository root. We have
`ktlint` running on pre-commit and CI on changed files, but these
violations slipped through the cracks.
* Implement `crate-hasher` utility to hash crate source files
* Add `generate-version-manifest` subcommand to publisher tool
* Incorporate `generate-version-manifest` into the build system
* Start making the PR bot run through the Docker build image
* Use nightly clippy in tools CI when necessary
* Fix transient failure when acquiring the build image
* Include examples repo revision in `versions.toml`
* Fix nightly clippy lint in `cargo-api-linter`
* Track examples revision in sync tool
* Make it easier to run Docker CI locally
Establishes a Docker build image with build tools precompiled into it to improve CI time and reduce GitHub Actions workflow configuration. Also makes it possible to run the exact CI workflows in local development.
This commit allows for the `modules` and `cargoCommands` properties to
be specified when running the `codegen-test` and `codegen-server-test`
Gradle modules. The `modules` property allows one to only generate a
subset of the integration test services, while the `cargoCommands`
property allows one to specify the Cargo commands to be run on the
generated Rust crates.
This functionality can be useful to reduce development iteration cycles. For
instance, to only run `cargo test` on the integration test
`simple.smithy`:
```sh
./gradlew codegen-test:build -P cargoCommands='test' -P modules='simple'
```
This commit also refactors the buildscripts of the `codegen-test` and
the `codegen-server-test` modules, extracting shared functionality to
`buildSrc/src/main/kotlin/CodegenTestCommon.kt`.
* Move the publisher tool from `aws-sdk-rust`
* Add `rust-runtime:assemble` target to generate a publishable bundle
* Run `fix-manifests` on assemble output
* Produce publish-ready Smithy runtime bundle during CI
* Allow publish from any arbitrary directory
* Add safe-guard to prevent accidental publish from local dev
* Fix unit test target
* Incorporate feedback
* Add `buildSrc` tests and publisher tool to CI
* Add link to developer guide
* Update buildSrc/src/main/kotlin/DocsLandingPage.kt
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Improve generated docs for crates and clients
This commit makes a first pass at improving the documentation rendered for AWS crates for both generated crates as well as `Client` structs.
* Fixups
* CR feedback
* Rev2 of Github Pages Landing Page
- Generate `index.md` instead so it works directly with Jekyll
- Fix some bugs in table generation and generate a narrower table that doesn't need side-to-side scrolling
* Fix trailing whitespace in generated markdown
* Cleanup generated markdown
* fix aws-config link
Co-authored-by: Zelda Hessler <zelda.hessler@pm.me>
* Generate a Docs Landing Page
This commit refactors a portion of the build into `buildSrc` and adds a job to generate a docs landing page. The generated SDK now contains a `docs.md` that contains top level links to all services.
* Add copyright headers
* Order by human name
Zelda Hessler