_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: david-perez <d@vidp.dev>
## Motivation and Context
This PR upgrades the MSRV to 1.68.2.
## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: 82marbag <69267416+82marbag@users.noreply.github.com>
## Motivation and Context
Addresses https://github.com/awslabs/smithy-rs/issues/2104
## Description
This PR allows us to build `aws-sdk-rust` even with unknown traits
present in the AWS models. This requires `Smithy` 1.29.0 that includes
[support](https://github.com/awslabs/smithy/pull/1685) for an extension
property,
[allowUnknownTraits](https://smithy.io/2.0/guides/building-models/gradle-plugin.html?highlight=allowunknowntraits#smithy-extension-properties).
## Testing
Manually added a unknown trait into an AWS model like so:
```
diff --git a/aws/sdk/aws-models/sts.json b/aws/sdk/aws-models/sts.json
--- a/aws/sdk/aws-models/sts.json
+++ b/aws/sdk/aws-models/sts.json
@@ -69,6 +69,7 @@
"aws.auth#sigv4": {
"name": "sts"
},
+ "aws.protocols#awsQuery2": {},
"aws.protocols#awsQuery": {},
```
and confirmed `/gradlew :aws:sdk:assemble` built SDK successfully. Also
verified that without `allowUnknownTraits = true` in
`aws/sdk/build.gradle.kts`, `/gradlew :aws:sdk:assemble` failed with the
above unknown trait.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
* Upgrade to Smithy 1.28.1
* fix a couple of failing protocol tests
* Adhere to new ValidationException message format
* Add RestJsonMalformed to FailingTest
* Add various fixes
---------
Co-authored-by: Harry Barber <hlbarber@amazon.co.uk>
Co-authored-by: Harry Barber <106155934+hlbarber@users.noreply.github.com>
* Upgrade to Smithy 1.27.1
* Fix bugs exposed by 1.27.1
* Fix rules engine customization
* update rfc3339 parsing test
* refactor timestamp format & remove upstreamed test
* Update changelog
* rename test
* fix datetime parsing on server and clients
* fix up a few more failing tests
* precommit lints
* Fix bad merge
* Upgrade Kotlin to 1.7.21
* Upgrade Ktlint to 0.48.2
* Run `pre-commit run --all-files` to fix broken Ktlints
* Fix string comparison broken by code formatting
* Add `RequestId` trait
* Implement `RequestId` for generated AWS client errors
* Move `RustWriter.implBlock` out of `StructureGenerator`
* Create structure/builder customization hooks
* Customize `_request_id` into AWS outputs
* Set request ID on outputs
* Refactor SDK service decorators
* Refactor S3's extended request ID implementation
* Combine `Error` and `ErrorKind`
* Add test for service error conversion
* Move error generators into `codegen-client` and fix tests
* Re-export `ErrorMetadata`
* Add request IDs to trace logs
* Simplify some error trait handling
* Rename `ClientContextParamDecorator` to `ClientContextConfigCustomization`
* Add deprecated alias to guide customers through upgrading
* Rename the `ErrorMetadata` trait to `ProvideErrorMetadata`
* Rename `aws_smithy_types::Error` to `ErrorMetadata`
* What happens if we comment out the runtime crate version from gradle.properties?
* Allow running the release and the CI workflows from an arbitrary commit.
* Does a fake version work?
* Pass `git_ref` from the release workflow.
* It needs to be a valid semver version.
* Sketch new command to upgrade version in gradle.properties
* Command implementation
* Plug the new publisher command into the `release` action.
* Plumb end-to-end
* Fix copyright header.
* Fix lint.
* Temporarily comment out the sanity check.
* Ignore sanity check
* Add a command that prints out the template for CHANGELOG.next.toml
* Add branch check + empty TOML generation.
* Add copyright headers.
* Fix imports.
* Remove sanity check.
* Move script to a file.
* Add a check to validate the tag.
* Remove second build step.
* Move to .github/scripts folder.
* Make the script easier to run locally
* Fail if anything fails.
* Add comment.
* Update .github/scripts/get-or-create-release-branch.sh
Co-authored-by: david-perez <d@vidp.dev>
* Update .github/scripts/get-or-create-release-branch.sh
Co-authored-by: david-perez <d@vidp.dev>
* Update .github/scripts/get-or-create-release-branch.sh
Co-authored-by: david-perez <d@vidp.dev>
* Update .github/workflows/ci.yml
Co-authored-by: david-perez <d@vidp.dev>
* Remove touch.
* Fix indentation and branch name.
* Update .github/workflows/ci.yml
Co-authored-by: david-perez <d@vidp.dev>
* Update .github/workflows/release.yml
Co-authored-by: david-perez <d@vidp.dev>
* Update .github/workflows/release.yml
Co-authored-by: david-perez <d@vidp.dev>
* Explicit flags.
* Use the path that was provided.
* Format
---------
Co-authored-by: david-perez <d@vidp.dev>
* Introduce a flag in gradle.properties to enable/disable testing (and the compilation of the associated dependencies).
* Feature flag testing in all non-test-only build.gradle.kts files.
* Remove dokka
* Replace our base64 implementation with one based on `base64-simd`
* Bump Rust MSRV to 1.62.1
* Unmark base64-related tests as failing
* Use `usize` for `base64::encoded_length`
* Run Rust 1.62.1 on CI
* Don't use external `base64_simd::Error` type in public interface
* Add initial criterion bench setup for base64
* Compare `base64_simd::encode` vs Smithy's `encode`
* Use string length for `BenchmarkId`
* Compare `base64_simd::decode` vs Smithy's `decode`
* Compare `base64_simd::encoded_length` vs Smithy's `encoded_length`
* Run clippy --fix for Rust 1.62.0 and format the resulting code
* Run clippy --fix on all targets
* Run clippy --fix with all features enabled
* Avoid extra allocation
* Use more idiomatic assert
* Ignore noisy lint
* Update pyo3 and pyo3-asyncio to 0.17.0
* Implement Eq on aws-smithy-checksums::Error
* Implement Eq on Protocol
* Replace conditionals with range-containment
* Implement Eq on types in aws-smithy-types
* Implement Eq on types in aws-smithy-http-server-python
* Implement Eq on types in aws-smithy-eventstream
* Implement Eq on types in aws-smithy-xml
* Implement Eq on aws-sigv4
* Update CI to use Rust 1.62.0
* Add Eq for generated types that implement PartialEq
* Allow clippy::needless_return in generated code
* Remove unnecessary reborrow in http_serde
* Remove unnecessary borrow in operation_deser
* Add CHANGELOG entries
* Revert "Add Eq for generated types that implement PartialEq"
This reverts commit 5169bd95aa.
* Update pyo3 and pyo3-asyncio in generated code for python server
* Allow clippy::derive_partial_eq_without_eq on structs and builders
* Run clippy on tools
* Fix accidental move in generated code
* Revert "Allow clippy::derive_partial_eq_without_eq on structs and builders"
This reverts commit 068c63ca20.
* Fix another accidental move in generated code
* Undo unwanted change to model
* Re-add reborrow in HttpBindingGenerator
* Fix clippy::format-push-string in changelogger
* Fix more uses of str.push_str(&format!(...))
* Remove unnecessary parenthesis
* Run ktlint
* Update aws/rust-runtime/aws-http/src/content_encoding.rs
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-http/src/content_encoding.rs
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Ignore doctest for non-exported macro
Rust 1.62 introduced a breaking change where doctests for non-exported
macros are now run by default. These don't compile because the macro
can't be imported in it.
See https://github.com/rust-lang/rust/issues/97030 for more info.
* Run cargo fmt
* Use $crate instead of crate in macro
* Revert "Implement Eq on types in aws-smithy-types"
This reverts commit c45a6b5a56.
* Revert "Implement Eq on types in aws-smithy-eventstream"
This reverts commit 78f4b07344.
* Revert "Implement Eq on types in aws-smithy-xml"
This reverts commit 590f01af73.
* Revert "Implement Eq on aws-sigv4"
This reverts commit d78bb62124.
* Revert "Implement Eq on types in aws-smithy-http-server-python"
This reverts commit f2cd901884.
* Revert "Implement Eq on aws-smithy-checksums::Error"
This reverts commit 5da170405e.
Co-authored-by: Julian Antonielli <julianantonielli@gmail.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Fix `aws:sdk-codegen-test` build
* Add config for SDK endpoint/readme generation
* Rename `sdk-codegen-test` to `sdk-adhoc-test` and update README
* Add `sdk-adhoc-tests` to CI
* Refactor CI scripts
The `check-rust-runtime-and-tools` script was getting beyond 20
minutes to run, so this commit splits it into `check-aws-config`,
`check-rust-runtimes`, and `check-tools`.
This also enables most of the checks to run before the smoke test SDK is
generated, which should deliver more immediate feedback on PRs.
* Give the canary its own CI step
* Exclude sigv4 test suite from pre-commit file endings hook
* Upgrade Ktlint
* Make Gradle ktlint target consistent with pre-commit
They were previously running two separate rulesets since pre-commit
pulls in the standard ruleset, and that wasn't pulled into the Gradle
dependencies.
* Fix Gradle Ktlint paths
* Establish editorconfig for Ktlint
* Fix Ktlint lints
* Enable `trailing-commas` rule
* Fix block quote indentation
* Update `hydrate-readmes` to take versions from `versions.toml`
* Add cleanup TODO comments to `changelogger`
* Update SDK readme template
* Split up `generate-version-manifest` subcommand
* Eliminate the `aws.sdk.version` property
* Fallback to model hash comparison if model metadata claims no changes
* Add `acquire-base-image` to `release.yml`
* Use empty model metadata for SDK generation in CI
* Fix the `aws-config` version number in SDK crate readmes
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
It's currently defaulting to 512 MiB. I often see this message in my
build logs:
> Daemon will be stopped at the end of the build after running out of JVM memory
I've done some (unscientific) profiling with this setting enabled, and
max heap size can peak at ~600 MiB, so I see no reason to increase it
more than 1024 MiB.
This commit upgrades:
* Kotlin to 1.6.20,
* Kotest to 5.2.3,
* Ktlint to 0.45.2; and
* Jacoco to 0.8.8.
Perhaps the only noteworthy change is that Ktlint now has a lint to
detect that the name of a class/object defined solely in a file has to
coincide with the filename.
* Upgrade to Smithy 1.16.2
* Remove extra test fixed in Smithy 1.16.2
* Upgrade to Smithy 1.16.3
* Remove extra test fixed in Smithy 1.16.3
* Remove failing expectation from server tests
* Upgrade to Smithy 1.17.0
* Upgrade to Smithy 1.18.1
* Upgrade to Smithy 1.19.0
* Upgrade to Smithy 1.21.0
* Update changelog
* Fix Gradle repositories
AWS SDK Rust Bot