* Upgrade Kotlin to 1.7.21
* Upgrade Ktlint to 0.48.2
* Run `pre-commit run --all-files` to fix broken Ktlints
* Fix string comparison broken by code formatting
* feat(aws-types): add api key to configuration
* chore: set package version to 0.52.0
* feat(aws-smithy-types): create auth types
* chore: use auth from smithy types
* chore: fix return self type
* chore: create http auth definition type
* chore: add constructor for http auth definition
* chore: ensure properties are not moved
* chore: create convenience constructors
* chore: add some todo comments
* feat: move query writer to aws-smithy-http crate
* chore(codegen): expose smithy http tower dependency
* chore: remove setters for auth definition
* chore: fix logical error for scheme not allowed
* chore: add constructor for basic and digest auth
* chore: allow equality comparision for api key
* Revert "chore: set package version to 0.52.0"
This reverts commit da660fcf16.
* chore: fix additional references to querywriter
* chore: implement from string for api key struct
* chore: disallow none api key in sdk config
* chore: fix formatting
* chore: add unit tests for auth types
* chore: add auth api key to external types
* chore: make query writer doc hidden
* feat: create aws-smithy-http-auth crate
* chore: use zeroing for auth api key
* chore: use builder pattern for auth definition
* chore: restructure http auth package
* chore: define default lint warning
* chore: include http auth in runtime common list
* chore: define setter for optional scheme
* chore: should panic while building auth definition
* chore: return missing required field error
* chore: make few code simplications for api key
* Revert "chore: add auth api key to external types"
This reverts commit b2318b0230.
* chore: revert api key from sdk config
* chore: panic on missing required field
* Opt out of `clippy::derive_partial_eq_without_eq`
---------
Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Copy non-service integration tests into SDK root tests directory
For tests in `aws/sdk/integration-tests` that are not named after a
service module, copy them into the SDK's root `tests/` directory so that
they are run as part of CI on the entire SDK.
* Add missing fields to integration test manifests
* Remove tests from root workspace
* Explicitly exclude the root tests from the root workspace
* Workaround bug in smithy-rules-engine
This PR works around a bug in smithy-rules-engine `Parameter.toBuilder()` — the method was discarding the default value (and other fields). This was fixed here: https://github.com/awslabs/smithy/pull/1571/files. Until that's merged, this works around it by copying the value manually. This also adds an SDK adhoc test that includes an endpoint test.
* change to TODO comment
* Add the `aws-credential-types` crate
This commit adds a new crate `aws-credential-types` to the `rust-runtime`
workspace. This lays the groundwork for being able to create a
`LazyCachingCredentialsProvider` outside the `aws-config` crate according
to the proposed solution in https://github.com/awslabs/smithy-rs/pull/2082.
We have moved the following into this new crate:
- Items in aws_types::credentials and and their dependencies
- Items in aws_config::meta::credentials and their dependencies
Finally, the crate comes with auxiliary files that are present in the
other crates in the `rust-runtime` workspace such as `external-types.toml`.
* Make `aws-types` depend on `aws-credential-types`
The credentials module has been moved from the `aws-types` crate to the
`aws-credential-types` crate. This leads to some of the items in the
`aws-types` crate adjusting their use statements to point to
`aws-credential-types`.
The `TimeSource` struct has also been moved to `aws-credential-types`
because it is used by `LazyCachingCredentialsProvider`. We have decided
to move it instead of duplicating it because `aws-config` was creating
a `TimeSource` from `aws-types` and then passing it to the builder for
`LazyCachingCredentialsProvider`. If we had duplicated the implementation
of `TimeSource` in `aws-credential-types`, two `TimeSource` implementations
would have been considered different types and the said use case in
`aws-config` would have been broken.
* Make `aws-config` depend on `aws-credential-types`
The `cache` module and modules in `meta::credentials` (except for `chain`)
have been moved to `aws-credential-types`. Again, the goal of restructuring
is to allow `LazyCachingCredentialsProvider` to be created outside the
`aws-config` crate. While doing so, we try not moving all the default
credential provider implementations.
* Make `aws-http` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `aws-inlineable` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `aws-sig-auth` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Emit `aws-credential-types` to the build directory
This commit adds `aws-credential-types` to AWS_SDK_RUNTIME so that
the build command `/gradlew :aws:sdk:assemble` can generate the crate
into sdk/build/aws-sdk.
* Make codegen aware of `aws-credential-types`
This commit allows the codegen to handle the `aws-credential-types` crate.
The items that have been moved from `aws-types` should now be prefixed with
`aws-credential-types` when generating fully qualified names.
* Make `dynamo-tests` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `s3-tests` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `s3control` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Update external-types.xml in rust-runtime crates
This commit fixes CI failures related to `cargo check-external-types`
in ec994be.
* Update the file permission on additional-ci
* Remove unused dependency from aws-credential-types
* Clean up features for aws-credential-types
This commit fixes a CI failure where the feature hardcoded-credentials
needed other features, aws-smithy-async/rt-tokio and tokio/rt, for the
test code to compile with --no-default-features.
* Update sdk-external-types.toml
* Update aws/rust-runtime/aws-credential-types/Cargo.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/README.md
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/README.md
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/additional-ci
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/src/lib.rs
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Reduce re-exports from `aws-credential-types`
This commit reduces the number of re-exports from `aws-credential-types`.
The rationale here is that if we add more items to this crate later on,
we may get some name collisions in root. Since this crate is not used by
our customers directly, it is acceptable for items to take a bit of typing
to get to.
* Fix broken intra doc link
This commit fixes a broken intra doc link that went unnoticed because the
offending link was behind the feature `hardcoded-credentials`.
* Update aws/rust-runtime/aws-credential-types/README.md
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Rename variants of `aws_credential_types::time_source::Inner`
This commit addresses https://github.com/awslabs/smithy-rs/pull/2108#discussion_r1053637722
* Split the unit test for `time_source` into two
This commit addresses https://github.com/awslabs/smithy-rs/pull/2108#discussion_r1053638381
* Update CHANGELOG.next.toml
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* update: CargoDependency companion fn names for smithy runtime crates
rename: CargoDependency.asType to CargoDependency.toType
fix: errors in InlineDependency doc comment
formatting: run import optimizer for all kotlin files
fix: gradle issue with incorrectly named tasks
* fix: server test broken by removal of rustName
This change provides compatibility with the latest Gradle 8.0 nightly.
Since the top-level code coverage stuff was already broken and
apparently no one noticed, I just deleted it. The coverage reports for
specific subprojects still work fine.
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
When registering the Cargo commands tasks. The `aws:sdk` is the only
Gradle subproject that depends on the Cargo commands tasks, but that
does not register the `modifyMtime` task.
Partially addresses #1596.
* Exclude sigv4 test suite from pre-commit file endings hook
* Upgrade Ktlint
* Make Gradle ktlint target consistent with pre-commit
They were previously running two separate rulesets since pre-commit
pulls in the standard ruleset, and that wasn't pulled into the Gradle
dependencies.
* Fix Gradle Ktlint paths
* Establish editorconfig for Ktlint
* Fix Ktlint lints
* Enable `trailing-commas` rule
* Fix block quote indentation
Include aws-smithy-http-server and aws-smithy-http-server-python in the list of crates that are automatically published to crates.io on each release of smithy-rs.
* Update `hydrate-readmes` to take versions from `versions.toml`
* Add cleanup TODO comments to `changelogger`
* Update SDK readme template
* Split up `generate-version-manifest` subcommand
* Eliminate the `aws.sdk.version` property
* Fallback to model hash comparison if model metadata claims no changes
* Add `acquire-base-image` to `release.yml`
* Use empty model metadata for SDK generation in CI
* Fix the `aws-config` version number in SDK crate readmes
* add: support for unsigned aws-chunked content encoding
* add: aws-smithy-checksums to CrateSet.key
* refactor: API of AwsChunkedBodyOptions
update: AwsChunkedBody to note we only support single chunks
remove: unnecessary `Option`s
remove: overly smart total_length_of_trailers_in_bytes in trailers_as_aws_chunked_bytes
update: use "where"-style declaration for `impl<Inner> Body for AwsChunkedBody<Inner>`
add: helpful data to trace logging
add: trailer len double check in AwsChunkedBody::poll_data
add: test for trailer len double check
add: assert to size_hint
fix: incorrect body emitted when body is empty
add: test for empty encoded body
* Update CHANGELOG.next.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* update: CHANGELOG.next.toml message
* refactor: aws-chunked trailer encoding and size calculation
update: when aws-chunked formatting a `HeaderMap`, header names with multiple values will be written out one value per line
remove: unnecessary AwsChunkedBodyOptions::stream_length method
add: trace fields
refactor: make inserting the final body CRLF more explicit
add: test to ensure trailer encoding and trailer len calculation stay in sync
* udpate: emit error instead of panicking for incorrect trailer length
* add: test ensuring trailer serialization and length calculation stay in sync for empty HeaderMap
* update: avoid repeated allocations in trailers_as_aws_chunked_bytes
fix: don't convert to str before getting len of HeaderValue
* add: aws-chunked body test for inner bodies that emit intermittent chunks
add: double check that stream_length used to create an `AwsChunkedBody` is correct.
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Remove smoketest/fullsdk model classifications
* Generate full SDK for CI using models in aws-sdk-rust
* Move the changelog renderer out of `sdk-lints` and add changelog splitter
* Save revision information into split off SDK changelog
* Replace common git impl with `sdk-sync`'s impl
* Filter SDK changelog entries on changelog render
* Add smithy-rs release workflow
* Add initial next SDK changelog
* Update `sdk-sync` to understand new models location
When `generateSmithyBuild` task is up to date, the hashes of the build
are not calculated, so `modifyMtime` fails, because it expects them to
be registered in Gradle's project properties. This can happen if you run
a command like `./gradlew codegen-server-test:test` twice consecutively.
In this case, we should skip the `modifyMtime` task altogether.
Everything is up to date, so no codegen will run, and no artifacts will
change.
Avoid unnecessary Cargo crate rebuilds
This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.
Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:
```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```
So the files get recreated each time.
While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.
Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.
With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.
This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.
Closes#1378.
Closes#1412.
This commit upgrades:
* Kotlin to 1.6.20,
* Kotest to 5.2.3,
* Ktlint to 0.45.2; and
* Jacoco to 0.8.8.
Perhaps the only noteworthy change is that Ktlint now has a lint to
detect that the name of a class/object defined solely in a file has to
coincide with the filename.
* Upgrade to Smithy 1.16.2
* Remove extra test fixed in Smithy 1.16.2
* Upgrade to Smithy 1.16.3
* Remove extra test fixed in Smithy 1.16.3
* Remove failing expectation from server tests
* Upgrade to Smithy 1.17.0
* Upgrade to Smithy 1.18.1
* Upgrade to Smithy 1.19.0
* Upgrade to Smithy 1.21.0
* Update changelog
* Fix Gradle repositories
This is the result of running `ktlint` from the repository root. We have
`ktlint` running on pre-commit and CI on changed files, but these
violations slipped through the cracks.
* Implement `crate-hasher` utility to hash crate source files
* Add `generate-version-manifest` subcommand to publisher tool
* Incorporate `generate-version-manifest` into the build system
* Start making the PR bot run through the Docker build image
* Use nightly clippy in tools CI when necessary
* Fix transient failure when acquiring the build image
* Include examples repo revision in `versions.toml`
* Fix nightly clippy lint in `cargo-api-linter`
* Track examples revision in sync tool
* Make it easier to run Docker CI locally
Establishes a Docker build image with build tools precompiled into it to improve CI time and reduce GitHub Actions workflow configuration. Also makes it possible to run the exact CI workflows in local development.
This commit allows for the `modules` and `cargoCommands` properties to
be specified when running the `codegen-test` and `codegen-server-test`
Gradle modules. The `modules` property allows one to only generate a
subset of the integration test services, while the `cargoCommands`
property allows one to specify the Cargo commands to be run on the
generated Rust crates.
This functionality can be useful to reduce development iteration cycles. For
instance, to only run `cargo test` on the integration test
`simple.smithy`:
```sh
./gradlew codegen-test:build -P cargoCommands='test' -P modules='simple'
```
This commit also refactors the buildscripts of the `codegen-test` and
the `codegen-server-test` modules, extracting shared functionality to
`buildSrc/src/main/kotlin/CodegenTestCommon.kt`.
* Move the publisher tool from `aws-sdk-rust`
* Add `rust-runtime:assemble` target to generate a publishable bundle
* Run `fix-manifests` on assemble output
* Produce publish-ready Smithy runtime bundle during CI
* Allow publish from any arbitrary directory
* Add safe-guard to prevent accidental publish from local dev
* Fix unit test target
* Incorporate feedback
* Add `buildSrc` tests and publisher tool to CI
* Add link to developer guide
* Update buildSrc/src/main/kotlin/DocsLandingPage.kt
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Improve generated docs for crates and clients
This commit makes a first pass at improving the documentation rendered for AWS crates for both generated crates as well as `Client` structs.
* Fixups
* CR feedback
* Rev2 of Github Pages Landing Page
- Generate `index.md` instead so it works directly with Jekyll
- Fix some bugs in table generation and generate a narrower table that doesn't need side-to-side scrolling
* Fix trailing whitespace in generated markdown
* Cleanup generated markdown
* fix aws-config link
Co-authored-by: Zelda Hessler <zelda.hessler@pm.me>
* Generate a Docs Landing Page
This commit refactors a portion of the build into `buildSrc` and adds a job to generate a docs landing page. The generated SDK now contains a `docs.md` that contains top level links to all services.
* Add copyright headers
* Order by human name
John DiSanti