Commit Graph

49 Commits

Author SHA1 Message Date
John DiSanti f310668417
Stop rebuilding `sdk-lints` all the time (#2861)
I noticed `sdk-lints` was getting recompiled just about every time I ran
`git commit` as part of the pre-commit hooks. It looks like a compiler
flag was added to the `ExecRustBuildTool` task type that isn't actually
used by any tools. Removing it should eliminate compiler flag conflicts
between the gradle targets and pre-commit.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-07-20 13:15:34 +00:00
John DiSanti ac95a5d1f0
Migrate to a new SDK example workspace structure (#2811)
## Motivation and Context
When the WebAssembly SDK example was added some months ago, we changed
the build process to make each SDK example its own Cargo workspace. This
allowed the `.cargo/config.toml` that changed the compiler target to
work correctly. However, this has led to other issues: dependency
compilation is no longer shared between examples which greatly increases
the time it takes for that CI step to run, and now it is even running
out of disk space on the GitHub Actions runners.

This PR adds support for a new workspace layout where the
[`aws-doc-sdk-examples`](https://github.com/awsdocs/aws-doc-sdk-examples)
repo gets to decide how the workspaces are logically grouped. If a
`Cargo.toml` file exists at the example root, then the build system
assumes that the _old_ "one example, one workspace" layout should be
used. If there is no root `Cargo.toml`, then it assumes the new layout
should be used.

The `sdk-versioner` tool had to be adapted to support more complex
relative path resolution to make this work, and the `publisher
fix-manifests` subcommand had to be fixed to ignore workspace-only
`Cargo.toml` files.

The build system in this PR needs to work for both the old and new
examples layout so that the `sdk-sync` process will succeed. #2810 has
been filed to track removing the old example layout at a later date.


[aws-doc-sdk-examples#4997](https://github.com/awsdocs/aws-doc-sdk-examples/pull/4997)
changes the workspace structure of the actual examples to the new one.

## Testing
- [x] Generated a full SDK with the old example layout, manually
examined the output, and spot checked that some examples compile
- [x] Generated a full SDK with the new example layout, manually
examined the output, and spot checked that some examples compile
- [x] Examples pass in CI with the old example layout
- [x] Examples pass in CI with the new example layout

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-06-30 17:46:11 +00:00
John DiSanti c8ba2d5b80
Fix auth failures in codegen-client-tests in orchestrator mode (#2812)
## Motivation and Context
In orchestrator mode, most `codegen-client-test` tests were failing due
to being unable to find a matching auth scheme, or due to some of the
test models referencing the `@sigv4` trait. This PR fixes all of those
failures, and adds the `smithy.runtime.mode` flag to
`codegen-client-test` as well.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-06-28 17:08:00 +00:00
Russell Cohen 8c045d271c
Add support for TimeStreamWrite and TimeStreamQuery (#2707)
TODO:
- [x] docs
- [x] integration test (canary even?)
- [x] customize README for timestream
## Motivation and Context
- #613 
- https://github.com/awslabs/aws-sdk-rust/issues/114


## Description
This adds support for TSW and TSQ by adding endpoint discovery as a
customization. This is made much simpler by the fact that endpoint
discovery for these services **has no parameters** which means that
there is no complexity from caching the returned endpoint.

Customers call `.enable_endpoint_discovery()` on the client to create a
version of the client with endpoint discovery enabled. This returns a
new client and a Reloader from which customers must spawn the reload
task if they want endpoint discovery to rerun.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-05-31 18:08:51 +00:00
John DiSanti 9bfe936fbc
Make the SDK ad hoc tests pass against the orchestrator (#2708)
## Motivation and Context
This PR refactors the client protocol test generator machinery to use a
client instead of calling `make_operation` directly, and then fixes the
ad hoc tests for the orchestrator.

The ad hoc tests revealed that overriding the signing region/service via
endpoint config was lost when porting SigV4 signing to the orchestrator,
so this PR updates the SigV4 `HttpRequestSigner` implementation to
restore this functionality. It is doing this in the signer directly
rather than via an interceptor since it should only run this logic when
SigV4 is the selected auth scheme.

Other notable changes:
- Adds `--no-fail-fast` arg to `cargoTest` targets so that all Rust
tests run in CI rather than stopping on the first failure
- Changes `EndpointResolver::resolve_and_apply_endpoint` to just
`resolve_endpoint` so that the orchestrator can place the endpoint
config into the request state, which is required for the signer to make
use of it
- Adds a `set_region` method to SDK service configs
- Deletes the API Gateway model and integration test from the SDK smoke
test since it is covered by the ad hoc tests
- Adds a comment explaining where the API Gateway model comes from in
the ad hoc tests
- Adds a `smithy.runtime.mode` Gradle property to `aws:sdk` and
`aws:sdk-adhoc-test` to trivially switch between middleware and
orchestrator when testing/generating locally

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2023-05-19 16:58:28 +00:00
Thomas Cameron 9f10be04b4
Add `RUSTFLAGS = aws_sdk_unstable` as environment variable when running Cargo from kotlin (#2614)
## Motivation and Context
This PR set `aws_sdk_unstable` to RUSTFALGS when running cargo from
kotlin.

It is required to enable test gated features introduced on RFC30.

No breaking changers are introduced.

## Description
add `RUSTFLAGS = aws_sdk_unstable` when running cargo from kotlin.


## Parent PR
This PR is listed as one of prerequisite PRs on this PR.

- https://github.com/awslabs/smithy-rs/pull/2615

## Testing
NA.

----
*By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice.*
2023-04-26 14:54:48 +00:00
Matteo Bigoi 3862ca0a37
Initial implementation of Typescript server bindings (#2277)
* A barely working code generation in typescript

* Extract shared socket into feature inside aws-smithy-http-server

* Building a fully functional application, I think

* Add NAPI build.rs

* Refactor all names to use typescript instead of js

* Add (hopefully) the PR bot for TS

* Clippy fixes

* Fix documentation

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>

* set_reuse_port in socket not on windows

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>

* Add example implementation

* Allow the new application to build

* Remove all occurrences of Python

* Simplify README

* Fix issue with the codegen-diff-revision script

* Try to prevent the ci-lint to bother us with TODO at this stage of development

* Remove codegen-client from typescript dependencies

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>

* Add CODEOWNERS and fix some other linting issues

* Add license

* Prevent from running tests on typescript in Windows

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>

* Initial work to remove error from this PR

* Update to call_async

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>

* types/node in package.json

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>

* Generate app.ts

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>

* Improve makefile

* Adapting code to the latest changes and removing runtime dependency (for now).

* Removing rust-runtime/aws-smithy-http-server-typescript.

* Making CI happy.

* Restoring ServerCodegenDecorator to be like main.

* Adding back the aws-smithy-http-server-typescript crate back.

* Removing index.js file.

---------

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
Co-authored-by: 82marbag <69267416+82marbag@users.noreply.github.com>
Co-authored-by: Alberto Pose <albepose@amazon.com>
2023-04-06 15:27:17 +00:00
John DiSanti 9006fe8775
Add SigV4 support to the orchestrator (#2533)
* Create the `aws-runtime` and `aws-runtime-api` crates

* Implement SigV4 in the orchestrator

* CI fixes

* Connect the service config http connector to the orchestrator

* Make it possible to register interceptors, and get signing to pass tests

* Fix allowed types lint

* Add missing docs.rs metadata
2023-04-05 17:33:03 +00:00
ysaito1001 c901c0b1c6
Exclude examples from the workspace (#2535)
This commit automates the fix made in https://github.com/awslabs/aws-sdk-rust/pull/767.
When `aws-sdk-rust` is generated, `examples` will be excluded from the
workspace.

Co-authored-by: Yuki Saito <awsaito@amazon.com>
2023-04-04 01:51:37 +00:00
John DiSanti 48bd719743
Remove `enableNewCrateOrganizationScheme` codegen flag (#2507) 2023-03-29 16:57:32 +00:00
Zelda Hessler bde362e8b9
Feature: Add smithy orchestrator (#2447)
* add: WIP for smithy orchestrator

* add: publish = false to the new, unreleased crates

* undo: more extraneous changes

* fix: correctly declare `publish = false`

* feature: add layered config bag

* Update aws/sdk/integration-tests/smithy_orchestrator/Cargo.toml

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* add: READMEs to new crates
add: external-types.toml to new crates
add: LICENSEs to new crates
update: crate organization and naming
update: interceptor errors per RFC 22
update: token bucket errors per RFC 22

* fix: bad imports in integration test

* fix: error struct naming

* fix: err source issue

* fix: clippy lints
fix: TODOs with no subject

* fix: broken token bucket test
add: docs.rs metadata to cargo toml of new crates

* update: crate naming

* fix: broken test import

* fix: cargo doc issues

* update: aws-smithy-runtime external-types.toml

* remove: unused deps
fix: move allowed external types to correct crate

* add: exception for aws_smithy_types

---------

Co-authored-by: Russell Cohen <rcoh@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2023-03-14 21:44:43 +00:00
John DiSanti 358d13a09a
Enable the crate reorganization for generic clients (#2448) 2023-03-13 19:00:11 -07:00
John DiSanti 2f60a5e090
Update top level docs for crate reorganization (#2432) 2023-03-10 10:49:25 -08:00
John DiSanti 85d2ace0d6
Upgrade Kotlin and Ktlint (#2392)
* Upgrade Kotlin to 1.7.21

* Upgrade Ktlint to 0.48.2

* Run `pre-commit run --all-files` to fix broken Ktlints

* Fix string comparison broken by code formatting
2023-02-22 18:21:06 +00:00
Eduardo de Moura Rodrigues 1d1e68aeba
feat(smithy-http-auth): add api key auth types (#2153)
* feat(aws-types): add api key to configuration

* chore: set package version to 0.52.0

* feat(aws-smithy-types): create auth types

* chore: use auth from smithy types

* chore: fix return self type

* chore: create http auth definition type

* chore: add constructor for http auth definition

* chore: ensure properties are not moved

* chore: create convenience constructors

* chore: add some todo comments

* feat: move query writer to aws-smithy-http crate

* chore(codegen): expose smithy http tower dependency

* chore: remove setters for auth definition

* chore: fix logical error for scheme not allowed

* chore: add constructor for basic and digest auth

* chore: allow equality comparision for api key

* Revert "chore: set package version to 0.52.0"

This reverts commit da660fcf16.

* chore: fix additional references to querywriter

* chore: implement from string for api key struct

* chore: disallow none api key in sdk config

* chore: fix formatting

* chore: add unit tests for auth types

* chore: add auth api key to external types

* chore: make query writer doc hidden

* feat: create aws-smithy-http-auth crate

* chore: use zeroing for auth api key

* chore: use builder pattern for auth definition

* chore: restructure http auth package

* chore: define default lint warning

* chore: include http auth in runtime common list

* chore: define setter for optional scheme

* chore: should panic while building auth definition

* chore: return missing required field error

* chore: make few code simplications for api key

* Revert "chore: add auth api key to external types"

This reverts commit b2318b0230.

* chore: revert api key from sdk config

* chore: panic on missing required field

* Opt out of `clippy::derive_partial_eq_without_eq`

---------

Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2023-02-13 23:07:06 +00:00
John DiSanti 7ad342aabe
Copy non-service integration tests into SDK root tests directory (#2255)
* Copy non-service integration tests into SDK root tests directory

For tests in `aws/sdk/integration-tests` that are not named after a
service module, copy them into the SDK's root `tests/` directory so that
they are run as part of CI on the entire SDK.

* Add missing fields to integration test manifests
* Remove tests from root workspace
* Explicitly exclude the root tests from the root workspace
2023-01-27 21:13:25 +00:00
david-perez 4fbfc07336
Make `RequireRustBuildTool` Gradle task use `--locked` when `cargo install`ing (#2210) 2023-01-18 15:26:06 +00:00
Russell Cohen fd8b2d4efa
Workaround bug in smithy-rules-engine (#2179)
* Workaround bug in smithy-rules-engine

This PR works around a bug in smithy-rules-engine `Parameter.toBuilder()` — the method was discarding the default value (and other fields). This was fixed here: https://github.com/awslabs/smithy/pull/1571/files. Until that's merged, this works around it by copying the value manually. This also adds an SDK adhoc test that includes an endpoint test.

* change to TODO comment
2023-01-09 16:37:12 +00:00
ysaito1001 bb5bbeb5cf
Move types for AWS SDK credentials to a separate crate (#2108)
* Add the `aws-credential-types` crate

This commit adds a new crate `aws-credential-types` to the `rust-runtime`
workspace. This lays the groundwork for being able to create a
`LazyCachingCredentialsProvider` outside the `aws-config` crate according
to the proposed solution in https://github.com/awslabs/smithy-rs/pull/2082.

We have moved the following into this new crate:
- Items in aws_types::credentials and and their dependencies
- Items in aws_config::meta::credentials and their dependencies

Finally, the crate comes with auxiliary files that are present in the
other crates in the `rust-runtime` workspace such as `external-types.toml`.

* Make `aws-types` depend on `aws-credential-types`

The credentials module has been moved from the `aws-types` crate to the
`aws-credential-types` crate. This leads to some of the items in the
`aws-types` crate adjusting their use statements to point to
`aws-credential-types`.

The `TimeSource` struct has also been moved to `aws-credential-types`
because it is used by `LazyCachingCredentialsProvider`. We have decided
to move it instead of duplicating it because `aws-config` was creating
a `TimeSource` from `aws-types` and then passing it to the builder for
`LazyCachingCredentialsProvider`. If we had duplicated the implementation
of `TimeSource` in `aws-credential-types`, two `TimeSource` implementations
would have been considered different types and the said use case in
`aws-config` would have been broken.

* Make `aws-config` depend on `aws-credential-types`

The `cache` module and modules in `meta::credentials` (except for `chain`)
have been moved to `aws-credential-types`. Again, the goal of restructuring
is to allow `LazyCachingCredentialsProvider` to be created outside the
`aws-config` crate. While doing so, we try not moving all the default
credential provider implementations.

* Make `aws-http` depend on `aws-credential-types`

This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.

* Make `aws-inlineable` depend on `aws-credential-types`

This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.

* Make `aws-sig-auth` depend on `aws-credential-types`

This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.

* Emit `aws-credential-types` to the build directory

This commit adds `aws-credential-types` to AWS_SDK_RUNTIME so that
the build command `/gradlew :aws:sdk:assemble` can generate the crate
into sdk/build/aws-sdk.

* Make codegen aware of `aws-credential-types`

This commit allows the codegen to handle the `aws-credential-types` crate.
The items that have been moved from `aws-types` should now be prefixed with
`aws-credential-types` when generating fully qualified names.

* Make `dynamo-tests` depend on `aws-credential-types`

This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.

* Make `s3-tests` depend on `aws-credential-types`

This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.

* Make `s3control` depend on `aws-credential-types`

This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.

* Update external-types.xml in rust-runtime crates

This commit fixes CI failures related to `cargo check-external-types`
in ec994be.

* Update the file permission on additional-ci

* Remove unused dependency from aws-credential-types

* Clean up features for aws-credential-types

This commit fixes a CI failure where the feature hardcoded-credentials
needed other features, aws-smithy-async/rt-tokio and tokio/rt, for the
test code to compile with --no-default-features.

* Update sdk-external-types.toml

* Update aws/rust-runtime/aws-credential-types/Cargo.toml

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-credential-types/README.md

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-credential-types/README.md

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-credential-types/additional-ci

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-credential-types/src/lib.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Reduce re-exports from `aws-credential-types`

This commit reduces the number of re-exports from `aws-credential-types`.
The rationale here is that if we add more items to this crate later on,
we may get some name collisions in root. Since this crate is not used by
our customers directly, it is acceptable for items to take a bit of typing
to get to.

* Fix broken intra doc link

This commit fixes a broken intra doc link that went unnoticed because the
offending link was behind the feature `hardcoded-credentials`.

* Update aws/rust-runtime/aws-credential-types/README.md

Co-authored-by: Zelda Hessler <zhessler@amazon.com>

* Rename variants of `aws_credential_types::time_source::Inner`

This commit addresses https://github.com/awslabs/smithy-rs/pull/2108#discussion_r1053637722

* Split the unit test for `time_source` into two

This commit addresses https://github.com/awslabs/smithy-rs/pull/2108#discussion_r1053638381

* Update CHANGELOG.next.toml

Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-12-21 19:12:53 +00:00
Thomas Cameron 1deb644afe
Check all features in gradle `cargoCheck`/`cargoTest` targets (#2100) 2022-12-16 21:31:19 +00:00
david-perez 66fd6c9f62
Rename command to generate RustDocs in `CodegenTestCommon.kt` (#2094)
The command is `cargo doc`, not `cargo docs`.
2022-12-15 12:06:37 +00:00
Zelda Hessler 4aca7b32fa
update: CargoDependency companion fn names for smithy runtime crates (#1996)
* update: CargoDependency companion fn names for smithy runtime crates
rename: CargoDependency.asType to CargoDependency.toType
fix: errors in InlineDependency doc comment
formatting: run import optimizer for all kotlin files
fix: gradle issue with incorrectly named tasks

* fix: server test broken by removal of rustName
2022-11-17 19:19:56 +00:00
Ryan Schmitt 884f441873
Fix Gradle deprecation warnings (#1978)
This change provides compatibility with the latest Gradle 8.0 nightly.
Since the top-level code coverage stuff was already broken and
apparently no one noticed, I just deleted it. The coverage reports for
specific subprojects still work fine.
2022-11-11 13:04:30 -08:00
Zelda Hessler 1d9aedea21
fix: compiler warning in docs index generator (#1900) 2022-10-25 16:17:50 +00:00
John DiSanti 96e9f61fee
Establish the `codegen-core` module (#1697)
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
2022-09-07 09:31:46 -07:00
david-perez 3a2de293f4
Don't depend on the `modifyMtime` task if it has not been registered (#1607)
When registering the Cargo commands tasks. The `aws:sdk` is the only
Gradle subproject that depends on the Cargo commands tasks, but that
does not register the `modifyMtime` task.

Partially addresses #1596.
2022-08-02 10:55:35 +00:00
John DiSanti 852c2ec260
Upgrade Ktlint and fix lint configuration (#1572)
* Exclude sigv4 test suite from pre-commit file endings hook
* Upgrade Ktlint
* Make Gradle ktlint target consistent with pre-commit

They were previously running two separate rulesets since pre-commit
pulls in the standard ruleset, and that wasn't pulled into the Gradle
dependencies.

* Fix Gradle Ktlint paths
* Establish editorconfig for Ktlint
* Fix Ktlint lints
* Enable `trailing-commas` rule
* Fix block quote indentation
2022-07-27 18:04:07 +00:00
Fahad Zubair f1af4435ec
aws-smithy-http-server crates to be published as part of release cycle (#1563)
Include aws-smithy-http-server and aws-smithy-http-server-python in the list of crates that are automatically published to crates.io on each release of smithy-rs.
2022-07-21 22:50:22 +01:00
John DiSanti 6143d90197
Enable independent crate versioning for SDK crates (#1540)
* Update `hydrate-readmes` to take versions from `versions.toml`
* Add cleanup TODO comments to `changelogger`
* Update SDK readme template
* Split up `generate-version-manifest` subcommand
* Eliminate the `aws.sdk.version` property
* Fallback to model hash comparison if model metadata claims no changes
* Add `acquire-base-image` to `release.yml`
* Use empty model metadata for SDK generation in CI
* Fix the `aws-config` version number in SDK crate readmes
2022-07-19 12:27:45 -07:00
John DiSanti c220b03a18
Use endpoint config from aws-sdk-rust (#1543) 2022-07-12 09:43:00 -07:00
Zelda Hessler b77c66cbf7
Feature: add support for aws-chunked content encoding (#1501)
* add: support for unsigned aws-chunked content encoding

* add: aws-smithy-checksums to CrateSet.key

* refactor: API of AwsChunkedBodyOptions
update: AwsChunkedBody to note we only support single chunks
remove: unnecessary `Option`s
remove: overly smart total_length_of_trailers_in_bytes in trailers_as_aws_chunked_bytes
update: use "where"-style declaration for `impl<Inner> Body for AwsChunkedBody<Inner>`
add: helpful data to trace logging
add: trailer len double check in AwsChunkedBody::poll_data
add: test for trailer len double check
add: assert to size_hint
fix: incorrect body emitted when body is empty
add: test for empty encoded body

* Update CHANGELOG.next.toml

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* update: CHANGELOG.next.toml message

* refactor: aws-chunked trailer encoding and size calculation
update: when aws-chunked formatting a `HeaderMap`, header names with multiple values will be written out one value per line
remove: unnecessary AwsChunkedBodyOptions::stream_length method
add: trace fields
refactor: make inserting the final body CRLF more explicit
add: test to ensure trailer encoding and trailer len calculation stay in sync

* udpate: emit error instead of panicking for incorrect trailer length

* add: test ensuring trailer serialization and length calculation stay in sync for empty HeaderMap

* update: avoid repeated allocations in trailers_as_aws_chunked_bytes
fix: don't convert to str before getting len of HeaderValue

* add: aws-chunked body test for inner bodies that emit intermittent chunks
add: double check that stream_length used to create an `AwsChunkedBody` is correct.

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-07-05 11:36:14 -05:00
John DiSanti 52c1174ac3
Decouple `aws-sdk-rust` release from `smithy-rs` release (#1495)
* Remove smoketest/fullsdk model classifications
* Generate full SDK for CI using models in aws-sdk-rust
* Move the changelog renderer out of `sdk-lints` and add changelog splitter
* Save revision information into split off SDK changelog
* Replace common git impl with `sdk-sync`'s impl
* Filter SDK changelog entries on changelog render
* Add smithy-rs release workflow
* Add initial next SDK changelog
* Update `sdk-sync` to understand new models location
2022-06-27 19:16:48 +00:00
david-perez a91b813e43
Fix `modifyMtime` task when `generateSmithyBuild` task is up to date (#1471)
When `generateSmithyBuild` task is up to date, the hashes of the build
are not calculated, so `modifyMtime` fails, because it expects them to
be registered in Gradle's project properties. This can happen if you run
a command like `./gradlew codegen-server-test:test` twice consecutively.

In this case, we should skip the `modifyMtime` task altogether.
Everything is up to date, so no codegen will run, and no artifacts will
change.
2022-06-17 16:12:56 +00:00
david-perez 40f22207ad
Avoid unnecessary Cargo crate rebuilds (#1422)
Avoid unnecessary Cargo crate rebuilds

This commit modifies the Gradle buildscripts to avoid unnecessary Cargo
rebuilds of the generated crates, decreasing development iteration
cycles.

Prior to this commit, if you ran a crate-generating command _twice_ on
any of the `codegen-test`, `codegen-server-test`, `sdk-codegen-test`,
and `codegen-server-test:python` subprojects (without making any changes
to the codegen code), both invocations would take the same time: Cargo
would recompile the crate and its dependencies, even when the generated
crate is identical. This is because Gradle deletes everything under
`buildDir` when running the `generateSmithyBuild` task:

```
> Task :codegen-server-test:smithyBuildJar
Deleting stale output file: /local/home/davidpz/workplace/smithy-ws/src/SmithyRsSource/codegen-server-test/build/smithyprojections/codegen-server-test
```

So the files get recreated each time.

While developing, it is likely that only a small number of the generated
crate files are modified across rebuilds. [Cargo uses
`mtime`](https://github.com/rust-lang/cargo/issues/6529) (among other
factors) to determine whether it needs to recompile a unit. Indeed,
running with `CARGO_LOG=cargo::core::compiler::fingerprint=trace` yields
`err: current filesystem status shows we're outdated`. This commit adds
a Gradle task that compares the hashes of the newly generated files with
the (previously cached) old ones, and restores their `mtime`s if the
hashes coincide.

Another issue was causing unnecessary crate rebuilds. Prior to this
commit, we were sending `RUSTFLAGS=-D warnings` when invoking Cargo.
However, a common thing to do after generating a crate is to open its
contents in an editor. The editor's `rust-analyzer` would compile the
crate and its dependencies without the `RUSTFLAGS` we had used earlier.
The next time you rebuilt the crate, Cargo would claim `err: RUSTFLAGS
has changed: previously [], now ["-D", "warnings"]` and recompile
everything again. This commit refactors the Gradle tasks so as to not
send these flags when invoking Cargo, instead generating a
`.cargo/config.toml` containing these flags. This way, `rust-analyzer`
also picks them up and does not need to recompile the crates.

With both patches, Cargo avoids unnecessary crate rebuilds. All in all,
the second invocation of a `./gradlew --info -P modules='simple' -P
cargoCommands='test' codegen-server-test:build` command now takes 15
seconds less than the first invocation on my `c5.4xlarge` machine; Cargo
does not need to do _any_ work on the second invocation.

This commit also refactors the `build.gradle.kts` files of the `sdk`,
`sdk-codegen-test`, `codegen-test`, `codegen-server-test`, and
`codegen-server-test:python` subprojects to make them DRYer and more
consistent. The last 4 subprojects' buildscripts are now much shorter,
with all the common logic having been moved to `CodegenTestCommon.kt`.
Note that we have made the last 4 subprojects' `cargo check` and `cargo
doc` invocations use the same set of flags than in the `sdk` subproject
for consistency.

Closes #1378.
Closes #1412.
2022-06-15 22:06:25 +00:00
John DiSanti 26316db7b2
Implement feature-gated independent SDK crate versioning (#1435)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-06-14 01:19:40 +00:00
John DiSanti a0539e20b0
Fix `SPDX-License-Identifier` in header comments (#1377)
* Fix misc whitespace with pre-commit

* Revise copyright check in `sdk-lints`

* Fix `SPDX-License-Identifier` in header comments
2022-05-09 09:50:46 -05:00
david-perez c9e6ab9a5f
Upgrade Kotlin, Kotest, `ktlint` and Jacoco (#1331)
This commit upgrades:

* Kotlin to 1.6.20,
* Kotest to 5.2.3,
* Ktlint to 0.45.2; and
* Jacoco to 0.8.8.

Perhaps the only noteworthy change is that Ktlint now has a lint to
detect that the name of a class/object defined solely in a file has to
coincide with the filename.
2022-04-22 21:00:00 +00:00
John DiSanti 27acbc6d30
Upgrade Smithy (#1330)
* Upgrade to Smithy 1.16.2

* Remove extra test fixed in Smithy 1.16.2

* Upgrade to Smithy 1.16.3

* Remove extra test fixed in Smithy 1.16.3

* Remove failing expectation from server tests

* Upgrade to Smithy 1.17.0

* Upgrade to Smithy 1.18.1

* Upgrade to Smithy 1.19.0

* Upgrade to Smithy 1.21.0

* Update changelog

* Fix Gradle repositories
2022-04-21 09:22:35 -04:00
david-perez a870d2ad05
Run `ktlint` from repository root (#1327)
This is the result of running `ktlint` from the repository root. We have
`ktlint` running on pre-commit and CI on changed files, but these
violations slipped through the cracks.
2022-04-19 17:56:00 +02:00
John DiSanti 7245df1c1c
Generate a `versions.toml` with the SDK (#1311)
* Implement `crate-hasher` utility to hash crate source files
* Add `generate-version-manifest` subcommand to publisher tool
* Incorporate `generate-version-manifest` into the build system
* Start making the PR bot run through the Docker build image
* Use nightly clippy in tools CI when necessary
* Fix transient failure when acquiring the build image
* Include examples repo revision in `versions.toml`
* Fix nightly clippy lint in `cargo-api-linter`
* Track examples revision in sync tool
* Make it easier to run Docker CI locally
2022-04-12 16:55:32 -07:00
John DiSanti 6fa7b36812
Use Docker build image in CI (#1255)
Establishes a Docker build image with build tools precompiled into it to improve CI time and reduce GitHub Actions workflow configuration. Also makes it possible to run the exact CI workflows in local development.
2022-03-28 11:05:08 -07:00
david-perez c3ef017d41
Allow for specified Cargo commands to be run on a subset of the integration tests (#1165)
This commit allows for the `modules` and `cargoCommands` properties to
be specified when running the `codegen-test` and `codegen-server-test`
Gradle modules. The `modules` property allows one to only generate a
subset of the integration test services, while the `cargoCommands`
property allows one to specify the Cargo commands to be run on the
generated Rust crates.

This functionality can be useful to reduce development iteration cycles. For
instance, to only run `cargo test` on the integration test
`simple.smithy`:

```sh
./gradlew codegen-test:build -P cargoCommands='test' -P modules='simple'
```

This commit also refactors the buildscripts of the `codegen-test` and
the `codegen-server-test` modules, extracting shared functionality to
`buildSrc/src/main/kotlin/CodegenTestCommon.kt`.
2022-02-18 14:14:36 +01:00
Russell Cohen a05e968038
Fix bug where index.md is sometimes deleted (#1056)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-01-10 15:48:38 -05:00
John DiSanti 17818ca7e5
Produce two publishable bundles in CI (#986)
* Move the publisher tool from `aws-sdk-rust`

* Add `rust-runtime:assemble` target to generate a publishable bundle

* Run `fix-manifests` on assemble output

* Produce publish-ready Smithy runtime bundle during CI

* Allow publish from any arbitrary directory

* Add safe-guard to prevent accidental publish from local dev

* Fix unit test target

* Incorporate feedback

* Add `buildSrc` tests and publisher tool to CI
2021-12-16 13:27:21 -08:00
Russell Cohen 2e7ed94351
Fix numerous todos & attach them to issues (#973)
* Fix numerous todos & attach them to issues

* Update changelog

* remove unused import in sigv4
2021-12-16 16:27:47 +00:00
Russell Cohen c8e9f19d0c
Add link to developer guide (#969)
* Add link to developer guide

* Update buildSrc/src/main/kotlin/DocsLandingPage.kt

Co-authored-by: John DiSanti <jdisanti@amazon.com>

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2021-12-13 21:23:30 +00:00
Russell Cohen b334ce2b51
Improve generated docs for crates and clients (#898)
* Improve generated docs for crates and clients

This commit makes a first pass at improving the documentation rendered for AWS crates for both generated crates as well as `Client` structs.

* Fixups

* CR feedback
2021-11-26 21:28:58 +00:00
Russell Cohen 476e988d43
Rev2 of Github Pages Landing Page (#859)
* Rev2 of Github Pages Landing Page

- Generate `index.md` instead so it works directly with Jekyll
- Fix some bugs in table generation and generate a narrower table that doesn't need side-to-side scrolling

* Fix trailing whitespace in generated markdown

* Cleanup generated markdown

* fix aws-config link

Co-authored-by: Zelda Hessler <zelda.hessler@pm.me>
2021-11-15 15:49:10 +00:00
Russell Cohen ada03d4ca1
Generate a Docs Landing Page (#853)
* Generate a Docs Landing Page

This commit refactors a portion of the build into `buildSrc` and adds a job to generate a docs landing page. The generated SDK now contains a `docs.md` that contains top level links to all services.

* Add copyright headers

* Order by human name
2021-11-12 17:47:48 +00:00