Commit Graph

370 Commits

Author SHA1 Message Date
Russell Cohen 3d408352b3
Use rustls for reqwest (#2403)
* Use rustls for reqwest

This also adds a CI check to ensure that we can build the SDK on MUSL

* Fix MUSL build in image

* Fix build image
2023-02-23 19:56:24 +00:00
Russell Cohen ba3b937586
Add presigned URL canary & ep2 test (#2379)
* Add presigned URL canary & ep2 test

* fix canary tests
2023-02-16 19:11:36 +00:00
John DiSanti f7ac844bc5
Add generic client codegen diff to PR bot (#2383) 2023-02-16 15:52:40 +00:00
Luca Palmieri 10bb26c852
Add tests to make sure the publisher works as expected. (#2365)
* Add tests to make sure the publisher works as expected.

* Fix lint.
2023-02-14 15:36:14 +00:00
Luca Palmieri ca3ef20ccc
Fix version regex. (#2364) 2023-02-14 13:27:37 +00:00
Zelda Hessler aa07854f40
Fix S3 canary match statement (#2358)
* fix: s3 canary error match

* update: use Error.is_ to check for "no such key" error in canary

* remove: leftover import
2023-02-13 23:32:59 +00:00
John DiSanti d48878ee01
Implement request ID access for SDK clients RFC (#2129)
* Add `RequestId` trait
* Implement `RequestId` for generated AWS client errors
* Move `RustWriter.implBlock` out of `StructureGenerator`
* Create structure/builder customization hooks
* Customize `_request_id` into AWS outputs
* Set request ID on outputs
* Refactor SDK service decorators
* Refactor S3's extended request ID implementation
* Combine `Error` and `ErrorKind`
* Add test for service error conversion
* Move error generators into `codegen-client` and fix tests
* Re-export `ErrorMetadata`
* Add request IDs to trace logs
* Simplify some error trait handling
* Rename `ClientContextParamDecorator` to `ClientContextConfigCustomization`
* Add deprecated alias to guide customers through upgrading
* Rename the `ErrorMetadata` trait to `ProvideErrorMetadata`
* Rename `aws_smithy_types::Error` to `ErrorMetadata`
2023-02-10 17:36:48 -08:00
Nugine de186678ee
Bump Rust MSRV to 1.63.0 (#2222)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2023-02-10 12:59:31 -08:00
John DiSanti 13b10d51a8
Add RFC for improving request ID access in SDK clients (#2083) 2023-02-10 19:07:56 +00:00
Burak e84ef6c7e2
Python: Type-stub generation for SSDKs (#2149)
* Initial Python stub generation

* Handle default values correctly

* Only generate `__init__` for classes that have constructor signatures

* Preserve doc comments

* Make context class generic

* Put type hint into a string to fix runtime error

* Run `mypy` on CI

* Use `make` to build Python SSDKs while generating diffs

* Escape Python types in Rust comments

* Only mark class methods with

* Sort imports to minimize diffs

* Add type annotations for `PySocket`

* Dont extend classes from `object` as every class already implicitly extended from `object`

* Use `vars` instead of `inspect.getmembers` to skip inherited members of a class

* Fix linting issues

* Add some tests for stubgen and refactor it

* Add type annotations to `PyMiddlewareException`

* Fix tests on Python 3.7

Python 3.7 doesn't support reading signatures from `__text_signature__`
for non-builtin functions (i.e. C/Rust functions). For testing we're using
regular Python syntax for defining signature.

* Provide default values for `typing.Optional[T]` types in type-stubs

* Update `is_fn_like` to cover more cases

* Remove `tools/smithy-rs-tool-common/`

* Make `DECORATORS` an array instead of a list

* Ignore missing type stub errors for `aiohttp`
2023-02-10 14:00:53 +00:00
Zelda Hessler bae93805de
Update pretty_assertions (#2332)
* update: pretty_assertions

* update: just use v1.3
2023-02-09 11:02:19 -06:00
Luca Palmieri bf678fd7d2
[Release workflow] Workaround for the ownership bug (#2318)
* First publish everything, then try to correct ownership.

* Avoid trying to remove teams, since we know it won't succeed.

* Fix warning and update dependencies.

* Avoid allocation unless necessary.

* Get full error representation for retryable failures.

* We want to hash the file content, not the file path.
2023-02-09 13:16:41 +00:00
Luca Palmieri b9b80c3014
Retrieve the output from outside the Docker context (#2300)
Co-authored-by: AWS SDK Rust Bot <97246200+aws-sdk-rust-ci@users.noreply.github.com>
2023-02-03 13:58:40 +00:00
Luca Palmieri 47f6bd8613
Clone does not preserve uncommitted changes. This was leading to the gradle.properties update being lost. (#2299) 2023-02-03 13:02:24 +00:00
Luca Palmieri e1991fa7c7
[release-branches] Fix working directory (#2298)
* Fix working directory.

* Build the Docker image using the latest commit on the invocation branch.

* We need a Docker image with the same SHA later in the process.
2023-02-03 12:26:20 +00:00
Luca Palmieri 85d8819e85
Collect more diagnostics. (#2297) 2023-02-03 11:33:34 +00:00
Luca Palmieri 3ccb2d7910
Set user name and email when committing (#2290) 2023-02-02 17:20:18 +00:00
Luca Palmieri d9efa6bdd7
Commit if modified (#2289)
* Fix paths.

* Fix commit logic - it should commit when there were changes.
2023-02-02 16:34:32 +00:00
Luca Palmieri 27429c8b79
Fix paths. (#2288) 2023-02-02 15:53:19 +00:00
Luca Palmieri bdabbf0835
Persist the modified gradle.properties outside of the Docker context (#2287)
* Fix if condition.

* Make sure that the changes are visible to the push step after the Docker action has executed by persisting the modified repository as an artifact.

* Give a name to the argument.
2023-02-02 14:42:49 +00:00
Luca Palmieri 20c4839cf4
We need to checkout in the `smithy-rs` folder because that's an assumption baked into the definition of the docker-build action. (#2284) 2023-02-02 12:43:26 +00:00
John DiSanti a6ef0788ec
Remove teams from `publisher` ownership list (#2257) 2023-02-01 22:44:09 +00:00
John DiSanti bed7b97ff8
Reduce Docker image rebuilds (#2269)
* Move `acquire-build-image` into `.github`
* Move the `tools-hash` script into `.github`
* Upload to ECR from PRs as well
* Move build tools into `tools/ci-build/`
* Move CI scripts out of `ci-build`
* Split CI for forks/non-forks
* Remove `fetch-depth: 0` from PR workflows
2023-02-01 18:41:31 +00:00
Luca Palmieri d9c0b2e636
Change the release flow to use release branches (#2253)
* What happens if we comment out the runtime crate version from gradle.properties?

* Allow running the release and the CI workflows from an arbitrary commit.

* Does a fake version work?

* Pass `git_ref` from the release workflow.

* It needs to be a valid semver version.

* Sketch new command to upgrade version in gradle.properties

* Command implementation

* Plug the new publisher command into the `release` action.

* Plumb end-to-end

* Fix copyright header.

* Fix lint.

* Temporarily comment out the sanity check.

* Ignore sanity check

* Add a command that prints out the template for CHANGELOG.next.toml

* Add branch check + empty TOML generation.

* Add copyright headers.

* Fix imports.

* Remove sanity check.

* Move script to a file.

* Add a check to validate the tag.

* Remove second build step.

* Move to .github/scripts folder.

* Make the script easier to run locally

* Fail if anything fails.

* Add comment.

* Update .github/scripts/get-or-create-release-branch.sh

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/scripts/get-or-create-release-branch.sh

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/scripts/get-or-create-release-branch.sh

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/workflows/ci.yml

Co-authored-by: david-perez <d@vidp.dev>

* Remove touch.

* Fix indentation and branch name.

* Update .github/workflows/ci.yml

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/workflows/release.yml

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/workflows/release.yml

Co-authored-by: david-perez <d@vidp.dev>

* Explicit flags.

* Use the path that was provided.

* Format

---------

Co-authored-by: david-perez <d@vidp.dev>
2023-02-01 17:10:18 +00:00
John DiSanti c04c6ab4d7
Add `Data limit exceeded` to build image throttle messages (#2260) 2023-01-30 20:12:29 +00:00
John DiSanti 7ad342aabe
Copy non-service integration tests into SDK root tests directory (#2255)
* Copy non-service integration tests into SDK root tests directory

For tests in `aws/sdk/integration-tests` that are not named after a
service module, copy them into the SDK's root `tests/` directory so that
they are run as part of CI on the entire SDK.

* Add missing fields to integration test manifests
* Remove tests from root workspace
* Explicitly exclude the root tests from the root workspace
2023-01-27 21:13:25 +00:00
John DiSanti e6c3a4b98a
Replace PR bot's diff tool with one that supports pagination (#2245)
* Implement paginated diff to html tool
* Wire up the new diff tool to PR bot
2023-01-27 18:23:39 +00:00
Russell Cohen 41ca31b85b
Add CI action to test aws-sdk-services (#2251)
* run cargo test --all-features instead

* Add check-only option, change check to test

* Support nested APIs in operation input tests
2023-01-25 15:55:05 -05:00
Zelda Hessler 582ae85532
update: aws-config README (#2221)
* update: aws-config README
* update: README linter to allow aws-config readme changes
2023-01-18 17:23:38 +00:00
John DiSanti d56fd27614
Remove `sdk-sync` (#2219) 2023-01-17 20:04:22 +00:00
John DiSanti d2b68191fc
Slow down crate publish more (#2214) 2023-01-17 18:49:46 +00:00
Russell Cohen 84b810ca4a
pin ignore to older version with MSRV (#2173)
* pin ignore to older version with MSRV

* Pin `ignore` to `0.4.18` (for real)

Co-authored-by: Julian Antonielli <julianantonielli@gmail.com>
2023-01-06 11:47:48 +00:00
Russell Cohen 7a6ddfbee3
get info logs for the canary (#2167) 2023-01-04 22:26:07 +00:00
Russell Cohen a84e61af63
Add instructions for running the canary locally & cleanup canary code (#2072)
* Add instructions for running the canary locally & cleanup the canary code

* Fix ec2 canary
2022-12-30 19:05:41 +00:00
Luca Palmieri 81028c658b
Add required fields to the dummy package Cargo.toml (#2148) 2022-12-29 16:37:43 +00:00
Luca Palmieri c49d596163
Claim names of unpublished crates (#2137)
* Build a list of crates that have not been published yet.

* Add publishing step

* Refactor.

* Add CI job.

* Lints

* Style.
2022-12-28 18:55:48 +00:00
Luca Palmieri 6dbefd2db9
Perform some optimisations on the CI Docker image (#2138)
* Perform some optimisations on the CI Docker image:
- Break down the `install_rust` stage into smaller stages that can advance concurrently;
- Invoke `yum updateinfo` once, in bare_base_image;
- Use the unstable sparse registry feature to speed up `cargo`'s registry operations.

* Fix ARG variable.

* Enable buildkit.
2022-12-28 17:12:07 +00:00
John DiSanti 3499c11b65
Fix Lambda bundle naming for the SDK canary (#2135)
* Add release tag and Rust version to canary bundle file name
  Update canary CDK dependencies
  Update Lambda name in logging permissions
2022-12-23 16:23:24 +00:00
Burak 2cc7c24be4
Python: Use Maturin to build SDKs (#2025)
* Remove `lib` prefix from generated module names

* Build Pokemon service with `Maturin`

* Fix Maturin build on CI

* Generate minimal `pyproject.toml` for generated SDKs to build from source using Maturin

* Fix `ktlint` issues

* Bring back type stubs for Pokemon service

* Update instructions for Lambda

* Make `build-wheel` and `build-wheel-release` to depend on `codegen`
2022-12-14 12:12:58 +00:00
david-perez 30c6828d9b
Run TODO lint against more files (#2078)
This change would have prevented me from committing what was reverted in
https://github.com/awslabs/smithy-rs/issues/2076
2022-12-13 11:22:12 +00:00
Zelda Hessler f3e05625e1
Fix clippy lints and run formatter (#2080)
* fix: clippy lints in integration tests
format: integration tests

* fix: clippy lints in runtime crates
format: runtime crates

* fix: clippy lints in tools
format: tools

* fix: undo clippy lint change that doesn't compile with tracing v0.1
2022-12-09 19:00:02 +00:00
John DiSanti e1de6fa85b
Upgrade cargo tools (#2032)
* Upgrade `cargo-check-external-types` to 0.1.6
* Upgrade `cargo-deny` to 0.13.5
* Upgrade `cargo-udeps` to 0.1.35
* Upgrade `cargo-hack` to 0.5.23
* Upgrade `cargo-minimal-versions` to 0.1.8
2022-11-29 19:45:40 +00:00
Zelda Hessler 9a4c1f304f
Fix Canary (#2016)
* add: "release-2022-10-26" to canary runner PINNED_SMITHY_RS_VERSIONS
* update: canary-runner README.md
2022-11-28 22:42:40 +00:00
Zelda Hessler 07967dadca
fix: missing Tokio feature in S3 integration test (#2017)
fix: remove extra "the" in CDK readme
2022-11-21 23:07:32 +00:00
Zelda Hessler d99a0dbbe0
fix: use new version of deprecated fn (#1994)
update: crate-hasher sha256 dep to 1.1
2022-11-17 18:02:31 +00:00
John DiSanti 9f0bc36fa7
Make `SdkError::into_service_error` infallible (#1974) 2022-11-16 02:17:44 +00:00
John DiSanti fba88df3ee
Revamp errors in `aws-smithy-http` (#1884) 2022-11-15 19:18:26 +00:00
Julian Antonielli 1844660c61
Remove our `base64` implementation in favour of `base64-simd` (#1938)
* Replace our base64 implementation with one based on `base64-simd`

* Bump Rust MSRV to 1.62.1

* Unmark base64-related tests as failing

* Use `usize` for `base64::encoded_length`

* Run Rust 1.62.1 on CI

* Don't use external `base64_simd::Error` type in public interface

* Add initial criterion bench setup for base64

* Compare `base64_simd::encode` vs Smithy's `encode`

* Use string length for `BenchmarkId`

* Compare `base64_simd::decode` vs Smithy's `decode`

* Compare `base64_simd::encoded_length` vs Smithy's `encoded_length`
2022-11-09 12:08:17 +00:00
John DiSanti 55a66a21ad
Fix canary Lambda invoke success check (#1960)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-11-08 16:35:31 +00:00
John DiSanti e9f876a166
Upgrade `cargo-check-external-types` to 0.1.5 (#1882) 2022-10-31 16:42:56 -07:00
John DiSanti b499d9345c
Slow down crate publish more (#1911)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-10-27 18:07:33 +00:00
John DiSanti ab703195bb
Fix case sensitivity bug in `changelogger` (#1866) 2022-10-18 19:39:12 +00:00
ysaito1001 e4e7ef2450
Add myself to the maintainers list (#1870)
Co-authored-by: Saito <awsaito@c889f3b5ddc4.ant.amazon.com>
2022-10-18 16:14:01 +00:00
John DiSanti aacaf160b4
Mark `release-2022-10-13` as yanked to fix SDK canary (#1865) 2022-10-18 09:11:23 -05:00
Luca Palmieri 18d969e160
Upgrade to Rust 1.62.0 (#1825)
* Run clippy --fix for Rust 1.62.0 and format the resulting code

* Run clippy --fix on all targets

* Run clippy --fix with all features enabled

* Avoid extra allocation

* Use more idiomatic assert

* Ignore noisy lint

* Update pyo3 and pyo3-asyncio to 0.17.0

* Implement Eq on aws-smithy-checksums::Error

* Implement Eq on Protocol

* Replace conditionals with range-containment

* Implement Eq on types in aws-smithy-types

* Implement Eq on types in aws-smithy-http-server-python

* Implement Eq on types in aws-smithy-eventstream

* Implement Eq on types in aws-smithy-xml

* Implement Eq on aws-sigv4

* Update CI to use Rust 1.62.0

* Add Eq for generated types that implement PartialEq

* Allow clippy::needless_return in generated code

* Remove unnecessary reborrow in http_serde

* Remove unnecessary borrow in operation_deser

* Add CHANGELOG entries

* Revert "Add Eq for generated types that implement PartialEq"

This reverts commit 5169bd95aa.

* Update pyo3 and pyo3-asyncio in generated code for python server

* Allow clippy::derive_partial_eq_without_eq on structs and builders

* Run clippy on tools

* Fix accidental move in generated code

* Revert "Allow clippy::derive_partial_eq_without_eq on structs and builders"

This reverts commit 068c63ca20.

* Fix another accidental move in generated code

* Undo unwanted change to model

* Re-add reborrow in HttpBindingGenerator

* Fix clippy::format-push-string in changelogger

* Fix more uses of str.push_str(&format!(...))

* Remove unnecessary parenthesis

* Run ktlint

* Update aws/rust-runtime/aws-http/src/content_encoding.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Update aws/rust-runtime/aws-http/src/content_encoding.rs

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* Ignore doctest for non-exported macro

Rust 1.62 introduced a breaking change where doctests for non-exported
macros are now run by default. These don't compile because the macro
can't be imported in it.
See https://github.com/rust-lang/rust/issues/97030 for more info.

* Run cargo fmt

* Use $crate instead of crate in macro

* Revert "Implement Eq on types in aws-smithy-types"

This reverts commit c45a6b5a56.

* Revert "Implement Eq on types in aws-smithy-eventstream"

This reverts commit 78f4b07344.

* Revert "Implement Eq on types in aws-smithy-xml"

This reverts commit 590f01af73.

* Revert "Implement Eq on aws-sigv4"

This reverts commit d78bb62124.

* Revert "Implement Eq on types in aws-smithy-http-server-python"

This reverts commit f2cd901884.

* Revert "Implement Eq on aws-smithy-checksums::Error"

This reverts commit 5da170405e.

Co-authored-by: Julian Antonielli <julianantonielli@gmail.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-10-17 09:32:12 +01:00
John DiSanti 046edd54ab
Remove `cargo-check-external-types` (#1820)
This tool now has its own repository:
https://github.com/awslabs/cargo-check-external-types
2022-10-11 19:46:13 +00:00
John DiSanti 38c32b792e
Update tool dependencies (#1808) 2022-10-04 11:09:19 -07:00
Matteo Bigoi 89a6e5894a
Add LukeMathWalker and jjant to the contributor list (#1791) 2022-09-30 17:40:31 +00:00
John DiSanti 49cc08da22
Move core tests from `codegen-client` into `codegen-core` (#1776)
* Move `testutil` into `core` where possible
* Move `rustlang` tests into `core`
* Move remaining `util` tests to `core`
* Move and refactor `TopLevelErrorGeneratorTest` to not require `RustCodegenPlugin`
* Move protocols into `core`
* Move `generator` tests into `core`
* Move protocol ser/de tests into `core`
* Suppress deprecation warning in `TopLevelErrorGeneratorTest`
* Move `InlineFunctionNamerTest` into `core`
* Move model transform tests into `core`
* Rename `SymbolBuilderTest` to `SymbolVisitorTest` to match class under test
* Move server classes out of client
* Move core codegen tests into their own CI step
2022-09-28 17:46:28 +00:00
Nicolas Farrier ff13b087fe
Run `cargo clippy` when building the Pokémon Service (#1742) 2022-09-21 11:26:02 +00:00
david-perez b4ac306fc2
Use Rust 2021 edition with `rustfmt` in CI (#1728)
We've long migrated to the 2021 edition everywhere. See PRs #1388, #1328,
and #1268.

See issue #1332 too.
2022-09-13 13:10:54 +02:00
John DiSanti 96e9f61fee
Establish the `codegen-core` module (#1697)
* Create the `codegen-core` module
* Move the `Version` class into `codegen-core`
* Rename `codegen` to `codegen-client`
* Rename `codegen-test` to `codegen-client-test`
* Move shared test models to common location
* Fix Smithy validation errors in `misc.smithy`
2022-09-07 09:31:46 -07:00
John DiSanti 40c0b04d23
Use the same smithy-rs revision for all commits in `sdk-sync` (#1694) 2022-09-02 01:45:49 +00:00
John DiSanti ce1ffa412b
Slow down crate publish (#1691)
Crate publish used to be slow due to the verification step that `cargo`
would run, which was basically a full compile of the crate prior to
uploading it to crates.io. Now that the `publisher` tool passes
`--no-verify` to `cargo`, it publishes fast enough to get throttled.

This change completely removes the publishing parallelism, adds a fixed
time between publishing each crate, and increases the publish retry
backoff time.
2022-09-01 22:49:10 +00:00
John DiSanti 600c26da76
Add license checks to CI (#1687) 2022-09-01 14:40:29 -07:00
John DiSanti 422f5777c2
Update maintainers list for the changelog (#1686) 2022-08-31 17:23:16 -07:00
John DiSanti 31c5a91a25
Upgrade min versions of tokio, tracing-subscriber, and criterion (#1662)
* Upgrade minimum Tokio version to 1.8.4

This addresses RUSTSEC-2021-0072 and RUSTSEC-2021-0124.
The SDK, runtime crates, and server are set to the minimum secure
version of Tokio to allow a larger range of versions to be used by
consumers of those libraries. The tools and the Python server are
both set to the latest Tokio version since they are not intended
to be consumed as a libraries.

* Upgrade `tracing-subscriber` to 0.3.15

* Upgrade `criterion` to 0.3.6

This addresses RUSTSEC-2021-0093 by upgrading `criterion`, which
upgrades `rayon`, which upgrades `crossbeam-deque`.
2022-08-31 21:19:04 +00:00
John DiSanti a0f67c375c
Retain old SDK changelog entries for multiple smithy-rs releases (#1661) 2022-08-31 19:33:51 +00:00
Zelda Hessler a65b1aaa8d
Update MSRV to 1.61.0 (#1672) 2022-08-25 15:43:35 -07:00
John DiSanti 1c02c08b16
Fix crash for certain re-exports in `cargo-check-external-types` (#1654)
The visitor was examining the inner contents of the entire re-exported
type, which, for a type that is re-exported within the same crate, is
the correct behavior. Its the wrong behavior when exporting types from
other crates, however, since it doesn't matter what is inside that
external type, and rustdoc doesn't include information about inner
pieces of the external type in the JSON output.
2022-08-23 17:04:31 +00:00
Matteo Bigoi 5455c4e9cc
Fix bug in coroutine scheduling. Add ByteStream implementation and simplify Python symbol visitor. (#1633)
* Fix bug in coroutine scheduling. Add ByteStream implementation and simplify Python symbol visitor.

* Bug in coroutine scheduling:

In the previous iteration, the runtime crate aws-smithy-http-server-python
was exposing the python application implementation as a struct, which
was wrapped by the codegenerated App to allow to dynamically building
the router.

This caused scheduling of coroutines (handlers prefixed with async def)
to block becuse we were passing the Python eventloop of the parent
process that was stored pre-fork().

This commit changes the runtime PyApp to become a trait, allowing us to
dynamically build the router post-fork() and with the right event loop.

This change also allowed us to remove a bunch of unnecessary Arc(s).

* Add ByteStream implementation

Implementation of a ByteStream type for Python that can be roughly used like this:

let b = await ByteStream.from_path("file.txt")
async for chunk in b:
    print(chunk)

Implement futures_core::stream::Stream for Python ByteStream wrapper.

The BytesStream implementation in python can now use a sync Mutex from
parking_lot because we are now using pyo3_asyncio::tokio::local_future_into_py()
to read a chunk, which supports !Send futures.

This allows to simply forward the implementation of Stream (poll_next()
and size_hint()) directly to our inner SDK ByteStream.

* Simplify Python symbol visitor

Inherit from Symbol visitor and override just what is needed to swap Python complex
types.

Signed-off-by: Bigo <1781140+crisidev@users.noreply.github.com>
2022-08-19 19:09:10 +01:00
John DiSanti 0d1dc51b0f
Implement more checks in `cargo-check-external-types` (#1645)
Adds support for unions, and marks unstable Rust language features as unsupported.
2022-08-18 21:41:35 +00:00
Zelda Hessler 02296d85a5
update: canary-runner deps (#1639)
* update: canary-runner deps
* rename: use Args instead of Opt to match other CLI tools
2022-08-18 18:39:33 +00:00
Taiki Endo a2f17cff31
Support raw pointer in cargo-check-external-types (#1643) 2022-08-17 20:40:54 +00:00
John DiSanti e179bea625
Don't compile crates a second time during `cargo publish` (#1644) 2022-08-17 19:28:55 +00:00
John DiSanti 275a6f8ad0
Fix git root check for submodules (#1640) 2022-08-17 10:31:34 -07:00
Burak 7731b3f1e0
Override codegen version commit hash in PR diffs (#1636)
* Override codegen version commit hash in PR diffs

* Move version commit hash override to shell script
2022-08-17 11:24:19 +01:00
John DiSanti 0f47f69ddf
Fix changelog template (#1638) 2022-08-16 18:59:10 +00:00
david-perez fd98756160
Add `awslabs/smithy-rs-server` as a crate owner in the `publisher` tool (#1619)
This commit adds the SDK and server groups to smithy-rs runtime crates,
but only the SDK group on `AwsRuntime` and `AwsSdk` crates.

With this change, the `publisher` tool won't fail when publishing the
`aws-smithy-http-server` crate.
2022-08-15 16:56:57 -07:00
John DiSanti 86debc676d
Improve `cargo-check-external-types` README (#1630) 2022-08-11 20:11:04 +00:00
John DiSanti 18c5062cc6
Break `cargo-check-external-types` dependency on `smithy-rs-tool-common` (#1629) 2022-08-11 09:12:19 -07:00
John DiSanti cbbbec4a48
Check external types for runtime and SDK crates in CI (#1625)
* Rename `cargo-api-linter` to `cargo-check-external-types`
* Run `cargo-check-external-types` on Smithy client/shared runtime crates
* Run `cargo-check-external-types` on SDK runtime crates
* Fix bug in determining crate name
* Run `cargo-check-external-types` on generated SDK crates
2022-08-10 18:33:43 -07:00
John DiSanti 6e1d1f6c18
Fix adhoc SDK codegen and refactor SDK CI (#1624)
* Fix `aws:sdk-codegen-test` build
* Add config for SDK endpoint/readme generation
* Rename `sdk-codegen-test` to `sdk-adhoc-test` and update README
* Add `sdk-adhoc-tests` to CI
* Refactor CI scripts

The `check-rust-runtime-and-tools` script was getting beyond 20
minutes to run, so this commit splits it into `check-aws-config`,
`check-rust-runtimes`, and `check-tools`.

This also enables most of the checks to run before the smoke test SDK is
generated, which should deliver more immediate feedback on PRs.

* Give the canary its own CI step
2022-08-10 09:08:08 -07:00
John DiSanti 462aef6394
Fix SDK integration tests in local dev and add CI step to verify (#1600) 2022-08-08 16:29:22 -07:00
John DiSanti d9d49a1b2a
Add `changelogger` tool to CI checks (#1615) 2022-08-08 20:30:34 +00:00
John DiSanti 5d69e73092
Upgrade `cargo-api-linter` to the latest nightly Rust (#1579)
* Upgrade `cargo-api-linter` to the latest Rust nightly
* Upgrade Rust nightly used by CI
* Handle failure to provide error context better
* Fix `aws-config` external type approval list
* Fix error context pathing issue
2022-08-03 20:12:31 +00:00
John DiSanti 5fb56c8446
Set minimum `hyper` version to `0.14.12` (#1602) 2022-07-29 09:01:06 -05:00
John DiSanti 6e031dcd4f
Fix the SDK canary (#1588)
* Update `generate-matrix` to return SDK release tags instead of versions
* Upgrade the SDK used in the canary runner
* Move `build-bundle` into `canary-runner` to use `versions.toml`
* Make release tag matrix output singular
* Delete old `build-bundle` and update README
* Add canary projects to standard tool CI checks
* Generate `Cargo.toml` before testing `canary-lambda`
2022-07-28 21:52:07 +00:00
John DiSanti 2670831154
Fix the check manifests tool (#1576) 2022-07-26 19:06:08 +00:00
John DiSanti 24f0749e93
Add support for removing crates from the SDK (#1567) 2022-07-25 21:20:17 +00:00
John DiSanti 9f446647ae
Clean up independent crate versioning migration (#1566) 2022-07-25 19:34:07 +00:00
82marbag 5ee95433d8
Update JAVA_HOME (#1564)
Recently CI is failing with

> ERROR: JAVA_HOME is set to an invalid directory: /usr/lib/jvm/jre-11-openjdk
> Please set the JAVA_HOME variable in your environment to match the
> location of your Java installation.

Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
2022-07-21 17:38:01 +00:00
John DiSanti 6143d90197
Enable independent crate versioning for SDK crates (#1540)
* Update `hydrate-readmes` to take versions from `versions.toml`
* Add cleanup TODO comments to `changelogger`
* Update SDK readme template
* Split up `generate-version-manifest` subcommand
* Eliminate the `aws.sdk.version` property
* Fallback to model hash comparison if model metadata claims no changes
* Add `acquire-base-image` to `release.yml`
* Use empty model metadata for SDK generation in CI
* Fix the `aws-config` version number in SDK crate readmes
2022-07-19 12:27:45 -07:00
Fahad Zubair 9ede23b123
Server SDK CHANGELOG.md file is created separately by the changelogger tool (#1513)
The affected target of a change (client, server, all) is shown in CHANGELOG.md

The meta tag for a change log entry in CHANGELOG.next.toml supports adding the affected target, as an example:
[smithy-rs]
...
meta = { "breaking" = false, "tada" = false, "bug" = false, "target" = "server" }
...

Possible values for target: [client, server, all]
2022-07-19 11:41:52 +01:00
John DiSanti 090b57d158
Clean up after release decoupling (#1518)
* Retain only the smoke test AWS models
* Address release decoupling TODO comments
2022-07-05 18:02:46 +00:00
John DiSanti 46984ab91d
Fix Docker build-image `cargo-udeps` build (#1519) 2022-06-30 12:49:23 -07:00
Matteo Bigoi 5312ff1132
Export a usable Python implementation via shared library (#1476)
This PR add several functionalities to the Python/Rust code-generation, making it usable for models that are not using Streaming and Union shapes.

    * Export codegenerated shapes as shared library that can be imported by the Python interpreter.
    * Add complete end-to-end testing running in CI.
    * Register signals and handle workers processes lifecycle.
    * Use a custom SymbolProvider to override non-primitive types that are exposed to Python.
    * Implement support for DateTime.
    * Cast Python exception into Rust error types.
    * End to end testing suite with a Rust client calling a real Python server.
2022-06-30 16:11:58 +00:00
John DiSanti dc8f1743b1
Retry Gradle daemon startup failures in `sdk-sync` (#1504) 2022-06-28 08:32:34 -07:00
John DiSanti 52c1174ac3
Decouple `aws-sdk-rust` release from `smithy-rs` release (#1495)
* Remove smoketest/fullsdk model classifications
* Generate full SDK for CI using models in aws-sdk-rust
* Move the changelog renderer out of `sdk-lints` and add changelog splitter
* Save revision information into split off SDK changelog
* Replace common git impl with `sdk-sync`'s impl
* Filter SDK changelog entries on changelog render
* Add smithy-rs release workflow
* Add initial next SDK changelog
* Update `sdk-sync` to understand new models location
2022-06-27 19:16:48 +00:00
John DiSanti 6bc8154fae
Change `sdk-sync` failure to warning to help release decoupling (#1494) 2022-06-23 10:44:08 -07:00
John DiSanti c3f37303fa
Remove deprecated v1 arg format from `sdk-versioner` (#1456) 2022-06-14 18:52:27 +00:00
John DiSanti 4a8da5f7a6
Fix issues with setting crate ownership during publish (#1454) 2022-06-14 17:42:38 +00:00
John DiSanti 66ff2bd0e7
Pin the `clap` version used to prevent future deprecation warnings (#1464)
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
2022-06-14 09:29:35 -05:00
John DiSanti 26316db7b2
Implement feature-gated independent SDK crate versioning (#1435)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-06-14 01:19:40 +00:00
John DiSanti 9156aca9fd
Add observability and more configuration to `sdk-sync` (#1452)
* Make `sdk-sync` Gradle heap/metaspace constraints configurable
* Dump process info on `sdk-sync` codegen failure
* Periodically log progress information in `sdk-sync`
* Enable verbose GC for codegen and use serial GC
2022-06-13 22:01:28 +00:00
John DiSanti 2986c93b0d
Configure Gradle parallelism and Kotlin compiler in `sdk-sync` (#1448) 2022-06-08 20:40:55 +00:00
John DiSanti f1a4782614
Reintroduce `sdk-sync` parallelism by disabling Smithy parallelism (#1443)
* Revert "remove parallelism from the sync tool (#1436)"

This reverts commit f0568e1f18.

* Reintroduce `sdk-sync` parallelism by disabling Smithy parallelism

* Make the Smithy parallelism configurable
2022-06-06 19:17:31 +00:00
Russell Cohen 242e6bcd67
make build image commit has configurable (#1437) 2022-06-02 19:20:31 +00:00
Russell Cohen f0568e1f18
remove parallelism from the sync tool (#1436)
smithy-build actually utilizes `ForkJoinPool.common()` when executing build projections. Because each service is its own projection, the build internally is _already_ running in parallel. Because of this, running the sync in parallel produces uneccessary memory pressure.

This change:
- removes parallelism
- runs gradle `--no-daemon --info` for better log output
2022-06-02 17:30:10 +00:00
Zelda Hessler b72c807f84
add: echo-server tool w/ README (#1432) 2022-06-02 11:25:11 -05:00
John DiSanti 3df09cc1d2
Fix release issues by introducing backwards compat into `sdk-versioner` (#1434)
The `sdk-sync` tool is running into issues during release due to the
`sdk-versioner` refactor commit since it is using a Docker build image
with the tools precompiled into it. Either the `sdk-versioner` binary is
too new and can't code generate the old commits, or it's too old and
can't generate the new commits.

This reintroduces the old `sdk-versioner` behavior so that it can
succeed against both sets of commits to get a release out.
2022-06-02 09:16:36 -04:00
John DiSanti f8867814ba
Refactor `sdk-versioner` to operate off the versions manifest (#1420) 2022-05-31 17:07:49 -07:00
John DiSanti a5790f7e5f
Implement release-aware yank tooling (#1407) 2022-05-26 17:28:36 -07:00
John DiSanti 7035f48d8e
Add flag to bypass confirmation prompt for crate publish (#1408) 2022-05-25 17:38:14 -07:00
John DiSanti f4a053bee0
Add release metadata to `versions.toml` (#1400) 2022-05-23 23:31:23 +00:00
Matteo Bigoi 5438979a2e
Add 82marbag to the list of maintainers (#1402) 2022-05-23 10:26:40 -05:00
Matteo Bigoi ebabb98745
Implement Python runtime crate and shared socket (#1399)
* Add the Python runtime crate `aws-smithy-http-server-python`
* Implement SharedSocket support.
* Implement logging to tracing support.
2022-05-20 17:08:57 +00:00
John DiSanti 2cd82cb862
Remove individual devs from crate ownership in crates.io (#1396) 2022-05-18 16:26:30 -07:00
Matteo Bigoi 5881feaf8f
[Server] Implement Gradle subprojects and scaffolding for the Python server. (#1366) 2022-05-18 14:26:29 +00:00
John DiSanti 6fe28ceeee
Remove `smithy-rs-sync` tool (#1393) 2022-05-17 08:53:57 -07:00
John DiSanti 4b3f9e06c4
Add retry to the `publish` tool (#1376) 2022-05-12 11:42:25 -07:00
John DiSanti d50093bb41
Add comment about `SMITHY_RS_DOCKER_BUILD_IMAGE` (#1383)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-05-12 08:01:06 -04:00
John DiSanti e1e9a29d5d
Fix some bugs in the `sdk-sync` tool (#1382) 2022-05-11 16:46:04 -07:00
John DiSanti 83af60855f
Add option to create a release manifest to changelog tool (#1384) 2022-05-11 15:20:27 -07:00
John DiSanti ad569b17d0
Add locale env vars to build image (#1380)
Co-authored-by: Russell Cohen <rcoh@amazon.com>
2022-05-10 15:14:49 +00:00
Russell Cohen c8fe6e2adf
Fix several minimal version bugs and add it to CI (#1365)
* Fix several minimal version bugs and add it to CI

* Fix docker image

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-05-09 17:14:14 +00:00
John DiSanti a0539e20b0
Fix `SPDX-License-Identifier` in header comments (#1377)
* Fix misc whitespace with pre-commit

* Revise copyright check in `sdk-lints`

* Fix `SPDX-License-Identifier` in header comments
2022-05-09 09:50:46 -05:00
John DiSanti ae9a5ed50a
Add support for automated model updates to the changelog tool (#1360) 2022-05-03 10:38:57 -07:00
John DiSanti 959d42931e
Refactor the `sdk-sync` tool to sync commits in parallel (#1357)
* Fix tests on MacOS

* Sync smithy-rs commits in parallel
2022-05-02 13:25:44 -07:00
Russell Cohen 9b6210c105
Release (#1351)
* Bump version numbers in gradle.properties

* Add auto-changelog script

* Update models, holding back S3

* Update CI to 1.58.1
2022-04-28 22:24:21 +00:00
John DiSanti 734acc175c
Update locked dependencies (#1349) 2022-04-27 15:24:30 -07:00
John DiSanti 0c011d635c
Fix local CI on M1 Macs (#1346)
The `acquire-build-image` script was coded up with the assumption that
it would only work with x86_64 build images. This commit revises that
script and the `Dockerfile` to correctly work on ARM64 architectures as
well.
2022-04-27 16:45:15 +00:00
John DiSanti 9699e04f22
Implement new `sdk-sync` tool (#1336)
* Copy `smithy-rs-sync` into new `sdk-sync`
* Refactor to use `Git` trait instead of `git2`
* Abstract out gradle commands
* Abstract out filesystem operations
* Add a mocked end to end test
* Load `versions.toml` for revision information
* Sync example commits into one aggregate commit
* Reorganize tests
* Only include smithy-rs commits that change the SDK
* Add `sdk-sync` to Docker build image
* Sync models
* Greatly increase debug logging
* Add git invocation tests
* Fix running in CI detection for Docker build image
* Rename `e2e_test` to `mock_e2e_test`
* Add full E2E test and fix a bug with change detection
2022-04-22 17:12:09 -07:00
John DiSanti 7675210fbc
Retry TLS handshake errors in `acquire-build-image` (#1334) 2022-04-21 13:07:13 -07:00
Zelda Hessler 52b808ec13
update: MSRV from 1.56.1 to 1.58.1 (#1325)
* update: MSRV from 1.56.1 to 1.58.1

* update: CHANGELOG.next.toml
fix: cdk rust issues

* Update CHANGELOG.next.toml

* Update CHANGELOG.next.toml
2022-04-15 14:01:18 -05:00
John DiSanti e9b00adca3
Fix tools dockerignore file (#1319) 2022-04-13 16:46:46 -04:00
John DiSanti 3a8b9f054e
Split `ci.yml` to avoid max workflow depth error (#1316) 2022-04-13 10:50:18 -07:00
John DiSanti 7245df1c1c
Generate a `versions.toml` with the SDK (#1311)
* Implement `crate-hasher` utility to hash crate source files
* Add `generate-version-manifest` subcommand to publisher tool
* Incorporate `generate-version-manifest` into the build system
* Start making the PR bot run through the Docker build image
* Use nightly clippy in tools CI when necessary
* Fix transient failure when acquiring the build image
* Include examples repo revision in `versions.toml`
* Fix nightly clippy lint in `cargo-api-linter`
* Track examples revision in sync tool
* Make it easier to run Docker CI locally
2022-04-12 16:55:32 -07:00
david-perez 3c25664762
Migrate from `structopt` to `clap` v3 (#1293)
`clap` v3 added support for a `structopt`-like parser derive usage.
`structopt` is now in maintenance mode
(https://docs.rs/structopt/latest/structopt/#maintenance); it is
recommended to migrate to `clap`.

Structs to parse CLI arguments have been renamed to `Args` for
consistency.
2022-04-11 12:34:32 -07:00
John DiSanti fc6af2071f
Combine PR bot messages into one (#1299) 2022-04-07 09:53:18 -07:00
John DiSanti 064467c99d
Enable deploying a minimal set of resources for running the canary (#1306) 2022-04-06 13:07:45 -07:00
John DiSanti 102f30421e
Add more debug info to `acquire-build-image` (#1305) 2022-04-06 09:21:02 -07:00
John DiSanti 6e63635e97
Enable running the canary with a path to a generated SDK (#1303) 2022-04-05 16:29:32 -07:00
John DiSanti e78d40f0d5
Add throttling retry to Docker build image pull (#1297) 2022-04-04 17:13:23 -07:00
david-perez 70de093507
Enable "TODOs" lint for server code (#1292)
Stale TODOs have been removed and new tracking issues have been cut to
provide context to the others.
2022-04-04 20:21:54 +02:00
John DiSanti 775d227c5f
Fix GitHub Actions workflow reference in `ci-main` (#1286) 2022-04-01 09:23:50 -07:00
John DiSanti 3e46543139
Simplify CI with composite actions and new image tagging approach (#1283)
- Refactors CI to use composite actions so that there's less duplication in the workflow yaml files.
- Switches to a different image tagging approach for the Docker build image: now the git hash of the `tools/` directory is the image tag rather than the commit hash of `HEAD`. This will result in more image reuse if the tools haven't changed, and makes the logic for deciding whether a new image must be built much simpler.
- When running CI on main, ensures the Docker build image is built and uploaded to ECR before running the CI tests.
2022-03-31 17:40:35 -07:00
John DiSanti 6fb5acc9bb
Update node dependencies (#1282) 2022-03-31 17:24:35 +00:00
John DiSanti 63b1de1a93
Update pinned nightly to fix `cargo udeps` checks (#1285)
* Update pinned nightly to fix `cargo udeps` checks

* Update `api-linter` to work on `nightly-2022-03-29`

* Make it possible to test tool modifications in a pull request
2022-03-31 09:30:04 -07:00
John DiSanti 7a5d179220
Fix path on git command (#1281) 2022-03-28 13:48:29 -07:00
John DiSanti 58fd455af3
Fix image acquisition when CI is run outside of a pull request (#1280) 2022-03-28 12:36:06 -07:00
John DiSanti 6fa7b36812
Use Docker build image in CI (#1255)
Establishes a Docker build image with build tools precompiled into it to improve CI time and reduce GitHub Actions workflow configuration. Also makes it possible to run the exact CI workflows in local development.
2022-03-28 11:05:08 -07:00
Zelda Hessler b2c989630f
Update to Rust Edition 2021 (#1268)
* update: set rust edition to 2021
update: strict param of intoCrate to set clippy to set #![deny(clippy::all)]

* add: changelog entry
2022-03-22 21:29:04 +00:00
John DiSanti d823f61156
Update pinned nightly and rustdoc format version (#1236) 2022-03-04 11:28:31 -08:00
John DiSanti b989a8d059
Fix presigning bug with `content-length` and `content-type` in S3 (#1216)
* Fix presigning bug with `content-length` and `content-type` in S3

* Fix presigning test in `aws-sigv4`

* Update changelog

* Clean when generating code for diff preview

* Incorporate feedback

* Fix server codegen

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-02-24 11:30:45 -06:00
John DiSanti 4f183f71fe
Pin nightly to `2022-02-22` to fix CI (#1221) 2022-02-24 09:33:15 -06:00
John DiSanti 375e0b2b82
Hide external buffer types used by `primitive::Encoder` in aws-smithy-types (#1209)
* Fix exit status in `api-linter`

* Run `api-linter` against `aws-smithy-types` in CI

* Hide external buffer types used by `primitive::Encoder`

* Unpin nightly in CI

* Make `Encoder` a struct and split out an inner enum

* Update changelog
2022-02-22 20:05:02 +00:00
John DiSanti 458b413be5
Upgrade CDK infrastructure to CDK 2 (#1214)
* Upgrade EKS integ test infrastructure to CDK 2

* Upgrade ci-cdk to CDK 2
2022-02-22 19:31:04 +00:00
John DiSanti 72de0c69d9
Implement tool to identify external types used in public APIs (#1172)
* Implement tool to identify external types used in public APIs

* Allow specifying which nightly to use

* Move `visitor::is_allowed_type` into `Config`

* Add doc comments

* Use `tracing_attributes::instrument`

* Rename `RefType` to `ErrorLocation`

* Add ability to output a Markdown table

* Allow customization of crate features in analysis

* Work around rustdoc ICE in `aws-smithy-json`

* Integrate with cargo

* Add a readme

* Fix some comments

* Rename `ContextStack` to `Path`

* Verify rustdoc output format version

* Improve error message
2022-02-18 15:17:20 -08:00
John DiSanti 329c036e6d
Make `sdk-versioner` work with example git dependencies (#1198) 2022-02-17 17:35:35 +00:00
John DiSanti a0019ba05a
Fix canary failure caused by the runner modifying Cargo.lock (#1185) 2022-02-15 14:58:08 -08:00
John DiSanti 03a2692fd2
Fix git repo root detection in tools (#1169) 2022-02-08 17:45:57 -08:00
John DiSanti 6b4e2baeb5
Fix the canary runner (#1155)
* Downgrade edition on `smithy-rs-tool-common` so it works with Rust 1.54

* Add more context to canary runner failures

* Fix path to canary-lambda

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-02-04 09:07:06 -08:00
John DiSanti 11c61f65c6
Fix SDK canary by pinning SDK versions to smithy-rs versions (#1145)
* Move canary-runner tool from aws-sdk-rust

* Commonize git shell operations

* Centralize package categorization logic

* Pin smithy-rs revisions to SDK versions for the canary

* Incorporate feedback
2022-02-02 16:53:02 -08:00
John DiSanti a183f1246b
Move re-exports into `types` and re-export `AggregatedBytes` (#1085)
* Move re-exports into `types` and re-export `AggregatedBytes`

* Remove extra newline from changelog generator

* Only re-export types when they're used

* Re-export `Config` and `Client` with `#[doc(inline)]`

* Update changelog

* Fix the canary build

* Don't check examples in full services check
2022-01-28 10:11:42 -08:00
John DiSanti 32923a3398
Include examples in SDK auto-sync (#1121) 2022-01-26 17:49:19 +00:00
John DiSanti 39e33163e6
Take SDK examples from `awsdocs/aws-doc-sdk-examples` (#1118)
* Split SDK service checks from example checks

* Delete examples

* Take examples from `aws-doc-sdk-examples`

* Fix example checkout

* Fix example manifests after relocation

* Update changelog
2022-01-25 22:24:13 +00:00
Russell Cohen a411bc3b34
Endpoint docs & aws-config spans (#1087)
* Add more details to custom endpoint docs

* Cleanup aws-config spans

* Cleanup aws-config spans

* Cleanups & update changelog
2022-01-25 21:16:42 +00:00
John DiSanti 1feb823157
Specify origin in rebase operation (#1117) 2022-01-25 19:03:58 +00:00
John DiSanti c273bb0206
Rebase `next` off `main` before syncing (#1115)
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-01-25 09:53:45 -08:00
John DiSanti 67ec04e460
Generate readme/changelog as build artifact and prevent example publish (#1109)
* Upgrade the publisher tool to use `clap` v3

* Instantiate the `aws-sdk-rust` README from a template during build

* Copy the SDK changelog into build artifacts at build time

* Disallow publish for example crates

* Only retrieve the current tag when needed

* Fix readme

* Rename inflate to hydrate
2022-01-24 15:12:35 -08:00
John DiSanti 79c1ec1361
Improve publisher tool (#1104)
* Display crate version numbers in publish summary

* Add additional validations to `fix-manifests` sub-command

* Update `yank` sub-command to take a repository location

* Incorporate feedback

* Fix `aws-smithy-http-server` version number

* Make tag check more functional
2022-01-21 22:21:16 +00:00
Russell Cohen 88aaa3308b
Sync script needs gradle clean (#1100)
* Sync script needs gradle clean

* Update changelogs
2022-01-20 12:30:47 -05:00
John DiSanti 3ad51ad6cf
Fix canary target path (#1091) 2022-01-19 23:21:39 +00:00
Russell Cohen 3e15e4a90c
Add SSO to the list of circular dep supressions (#1089)
* Add SSO to the list of circular dep supressions

* Update CHANGELOG.next.toml

Co-authored-by: John DiSanti <jdisanti@amazon.com>

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-01-19 22:08:53 +00:00
John DiSanti da214d905b
Add tool for bulk updating references to the AWS Rust SDK (#1080) 2022-01-19 18:25:40 +00:00
John DiSanti fd6788ff78
Simplify Canary Lambda CI (#1084)
* Simplify Canary Lambda CI

* Remove `Cargo.toml` check and make the script recursive
2022-01-19 17:42:23 +00:00
John DiSanti 7b061f40a6
Fix the canary Lambda build and add it to CI (#1079) 2022-01-18 13:16:28 -08:00
Russell Cohen 21ffa905db
Fix paginator bug where `None` was returned immediately (#1083)
* Fix paginator bug where `None` was returned immediately

The escape hatch added by aws-sdk-rust#391 did not properly handle the case where the first response was `None` and _not_ the empty string. This diff:
- Checks for emptiness for both maps and strings
- Fixes the check so that an initial `None` input does not cause an incorrect paginator error

* Update changelog

* Apply suggestions from code review

Co-authored-by: Zelda Hessler <zhessler@amazon.com>

* rustfmt

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-01-18 12:46:53 -05:00
Russell Cohen 232d09f4b2
Collect canary tasks so that they are run in parallel (#1076) 2022-01-14 14:11:18 -05:00
John DiSanti 88e7089388
Fix the PR bot (#1073) 2022-01-13 17:05:40 -08:00
John DiSanti 4bbc677c5a
Fix several warnings and reduce Rust target clobbering (#1069) 2022-01-13 16:45:04 -08:00
Russell Cohen 78cad9e371
Assorted canary Improvements (#1061)
* Assorted canary improvements

1. Add a timeout for individual canaries
2. Add an Ec2 canary.
3. Add functionality to enable/disable canaries based on SDK version

* Refactor page size
2022-01-11 20:05:35 +00:00
Russell Cohen db5aac4bcd
make references into links to make changelogs more navigable (#1059) 2022-01-11 14:43:08 -05:00
John DiSanti f1329022fb
Allow the canary runner to log metrics (#1035)
* Allow the canary runner to log metrics

* Add instructions on adding a new canary

* Add note about deployment to readme
2022-01-10 15:51:52 -08:00
Russell Cohen 96495b4c63
Fix bug in publisher topological sort (#1049)
* Fix bug in publisher topological sort

* Update tools/publisher/src/sort.rs

Co-authored-by: Zelda Hessler <zhessler@amazon.com>

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
2022-01-07 17:38:34 -05:00
Zelda Hessler 862d4cd0ff
remove: step to checkout branch to sync to (#1048)
* remove: step to checkout branch to sync to

* remove: step to checkout branch to sync to
2022-01-06 20:36:40 +00:00
Zelda Hessler 5a1990791d
update: insecure deps (#1039)
* update: insecure deps
add: extra context to timeout test failure

* Update aws/rust-runtime/aws-config/Cargo.toml

* update: tracing-subscriber dependents
remove: tracing-subscriber fmt feature (it's default)

* update: tracing-subscriber dependents
remove: tracing-subscriber fmt feature (it's default)

* update: hyper dependents to be less specific
2022-01-06 09:02:55 -06:00
Zelda Hessler 2f701abc59
Fix: strange commit behavior in sync tool (#1042)
* update: create mirror commit now works by shelling out
update: rm -rf instead of gradle clean
update: find_last_commit to work by shelling out

* add: apologetic comment regarding shelling out
2022-01-05 22:01:56 +00:00
Russell Cohen a77c5ca2a9
Add support for Paginators (#1006)
* Add Support for Paginators

This commit adds pagination support for both clients. It does this via the `FnStream` abstraction which combines a rendezvous-channel with a closure to produce a stream. When the stream is polled, the underlying closure is polled to advance the computation which then pushes data into the channel.

Two paginators are generated:
1. `.paginate()` which produces a stream of `Result<Page, Err>`
2. `.paginate().items()` which produces a stream of `Result<Item, Err>` where items are flattened from the individual pages. This uses `TryFlatMap` to support conveniently generating a flat page iterator.

In addition, docs are generated to point customers towards the paginators.

* Add RFC, more tests

* backout unrelated changes

* Fix paginators for glacier

* Fix clippy error

* Add test for paginators that point to maps & fix apigateway paginator

* Fix docs

* remove extraeneous println

* Fix DynamoDB examples

* Disallow paginating operations with the idempotency token trait

* Apply suggestions from code review

misc typos / cleanups

Co-authored-by: Zelda Hessler <zhessler@amazon.com>

* CR feedback

* Fix typo

* Update changelog

* Apply suggestions from code review

Co-authored-by: John DiSanti <jdisanti@amazon.com>

* CR feedback round 1

* Rename paginate to into_paginator()

* update pr bot comment

Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2022-01-05 17:46:57 +00:00
Zelda Hessler 34520b1b0a
fix: copy contents and not folder (#1033)
add: bot prefix to sync commits
2022-01-04 13:55:06 -05:00
Zelda Hessler a9f1b5aa26
remove: step that sets git hash incorrectly (#1032) 2022-01-04 17:38:24 +00:00
John DiSanti 588955440a
Fix bug in codegen diff that omitted new files (#1027)
Co-authored-by: Russell Cohen <rcoh@amazon.com>
2022-01-04 11:29:05 -05:00
Zelda Hessler 1b8707f662
Fix sync tool bugs (#1028)
* fix: filter out empty lines
update: test with newlines

* add: git repo check and warnings
update: println -> eprintln
feature: allow commit total to be limited
add: warning for leftover commits to sync
fix: create commit hash file if none exists
fix: get working dir in a github-actions-friendly way
update: add more context
fix: clippy lints

* update: simplify file writing
update: remove redundant lifetimes

Co-authored-by: Russell Cohen <rcoh@amazon.com>
2022-01-04 15:58:44 +00:00
Zelda Hessler f04a9e8946
Fix: Sync script handwritten files working directory bug (#1025)
* Fix bug where the sync script didn't work when the directories were different

* remove: leftover comment

* remove: no-longer-accurate comment

Co-authored-by: Russell Cohen <rcoh@amazon.com>
2022-01-03 16:03:49 +00:00
Zelda Hessler 6e31166c82
fix: don't panic on missing key in manifest (#1018) 2021-12-30 17:47:20 +00:00
Zelda Hessler 11de3f668d
update: canonicalize repo paths (#1016)
* update: canonicalize repo paths
update: fix path log text alignment

* add: more logging for HandwrittenFiles-related functions
add: warning for 'generated' handwritten dotfile

* fix: warning for 'generated' handwritten dotfile
2021-12-29 23:12:50 +00:00
Zelda Hessler 4fdb6bbe7a
fix: exit with error code on failure (#1014)
add: print directories at start
2021-12-29 20:15:53 +00:00
Zelda Hessler b828dc80fe
add: tool to sync aws-sdk-rust/next with smithy-rs (#1004)
* add: tool to sync aws-sdk-rust/next with smithy-rs

* fix: lint issues

* Update tools/smithy-rs-sync/src/main.rs

Co-authored-by: david-perez <d@vidp.dev>

* refactor: simplify commands by using a helper fn
update: use eprintln instead of println
update: note number of successfully mirrored commits in success message
update: use fs::read_to_string
update: use .display() instead of .to_string_lossy() for displaying paths
update: ensure absolute paths in fn copy_sdk
update: use iters instead of for loop in fn commits_to_be_applied

* add: fn for deleting files/ folders not in allowlist

* add: missing copyright header
remove: TODO

* refactor: handwritten to work with gitignore
add: check we're not generating files marked as handwritten
add: fn to list handwritten files
add: test for find_handwritten_files_and_folders
fix: clippy lint
add: timer for SDK cleanout
remove: once_cell dep
sort: deps in Cargo.toml

* fix: set smithyrs-githash per mirror commit
update: set bot as committer

* refactor: HandwrittenFiles to return iters

* add: Russell's error hint

Co-authored-by: david-perez <d@vidp.dev>
Co-authored-by: AWS SDK Rust Bot <aws-sdk-rust-primary@amazon.com>
2021-12-28 13:09:43 -06:00
Russell Cohen e559e4cf59
Add Route53 Hosted Zone Trimming (#999)
* Add Route53 Hosted Zone Trimming

Route53 requires a customization where `/hostedzone/` is stripped from hosted zone arguments when round-tripped. This adds a trait and customization to the route53 model to trim hosted zone prefixes.

* Add missing copyright header

* Also allow no leading slash

* Add route53 to smoke test because it has unit tests

* Update changelog

* Add missing call to flush()

* Remove unused env_logger depedendency
2021-12-21 19:19:32 +00:00