Commit Graph

404 Commits

Author SHA1 Message Date
Landon James 21b3995a2f
Update canary build tool (#3701)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->

Our WASI canary was breaking with the newest Rust version due to the new
WASI build targets:
https://blog.rust-lang.org/2024/04/09/updates-to-rusts-wasi-targets.html

## Description
<!--- Describe your changes in detail -->
Updating the version of `cargo-component` used to build our WASI canary.
Made the same update in the `aws-sdk-rust` repo:
https://github.com/awslabs/aws-sdk-rust/pull/1168

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
This same change was made in the canary workflow in the `aws-sdk-rust`
repo and run successfully there:
* PR: https://github.com/awslabs/aws-sdk-rust/pull/1168
* Canary run:
https://github.com/awslabs/aws-sdk-rust/actions/runs/9575014647

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-19 16:36:57 +00:00
ysaito1001 3177a6b12c
Update dependent library versions (#3698)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-06-14 18:45:54 +00:00
ysaito1001 079790e366
Fix `dubious ownership` error in runtime-versioner unit tests (#3668)
## Motivation and Context
In a Docker container we use in our release pipeline, we've got the
following error in `runtime-versioner` unit tests
```
     Running tests/test_audit.rs (src/smithy-rs/tools/target/debug/deps/test_audit-9e2b5cd76b506229)

running 6 tests
Cloning into 'test_base'...
Cloning into 'test_base'...
Cloning into 'test_base'...
Cloning into 'test_base'...
Cloning into 'test_base'...
Cloning into 'test_base'...
fatal: detected dubious ownership in repository at '/tmp/.tmpjwZMOW/test_base.git'
To add an exception for this directory, call:

    git config --global --add safe.directory /tmp/.tmpjwZMOW/test_base.git
fatal: detected dubious ownership in repository at '/tmp/.tmp9Aw4By/test_base.git'
```

`runtime-versioner` unit tests [unpacks a tar file for a fake git
repo](cd0f0bac51/tools/ci-build/runtime-versioner/test-common/src/lib.rs (L31-L39)),
but for some reason, the resulting file had a strange owner:group
assigned to it
```
drwxr-xr-x 6  504 games  4096 Dec 20 00:46 test_base.git
``` 
The tests have been passing but they started breaking yesterday for some
reason. To side step this issue, this PR will ensure the resulting file
has the correct current user.

## Testing
Verified running unit tests in a container with the fix and confirmed it
passed.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-05-29 08:41:29 -04:00
Landon James 68704b1062 Bumping MSRV to 1.76.0 2024-05-20 11:58:10 -07:00
Landon James c45dc122a1
Updating docs for app_name on ConfigLoader (#3645)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Docs indicating that the `AppName` can be loaded from an environment
variable or profile key were buried in the default provider and would be
hard to find for users.

## Description
<!--- Describe your changes in detail -->
This change adds documentation to the `app_name` method on
`ConfigLoader` that makes the options for loading the `AppName` more
explicit.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Ran `cargo doc` to ensure the docs were generated correctly

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
2024-05-16 20:10:48 +00:00
ysaito1001 945407481f
Fix panic when failing to create `Duration` for exponential backoff from a large float (#3621)
## Motivation and Context
Avoids panic when [Duration for exponential
backoff](https://github.com/smithy-lang/smithy-rs/blob/main/rust-runtime/aws-smithy-runtime/src/client/retries/strategy/standard.rs#L150)
could not be created from a large float.

## Description

[Duration::from_secs_f64](https://doc.rust-lang.org/core/time/struct.Duration.html#method.from_secs_f64)
may panic. This PR switches to use a fallible sibling
`Duration::try_from_secs_f64` to avoid panics. If
`Duration::try_from_secs_f64` returns an `Err` we fallback to
`max_backoff` for subsequent retries. Furthermore, we learned from
internal discussion that jitter also needs to be applied to
`max_backoff`. This PR updates `calculate_exponential_backoff` to handle
all the said business logic in one place.

## Testing
- Added a unit test
`should_not_panic_when_exponential_backoff_duration_could_not_be_created`
- Manually verified reproduction steps provided [in the original
PR](https://github.com/awslabs/aws-sdk-rust/issues/1133)
<details> 
<summary>More details</summary>

```
#[tokio::test]
async fn repro_1133() {
    let config = aws_config::from_env()
        .region(Region::new("non-existing-region")) // forces retries
        .retry_config(
            RetryConfig::standard()
                .with_initial_backoff(Duration::from_millis(1))
                .with_max_attempts(100),
        )
        .timeout_config(
            TimeoutConfig::builder()
                .operation_attempt_timeout(Duration::from_secs(180))
                .operation_timeout(Duration::from_secs(600))
                .build(),
        )
        .load()
        .await;

    let client: Client = Client::new(&config);
    let res = client
        .list_objects_v2()
        .bucket("bucket-name-does-not-matter")
        .send()
        .await;

    dbg!(res);
}
```

Without changes in this PR:
```
---- repro_1133 stdout ----
thread 'repro_1133' panicked at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/time.rs:741:23:
can not convert float seconds to Duration: value is either too big or NaN
stack backtrace:
...

failures:
    repro_1133

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out; finished in 338.18s
```

With changes in this PR:
```
// no panic
---- repro_1133 stdout ----
res = Err(
    TimeoutError(
        TimeoutError {
            source: MaybeTimeoutError {
                kind: Operation,
                duration: 600s,
            },
        },
    ),
)
```
</details> 

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

## Appendix
<details> 
<summary>runtime-versioner bug fix</summary>

This PR also fixes a limitation in `runtime-versioner audit`. I included
the fix in the PR because the issue occurred with special conditions,
and we don't get to reproduce it every day. The way the issue manifests
is this.

1. We have a branch from the main whose latest release tag at the time
was `release-2024-04-30`
2. The main has moved ahead and a new `smithy-rs` release has been made
`release-2024-05-08`
3. We perform `git merge main`, pre-commit hooks run, and we then get
`audit` failures from `runtime-versioner`:
```
2024-05-10T16:54:36.434968Z  WARN runtime_versioner::command::audit: there are newer releases since 'release-2024-04-30'
aws-config was changed and version bumped, but the new version number (1.4.0) has already been published to crates.io. Choose a new version number.
aws-runtime was changed and version bumped, but the new version number (1.2.2) has already been published to crates.io. Choose a new version number.
aws-smithy-runtime-api was changed and version bumped, but the new version number (1.6.0) has already been published to crates.io. Choose a new version number.
aws-smithy-types was changed and version bumped, but the new version number (1.1.9) has already been published to crates.io. Choose a new version number.
aws-types was changed and version bumped, but the new version number (1.2.1) has already been published to crates.io. Choose a new version number.
Error: there are audit failures in the runtime crates
```
This happens because when the latest `main` is being merged to our
branch, `runtime-versioner audit` should use `previous_release_tag`
`release-2024-05-08` to perform audit but [pre-commit hooks run the
tool](https://github.com/smithy-lang/smithy-rs/blob/main/.pre-commit-hooks/runtime-versioner.sh#L8)
using the latest previous release tag that can be traced back from
`HEAD` of our branch, which is `release-2024-04-30`. Hence the error.

The fix adds an environment variable
`SMITHY_RS_RUNTIME_VERSIONER_AUDIT_PREVIOUS_RELEASE_TAG` to specify a
previous release tag override, in addition to a `--previous-release-tag`
command-line argument.
Furthermore, the fix has relaxed certain checks in `audit`. Taking our
example for instance, when `HEAD` is now behind `release-2024-05-08`,
it's OK to fail even if `release-2024-05-08` is not the ancestor of
`HEAD` (as stated above, `git merge-base --is-ancestor` does not know
that while main is being merged) as long as `release-2024-04-28` (the
latest release seen from `HEAD`) is the ancestor of
`release-2024-05-08`.

```
   release-2024-04-28               release-2024-05-08           
            │                                │                   
────────────┼───────────────┬────────────────┼───────x─────► main
            │               │HEAD            │       x           
                            │ of                     x           
                            │branch                  x           
                            │                        x           
                            │                        x           
                            │              xxxxxxxxxxx           
                            │              x                     
                            │              x  git merge main     
                            │              x                     
                            │              ▼                     
                            │                                    
                            └──────────────────► feature branch  
```
To use the fix, set the environment variable with the new release tag
and perform `git merge main`:
```
➜  smithy-rs git:(ysaito/fix-panic-in-exp-backoff) ✗ export SMITHY_RS_RUNTIME_VERSIONER_AUDIT_PREVIOUS_RELEASE_TAG=release-2024-05-08
➜  smithy-rs git:(ysaito/fix-panic-in-exp-backoff) ✗ git merge main
     ...
     Running `/Users/awsaito/src/smithy-rs/tools/target/debug/runtime-versioner audit`
2024-05-10T19:32:26.665578Z  WARN runtime_versioner::tag: expected previous release to be 'release-2024-04-30', but 'release-2024-05-08' was specified. Proceeding with 'release-2024-05-08'.
SUCCESS
```

</summary>

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-05-13 16:53:27 +00:00
Russell Cohen f15e7b3d59
Docs Improvements (#3600)
## Motivation and Context
- Add READMEs to internal tools
- Delete completely outdated documentation from the design folder

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2024-04-26 15:50:38 +00:00
John DiSanti 13c04dafcf
Revert cargo-semver-checks version due to performance regression (#3594)
It seems upgrading cargo-semver-checks to 0.30 made it take
significantly longer than before (going from less than 20 minutes up to
between 1-4 hours). Reverting it back to 0.24.1 for now to keep CI
faster.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-19 19:40:24 +00:00
ysaito1001 1f5cb697d1
Update JDK to 17 (#3592)
## Motivation and Context
Upgrade JDK to 17 in the repository

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-19 15:09:21 +00:00
ysaito1001 6b59ba26c4
Fix compiling S3 crate for wasm (#3590)
## Motivation and Context
Running `cargo build --target wasm32-unknown-unknown
--no-default-features` on an S3 crate has stopped working since
https://github.com/smithy-lang/smithy-rs/pull/3465 with the following
error:
```
error: the wasm*-unknown-unknown targets are not supported by default, you may need to enable the "js" feature. For more information see: https://docs.rs/getrandom/#webassembly-support
   --> /Users/[REDACTED]/.cargo/registry/src/index.crates.io-6f17d22bba15001f/getrandom-0.2.14/src/lib.rs:352:9
    |
352 | /         compile_error!("the wasm*-unknown-unknown targets are not supported by \
353 | |                         default, you may need to enable the \"js\" feature. \
354 | |                         For more information see: \
355 | |                         https://docs.rs/getrandom/#webassembly-support");
    | |________________________________________________________________________^
```

To address the issue, this PR updates an S3's dependency on `ahash` in a
way that disables default features.

## Testing
Updated the existing test `integration-tests/webassembly` so that
`check-aws-sdk-standalone-integration-tests` will run `cargo check`
`aws-sdk-s3` against both `wasm32-wasi` and `wasm32-unknown-unknown`
(the updated check would break if we removed `default-features = false`
from the `ahash` dependency).

## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-18 21:19:22 +00:00
John DiSanti b52ba10f79
Update dependencies and build tools (#3554)
This PR updates all the dependency lock files, upgrades the S3
throughput benchmark to SDK 1.x, and updates the build image
dependencies.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-12 23:11:46 +00:00
John DiSanti b1cbce65a5
Upgrade MSRV to 1.75 (#3553)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-11 23:20:06 +00:00
John DiSanti 979c0cac8a
Fix runtime-versioner audit edge case with SDK releases (#3581)
In the time gap between smithy-rs and SDK releases, the
runtime-versioner can incorrectly allow changes to aws/rust-runtime
crates without version bumping. This PR fixes this issue by detecting
this edge case.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-11 20:39:59 +00:00
John DiSanti 1ae508d026
Update maintainers (#3574)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-10 12:54:37 -07:00
John DiSanti 1bb03e5414
Remove dead subcommand from publisher and switch to regex-lite (#3560)
This PR removes the now unused upgrade-runtime-crates-version subcommand
from publisher, and also switches it to regex-lite since it doesn't need
the full regex feature set.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: ysaito1001 <awsaito@amazon.com>
2024-04-10 19:22:39 +00:00
John DiSanti 69efe3a728
Don't generate a SDK for runtime-versioner patching (#3540)
This PR removes the requirement of generating a SDK in order to patch
smithy-rs runtime crates into an already released SDK version for
testing. This is possible now that all the runtime crates are
independently versioned. The tool also takes multiple paths as input for
the runtime crate locations. This makes it possible to pass in the
`rust-runtime` and `aws/rust-runtime` directories from smithy-rs (done
by default by one of the subcommands).

Some of the publisher tool's package resolution code is moved into
smithy-rs-tool-common so it can be better shared across the tools, and
as part of this, the version in the package handle was made optional.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-09 00:41:21 +00:00
ysaito1001 aff489bb21
Run canary as part of CI (#3525)
## Motivation and Context
CI now runs
[canary](https://github.com/smithy-lang/smithy-rs/tree/main/tools/ci-cdk/canary-lambda/src/latest)
on pull requests.

## Description
Historically, canary has been running only during releases and daily in
the
[aws-sdk-rust](https://github.com/awslabs/aws-sdk-rust/actions/workflows/canary.yaml)
repository. This presents a problem of action at a distance where a
potential bug that may hurt canary won't immediately be caught when a PR
is merged to `main` in `smithy-rs` (such as code under code under
`#[cfg(debug_assertions)]` was not used during canary's lambda execution
and the definition of an item only used under that resulted in unused
warning). This PR will address that problem.

PRs from forked repositories cannot run the `Canary` job (it will be
skipped). A maintainer can run it on their behalf within
`smithy-lang:smithy-rs` using a newly added [manual
workflow](https://github.com/smithy-lang/smithy-rs/pull/3525/files#diff-1b1b5b27850107cb97519c98de99d35a49b90277ccb52201b842e9b81feb5d47).

After we merge this to main, we will update the branch protection so
that the `Canary` job is required for merge, in addition to ` Matrix
Success ` (this will not affect PRs from forked repositories since the
`Canary` job will be skipped a skipped job will [report its status as
Success](https://docs.github.com/en/actions/using-jobs/using-conditions-to-control-job-execution),
but a manual canary run should pass).

## Testing
- Verified a PR in `smithy-lang:smithy-rs` ran the canary
([passed](https://github.com/smithy-lang/smithy-rs/actions/runs/8473703578/job/23219021189?pr=3525)).
- Verified a PR in a `smithy-rs` fork cannot run the canary and
[indicated that a maintainer needs to run it
manually](https://github.com/smithy-lang/smithy-rs/actions/runs/8473799146/job/23218944122?pr=3528)
- Verified a maintainer can manually invoke the canary using a PR from a
fork
([passed](https://github.com/smithy-lang/smithy-rs/actions/runs/8474050953))
- Verified internal release pipeline passed with [the changes to
canary-runner](https://github.com/smithy-lang/smithy-rs/pull/3525/files#diff-cc953ba68fec8cf937e1aa70181a901150fc5f1a74bf6ca1075d522c63d1eb6f)

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-08 23:44:07 +00:00
Russell Cohen 09ba40e66b
Add FIPS support to Hyper 1.0 Client (#3539)
## Description
This does several things:
1. Upgrade to RusTLS 0.23 which enables FIPS support
2. Add smoke test of the clients. This revealed a bug where https URLs
were not supported.

This is technically a breaking change because I added `non_exhaustive`
to the CryptoMode enum.

<!--- Describe your changes in detail -->

## Testing
New integration tests. I expect this to fail in CI since I'll need to
update the build image to match.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-02 18:02:13 +00:00
Russell Cohen 88405d6383
Upgrade CI to use docker-compose v2 (#3543)
## Motivation and Context
- https://github.com/actions/runner-images/issues/9557
- https://docs.docker.com/compose/migrate/

## Description
It seems like they should be drop-in compatible for our use case.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-04-02 16:12:02 +00:00
Russell Cohen 0b9a2b8e89
Remove flags from release (#3521)
## Motivation and Context
This removes the (unneeded) flags from the smithy-rs release job that
set the stable/unstable versions.

## Testing
- [x] ran a dry run on the branch

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-29 14:30:38 +00:00
Russell Cohen 58d40b7686
Update CI to check for semver hazards on pull requests (#3535)
## Motivation and Context
We've hit issues several times where semver hazards were not discovered
until release. Now that we've removed version arguments, we can now run
this test on PRs.


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-28 14:11:15 -07:00
Russell Cohen 327e31e580
disable content length enforcement (#3524)
https://github.com/smithy-lang/smithy-rs/issues/3523

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-28 14:54:37 +00:00
Zelda Hessler f97e6f3b10
fix the `./gradlew :aws:sdk:assemble` command and aws-config tests on Windows (#3522)
tracked in issue #3518 

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-28 03:10:32 +00:00
Russell Cohen 5c9379574d
Make runtime versioner ignore any additional flags (#3520)
## Motivation and Context
This is the first of two PRs to remove the concept of stable/unstable
versions from the release process. This PR will cause a full docker
rebuild, so I'm splitting it out to merge first to make iterating on the
second PR easier.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-27 18:16:47 +00:00
ysaito1001 f5f94a21e6
Split CDK app between `smithy-rs` and `aws-sdk-rust` (#3514)
## Motivation and Context
A prerequisite for running canary in smithy-rs CI

## Description
This PR is essentially refactoring so that `canary-stack.ts` library can
be used by both `smithy-rs` and `aws-sdk-rust`. CDK apps are then split
into `bin/smithy-rs` and `bin/aws-sdk-rust`; the former is used by our
local development and the latter is used by our internal release
pipeline.

Merging this PR to the main branch will not affect anything. However, we
will need to update our internal release pipeline to use
`bin/aws-sdk-rust/*` instead when we release new SDKs:
- `npm run build` -> `npx tsc && npx cdk --app "node
build/bin/aws-sdk-rust/canary.js" synth`
- `npx cdk bootstrap` -> `npx cdk bootstrap --app "ts-node
--prefer-ts-exts bin/aws-sdk-rust/canary.ts"`
- `npx cdk --app "node build/bin/canary-only.js" synth` -> `npx cdk
--app "node build/bin/aws-sdk-rust/canary-only.js" synth`
- `npx cdk --app "node build/bin/canary-only.js" deploy --outputs-file
cdk-outputs.json` -> `npx cdk --app "node
build/bin/aws-sdk-rust/canary-only.js" deploy --outputs-file
cdk-outputs.json`

Also, when this PR is merged, we're ready to deploy a new canary stack
`smithy-rs-canary-stack` used by smithy-rs CI.

## Testing
> we will need to update our internal release pipeline to use
`bin/aws-sdk-rust/*` instead

Verified internally that it passed the release pipeline.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-26 23:33:14 +00:00
Zelda Hessler 2d931fa8dc
Fix the runtime versioner and update tool deps (#3503)
This issue was caused by a runtime crate referencing another runtime
crate with a version dependency instead of a path dependency. I've added
an error message that tells us how to fix this if it comes up again:

```txt
crate `aws-smithy-mocks-experimental` depends on crate `aws-smithy-types` crate by version instead of by path. Please update it to use path dependencies for all runtime crates.
```

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-22 18:10:07 +00:00
ysaito1001 f341aa2264
Enable S3 Express canary by default (#3499)
## Motivation and Context
Now that canary [passed for the last three
days](https://github.com/awslabs/aws-sdk-rust/actions/runs/8335320052),
we can turn s3 express canary on by default.

## Testing
Verified internally that our release pipeline passes canary with s3
express canary enabled.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-20 15:02:06 +00:00
Russell Cohen b43c7ad2e7
Ignore aws-config/clippy.toml (#3489)
## Motivation and Context
Fix CI breakage during release.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-14 01:36:25 +00:00
Russell Cohen 987024bfef
Fix publisher check of published versions (#3483)
## Motivation and Context
Switch to the sparse index broke the publishers check if whether or not
a version was already published.

## Description
Fix check, add tests.

## Testing
- Ran test against the real sparse index.


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-12 20:28:34 +00:00
John DiSanti 16b01d054d
Fix new runtime crate handling in SDK patcher tool (#3475)
The SDK runtime patcher tool used during smithy-rs release to detect
semver breaks was breaking due to the new aws-smithy-wasm crate since it
had a bad assumption in the logic to determine if a runtime crate
changed. This PR corrects that logic to state the runtime crate doesn't
need patching if it didn't exist previously.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-11 20:50:52 +00:00
ysaito1001 f3b332da47
Add support for S3 Express One Zone (#3465)
## Motivation and Context
Allows the Rust SDK to use [S3 Express One
Zone](https://aws.amazon.com/s3/storage-classes/express-one-zone/)

## Description
The PR adds the said S3-specific functionality to the Rust SDK. The code
changes have already been reviewed by previous sub PRs, but it's worth
going through them again as a whole:
- https://github.com/smithy-lang/smithy-rs/pull/3386
- https://github.com/smithy-lang/smithy-rs/pull/3388
- https://github.com/smithy-lang/smithy-rs/pull/3390
- https://github.com/smithy-lang/smithy-rs/pull/3432
- https://github.com/smithy-lang/smithy-rs/pull/3433
- https://github.com/smithy-lang/smithy-rs/pull/3459
- https://github.com/smithy-lang/smithy-rs/pull/3457
- https://github.com/smithy-lang/smithy-rs/pull/3462

In addition to the PRs above, commit eebe8af increases the canary
lambda's memory size to 512MB from 128MB (also makes it configurable
through a command line arg for `canary-runner`). By default, lambda's
allowed memory size is 128MB but with the addition of `canary-wasm` in
main, canary lambda's memory usage will be 152MB, causing the lambda to
be killed by a signal during runtime. The commit addresses that issue.

## Testing
- Unit tests in
[aws/rust-runtime/aws-inlineable/src/s3_express.rs](7f8c28b703/aws/rust-runtime/aws-inlineable/src/s3_express.rs)
- Integration tests in
[aws/sdk/integration-tests/s3/tests/express.rs](7f8c28b703/aws/sdk/integration-tests/s3/tests/express.rs)
- Canary in smithy-rs#3462

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: AWS SDK Rust Bot <aws-sdk-rust-primary@amazon.com>
Co-authored-by: AWS SDK Rust Bot <97246200+aws-sdk-rust-ci@users.noreply.github.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
Co-authored-by: Russell Cohen <rcoh@amazon.com>
2024-03-11 16:01:48 +00:00
ysaito1001 9ac9fbc4ea
Fix `canary-wasm` not being able to locate generated crates (#3472)
## Motivation and Context
Fixes the issue of `canary-wasm` not being able to locate generated
crates in our release pipeline

## Description
The `canary-wasm` crate has
[Cargo.toml](https://github.com/smithy-lang/smithy-rs/blob/main/tools/ci-cdk/canary-wasm/Cargo.toml)
checked-in, whose SDK dependencies assume path locations with respect to
build artifacts within the `smithy-rs` repository. However, those
locations can be different in our release pipeline (the same reason why
there's no manifest file checked-in for
[canary-lambda](https://github.com/smithy-lang/smithy-rs/tree/main/tools/ci-cdk/canary-lambda)
and it needs to be generated on the fly by `canary-runner`'s
`build-bundle` subcommand).

To fix this, `canary-runner` now generates the manifest for
`canary-wasm`.

## Testing
- [x] Added unit tests for `canary-runner`
- [x] Verified in our release pipeline that canary built and passed

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-07 19:44:05 +00:00
Russell Cohen f68b5eeaa9
Add Initial Support for Hyper 1.0 (#3461)
## Motivation and Context
- #1925 

## Description
This adds a minimal Hyper client, focusing on not exposing any unstable
APIs. For this reason, the `Client::Builder` customization API is not
exposed anymore. We do this because at some point in the future, we will
likely move away from the hyper-util based Client.

The code for this was lifted directly from the Hyper 0.14 implementation
but updated for new traits.

However, this does come with some new valuable pieces:
1. Support for aws-lc (no FIPS yet)
2. Support for providing a custom DNS resolver

## Testing
- E2E test with Hyper. A Canary should also be added
(https://github.com/awslabs/aws-sdk-rust/issues/1089)

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-07 10:57:57 -08:00
ysaito1001 fceb914fb2
Allow runtime-versioner's audit to accept HTTP remote (#3456)
## Motivation and Context
Fixes the following error when a pre-commit hook `runtime-versioner`
runs in a repo created from HTTP remote:
```
Error: smithy-rs origin must be 'git@github.com:smithy-lang/smithy-rs.git' in order to get the latest release tags
```

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-03-01 21:50:35 +00:00
John DiSanti 4604aa5b5d
Replace the crates.io API with the sparse index (#3447)
The sparse index is preferred to the crates.io API for the checks we
need, according to the [documentation](https://crates.io/data-access).
When the tools were first implemented, the sparse index didn't exist, so
the API was used.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-28 20:47:46 +00:00
John DiSanti e6791567b4
Fix claim crate names workflow (#3445)
This fixes the claim crate names GitHub Actions workflow that runs on
changes to main.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-28 17:21:24 +00:00
Harry Barber 094924cac5
Add CHANGELOG support for multiple reference authors (#2012)
## Motivation and Context

Currently only a single author can be provided to a changelog entry.
This does not support the cases where a single PR was coauthored by
multiple authors and/or multiple PRs were authored by different
individuals.

## Description

- Accept a list of strings, in addition to a single string in the
`author` field. Each author in the list is added as a contributor to
every PR provided in references.
- Accept references of the form `{ "id": <ID>, "authors": <Author> }`,
in addition to `ID`. Each author given in `authors` is added as a
contributor to that PR. Authors common to all references will be
serialized in the same way as using the top-level `author` field.

## Notes

- This should not be a breaking change. Existing `CHANGELOG.next.toml`
should be deserialized and then serialized identically.

---------

Co-authored-by: Harry Barber <hlbarber@amazon.co.uk>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
2024-02-28 17:21:02 +00:00
Landon James d95cc86400
Add `aws-smithy-wasm` crate with WASI http client (#3409)
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
This change adds a new crate, `aws-smithy-wasm`, that exports a SDK
compatible WASI http client. This is a continuation of the work in #2520
using the now stabilized WASI 0.2.0 interfaces from the [wasi
crate](https://crates.io/crates/wasi). This supports, but does not
finalize the work for #2087

## Description
<!--- Describe your changes in detail -->

Add a new crate, `aws-smithy-wasm` which exports a function
`wasi_http_client` that will provide the user with a WASI compatible
http client. This client is implemented by using the
`wasi::http::outgoing_handler`
[ref](https://docs.rs/wasi/0.12.0+wasi-0.2.0/wasi/http/outgoing_handler/index.html)
along with some utility implementations of `TryFrom` to transform back
and worth between the types from the `http` crate and the `wasi::http`
types. It also exports a unit struct `WasmSleep` that impls the
`AsyncSleep` trait needed by the SDK.

## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
This is tested via an integration test in
`aws/sdk/integration-tests/webassembly` that uses the wasi http-client
to vuild a config and an operation (that is not sent). It is further
tested in a new canary (`wasm_canary`) that calls the S3
`list_objects_v2` API.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [X] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Russell Cohen <rcoh@amazon.com>
Co-authored-by: John DiSanti <john@vinylsquid.com>
2024-02-27 23:40:42 +00:00
John DiSanti b382ebc063
Upgrade cargo-check-external-types to 0.1.11 (#3413)
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-21 14:28:25 +00:00
Russell Cohen 40f44cee6c
Rust msrv 1.74 (#3410)
## Motivation and Context
Update MSRV to 1.74

## Description
Normal slate of things, mostly appeasing clippy.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2024-02-16 21:15:48 +00:00
Russell Cohen 30421e1333
Support unchanging versions in the versioner (#3414)
## Motivation and Context
The nested path dependencies in our generated runtime crates cause
issues when simulating a release. This strips those out in order to
support testing a release where some versions _don't_ change.

## Testing
https://github.com/smithy-lang/smithy-rs/actions/runs/7917462892


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-15 16:02:59 +00:00
Russell Cohen fa61448b35
Remove warning banner code (#3407)
## Motivation and Context
Remove warning banner related code. The test is broken so this code is
just causing a hassle and will never be used again.

- [ ] A generated code diff should be audited prior to merging.
----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-13 18:23:40 +00:00
Russell Cohen c43fc71c70
Fix the canary by using edit distance (#3406)
## Testing
- deployed to my personal account

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-12 21:01:33 +00:00
John DiSanti 0be5cb32bc
Add CI step to release workflow to check for semver hazards (#3383)
This CI step will check for subtle semver hazards with the `ConfigBag`
during release. Once all the runtime crates are independent, then this
check can be moved into normal CI.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-02-09 00:15:38 +00:00
Russell Cohen 8873666b0e
Add http = "1.0" support to the `http` request wrapper (#3373)
## Motivation and Context
- aws-sdk-rust#977
- smithy-rs#3365

## Description
Add `try_from` and `try_into` for HTTP 1.x to the HTTP request/response
wrapper. This is a stepping stone en route to supporting Hyper 1.0

## Testing
- [x] New unit tests

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: John DiSanti <jdisanti@amazon.com>
2024-01-25 17:32:34 +00:00
John DiSanti 5e20575e27
Make SDK runtime patch tool that supports SDK runtime crates (#3369)
A tool to patch in new versions of smithy-rs runtime crates was
introduced in #3327 so that semver breaks could better be detected for
upcoming releases. However, this tool didn't support patching in SDK
runtime crates, which is necessary for testing #3355.

This PR moves the tool into the runtime-versioner tool, and adjusts it
to support both smithy-rs and SDK runtime crates.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-24 01:55:43 +00:00
John DiSanti 366ccab1bc
Fix describe tag issue when finding ancestor tag (#3376)
The runtime-versioner's audit subcommand fails when the current commit
is the tagged release commit due to a bug in `ancestor_tag` where it was
failing to trim the output from `git describe --tags`. I fixed this
earlier in #3369 since I was running into it there, but that hasn't been
reviewed/merged yet. This issue is causing problems for the latest
smithy-rs/SDK release, so I've pulled it out into a separate PR.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-18 17:24:53 -08:00
John DiSanti 6ffd99000b
Implement runtime-versioner audit tool (#3332)
This PR implements a build tool to audit runtime crate versions as part
of CI and release. If a runtime crate doesn't have the special
`0.0.0-smithy-rs-head` version number, then it is assumed to be
independently versioned, and the audit tool will verify it is version
bumped when any changes are made to it.

Given that there isn't a complete/reliable semver checking tool for Rust
yet, this tool isn't smart. It will rely on devs to correctly bump the
version number when that is done.

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-11 23:20:36 +00:00
Russell Cohen b2b91af447
sdk-lints was using a yanked verion of 'cargo_toml' (#3360)
## Motivation and Context
update version of cargo_toml used by sdk-lints


----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-11 21:15:37 +00:00
Russell Cohen aae94aea55
Adds experimental mocks package. (#3308)
Publishing is disabled for this package currently.

## Motivation and Context
Testing is a huge pain. Currently. This method isn't ideal and can lack
accuracy in some cases but in quick and useful for many applications.

## Description
New mock package that allows code like this:
```rust
    let s3_404 = mock!(Client::get_object)
        .match_requests(|inp| {
            inp.bucket() == Some("test-bucket") && inp.key() != Some("correct-key")
        })
        .then_http_response(|| {
            http::Response::builder()
                .status(400)
                .body(SdkBody::from(S3_NO_SUCH_KEY))
                .unwrap()
                .try_into()
                .unwrap()
        });

    let s3_real_object = mock!(Client::get_object)
        .match_requests(|inp| {
            inp.bucket() == Some("test-bucket") && inp.key() == Some("correct-key")
        })
        .then_output(|| {
            GetObjectOutput::builder()
                .body(ByteStream::from_static(b"test-test-test"))
                .build()
        });

    let get_object_mocks = MockResponseInterceptor::new()
        .with_rule(&s3_404)
        .with_rule(&s3_real_object);

    let s3 = aws_sdk_s3::Client::from_conf(
        Config::builder()
            .with_test_defaults()
            .region(Region::new("us-east-1"))
            .interceptor(get_object_mocks)
            .build(),
    );
```

There is no binary level mocking and this is completely typesafe. IDE
inference works properly as well. The crimes that were committed were
small and self-contained.

## Testing
There is a unit test and it works.

## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
2024-01-11 20:02:28 +00:00