_This PR also updates `pre-commit ktlint` runner. Now it won't spit out
a bazillion debug logs when run_
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR changes the default runtime mode to orchestrator for generated
clients and the AWS SDK.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR fixes Timestream in the orchestrator implementation.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
During the orchestrator implementation, a bug was introduced that always
enabled the `test-util` feature on `aws-smithy-runtime` in the generated
crates. This led to several pieces of non-test code relying on test-only
code, specifically around `SystemTime` implementing `TimeSource`.
This PR fixes that issue, and also fixes another issue where the
`RuntimeComponentsBuilder` was being initialized without a name for the
service config.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR moves all the "runtime components", pieces that are core to the
operation of the orchestrator, into a separate `RuntimeComponents` type
for the orchestrator to reference directly.
The reason for this is so that these core components cannot be changed
by interceptors while the orchestrator is executing a request.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
We weren't already doing this but we should b/c it's needed to use the
`add_interceptor` and `set_interceptors` config methods
---
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR increases the max line length on generated client code to reduce
the likelihood of rustfmt failing to format files.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Adds `Metadata` to a config layer through `OperationRuntimePlugin`.
## Description
We have had a customer's use case where a service name and a operation
name are obtained from
[Metadata](ddba46086a/rust-runtime/aws-smithy-http/src/operation.rs (L17-L22)).
The end goal was to use their names as part of metrics collection.
Previously, it was done using `map_operation` on a
`CustomizableOperation`, e.g.
```
client
.some_operation()
.customize()
.await?
.map_operation(|operation| {
operation.try_clone().map(|operation| {
let (_, parts) = operation.into_request_response();
parts.metadata.map(|metadata| {
let service_name = metadata.service().to_string().to_uppercase();
let operation_name = metadata.name().to_string();
/*
* do something with `service_name` and `operation_name`
*/
})
});
Ok(operation)
})?
.send()
.await;
```
The orchestrator no longer supports `map_operation` on
`CustomizableOperation`. We therefore add `Metadata` to a config layer
through `OperationRuntimePlugin`. See the added integration test for how
`Metadata` is retrieved from within an interceptor.
## Testing
Added an integration-test to verify `Metadata` is properly set.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: ysaito1001 <awsaito@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
## Motivation and Context
Implements the actual meat of `config_override` introduced [as a
skeleton in the past](https://github.com/awslabs/smithy-rs/pull/2589).
## Description
This PR enables operation-level config via `config_override` on a
customizable operation (see
[config-override.rs](8105dd46b4/aws/sdk/integration-tests/s3/tests/config-override.rs)
integration tests for example code snippets).
The way it's implemented is through `ConfigOverrideRuntimePlugin`. The
plugin holds onto two things: a `Builder` passed to `config_override`
and a `FrozenLayer` derived from a service config (the latter is
primarily for retrieving default values understood by a service config).
The plugin then implements the `RuntimePlugin` trait to generate its own
`FrozenLayer` that contains operation-level orchestrator components.
That `FrozenLayer` will then be added to a config bag later in the
orchestrator execution in a way that it takes precedence over the
client-level configuration (see
[register_default_runtime_plugins](8105dd46b4/codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/generators/OperationGenerator.kt (L65-L71))).
A couple of things to note:
- The size of `ConfigOverrideRuntimePlugin::config` code-generated is
getting large. Maybe each customization defines a helper function
instead of inlining logic directly in `config` and we let the `config`
method call those generated helpers.
- The PR does not handle a case where `retry_partition` within
`config_override` since it's currently `#[doc(hidden)]`, e.g.
```
client
.some_operation()
.customize()
.await
.unwrap()
.config_override(Config::builder().retry_partition(/* ... */))
...
```
## Testing
- Added tests in Kotlin
[ConfigOverrideRuntimePluginGeneratorTest.kt](https://github.com/awslabs/smithy-rs/pull/2814/files#diff-04a76a43c2adb3a2ee37443157c68ec6dad77fab2484722b370a7ba14cf02086)
and
[CredentialCacheConfigTest.kt](https://github.com/awslabs/smithy-rs/pull/2814/files#diff-32246072688cd11391fa10cd9cb38a80ed88b587e95037225dbe9f1a482ebc5d).
~~These tests are minimal in that they only check if the appropriate
orchestrator components are inserted into a config override layer when a
user calls a certain builder method as part of `config_override`.~~
- Added integration tests
[config-override.rs](https://github.com/awslabs/smithy-rs/pull/2814/files#diff-6fd7a1e70b1c3fa3e9c747925f3fc7a6ce0f7b801bd6bc46c54eec44150f5158)
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
This PR fixes the S3 `alternative-async-runtime` retry tests. The retry
backoff time was being included in the operation attempt timeout, which
I think is undesirable as it makes retry config much harder to get right
since the backoff times are not predictable (they have randomness
incorporated into them). The overall operation timeout is the only one
that needs to incorporate backoff times.
In addition, this PR also:
- Updates READMEs for the `aws-smithy-runtime-api` and
`aws-smithy-runtime` crates
- Adds top-level crate docs to describe features to `aws-smithy-runtime`
- Copies `capture_test_logs` into `aws-smithy-runtime` so that it can be
used (just about) everywhere instead of just in `aws-config`
- Adds service/operation name to the tracing `invoke` span so it's
possible to tell which operation the events are for
- Makes the `Debug` impl for `Identity` useful
- Adds a ton of trace/debug spans and events to the orchestrator
- Fixes an issue in `aws-smithy-runtime` where a huge amount of the
orchestrator tests weren't being compiled due to a removed feature flag
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR updates the orchestrator to support connection poisoning for the
`hyper` connector. It also renames many usages of "connection" to
"connector" in order to make talking about these concepts less
confusing. In short:
```
/// A "connector" manages one or more "connections", handles connection timeouts, re-establishes
/// connections, etc.
///
/// "Connections" refers to the actual transport layer implementation of the connector.
/// By default, the orchestrator uses a connector provided by `hyper`.
```
One possibly controversial decision I made is that `reconnect_mode` is
now a standalone configurable field, rather that a sub-field of
`RetryConfig`. I think we should get together as a team and discuss what
kinds of things we want to allow users to do when configuring
connections.
My thoughts on this are that we should either:
- **A.** Make connection-related settings their own types instead of
including them within other config types
- **B.** Make a single config struct for all connector-related stuff/use
the preëxisting `ConnectorSettings` struct.
Personally, I'm partial to A.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
All the types in the `crate::endpoint` module are related to endpoint
configuration, so they should be moved into `crate::config::endpoint` to
be consistent with the rest of the generated crate organization. This PR
moves them for the orchestrator implementation, but retains them in
`crate::endpoint` when generating for middleware.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR refactors the fluent client so that the client runtime plugins
are created exactly once at client creation time, and reused thereafter.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR fixes the Smithy `@httpChecksumRequired` trait and idempotency
token auto-fill in the orchestrator implementation.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
#2190
## Description
<!--- Describe your changes in detail -->
add support for adaptive retries
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
I wrote some tests and I'm open to suggestions for more.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
In orchestrator mode, most `codegen-client-test` tests were failing due
to being unable to find a matching auth scheme, or due to some of the
test models referencing the `@sigv4` trait. This PR fixes all of those
failures, and adds the `smithy.runtime.mode` flag to
`codegen-client-test` as well.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR:
- Renames "anonymous auth" to "no auth"
- Removes fallback to other auth schemes when an identity fails to
resolve (this was not complying to the reference architecture)
- Adds the ability to opt out of credentials in `ConfigLoader`, and
removes defaulting of the shared credentials cache if no credentials
provider is given
- Makes `ConfigBagAccessors` work on `FrozenLayer` and `CloneableLayer`
- Fixes STS and aws-config tests
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR removes the non-storable `get`/`put` methods from `ConfigBag`
and `Layer`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Removes `HACK` that used to put `Handle` in config bag.
## Description
The `HACK` was previously placed in `ServiceRuntimePlugin::config`
because later in the orchestrator execution, we needed to access service
config fields (through `Handle`) to build endpoint `Params` within
`EndpointParamsInterceptor`. Now that service config fields are baked
into a frozen layer, `EndpointParamsInterceptor` can load those fields
directly from the config bag (to which the frozen layer has been added)
when constructing endpoint `Params`. We can therefore remove `HACK` at
this point.
## Testing
- [x] Passed tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
This PR:
- Makes the `#[doc(hidden)]` attributes on orchestrator-only config
functions be conditional based on whether the orchestrator is default or
not.
- Adds re-exports specific to the orchestrator.
- Centralizes `BoxError`.
- Changes organization of interceptor context types a bit.
- Reduces visibility of `Phase` to `pub(crate)`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Moves setting orchestrator components out of `ServiceRuntimePlugin` and
puts it in service config builders' build method.
## Description
This PR is the forth in a series of config refactoring. Here, we move
pieces of code out of `ServiceRuntimePlugin::config` method so that key
orchestrator components meant for the service-level config should only
be constructed once when a service config is created, e.g. during
builder's `build` method. Previously, those components were newly
created every time an operation is invoked.
Wherever `self.handle.conf...` is used, the PR has moved it from
`ServiceRuntimePlugin::config` to the builders' `build` method.
Note that there will be a separate PR to better handle auth resolver &
identity resolver in the context of the ongoing config refactoring.
## Testing
- [x] Passed tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
This allows testing, as well as making decisions while creating
builders. The references are not mutable, so these props remain read
only keeping the Builder invariants.
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
Some operations are built up using various logic (for instance, choosing
an S3 bucket based on a logged in user). It is a lot of overhead to test
at the mock level, and the Debug formatting is typically not stable.
<!--- If it fixes an open issue, please link to the issue here -->
Closes#2791
## Description
<!--- Describe your changes in detail -->
1. Add `get_foo(&self) -> &Option<Foo>` to Builders
2. Add `as_input(&self) -> &OperationInputBuilder` to Fluent Builders
3. Add `get_foo(&self) -> &Option<Foo>` to FluentBuilder which delegates
to Builders
## Testing
<!--- Please describe in detail how you tested your changes -->
Included unit tests, as well as [proof of concept
tests](017e8a2e40)
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [X] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [X] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
This PR will reduce fields in service config builders to contain only
`CloneableLayer` (except for `interceptors` and `identity_resolvers`).
## Description
This is the third PR in a series of config refactoring in the
orchestrator mode. Just like #2762, this PR reduces fields in service
config builders to contain `CloneableLayer`.
The code changes follow a straightforward pattern where builders'
setters are implemented via a config layer.
## Testing
Relies on the existing tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
This PR fixes codegen for the Smithy `@httpChecksum` trait when
generating in orchestrator mode, and also fixes an issue in the unit
tests where the wrong body was being tested to be retryable. The request
body should be retryable rather than the response body.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Incorporates suggestion made in
https://github.com/awslabs/smithy-rs/pull/2762#discussion_r1231462878
## Description
This PR renames `make_token` to `idempotency_token_provider` for clarity
wherever it is referred to in the fields and API in service configs and
their builders.
## Testing
Existing tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
Service config structs now only contain a
`aws_smithy_types::config_bag::FrozenLayer` in the orchestrator mode (no
changes for the middleware mode).
## Description
This PR reduces the individual fields of service configs to contain just
`FrozenLayer`. This makes service configs work more seamlessly with
runtime plugins in the orchestrator mode.
Note that service config _builder_ s still contain individual fields.
We're planning to make them just contain
`aws_smithy_types::config_bag::Layer`. To do that, though, we need to
make `Layer` cloneable and that will be handled in a separate PR. For
builders, the only change you will in the PR is that their `build`
method will put fields into a `Layer`, freeze it, and pass it to service
configs.
This PR is marked as a breaking change because it's based on [another
PR](https://github.com/awslabs/smithy-rs/pull/2728) that's also breaking
change.
## Testing
- [x] Passed tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
This PR categorizes orchestrator TODO comments into `Launch` and
`Cleanup`:
- `enableNewSmithyRuntimeLaunch`: Comment needs to be addressed before
orchestrator launch
- `enableNewSmithyRuntimeCleanup`: Comment needs to be addressed after
launch when removing middleware
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
We will eventually want to store additional runtime plugins on fluent
builders, clients, etc. To make this work, the RuntimePlugin trait needs
to assert Send/Sync
## Description
Small tweaks + test on the RuntimePlugin trait
## Testing
CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
This PR fixes presigning for Amazon Polly in the orchestrator
implementation.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This is a child PR of https://github.com/awslabs/smithy-rs/pull/2615.
## Description
- Adds `send_with` method to Fluent Builder.
## Prerequisite PRs
You can merge this first too reduce diffs.
- https://github.com/awslabs/smithy-rs/pull/2651
## Testing
NA
## Checklist
NA
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <johndisanti@gmail.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
This PR fixes the protocol tests in orchestrator mode, and adds
`--all-targets` to the orchestrator CI checks.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
addresses #2743
## Description
<!--- Describe your changes in detail -->
- add more standard retry tests
- add optional standard retries token bucket
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
tests are included
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
Update the RuntimePlugin trait based on discussion:
1. Methods are infallible
2. Split out `Config` and `Interceptors`
3. `ConfigBag` now has an explicit field `interceptor_state`
4. Refactor `ConfigBagAccessors` so that we can build the core around
the trait and keep the trait together with a `where Self` trick
## Description
- Update the `RuntimePlugin` trait
- Deal with resulting implications
## Testing
- [x] CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
Hides `Arc<dyn ResolveEndpoint>` from public API.
## Description
This PR replaces the occurrences of `Arc<dyn ResolveEndpoint>` with
`SharedEndpointResolver` to not expose bare `Arc`s in the public API.
## Testing
- [x] Passed tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
This PR fixes S3 presigning in the client orchestrator implementation. A
Polly presigning fix will come in a separate PR.
This PR also:
- Renames `ClientProtocolGenerator` to `OperationGenerator` to better
represent what its generating.
- Moves `OperationCustomization` into `codegen-client` since it is
client specific and unused by the server codegen.
- Deletes the `ProtocolGenerator` base class in `codegen-core` since it
wasn't adding any value.
- Adds the concept of stop points to the orchestrator so that
orchestration can be halted before transmitting a request.
- Adds `DisableInvocationIdInterceptor`,
`DisableRequestInfoInterceptor`, `DisableUserAgentInterceptor`, and
`HeaderSerializationSettings` config bag configs to facilitate
presigning requirements.
- Fixes compilation of all S3 and Polly tests when in orchestrator mode.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Add `SharedAsyncSleep` wrapping `Arc<dyn AsyncSleep>` and update call
sites of `Arc<dyn AsyncSleep>` to use `SharedAsyncSleep`
## Description
Public APIs that take/return a trait object for `AsyncSleep` have been
using `Arc<dyn AsyncSleep>` in the codebase. This has a downside of
exposing a bare `Arc`, making it difficult to change the APIs in the
future without breaking backwards compatibility. This PR adds a newtype
wrapper `SharedAsyncSleep` that hides `Arc<dyn AsyncSleep>` to alleviate
the said issue (the idiom of having `SharedXXX` has been seen in the
codebase, e.g. `SharedTimeSource`, `SharedCredentialsProvider`, and so
on).
## Testing
- [x] Passed tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Description
<!--- Describe your changes in detail -->
This PR adds support for the Standard retry strategy. The standard
strategy will be inserted into a service's config bag if a retry config
was set. Otherwise, unless another retry strategy was already set, a
`NeverRetryStrategy` will be set. This seemed like a reasonable default,
but I'm open to suggestions.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
tests are included
---
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
This change fixes many of the smaller issues I ran into during the
implementation of standard retries for the orchestrator. Merges from
`main` were getting difficult with my [other
PR](https://github.com/awslabs/smithy-rs/pull/2725) so I'm breaking
things up.
## Description
<!--- Describe your changes in detail -->
- when orchestrator attempt timeout occurs, error is now set in context
- update test connection to allow defining connection events with
optional latency simulation update orchestrator attempt loop to track
iteration count
- set request attempts from the attempt loop
- add comment explaining "rewind" step of making request attempts add
`doesnt_matter` method to `TypeErasedBox`, useful when testing update
tests to use the new `TypeErasedBox::doesnt_matter` method
- add more doc comments
- add `set_subsec_nanos` method to `DateTime`.
- I added this to make it easier to string-format a datetime that didn't
include the nanos.
- fix Invocation ID interceptor not inserting the expected header update
input type for `OperationError::other` to be more user-friendly
- add `test-util` feature to `aws-smithy-runtime-api`
- add `test-util` feature to `aws-runtime`
- fix presigining inlineable to pull in tower dep during codegen
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
tests have been updated where necessary
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Description
<!--- Describe your changes in detail -->
- update retry classifiers to act directly on the `InterceptorContext`
- add codegen method for inserting retry classifiers into the config bag
- update `InterceptorContext` to return options instead of panicking
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
Tests were updated as needed.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR moves types and traits in those modules to the
`aws-smithy-types` crate.
## Description
This is to prepare for the upcoming refactoring where `SdkConfig` and
service configs (and both of their builders) will be backed by
`ConfigBag`.
## Testing
- [x] Pass tests in CI
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] ~~I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates~~
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
TODO:
- [x] docs
- [x] integration test (canary even?)
- [x] customize README for timestream
## Motivation and Context
- #613
- https://github.com/awslabs/aws-sdk-rust/issues/114
## Description
This adds support for TSW and TSQ by adding endpoint discovery as a
customization. This is made much simpler by the fact that endpoint
discovery for these services **has no parameters** which means that
there is no complexity from caching the returned endpoint.
Customers call `.enable_endpoint_discovery()` on the client to create a
version of the client with endpoint discovery enabled. This returns a
new client and a Reloader from which customers must spawn the reload
task if they want endpoint discovery to rerun.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
- #2262
- #2087
- #2707
This adds `TimeSource` in SDK and service configs so we can effectively
control time when executing requests with the SDK at the client level.
_note:_ the breaking change is in a trait implementation of a struct
we're going to delete, not a real breaking change
## Description
- Add `SharedTimeSource` and use it in SdkConfig / `aws-config` /
<service>::Config
- Wire up the signer and other uses of `SystemTime::now()` that I could
find
- track down broken tests
## Testing
- [x] various unit tests that all still pass
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR makes generic clients default to the built-in rustls HTTPS
connector when no override is provided, but only for the new
orchestrator implementation.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR gets event streams working in the client orchestrator
implementation, and depends on #2671.
The orchestrator's `TypeErasedBox` enforces a `Send + Sync` requirement
on inputs and outputs. For the most part, this isn't an issue since
almost all generated inputs/outputs are `Send + Sync`, but it turns out
the `EventStreamSender` wasn't `Sync` due to an omission of the `Sync`
bound. Thus, this PR is a breaking change, as it adds a `Sync`
requirement for anyone who passes a stream to an event stream operation.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Render `CustomizableOperation` once in the `customizable` module instead
of rendering it in every operation's `builders` module.
## Description
This PR is a follow-up on #2706. The previous PR ported
`CustomizableOperation` in the middleware to the orchestrator but it had
a flaw in that the type was rendered per every operation. This was
clearly a codegen regression because the code for
`CustomizableOperation` is repeated many times and is rendered even if
no customization is requested for an operation.
This PR attempts to fix that problem. Just like `CustomizableOperation`
in the middleware, we render the new `CustomizableOperation` once in the
`customize` module per crate. To accomplish that, the new
`CustomizableOperation` uses a closure, called `customizable_send`, to
encapsulate a call to a fluent builder's `send` method and invokes it
when its own `send` method is called.
~~As a side note, this PR will not take care of the following. We have
[a separate PR](https://github.com/awslabs/smithy-rs/pull/2723) to fix
them and once we've merged it to this PR, they should be addressed
automatically (there will be merge conflicts, though):~~
- ~~Removing `send_orchestrator_with_plugin` (which is why a type
parameter `R` is present in the code changes, but it'll soon go away)~~
- ~~Incorrect signature of a closure in orchestrator's
`CustomizableOperaton::map_request`~~
## Testing
No new tests added as it's purely refactoring. Confirmed `sra-test`
passed (except for the one that's known to fail).
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
This PR creates a `DeferredSigner` implementation that allows for the
event stream message signer to be wired up by the signing implementation
later in the request lifecycle rather than by adding an event stream
signer method to the config.
Refactoring this brings the middleware client implementation closer to
how the orchestrator implementation will work, which unblocks the work
required to make event streams work in the orchestrator.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
* Show how to plug a connector
* For TLS: keep rustls, only
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Signed-off-by: Daniele Ahmed <ahmeddan@amazon.de>
## Motivation and Context
This PR gets the Glacier integration tests to pass against the
orchestrator implementation by porting the Glacier customizations from
`make_operation` changes to interceptors.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR removes all the middleware code from generated clients when the
`smithy.runtime.mode` flag is set to `orchestrator`. It also changes the
`aws-config` crate to use the fluent client instead of the Smithy client
for STS and SSO based credential providers.
The `polly` test no longer compiles in orchestrator mode with these
changes since presigning directly references middleware code. This will
get fixed in a later PR.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Description
This will allow integration tests under `aws/sdk/integration-tests` to
run in both smithy runtime modes. Prior to the PR, the integration tests
use `map_operation` to customize operations, which is not supported in
the orchestrator, preventing us from running them in the orchestrator
mode. Fortunately, all the usages of `map_operation` in the integration
tests involve setting a test request time and a test user agent in a
property bag.
This PR stops using `map_operation` in those tests and instead
introduces separate test helper methods in both runtime modes: one for
setting a test request and the other for setting a test user agent. They
allow the integration tests to compile and run in both modes.
Note that the integration tests in the orchestrator do not necessarily
pass at this time. We'll address the failures subsequently.
## Testing
Confirmed integration tests continue passing with the changes when run
in the middleware . Confirmed those complied (but not necessarily
passed) in the orchestrator.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
This PR refactors the client protocol test generator machinery to use a
client instead of calling `make_operation` directly, and then fixes the
ad hoc tests for the orchestrator.
The ad hoc tests revealed that overriding the signing region/service via
endpoint config was lost when porting SigV4 signing to the orchestrator,
so this PR updates the SigV4 `HttpRequestSigner` implementation to
restore this functionality. It is doing this in the signer directly
rather than via an interceptor since it should only run this logic when
SigV4 is the selected auth scheme.
Other notable changes:
- Adds `--no-fail-fast` arg to `cargoTest` targets so that all Rust
tests run in CI rather than stopping on the first failure
- Changes `EndpointResolver::resolve_and_apply_endpoint` to just
`resolve_endpoint` so that the orchestrator can place the endpoint
config into the request state, which is required for the signer to make
use of it
- Adds a `set_region` method to SDK service configs
- Deletes the API Gateway model and integration test from the SDK smoke
test since it is covered by the ad hoc tests
- Adds a comment explaining where the API Gateway model comes from in
the ad hoc tests
- Adds a `smithy.runtime.mode` Gradle property to `aws:sdk` and
`aws:sdk-adhoc-test` to trivially switch between middleware and
orchestrator when testing/generating locally
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
Necessary before we can implement retries.
## Description
<!--- Describe your changes in detail -->
I noticed that we weren't handling the flow quite right when errors
occurred. This PR fixes that and adds interceptor-based tests to ensure
things are working right. I still think we could use more tests but the
PR is already quite large.
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
This PR contains tests
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
Port [Customizable
Operation](https://github.com/awslabs/smithy-rs/pull/1647) to
orchestrator
## Description
This PR implements `CustomizableOperation` in the orchestrator. Just
like the counterpart in the middleware, it is created when the
`customize` method (in the orchestrator mode) on a fluent builder is
called. The `customize` method in the orchestrator could technically be
made a synchronous method because there is no need to create an
operation, which requires `async`, therefore making the `customize`
method in the middleware `async`. However, during the transition from
the middleware to the orchestrator, the integration tests
([example](31c152d9af/aws/sdk/integration-tests/s3/tests/signing-it.rs (L36)))
need to be able to run in both modes. For this reason, the `customize`
method in the orchestrator is temporarily marked as `async`.
Regarding methods defined on the new `CustomizableOperation`, they
include `mutate_request` and `map_request` from the counterpart in the
middleware. However, it did not port `map_operation` because there is no
operation to map on. Most use cases for `map_operation` is put things in
a property bag. The new `CustomizableOperation` provides an
`interceptor` method to accomplish the same, i.e putting things in a
config bag.
Finally, for integration tests to run in both modes, the code gen emits
the implementation of the `customize` method differently depending on
the active Smithy runtime mode, similar to what the implementation of
`send` method does.
## Testing
Added one `sra-test` for mutating a request.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
Operations named `Send` or `Sync` (and probably others) were colliding
with the types in the standard library prelude and causing compiler
errors. This PR adds tests that include all the type names from the Rust
prelude, and fixes the compiler errors they cause.
In the future, the `no_implicit_prelude` attribute can be added to
certain code generated modules to better enforce that there can't be
name collisions, but for now, the `tokio::test` macro doesn't compile
with that attribute enabled (and likely other macros from other
libraries).
## Checklist
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Description
This PR allows users to pass-in interceptors to a generic client.
Client-level configured interceptors are eventually added to
`client_interceptors` and operation-level interceptors to
`operation_interceptors`, both of which are fields in the
`aws-smithy-runtime-api::client::interceptors::Interceptors`. The
relevant code is generated only in the orchestrator mode.
The SDK registers a default set of (client-level & operation-level)
interceptors, and the passed-in interceptors by a user will run _after_
those default interceptors.
## Testing
- Added integration tests `operation_interceptor_test` and
`interceptor_priority` in `sra_test`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
This PR fixes the KMS tests when the generating code in orchestrator
mode by ensuring the `Future` returned by `send()` implements `Send`,
and also adds a codegen unit test for this so that it's checked at the
generic client level.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
The paginators were originally turned off for the orchestrator feature
flag since there were issues with them, but those issues seem to have
been resolved since. This PR re-enables them.
## Testing
- [x] Generated the AWS SDK in `orchestrator` mode and verified the
paginator tests pass
- [x] Added a unit test that validates the paginators compile for
generic clients in `orchestrator` mode
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
When setting the `enableNewSmithyRuntime` feature gate to
`both_default_middleware` or `both_default_orchestrator`, the generated
code would fail to compile due to the a checksum response decorator
assuming the property bag always existed. This PR makes that decorator
aware of the conditional existence of the property bag so that it can
elect to not customize the code for the orchestrator use case (which
will ultimately replace the decorator with an interceptor).
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR:
- Deletes `TraceProbe`
- Replaces the orchestrator's `Phase` helper with a couple of macros
- Introduces the concept of phases into `InterceptorContext` so that
input/output/error/request/response accessors don't need option wrappers
- Adds `TypeErasedError` so that `orchestrator::Error` can implement
`Error`
- Rewinds the interceptor context in the retry loop
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR ports @rcoh's addition of `Debug` in `ConfigBag` to the
`TypeErasedBox`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR implements timeouts in the orchestrator.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
- split out interceptors from config bag to simplify orchestrator
## Description
- update runtime plugin trait to split out interceptors
- remove interceptor generics
- update interceptors struct to remove locking and indirection
## Testing
- [x] ```./gradlew :aws:sra-test:assemble && (cd
aws/sra-test/integration-tests/aws-sdk-s3 && cargo test)```
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
## Motivation and Context
This PR adds the codegen logic to generate endpoint parameters in the
endpoint params interceptor.
Fixes#2644.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
So far we have been testing with the orchestrator on or off, with "on"
meaning that both the middleware and orchestrator implementations exist,
and you have to opt into the orchestrator via a `send_v2` function call
(instead of `send`). This approach makes it difficult to test against
the orchestrator exclusively, and also test the existing integration
tests with the orchestrator implementation.
This PR changes `enableNewSmithyRuntime` into a quad-state with:
- middleware
- both middleware and orchestrator, defaulting to middleware
- both middleware and orchestrator, defaulting to orchestrator
- orchestrator
This allows for generating a client with the orchestrator exclusively,
or generating both and defaulting to the orchestrator, to reduce testing
friction.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
When SigV4 signing was ported over to the orchestrator, the request time
got calculated once before the retry loop, which was incorrect. This PR
moves that request time calculation into the request signer so that it
happens on every attempt (unless there is a configured request time
override).
This PR also refactors auth to use the `ConfigBag` instead of a separate
signing properties `PropertyBag`.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
- `send_v2()` doesn't work for a very small number of reasons—allow it
one last final runtime_plugin so that we can use it to make an e2e
request
## Description
- update a few decorators to be deterministic
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [ ] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
## Motivation and Context
Port https://github.com/awslabs/smithy-rs/pull/2630 to the `main`
branch.
## Description
Adding `Debug` bound on `ResolveEndpoint` in
https://github.com/awslabs/smithy-rs/pull/2577 was a breaking change.
Unfortunately, the semvar check that ran in that PR barked at a
different thing (i.e. barked at moving `StaticUriEndpointResolver` and
`DefaultEndpointResolver` from `aws-smithy-runtime-api` to
`aws-smithy-runtime`). Eventually the breaking change got merged to the
`main` branch.
This PR reverts the change in question and implements the `Debug` trait
for `ResolveEndpoint` in a semvar non-breaking way.
## Testing
- [ ] Passed tests in CI
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
<!--- Why is this change required? What problem does it solve? -->
<!--- If it fixes an open issue, please link to the issue here -->
To retry a response, we must first classify it as retryable.
## Description
<!--- Describe your changes in detail -->
feature: add AWS error code classifier
feature: add x-amz-retry-after header classifier
feature: add smithy modeled retry classifier
feature: add error type classifier
feature: add HTTP status code classifier
add: tests for classifiers
remove: redundant `http` dep from `aws-http`
move: `NeverRetryStrategy` to smithy-runtime crate
add: RuntimePluginImpls codegen section for operation-specific runtime
plugin definitions
update: orchestrator retries to work with `ShouldAttempt`
add: retry classifier config bag accessor
add: raw response getter to SdkError
update: RetryStrategy trait signatures to use `ShouldAttempt`
add: `RetryClassifiers` struct for holding and calling retry classifiers
update: `RetryClassifierDecorator` to define orchestrator classifiers
add: `default_retry_classifiers` fn to codegen
update: `ServiceGenerator` to add feature flag for
aws-smithy-runtime/test-util
update: SRA integration test to insert retry classifier plugin
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
this change includes tests
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR adds support for Smithy's `@httpBasicAuth` and `@httpBearerAuth`
auth schemes, and ports the `@httpApiKeyAuth` scheme to the
orchestrator. This is prerequisite work for supporting Amazon
CodeCatalyst since that requires bearer auth.
This PR also fixes a bug in auth orchestrator that caused an error if no
identity is present for a scheme even when an identity for a lower
priority scheme is available.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
## Motivation and Context
This PR incorporates post-merge feedback left in #2577.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
- credentials providers may leak credentials in the HTTP body at the
debug level
## Description
This adds a test to aws-config that looks for leaked credentials in all
of our provider integration tests—since these test use AWS APIs under
the hood, this also serves to test AWS services in general.
To support this, `sensitive` was added to the ParseHttpResponse trait
and code was generated to take action based on this change.
- [x] Add environment variable to force logging of the body
- [x] consider if we want to suppress request body logging as well
## Testing
<!--- Please describe in detail how you tested your changes -->
<!--- Include details of your testing environment, and the tests you ran
to -->
<!--- see how your change affects other areas of the code, etc. -->
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Motivation and Context
This PR adds skeleton code for supporting operation level config behind
`enableNewSmithyRuntime`.
## Description
The approach is described in
https://github.com/awslabs/smithy-rs/pull/2527, where fluent builders
now have a new method, `config_override`, that takes a service config
builder. The builder implements the `RuntimePlugin` trait and once
`TODO(RuntimePlugins)` is implemented, allows itself to put field values
into a config bag within `send_v2`.
## Testing
Added a line of code to an ignored test in `sra_test`, which, however,
does check compilation.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
## Description
Docs were wrong, the function is called `send()`, not `call()`.
-----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
## Motivation and Context
This PR adds `DefaultEndpointResolver`, a default implementer of the
[EndpointResolver](1e27efe05f/rust-runtime/aws-smithy-runtime-api/src/client/orchestrator.rs (L182-L184))
trait, to the orchestrator.
## Description
Roughly speaking, the endpoint-related work is currently done by
`make_operation` and `SmithyEndpointStage`, where `make_operation`
constructs endpoint params and applies an endpoint resolver and the
`SmithyEndpointStage` later updates the request header based on the
resolved endpoint. In the orchestrator world, that work is handled by
`DefaultEndpointResolver::resolve_and_apply_endpoint`.
The way endpoint parameters and an endpoint prefix are made available to
`DefaultEndpointResolver::resolve_and_apply_endpoint` is done by
interceptors because they _may_ require pieces of information only
available in an operation input struct. The place that has access to
both an operation input struct and a config bag happens to be an
interceptor.
There are two interceptors `<Operation>EndpointParamsInterceptor` and
`<Operation>EndpointParamsFinalizerInterceptor` per operation. We pass
around endpoint params _builder_ through interceptors to allow it to be
configured with more information at a later point; An end point
parameters builder is first initialized within the
`ServiceRuntimePlugin` with the field values obtained from the service
config, and is stored in a config bag. The finalizer interceptor "seals"
the builder and creates the actual endpoint params before it is passed
to `DefaultEndpointResolver::resolve_and_apply_endpoint`. These
interceptors implement `read_before_execution` because they need to
access an operation input before it gets serialized (otherwise,
`context.input()` would return a `None`).
## Testing
Replaced `StaticUriEndpointResolver` with `DefaultEndpointResolver` in
`sra_test` and verified the test continued passing.
UPDATE: The test currently fails in the `main` branch (it returns a
`None` when extracting `SigV4OperationSigningConfig` from the config bag
in `OverrideSigningTimeInterceptor`, hence `unwrap` fails), and by
merging the `main` branch this PR no longer passes the test, but it does
not add new failures either.
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
## Motivation and Context
This is an attempt at fixing
https://github.com/awslabs/aws-sdk-rust/issues/784.
The service-level `Error` enum implements `std::error::Error` but does
not implement its `source()` method. This means that an error library
like `anyhow` or `eyre` won't be able to display the root cause of an
error, which is especially problematic for the `Unhandled` variant.
## Description
I modified `ServiceErrorGenerator` in the `codegen-client` crate and
replaced the line that output `impl std::error::Error for Error {}` with
an impl block that implements the `source()` method by delegating to the
inner error structure.
## Testing
I've added a simple unit test to `ServiceErrorGeneratorTest`.
## Checklist
<!--- If a checkbox below is not applicable, then please DELETE it
rather than leaving it unchecked -->
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the
smithy-rs codegen or runtime crates
- [x] I have updated `CHANGELOG.next.toml` if I made changes to the AWS
SDK, generated SDK code, or SDK runtime crates
----
_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._
* Add `RustType.Application` and `PythonType.Application` variants
* Use `RustType.Application` for event streaming symbols
* Call symbol provider to get `Receiver` type instead of hardcoding
* Add Python specific event streaming symbol provider
* Add event streaming wrapper generator
* Generate correct type information for event streaming types
* Add `CapturePokemon` operation to Python Pokemon service
* Add `InternalServerError` variant to all event streaming errors
* Use `PythonServerCargoDependency` for PyO3Asyncio imports
* Return an attribute error instead of panicking in `IntoPy` impls of wrappers
* Add `Sync` bound to `new` methods of wrappers
* Revert "Add `InternalServerError` variant to all event streaming errors"
This reverts commit b610cb27c7532102e3a3ec15a59da6c56b60c318.
* Add `PythonServerEventStreamErrorGenerator` to generate Python specific error types for unions
* Try to extract error type or inner type from incoming streaming value and ignore the value otherwise for now
* Allow missing type-stubs for `aiohttp`
* Propogate modelled errors through stream
* Raise modelled exceptions rather than sending through stream
* Allow senders from Python side to raise modelled exceptions
* Update `EventStreamSymbolProviderTest` to use `Application` type instead of `Opaque` type
* Fix `ktlint` issues
* Group up common variables in `codegenScope`
* Document `RustType.Application`
* Format `codegen-server-test/python/model/pokemon.smithy`
* Use `tokio-stream` crate instead of `futures` crate
* Use a better error message if user tries to reuse a stream
* Add some details about event streams to example Pokemon service
---------
Co-authored-by: Matteo Bigoi <1781140+crisidev@users.noreply.github.com>
* Create the `aws-runtime` and `aws-runtime-api` crates
* Implement SigV4 in the orchestrator
* CI fixes
* Connect the service config http connector to the orchestrator
* Make it possible to register interceptors, and get signing to pass tests
* Fix allowed types lint
* Add missing docs.rs metadata
No implementation of the `Protocol` interface makes use of the
`OperationShape` parameter in the `structuredDataParser` and
`structuredDataSerializer` methods.
* add: stubs to make sra test fully runnable (but not useful 😅)
* fix: disable build.gradle enableNewSmithyRuntime option
* fix: clippy lints
add: feature gate for test connection
* fix: doc comment
* move: connections module from API to runtime
* update: TODO message
add: exceptions for external types
* Fix external types lint
---------
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Add CI check for deterministic codegen
* Fix typo
* Mark files as executable
* Fix argument count from 1 to 0
* Fix diff script
* Fix error in generate_and_commit function
* Cleanup command output
* Fix codegen-diff script
* Sort member params to fix codegen non-determinism
* Convert interceptor fns to macro invocations
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Add unmodeled error to ServiceError and bring in EventLog
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Simplify and test `EventLog`
* Attempt to integrate the event log with the orchestrator
* fix: error type in integration test
update: retry handling in orchestrator
* update: set the runtime plugins to do nothing instead of panic when called on
* Introduce a construct for type erasure
* Eliminate several generics and add phased error handling
* Code generate the `ResponseDeserializer` trait impls
* CI fixes
* Reorganize the new runtime crates
* Create the `Identity` type
* Reorganize orchestrator traits and accessors
* Set up code generated test environment for the new runtime
* Add initial auth orchestration implementation
* Fix clippy lint
* Incorporate feedback
* Fix external types lint
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Upgrade to Smithy 1.27.1
* Fix bugs exposed by 1.27.1
* Fix rules engine customization
* update rfc3339 parsing test
* refactor timestamp format & remove upstreamed test
* Update changelog
* rename test
* fix datetime parsing on server and clients
* fix up a few more failing tests
* precommit lints
* Fix bad merge
* Add Connection Poisoning to aws-smithy-client
* Fix doc links
* Remove required tokio dependency from aws-smithy-client
* Remove external type exposed
* Rename, re-add tokio dependency
* Change IP to 127.0.0.1 to attempt to fix windows
* Add dns::Name to external types
* Remove non_exhaustive not needed
* Add client target to changelog
* Enable the crate reorg for the SDK
* Fix `aws-config`
* Fix SDK integration tests
* Fix event stream error module bug
* Fix doc test imports
* Fix some tests
* Fix external type check
* Fix the canary for both versions
* Fix codegen crash for certain SDK models
* Fix some doc compilation failures
* Update the changelog
* Fix presigning test merge conflict
* Remove `toEnumVariantName` from `RustSymbolProvider`
The `toEnumVariantName` function existed on symbol provider to work
around enum definitions not being shapes. In the future when we refactor
to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s
for each enum member. This change incrementally moves us to that future
by creating fake `MemberShape`s in the enum generator from the enum
definition.
* Fix escaping of `Self` in symbol providers
* Clean up an old hack
* Make `RustReservedWordsSymbolProvider` configurable
* Update changelog
* Incorporate feedback
* Add more client re-exports
This commit adds more client re-exports proposed in
https://github.com/awslabs/smithy-rs/issues/1759.
Specifically, it re-exports the following types:
- `aws_smithy_http::body::SdkBody`
- `aws_smithy_http::byte_stream::error::Error`
- `aws_smithy_http::operation::{Request, Response}`
* Update CHANGELOG.next.toml
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
* Split fluent client functions across multiple files
* Add (commented) test for large generated files to CI
* Split serialization/deserialization generated code across multiple files
* Remove extraneous newline from fluent client doc comments
* Add doc comments to `ProtocolFunctions`
* Simplify some doc comment generation
* Improve some function names
* Add hook for CodegenDecorators to insert a custom symbol provider
* rename {extraS=>s}ymbolProvider
* fix rebase conflict that led to invalid code
* Remove model parameter from CoreCodegenDecorator::symbolProvider
---------
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use `clientIntegrationTest` (WIP)
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` with `clientIntegrationTest`
* Refactor `ClientEventStreamUnmarshallerGeneratorTest` to use generic test cases
* Start refactoring `ServerEventStreamUnmarshallerGeneratorTest`
* Make `ServerEventStreamUnmarshallerGeneratorTest` tests work
* Uncomment other test models
* Allow unused on `parse_generic_error`
* Rename `ServerEventStreamUnmarshallerGeneratorTest`
* Make `EventStreamUnmarshallTestCases` codegenTarget-agnostic
* Refactor `ClientEventStreamMarshallerGeneratorTest`: Tests run but fail
* Refactor `ServerEventStreamMarshallerGeneratorTest`
* Move `.into()` calls to `conditionalBuilderInput`
* Add "context" to TODO
* Fix client unmarshall tests
* Fix clippy lint
* Fix more clippy lints
* Add docs for `event_stream_serde` module
* Fix client tests
* Remove `#[allow(missing_docs)]` from event stream module
* Remove unused `EventStreamTestTools`
* Add `smithy-validation-model` test dep to `codegen-client`
* Temporarily add docs to make tests compile
* Undo change in model
* Make event stream unmarshaller tests a unit test
* Remove unused code
* Make `ServerEventStreamUnmarshallerGeneratorTest` a unit test
* Make `ServerEventStreamMarshallerGeneratorTest` a unit test
* Make `ServerEventStreamMarshallerGeneratorTest` pass
* Make remaining tests non-integration tests
* Make event stream serde module private again
* Remove unnecessary clippy allowances
* Remove clippy allowance
* Remove docs for `event_stream_serde` module
* Remove docs for `$unmarshallerTypeName::new`
* Remove more unnecessary docs
* Remove more superfluous docs
* Undo unnecessary diffs
* Uncomment last test
* Make `conditionalBuilderInput` internal
* Add builder symbol/module resolution to symbol providers
* Back out breaking changes
* Reduce codegen diff
* Fix server example clippy lints
* Associate #2396 with TODO comments
* Improve doc comment
* Revert doc hidden change
* Conditionally re-export the Smithy Client Builder in clients
* Reorganize error type re-export
* Reorganize the `paginator` module
* Flatten the `presigning` module
* Hide the `http_body_checksum` module
The `toEnumVariantName` function existed on symbol provider to work
around enum definitions not being shapes. In the future when we refactor
to use `EnumShape` instead of `EnumTrait`, there will be `MemberShape`s
for each enum member. This change incrementally moves us to that future
by creating fake `MemberShape`s in the enum generator from the enum
definition.
* Add support for the awsQueryCompatible trait
This commit adds support for the awsQueryCompatible trait. This allows
services already supporting custom error codes through the AWS Query
protocol with the awsQueryError trait to continue supporting them after
the services switch to the AWS JSON 1.0 protocol.
* Add copyright header
* Fix clippy warning for clippy::manual-map
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Remove unused variables from `errorScope`
This commit addresses https://github.com/awslabs/smithy-rs/pull/2398#discussion_r1114763528
* Reorder arguments for test verification
This commit addresses https://github.com/awslabs/smithy-rs/pull/2398#discussion_r1114766817
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
* Upgrade Kotlin to 1.7.21
* Upgrade Ktlint to 0.48.2
* Run `pre-commit run --all-files` to fix broken Ktlints
* Fix string comparison broken by code formatting
* Constraint member types are refactored as standalone shapes.
* ModelModule to ServerRustModule.model
* Constraints are written to the correct module
* Code generates for non-public constrained types.
* Removed a comment
* Using ConcurrentHashmap just to be on the safe side
* Clippy warnings removed on constraints, k.into() if gated
* Wordings for some of the checks changed
* Test need to call rustCrate.renderInlineMemoryModules
* ktlintFormat related changes
* RustCrate need to be passed for server builder
* Param renamed in getParentAndInlineModuleForConstrainedMember
* pubCrate to publicConstrainedType rename
* PythonServer symbol builder needed to pass publicConstrainedTypes
* @required still remains on the member shape after transformation
* ConcurrentLinkedQueue used for root RustWriters
* runTestCase does not run the tests but just sets them up, hence has been renamed
* CHANGELOG added
---------
Co-authored-by: Fahad Zubair <fahadzub@amazon.com>
* feat(codegen): support for api key auth trait
* chore: update to new codegen decorator interface
* chore: include basic test
* chore: set api key into rest xml extras model
* chore: update test
* chore: refactor api key definition map
* feat(codegen): add api key decorator by default
* chore: add smithy-http-auth to runtime type
* chore: reference new smithy-http-auth crate
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Revert "chore: set api key into rest xml extras model"
This reverts commit 93b99c8703.
* chore: moved api key re-export to extras customization
* chore: include test for auth in query and header
* chore: fix linting
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/main/kotlin/software/amazon/smithy/rust/codegen/client/smithy/customizations/ApiKeyAuthDecorator.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/test/kotlin/software/amazon/smithy/rust/codegen/client/customizations/ApiKeyAuthDecoratorTest.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Update codegen-client/src/test/kotlin/software/amazon/smithy/rust/codegen/client/customizations/ApiKeyAuthDecoratorTest.kt
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* chore: add doc hidden to re-export
* chore: ensure extras are added only if it applies
* Revert "chore: add doc hidden to re-export"
This reverts commit 8a49e2b47b.
---------
Co-authored-by: Eduardo Rodrigues <eduardomourar@users.noreply.github.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: John DiSanti <johndisanti@gmail.com>
* Fix recursive constraint violations with paths over list and map shapes
There is a widespread assumption throughout the generation of constraint
violations that does not hold true all the time, namely, that a
recursive constraint violation graph has the same requirements with
regards to boxing as the regular shape graph.
Some types corresponding to recursive shapes are boxed to introduce
indirection and thus not generate an infinitely recursive type. The
algorithm however does not superfluously introduce boxes when the cycle
goes through a list shape or a map shape. Why list shapes and map
shapes? List shapes and map shapes get rendered in Rust as `Vec<T>` and
`HashMap<K, V>`, respectively, they're the only Smithy shapes that
"organically" introduce indirection (via a pointer to the heap) in the
recursive path. For other recursive paths, we thus have to introduce the
indirection artificially ourselves using `Box`. This is done in the
`RecursiveShapeBoxer` model transform.
However, the constraint violation graph needs to box types in recursive
paths more often. Since we don't collect constraint violations
(yet, see #2040), the constraint violation graph never holds
`Vec<T>`s or `HashMap<K, V>`s, only simple types. Indeed, the following simple
recursive model:
```smithy
union Recursive {
list: List
}
@length(min: 69)
list List {
member: Recursive
}
```
has a cycle that goes through a list shape, so no shapes in it need
boxing in the regular shape graph. However, the constraint violation
graph is infinitely recursive if we don't introduce boxing somewhere:
```rust
pub mod model {
pub mod list {
pub enum ConstraintViolation {
Length(usize),
Member(
usize,
crate::model::recursive::ConstraintViolation,
),
}
}
pub mod recursive {
pub enum ConstraintViolation {
List(crate::model::list::ConstraintViolation),
}
}
}
```
This commit fixes things by making the `RecursiveShapeBoxer` model
transform configurable so that the "cycles through lists and maps
introduce indirection" assumption can be lifted. This allows a server
model transform, `RecursiveConstraintViolationBoxer`, to tag member
shapes along recursive paths with a new trait,
`ConstraintViolationRustBoxTrait`, that the constraint violation type
generation then utilizes to ensure that no infinitely recursive
constraint violation types get generated.
For example, for the above model, the generated Rust code would now look
like:
```rust
pub mod model {
pub mod list {
pub enum ConstraintViolation {
Length(usize),
Member(
usize,
std::boxed::Box(crate::model::recursive::ConstraintViolation),
),
}
}
pub mod recursive {
pub enum ConstraintViolation {
List(crate::model::list::ConstraintViolation),
}
}
}
```
Likewise, places where constraint violations are handled (like where
unconstrained types are converted to constrained types) have been
updated to account for the scenario where they now are or need to be
boxed.
Parametrized tests have been added to exhaustively test combinations of
models exercising recursive paths going through (sparse and non-sparse)
list and map shapes, as well as union and structure shapes
(`RecursiveConstraintViolationsTest`). These tests even assert that the
specific member shapes along the cycles are tagged as expected
(`RecursiveConstraintViolationBoxerTest`).
* Address comments
* Add `RequestId` trait
* Implement `RequestId` for generated AWS client errors
* Move `RustWriter.implBlock` out of `StructureGenerator`
* Create structure/builder customization hooks
* Customize `_request_id` into AWS outputs
* Set request ID on outputs
* Refactor SDK service decorators
* Refactor S3's extended request ID implementation
* Combine `Error` and `ErrorKind`
* Add test for service error conversion
* Move error generators into `codegen-client` and fix tests
* Re-export `ErrorMetadata`
* Add request IDs to trace logs
* Simplify some error trait handling
* Rename `ClientContextParamDecorator` to `ClientContextConfigCustomization`
* Add deprecated alias to guide customers through upgrading
* Rename the `ErrorMetadata` trait to `ProvideErrorMetadata`
* Rename `aws_smithy_types::Error` to `ErrorMetadata`
* Refactor modules to be configurable in `codegen-core`
* Remove panicking default test symbol provider
* Remove as many references to Error/Types as possible
* Rename module constants
This allows "easily" converting to other custom validation exception
shapes by implementing a decorator.
As a simple example, this PR adds a
`CustomValidationExceptionWithReasonDecorator` decorator that is able to
convert into a shape that is very similar to
`smithy.framework#ValidationException`, but that has an additional
`reason` field. The decorator can be enabled via the newly added
`experimentalCustomValidationExceptionWithReasonPleaseDoNotUse` codegen
config flag.
This effectively provides a way for users to use custom validation
exceptions without having to wait for the full implementation of #2053,
provided they're interested enough to write a decorator in a JVM
language. This mechanism is _experimental_ and will be removed once full
support for custom validation exceptions as described in #2053 lands,
hence why the configuration key is strongly worded in this respect.
This commit also ports the mechanism to run codegen integration tests
within Kotlin unit tests for client SDKs to the server. See #1956 for
details. The custom validation exception decorator is tested this way.
Rename `RustCodegenServerPlugin` to `RustServerCodegenPlugin`, for
consistency with `RustClientCodegenPlugin`. This is a better name, since
the plugin is named `rust-server-codegen`.
This commit also renames `PythonCodegenServerPlugin` to
`RustServerCodegenPythonPlugin` for the same reasons.
This commit also contains other drive-by improvements made while working
on #2302.
* Prevent test dependencies from leaking into production
* refactor & fix tests
* fix tests take two
* fix more tests
* Fix missed called to mergeDependencyFeatures
* Add test
* fix glacier compilation
* fix more tests
* fix one more test
* Fix broken doc link to `Stream`
This commit fixes broken doc link to `Stream` in codegen clients. That
target is `tokio_stream::Stream`, which in turn is a re-export of
`futures_core::Stream`. Since we do not have a good way to link to the
re-export, we remove a hyper link and just put `Stream`.
* Update CHANGELOG.next.toml
---------
Co-authored-by: Yuki Saito <awsaito@amazon.com>
In server SDKs, these traits can be implemented by any shape _except_ if
the shape's closure contains:
1. A `float`, `double`, or `document` shape: floating point types in
Rust do not implement `Eq`. Similarly, [`document` shapes] may
contain arbitrary JSON-like data containing floating point values.
2. A [@streaming] shape: all the streaming data would need to be
buffered first to compare it.
Additionally, the `Hash` trait cannot be implemented by shapes whose
closure contains:
1. A `map` shape: we render `map` shapes as `std::collections::HashMap`,
which _do not_ implement `Hash`. See
https://github.com/awslabs/smithy/issues/1567.
In **client SDKs, these traits cannot be derived on any code-generated
Rust types corresponding to Smithy shapes**, since e.g. adding new
optional members to a structure [is a backwards-compatible change], and
doing so alters the semantics of these traits.
However, this commit does implement these traits for the
`aws_smithy_types::date_time::DateTime` and `aws_smithy_types::Blob`
runtime types.
This change is necessary to efficiently implement the `@uniqueItems`
constraint trait in server SDKs.
[`Eq`]: https://doc.rust-lang.org/std/cmp/trait.Eq.html
[`Hash`]: https://doc.rust-lang.org/std/hash/trait.Hash.html
[`document` shapes]: https://smithy.io/2.0/spec/simple-types.html#document
[@streaming]: https://smithy.io/2.0/spec/streaming.html
[is a backwards-compatible change]: https://smithy.io/2.0/guides/evolving-models.html#updating-structures
* Add support for operationInput tests
* More unfication, fix tests, docs
* Set endpoint_url only when endpoint_url is used
* Fix test-util feature
* CR feedback
* fix missing path
Most usages are in the server constrained types generators, where
`#[allow(dead_code)]` suffices and makes the intent clearer.
See https://stackoverflow.com/a/64556868
* Add the `aws-credential-types` crate
This commit adds a new crate `aws-credential-types` to the `rust-runtime`
workspace. This lays the groundwork for being able to create a
`LazyCachingCredentialsProvider` outside the `aws-config` crate according
to the proposed solution in https://github.com/awslabs/smithy-rs/pull/2082.
We have moved the following into this new crate:
- Items in aws_types::credentials and and their dependencies
- Items in aws_config::meta::credentials and their dependencies
Finally, the crate comes with auxiliary files that are present in the
other crates in the `rust-runtime` workspace such as `external-types.toml`.
* Make `aws-types` depend on `aws-credential-types`
The credentials module has been moved from the `aws-types` crate to the
`aws-credential-types` crate. This leads to some of the items in the
`aws-types` crate adjusting their use statements to point to
`aws-credential-types`.
The `TimeSource` struct has also been moved to `aws-credential-types`
because it is used by `LazyCachingCredentialsProvider`. We have decided
to move it instead of duplicating it because `aws-config` was creating
a `TimeSource` from `aws-types` and then passing it to the builder for
`LazyCachingCredentialsProvider`. If we had duplicated the implementation
of `TimeSource` in `aws-credential-types`, two `TimeSource` implementations
would have been considered different types and the said use case in
`aws-config` would have been broken.
* Make `aws-config` depend on `aws-credential-types`
The `cache` module and modules in `meta::credentials` (except for `chain`)
have been moved to `aws-credential-types`. Again, the goal of restructuring
is to allow `LazyCachingCredentialsProvider` to be created outside the
`aws-config` crate. While doing so, we try not moving all the default
credential provider implementations.
* Make `aws-http` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `aws-inlineable` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `aws-sig-auth` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Emit `aws-credential-types` to the build directory
This commit adds `aws-credential-types` to AWS_SDK_RUNTIME so that
the build command `/gradlew :aws:sdk:assemble` can generate the crate
into sdk/build/aws-sdk.
* Make codegen aware of `aws-credential-types`
This commit allows the codegen to handle the `aws-credential-types` crate.
The items that have been moved from `aws-types` should now be prefixed with
`aws-credential-types` when generating fully qualified names.
* Make `dynamo-tests` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `s3-tests` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Make `s3control` depend on `aws-credential-types`
This commit adjusts the use statements for the items that have been moved
from the `aws-types` crate to the `aws-credential-types` crate.
* Update external-types.xml in rust-runtime crates
This commit fixes CI failures related to `cargo check-external-types`
in ec994be.
* Update the file permission on additional-ci
* Remove unused dependency from aws-credential-types
* Clean up features for aws-credential-types
This commit fixes a CI failure where the feature hardcoded-credentials
needed other features, aws-smithy-async/rt-tokio and tokio/rt, for the
test code to compile with --no-default-features.
* Update sdk-external-types.toml
* Update aws/rust-runtime/aws-credential-types/Cargo.toml
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/README.md
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/README.md
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/additional-ci
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Update aws/rust-runtime/aws-credential-types/src/lib.rs
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Reduce re-exports from `aws-credential-types`
This commit reduces the number of re-exports from `aws-credential-types`.
The rationale here is that if we add more items to this crate later on,
we may get some name collisions in root. Since this crate is not used by
our customers directly, it is acceptable for items to take a bit of typing
to get to.
* Fix broken intra doc link
This commit fixes a broken intra doc link that went unnoticed because the
offending link was behind the feature `hardcoded-credentials`.
* Introduce type and trait for credential caching
This commit introduces the following items in the `aws-credential-types`
crate:
* CredentialsCache
* ProvideCachedCredentials
`CredentialsCache` is a struct a user will be interacting with when creating
a credentials cache; the user no longer creates a concrete credentials cache
directly, and instead it is taken care of by `CredentialsCache` behind the
scene.
`ProvideCachedCredentials` is a trait that will be implemented by concrete
credentials caches.
Furthermore, this commit renames the following structs according to the RFC
https://github.com/awslabs/smithy-rs/pull/1842:
* SharedCredentialsProvider -> SharedCredentialsCache
* LazyCachingCredentialsProvider -> LazyCredentialsCache
* Add `credentials_cache` to `SdkConfig` and to builder
This commit adds a new field `credentials_cache` to `SdkConfig`. It also
adds a new method `credentials_cache` to `sdk_config::Builder`. They
will help a `CredentialsCache` be threaded through from `ConfigLoader`
to a service specific client's `config::Builder`, which will be
implemented in a subsequent commit.
* Put `SharedCredentialsCache` into the property bag
This commit updates what goes into the property bag. Now that
`SharedCredentialsProvider` has been renamed to `SharedCredentialsCache`,
that's what goes into the property bag. Once a `SharedCredentialsCache`
is retrieved from the bag, credentials can be obtained by calling
`provide_cached_credentials`.
* Thread through `credentials_cache` to service client
This commit threads through `credentials_cache` to service client's
`Config` and its builder. The builder will be the single-sourced place
for creating a credentials cache.
* Update `aws-config` to use `CredentialsCache`
This commit updates `aws-config` to use `CredentialsCache`. Specifically,
* `ConfigLoader` now has `credentials_cache` to take `CredentialsCache`
* No more `LazyCachingCredentialsProvider` in `DefaultCredentialsChain`
* No more `LazyCachingCredentialsProvider` in `AssumeRoleProvider`
The second and third bullet points are a result of a credentials cache
being composed in a service client's `config::Builder` rather than
`DefaultCredentialsChain` or `AssumeRoleProvider` holding it as its field.
* Update sdk integration tests
This commit bulk updates the integration tests for SDK. Most updates
replace the previous `SharedCredentialsProvider::new` with `Arc::new`.
A more subtle but important change is to respect the `sleep_impl` field
within the build method of a Config builder, making sure to thread it
a default `LazyCredentialsCache` created within the build method. If we
missed this step, the default constructed `LazyCredentialsCache` would
later use the default Tokio sleep impl even during tests that exercise
different async runtime, causing them to fail.
* Update aws/rust-runtime/aws-credential-types/README.md
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* Rename variants of `aws_credential_types::time_source::Inner`
This commit addresses https://github.com/awslabs/smithy-rs/pull/2108#discussion_r1053637722
* Split the unit test for `time_source` into two
This commit addresses https://github.com/awslabs/smithy-rs/pull/2108#discussion_r1053638381
* Update CHANGELOG.next.toml
* Fix test failures in CI coming from `aws-inlineable`
This commit fixes test failures in CI coming from the integration test in
the `aws-inlineable` crate. Commit ea47572 should have included this
change.
* Update external-types TOML files
* Clean up offending use statements left after merging main
* Remove moved module wrongly brought in after merging main
* Remove `credentials_cache` from `Builder` for `DefaultCredentialsChain`
This commit removes a field `credentials_cache` from the `Builder` for
`DefaultCredentialsChain` as it no longer stores `LazyCredentialsCache`.
Furthermore, we have also removed methods on the builder that referred
to the field `credentials_cache`.
After this commit, certain use cases will be broken, i.e. when a user
sets timeout for loading credentials via `load_timeout` on the builder,
the configured timeout will be dropped on the floor because it will not
be threaded through the field `credentials_cache`. We will later provide
instructions for how to update those use cases with our new set of APIs.
* Remove `configure` from `LazyCredentialsCache` builder
This commit removes the `configure` method from the builder for
`LazyCredentialsCache`. We tried our best to keep it when we had moved
the builder from `aws-config` but had to modify the method signature to
destructure `ProviderCondig` to obtain the two fields out of it
(`ProviderConfig` lives in `aws-config` so cannot be passed to the builder
for `LazyCredentialsCache` which lives in `aws-credential-types`).
Given `configure` is technically meant for credentials providers, which
`LazyCredentialsCache` is not anymore, we might as well remove it and
accomplish the same effect by having customers use both `time_source` and
`set_sleep` on the builder instead.
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
* Use unwrap_or_else to simplify two assignments
This commit addresses https://github.com/awslabs/smithy-rs/pull/2122#discussion_r1056497419
* Add doc link to `CredentialsCache` to builder method
This commit addresses https://github.com/awslabs/smithy-rs/pull/2122#discussion_r1056500448.
In addition, it moves rustdoc for `LazyCredentialsCache` to `LazyBuilder`
as `LazyCredentialsCache` has been made `pub(crate)` from `pub` and
`LazyBuilder` is now a `pub` item instead.
* Make `CredentialsCache` configurable in `AssumeRoleProviderBuilder`
This commit addresses https://github.com/awslabs/smithy-rs/pull/2122#discussion_r1066400431.
It allows users to pass their `CredentialsCache` to the builder just as
they do in `SdkConfig`.
* Update CHANGELOG.next.toml
This commit addresses https://github.com/awslabs/smithy-rs/pull/2122#discussion_r1066400431.
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
* refactor: rust Attribute codegen
* update: post-refactor usage of Attribute in codegen
add: more attribute tests
update: don't render attributes missing required args
* fix: Python Server codegen broken by Attribute refactor
* fix: Python Server codegen broken by extra `:W`
* remove: TODOs
* update: codegen code to use new attribute syntax
* fix: RustMetadata creation in ConstrainedBlobGenerator
* add a test of credentials
* Add tests of S3 customizations
* Add fix from separate PR
* Add FIPS and DualStack
* Fix date issue + other cleanup
* refactor out awsSdkIntegrationTest
* use extendIf in a couple more places
* Add test of string built in
* fix generation of region related code
* CR feedback
* Move the allow lints customization into `codegen-core`
* Move the crate version customization into `codegen-core`
* Move "pub use" extra into `codegen-core`
* Move `EventStreamSymbolProvider` into `codegen-core`
* Move the streaming shape providers into `codegen-core`
* Refactor event stream marshall/unmarshall tests
* Break `codegen-server` dependency on `codegen-client`
* Split up `EventStreamTestTools`
* Move codegen context creation in event stream tests
* Restructure tests so that #1442 is easier to resolve in the future
* Add client/server prefixes to test classes
* Improve TODO comments in server event stream tests
* Use correct builders for `ServerEventStreamMarshallerGeneratorTest`
* Remove test cases for protocols that don't support event streams
* Introduce a flag in gradle.properties to enable/disable testing (and the compilation of the associated dependencies).
* Feature flag testing in all non-test-only build.gradle.kts files.
* Remove dokka
* wip
* Fix region decorator
* Update S3 tests to succeed
* Create 'endpoint_url' setters
* Fix SDK adhoc tests
* Fix endpoint tests
* Fix protocol test generator to have a stub endpoint resolver
* Fix some more tests
* Fix aws rust runtime tests
* Update generator to appease clippy
* CR feedback
* Fix compilation
* Fix tests
* Fix doc links
* Fix SDK integration tests
* Update changelog
* Fix s3 control by adding transformer
* Throw a specific exception if the service doesn't have ep rules
* Add codecatalyst to the list of custom services
This change creates `ClientCodegenDecorator` and
`ServerCodegenDecorator` in `codegen-client` and `codegen-server`
respectively to replace `RustCodegenDecorator`. Client/server
equivalents are created to replace `CombinedCodegenDecorator` as well.
This eliminates the need for the `supportsCodegenContext` method since
the decorator interface is no longer generic, so the `ServiceLoader` is
now powerful enough to differentiate.
The largest benefit, however, is that now clients and servers can have
separate customizations.
* refactor: RuntimeType creation and member resolution
refactor: RuntimeType companion functions
add: AwsCargoDependency object
refactor: move aws runtime crate fns into AwsRuntimeType
update: all code affected by the above changes
* update: split AwsCargoDependency into separate package
rename: runtimeCrate to smithyRuntimeCrate
remove: smithy prefix from RuntimeType
* fix: missing dep issue
update: RuntimeType import
* fix: missing dep issue in server
rename: pascal-case functions to be camel-case
* revert: changes to CodegenTestCommon.kt
* fix: error I introduced in a merge
fix: outdated doc comment
* Emit spans for implementers of map request middleware traits
* Instrument dispatch with its own span
* Fix trace span hierarchy
* Partially flatten the middleware span hierarchy
* Make `MapRequest::name` required
* Add sub-spans to the `load_response` span
* Split endpoint resolution middleware into two parts & refactor endpoint generation
* Endpoints 2.0 Integration pre-work
This PR does a 3 bits of pre-work ahead of ep2 integration:
1. Split endpoint resolution into two separate middlewares:
1. A smithy native middleware that applies URI and headers
2. An AWS middleware that applies the auth schemes
2. Add vendorParams support to the ProtocolTestGenerator so that protocol tests can insert a region.
3. Simplify endpoint resolution logic by allowing `make_operation` to fail when an endpoint cannot be resolved.
* Back out previous change to insert endpoint directly into the bag
* backout changes to property bag
* Update changelog & add more docs
* Fix AWS test
* Fix test
* Make enum forward-compatible
This commit implements the suggested approach described in
https://github.com/awslabs/smithy-rs/issues/627#issuecomment-995923092
The idea is that once the user writes a match expression against an enum
and assumes that an execution path comes to a particular match arm, we
should guarantee that when the user upgrades a version of SDK, the
execution path should come to the same match arm as before.
* Add unit test to ensure enums are forward-compatible
The test first mimics the user's interaction with the enum generated from
a model where the user writes a match expression on the enum, wishing to
hit the match arm corresponding to Variant3, which is not yet supported
by the model. The test then simulates a scenario where the user now has
access to the updated enum generated from the next version of the model
that does support Variant3.
The user's code should compile in both of the use cases.
* Generate rustdoc for enum's forward-compatibility
This commits generates rustdoc explaining to the users how they might
write a match expression against a generated enum so their code is
forward-compatible. While it is explained in
https://github.com/awslabs/smithy-rs/issues/627#issuecomment-995923092,
it can be helpful if the users can read it in rustdoc right below where
the enum's definition is shown.
* Make snippet in rustdoc text
This commit updates a rustdoc code snippet generated for an enum from
`rust,no_run` to `text`. We observed that the snippet fails to compile
in CI because the name of the enum was not fully qualified; code in
rustdoc is treated in a similar way that code in integration tests is
treated as being outside of a crate, but not part of the crate, which
means the name of the crate needs to be spelled out for a `use` statement
if we want to import the enum to the code in rustdoc. However, the name
of the crate is usually one-off, generated on the fly for testing, making
it not obvious to hard-code it or obtain it programmatically from within
Kotlin code.
* Suppress missing doc lint for UnknownVariantValue
This commit marks UnknownVariantValue as `#[allow(missing_docs)]` because
failing to do so will cause tests in CI to fail.
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/EnumGenerator.kt
Co-authored-by: Russell Cohen <rcoh@amazon.com>
* Generate UnknownVariantValue via forInlineFun
This commit attempts to create UnknownVariantValue using
RuntimeType.forInlineFun. Prior to this commit, it was generated per enum
but that caused multiple definitions of UnknownVariantValue in a single
file as there can be multiple enums in the file.
By using RuntimeType.forInlineFun, we can generate a single UnknownVariantValue
per module and the enums within the module can refer to the same instance.
This commit, however, fails to pass the unit tests in EnumGeneratorTest.
The issue seems to be that a RustWriter used in the tests does not end up
calling the finalize method, failing to render inline functions.
* Replace "target == CodegenTarget.CLIENT" with a helper
This commit addresses https://github.com/awslabs/smithy-rs/pull/1945#discussion_r1014390811
but uses an existing helper CodegenTarget.renderUnknownVariant.
* Update EnumGeneratorTests to use TestWorkspace
This commit addresses failures for unit tests in EnumGeneratorTests.
Now that we use RuntimeType.forInlineFcn in EnumGenerator, we need
to ensure that the inline functions should be rendered to a Rust
source file during testing. RustWriter, used in EnumGeneratorTests
prior to this commit, was not capable of doing so. We thus update
EnumGeneratorTests to use TestWorkspace by which we ensure that the
inline functions are rendered during testing.
* Make sure to use the passed-in variable for shapeId
This commit fixes a copy-and-paste error introduced in 712c983. The
function should use the passed-in variable rather than the hard-coded
literal "test#SomeEnum".
* Address https://github.com/awslabs/smithy-rs/pull/1945\#discussion_r1014389852
* Update CHANGELOG.next.toml
* Update CHANGELOG.next.toml
This commit addresses https://github.com/awslabs/smithy-rs/pull/1945#discussion_r1017235315
* Avoid potential name collisions by UnknownVariantValue
This commit addresses https://github.com/awslabs/smithy-rs/pull/1945#discussion_r1017237745.
It changes the module in which the `UnknownVariantValue` is rendered.
Before the commit, it was emitted to the `model` module but that might cause
name collisions in the future when a Smithy model has a shape named
`UnknownVariantValue`. After this commit, we render it to the `types` module.
* Move re-exports from lib.rs to types.rs
This commit moves re-export statements in client crates from `lib.rs` to
`types.rs`. The motivation is that now that we render the struct
`UnknownVariantValue` in a separate file for the `types` module, we can
no longer have a `pub mod types {...}` block in `lib.rs` as it cannot
coexist with `pub mod types;`.
* Add docs on UnknownVariantValue
This commit adds public docs on `UnknownVariantValue`. Since it is
rendered in `types.rs`, the users may not find the `Unknown` variant
of an enum in the same file and may wonder what this struct is for when
seeing it in isolation.
* Update CHANGELOG.next.toml
This commit addresses https://github.com/awslabs/smithy-rs/pull/1945#discussion_r1020414459
* Update the module documentation for types
This commit updates rustdoc for the types module to reflect that fact
that the module now contains not only re-exports but also an auxiliary
struct `UnknownVariantValue`.
* Add extensions to run code block depending on the target
This commit introduces extensions on CodegenTarget that execute the block
of code depending on the target is for client or for server. These
extensions are intended to replace if-else expressions throughout the
codebase explicitly checking whether the target is for client or for server.
We do not update such if-else expressions all at once in this commit.
Rather, we are planning to make incremental changes to update them
opportunistically.
* Respect the sensitive trait on enums
This commit fixes#1745. It allows enums to respect the sensitive trait.
When annotated as such, an enum will not display its data and instead
will show the redacted text upon debug print.
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/EnumGenerator.kt
Co-authored-by: John DiSanti <jdisanti@amazon.com>
* Move extensions into CodegenTarget as methods
This commit addresses https://github.com/awslabs/smithy-rs/pull/1945#discussion_r1021952411.
By moving extensions into the CodegenTarget enum, no separate import is
required to use them.
* Update codegen-core/src/test/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/EnumGeneratorTest.kt
Co-authored-by: david-perez <d@vidp.dev>
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/EnumGenerator.kt
Co-authored-by: david-perez <d@vidp.dev>
* Update codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/smithy/generators/EnumGenerator.kt
Co-authored-by: david-perez <d@vidp.dev>
* Update CHANGELOG.next.toml
* Configure the (in|ex)clusion of the Debug trait for containers per members' sensitive trait (#2029)
* Removes Debug for shape with sensitive trait
This commit centralizes a place where the codegen excludes the Debug
trait if a shape has the sensitive trait. Previously the exclusion was
handled locally in each shape, e.g. StructureGenerator and EnumGenerator.
However, that approach may overlook a certain shape we also need to treat
as such. We now handle the exclusion of the Debug trait in one place,
BaseSymbolMetadataProvider.
* Stop excluding the Debug trait locally
This commit updates EnumGenerator and StructureGenerator based on the
change made to BaseSymbolMetadataProvider in the previous commit.
Now that the exclusion of the Debug trait was centralized, those
classes in question no longer need to do so individually.
* Implement a custom Debug trait in BuilderGenerator
This commit implements a custom Debug trait in BuilderGenerator now that
the derived Debug trait is excluded from BaseSymbolMetadataProvider for
the structure container. The implementation of the custom Debug trait
pretty much follows that of StructureGenerator.
* Implement a custom Debug trait in UnionGenerator
This commit implements a custom Debug trait in BuilderGenerator now that
the derived Debug trait is excluded from BaseSymbolMetadataProvider for
the union container. The implementation of the custom Debug trait pretty
much follows that of EnumGenerator.
* Implement a custom Debug trait in ServerBuilderGenerator
This commit implements a custom Debug trait in ServerBuilderGenerator now
that the derived Debug trait is excluded from BaseSymbolMetadataProvider
for the structure container. The implementation of the custom Debug trait
pretty much follows that of StructureGenerator.
* Add the Copyright header
* Update CHANGELOG.next.toml
* Update Debug impl for UnionGenerator
This commit updates the implementation of a custom Debug trait impl for
UnionGenerator. Turns out that in a Union, a member target can be marked
as sensitive separately outside the Union. Therefore, the implementation
of a custom Debug trait has two cases depending on where the sensitive
trait appears, either it is applied to the whole Union or to a member
target.
* Peek at member sensitivity for Debug trait (in|ex)clusion
This commit addresses https://github.com/awslabs/smithy-rs/pull/2029#discussion_r1034205685.
With this change, structure shapes no longer need to exclude the Debug
trait unconditionally. The upshot is that we may be able to avoid a
custom Debug impl for a structure where the derived Debug will do, i.e.
when there is no sensitive trait either at a container level or at a
member level.
* Remove statement that does not seem to take effect
This commit addresses https://github.com/awslabs/smithy-rs/pull/2029#discussion_r1036288146
* Rename renderDebugImplForUnion -> renderFullyRedactedDebugImpl
This commit addresses https://github.com/awslabs/smithy-rs/pull/2029#discussion_r1036290722
* Rename renderDebugImplForUnionMemberWise -> renderDebugImpl
This commit addresses https://github.com/awslabs/smithy-rs/pull/2029#discussion_r1036291209
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: Saito <awsaito@c889f3b5ddc4.ant.amazon.com>
Co-authored-by: Russell Cohen <rcoh@amazon.com>
Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: John DiSanti <jdisanti@amazon.com>
Co-authored-by: david-perez <d@vidp.dev>
* Add Endpoint Resolver Implementation
This commit adds `EndpointDecorator`, standard libary + tests that can be used to add endpoints 2.0 to the AWS SDK.
It _does not_ actually wire these things up. We'll follow up with a PR that actually integrates everything.
* CR Feedback
* CR feedback II
* Overhaul RustModule system to support nested modules
* Cleanups / CR feedback
* Get server unit tests passing
* Fix compilation
* Remove unused import
* CR Feedback II
* customize isn't actually a reserved word—move it to the reserved member section
Zelda Hessler