Commit Graph

6473 Commits

Author SHA1 Message Date
Eric Anderson 92463f62bf interop-testing: Avoid Netty-specific APIs to creating channels
We prefer to test using the stable APIs, as they are what our users
should be using. Http2Client continues using NettyChannelBuilder because
it is intended to test grpc-netty.
2024-02-12 07:52:31 -08:00
Eric Anderson 372a535615 auth: Add support for Retryable interface
Retryable was added in google-auth-library 1.5.3 to make clear the
situations that deserve a retry of the RPC. Upgrading to that caused
problems because of transitive dependency issues syncing into Google so
it was reverted in 369f87be. google-auth-library 1.11.0 changed the
approach to avoid the transitive dependency updates. cl/601545581
upgraded to 1.22.0 inside Google. Bump to that version and swap away
from the imprecise IOException heuristic. go/auth-correct-retry

Fixes #6808
2024-02-09 15:15:18 -08:00
yifeizhuang ff34d51c73
api: stabilize server.getListenerSockets (#10910) 2024-02-09 14:40:54 -08:00
Anirudh Ramachandra 608bb8499c
Mark couple of helper functions in XdsClient as public. (#10871) 2024-02-09 12:53:03 -08:00
Anirudh Ramachandra 52b11c1d08
Expose the getOrCreate method via the InternalSharedXdsClientPoolProvider. This is needed for internal users to both set the bootstrap and interact with the XdsClient via the shared object pool (#10872) 2024-02-09 12:50:26 -08:00
Eric Anderson d568a8dc19 kokoro: Avoid using Kokoro's env_vars setting
It makes it hard to run ephemeral jobs, and we don't actually need it
here because we aren't changing the setting with manual runs.
2024-02-09 12:47:16 -08:00
Larry Safran 6847219657
Start 1.63.0 development cycle (#10906) 2024-02-08 13:11:38 -08:00
Eric Anderson 95b847e799 interop-testing: Use separate event loops in RetryTest
The RetryTest was flaky, and it seems to have been caused by the client
and server getting assigned to the same event loop. Separating the two
reduces the flake rate from ~3% to less than 0.1% (no flakes in a 1000).

While I was here fixing the executors, I reduced the number of threads
created and shut down the threads after they are no longer used. This
had no impact to the flake rate (no flakes in 1000).
2024-02-08 07:26:31 -08:00
Larry Safran 7ba0718bb9
Change log level from WARNING to FINER for expected exception (#10899)
* Change log level from WARNING to FINER.  Fixes #10816
2024-02-07 17:31:27 -08:00
Larry Safran 5faab7f47e
Revert "Fix retries that timeout hanging forever. (#10855)" (#10901)
This reverts commit c0a9d315d3.
2024-02-07 15:05:21 -08:00
yifeizhuang 03decafa1f
XdsClient is experimental (#10876) 2024-02-06 17:43:30 -08:00
yifeizhuang f6d9221b65
xds: hide TlsContextManager in XdsResourceType.Args (#10894) 2024-02-06 15:41:24 -08:00
Sergii Tkachenko 4f7ec131ec
xds: Googleapis proto sync to 2023-01-10 (#10896)
Sync googleapis protos to
googleapis/googleapis@114a745b28 for
consistency with Envoy and cncf/xds.

The same version is used in [envoy]
(62e7c59374/api/bazel/repository_locations.bzl (L69))
and [cncf/xds]
(0fa0005c9c/bazel/repository_locations.bzl (L23))
since Jan 2023.

Function-wise, this is a noop.
2024-02-05 18:52:55 -08:00
Sergii Tkachenko 68334a019a
xds: Envoy proto sync to 2024-01-24 (#10895)
`envoyproxy/envoy`: Sync protos to the latest imported version
147e6b9523
(commit 2024-01-24, cl/604403196).

Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler.
2024-02-05 17:21:51 -08:00
Larry Safran 374dbe9461
Fix flaky retry tests (#10887)
* Reorder tracing and actually closing listener to eliminate test flakiness
* Use real value rather than mock for flaky test
2024-02-05 10:54:55 -08:00
Anirudh Ramachandra 3202370684
Allow users to start watching xDS resources (#10864) 2024-02-01 09:21:00 -08:00
yifeizhuang a97f21b61e
xds: fix NPE in wrr in TF state (#10868) 2024-02-01 09:06:46 -08:00
Larry Safran c0a9d315d3
Fix retries that timeout hanging forever. (#10855)
Fixes #10336
2024-01-31 13:15:35 -08:00
yifeizhuang 8d280c97e3
xds: move filterRegistry and loadBalancerRegistry out of XdsResourceType.Args (#10843) 2024-01-31 09:45:37 -08:00
yifeizhuang 20abea47bc
xds: move tlsContextManager (#10859)
Minor refactor to the tlsContextManager to not expose itself on the xdsClientImpl constructor.
This is to allow people who plugins xdsTransportFactory to use the API easily.
2024-01-30 12:59:33 -08:00
Mateus Azis c2a3792a04
binder: Create a new RobolectricBinderSecurityTest (#10835)
Exercises async security policies codepaths.
2024-01-30 11:36:53 -08:00
Larry Safran ac428c14fd
Relax some timeouts that have been flaky for the automated CI testing. (#10858)
* Relax some timeouts that have been flaky for the automated CI testing.
2024-01-29 17:43:21 -08:00
Larry Safran ae5fdd5faa
Add check for empty addresses and if not there disable ipV6. (#10848)
* Add check for empty addresses and if not there dissable ipV6.
2024-01-29 17:20:11 -08:00
Larry Safran 7f4c16e068
internal:Happy eyeballs (#10731)
* implement happy eyeballs

---------

Co-authored-by: tonyjongyoonan <tonyjan@google.com>
2024-01-29 12:50:58 -08:00
yifeizhuang 8e1cc943b0
xds: change controlPlaneClient and loadReportClient to use xdsTransportFactory (#10829) 2024-01-25 17:05:12 -08:00
Anirudh Ramachandra 6d96e6588e
Allow users outside of io.grpc.xds package to create custom xDS resources (#10834)
Currently few of the interfaces needed to define and start a watch for a xDS resource are package private, which can't be used externally outside of io.grpc.xds. Exposing them outside allows users to define their own custom resources and start a watch along with the default supported resources.

Also as part of this change, move an Exception defined in the XdsClientImpl into XdsResourceType. As XdsClientImpl is an implementation package, it makes more sense to expose it via the XdsResourceType class.
2024-01-24 19:48:52 -08:00
Mateus Azis 3e8e56feea
Make AsyncSecurityPolicy.checkAuthorizationAsync public. (#10836)
I initially omitted the visibility modifier because this class began as an interface. Since it moved to an abstract class, we must make it public so it can be overriden by subclasses in the integrator's packages.

Part of #10566.
2024-01-22 15:54:48 -08:00
Mateus Azis 9b9e53359f
Add unit test coverage for PendingAuthListener. (#10806)
Part of #10566.
2024-01-22 15:06:16 -08:00
John Cormie 09acf2f6b7
transports can send a msg larger than their flow control window (#10842) 2024-01-22 10:30:32 -08:00
Terry Wilson c9db8fa4fe
buildscripts: Break up android-interop gradle builds (#10839)
Splits the :grpc-android-interop-testing:assembleDebug and
:grpc-android-interop-testing:assembleDebugAndroidTest build
targets with hopes of avoiding OOMs.
2024-01-22 09:05:46 -08:00
John Cormie fdd6f5001f
binder: fix up BinderClientTransportTest (#10822)
- Multiple test cases assumed all messages would arrive on a single MessageProducer but this isn't guaranteed by the API contract.
- testBadTransactionStreamThroughput_b163053382 was writing `serverCallsCompleted` on one thread and reading it on another without synchronization. A deeper problem was that waiting for the call to complete on the server doesn't mean messages are  immediately available on the client.
- Replaced 100ms polling loops with wait()/notifyAll()
- Close() InputStreams that we read as required by the MessageProducer#next contract.

Passes with --runs_per_test=1000
2024-01-19 17:47:08 -08:00
Eric Anderson 65a6b3bc21 java_grpc_library.bzl: Support runfiles for protoc and the plugin
To support runfiles, the rule has to track more than just the
executable. `files_to_run` has both the runfile and executable
information (as separate fields), as does `files`, (combined as depset).
So using those when able is inherently "safe." `files_to_run.executable`
is only the executable, so does not propagate dependency information,
so we make sure to pass `files` to the rule in addition.
(`files_to_run.executable` is formatted into a string, so it wouldn't
carry depset information anyway.)

As originally noticed in cl/597962426
2024-01-19 17:02:55 -08:00
Sergii Tkachenko 75492c8b36
services: Remove deprecated `io.grpc.services.BinaryLogs` (#10832)
* services: Remove deprecated `io.grpc.services.BinaryLogs`
* services: Remove //services:binarylog bazel target

Was deprecated more than 2 years ago in bab1fe3.
`io.grpc.protobuf.services.BinaryLogs` should be used instead.

Experimental tracking ticket:
https://github.com/grpc/grpc-java/issues/4017
2024-01-18 11:29:19 -08:00
yifeizhuang 005f8c006d
xds: add xdsTransportFactory interface (#10827)
This interface will be used to replace XdsChannelTransport.
2024-01-16 17:41:24 -08:00
Larry Safran a151099820
change since javadoc in ClientTransportFilter.java (#10817)
Change from 1.61 to 1.62 since it wasn't released in 1.61
2024-01-16 09:54:56 -08:00
Sergii Tkachenko be74bbdfe0
Bump gradle from 8.3 to 8.5 (#10791)
`./gradlew wrapper --gradle-version=8.5`

Changelog:
- [Gradle 8.4 Release Notes](https://docs.gradle.org/8.4/release-notes.html)
- [Gradle 8.5 Release Notes](https://docs.gradle.org/8.5/release-notes.html)
2024-01-12 16:01:48 -08:00
Sergii Tkachenko 1e69c712f7
Update README etc to reference 1.61.0 (#10821)
* Update README etc to reference 1.61.0
* Update README protoc references to 3.25.1
2024-01-12 11:48:14 -08:00
Eric Anderson 91cfa97c28 netty: Avoid volatile attributes on client-side
6efa9ee3 added `volatile` to `attributes` after TSAN detected a data
race that was added in 91d15ce4. The race was because attributes may be
read from another thread after `transportReady()`, and the
post-filtering assignment occurred after `transportReady()`. The code
now filters the attributes separately so they are updated before calling
`transportReady()`.

Original TSAN failure:
```
  Read of size 4 at 0x0000cd44769c by thread T23:
    #0 io.grpc.netty.NettyClientHandler.getAttributes()Lio/grpc/Attributes; NettyClientHandler.java:327
    #1 io.grpc.netty.NettyClientTransport.getAttributes()Lio/grpc/Attributes; NettyClientTransport.java:363
    #2 io.grpc.netty.NettyClientTransport.newStream(Lio/grpc/MethodDescriptor;Lio/grpc/Metadata;Lio/grpc/CallOptions;[Lio/grpc/ClientStreamTracer;)Lio/grpc/internal/ClientStream; NettyClientTransport.java:183
    #3 io.grpc.internal.MetadataApplierImpl.apply(Lio/grpc/Metadata;)V MetadataApplierImpl.java:74
    #4 io.grpc.auth.GoogleAuthLibraryCallCredentials$1.onSuccess(Ljava/util/Map;)V GoogleAuthLibraryCallCredentials.java:141
    #5 com.google.auth.oauth2.OAuth2Credentials$FutureCallbackToMetadataCallbackAdapter.onSuccess(Lcom/google/auth/oauth2/OAuth2Credentials$OAuthValue;)V OAuth2Credentials.java:534
    #6 com.google.auth.oauth2.OAuth2Credentials$FutureCallbackToMetadataCallbackAdapter.onSuccess(Ljava/lang/Object;)V OAuth2Credentials.java:525
    ...

  Previous write of size 4 at 0x0000cd44769c by thread T24:
    #0 io.grpc.netty.NettyClientHandler$FrameListener.onSettingsRead(Lio/netty/channel/ChannelHandlerContext;Lio/netty/handler/codec/http2/Http2Settings;)V NettyClientHandler.java:920
    #1 io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onSettingsRead(Lio/netty/channel/ChannelHandlerContext;Lio/netty/handler/codec/http2/Http2Settings;)V DefaultHttp2ConnectionDecoder.java:515
    ...
```
2024-01-12 07:29:48 -08:00
Sergii Tkachenko c346b41e22
RELEASING.md: Include author email in shortlog (#10794)
Makes it easier to track down / credit the author.
2024-01-11 11:37:19 -08:00
John Cormie 4352099c7f
binder: BinderTransportTest uses a realistic Executor. (#10803) 2024-01-11 10:49:03 -08:00
Eric Anderson 100d5a55fd xds: Add EC key support
io.grpc.util.CertificateUtils does much of the same thing as xds's
CertificateUtils, but also supports EC keys. The xds code pre-dates the
grpc-util class, so it isn't surprising it wasn't using it.

There's a good number of usages of the xds CertificateUtils, so I just
got rid of the duplicate implementation, but didn't yet bother changing
callers io.grpc.util.
2024-01-10 15:17:36 -08:00
Eric Anderson cb03bd2346 java_grpc_library.bzl: Support Auto Exec Groups
Tested with --incompatible_auto_exec_groups=true with Bazel 7
2024-01-10 13:46:20 -08:00
Terry Wilson 6605649c28
okhttp: Ignore known conscrypt socket close issue (#10811)
This stops an exception from being thrown when a known Conscrypt synchronization issue happens.
2024-01-10 11:01:42 -08:00
Terry Wilson 516f411e02
android-interop-testing: Fix tester activity layout (#10807)
Add missing inputType and lint ignore rule.
2024-01-09 13:37:06 -08:00
Sergii Tkachenko d180e6cfa2
xds: (minor) address Java Code Clarity recommendations in CSDS (#10804)
Redoes https://github.com/grpc/grpc-java/pull/8028
Ref cl/365582959
2024-01-08 12:30:46 -08:00
yifeizhuang 6efa9ee37d
fix tsan data race in attributes (#10805) 2024-01-08 10:54:06 -08:00
Larry Safran 4e163361d4
util:create a test for MultiChildLoadBalancer and Endpoint (#10771)
* Add a test for MultiChildLB doing some basic checking and using multiple addresses for an eag

* Add tests for Endpoint
2024-01-05 17:09:35 -08:00
Larry Safran 4b2e5eddd2
api:Provide a default implementation for new method added to ManagedTranport.Listener to support ClientTransportFilters (#10795)
* Provide a default implementation for new method added to ManagedTransport.Listener to support ClientTransportFilters
* Relax test constraint to reduce flakiness due to timing.
* Add test for listener.filterTransport.
2024-01-04 14:19:13 -08:00
yifeizhuang f01c110f77
fix flaky xds test due to verification race (#10798) 2024-01-04 11:31:53 -08:00
John Cormie 5e07310d06
binder: oneway txns cannot arrive out of order (#10754)
See https://developer.android.com/reference/android/os/IBinder#FLAG_ONEWAY
2024-01-03 22:31:51 -08:00