Commit Graph

6486 Commits

Author SHA1 Message Date
Eric Anderson 3994b15fea buildscripts: Avoid vswhere crashing Gradle
I'm trying to upgrade to a newer Windows Kokoro image, but the new one
has an old vswhere installed that breaks Gradle. Our old image doesn't
have vswhere at all. If vswhere isn't found, this rename prints some
errors, but the bat script continues executing. So this change is
compatible with both the older and newer image.
2024-02-16 15:12:14 -08:00
Eric Anderson f4cc166f18 xds: Copy data in least request to avoid picker data race
In 0d39bf50 the ReadyPicker was changed holding List<Subchannel> to
List<ChildLbState>, but ChildLbState mutates over time and is not
synchronized. We want the picker to have a snapshot of the data, so copy
the data from ChildLbState instead of using it directly.

Unfortunately the tests depended on the ChildLbState a bit, so we need
to save the EAG only to use it in tests. That's okay for now, but in the
future we'll probably want to remove that unnecessary memory usage.
2024-02-16 13:44:25 -08:00
Eric Anderson a231d80756 Remove semi-circular dependency between core and util
Add the 'fake' dependency to grpc-netty instead of grpc-core.
grpc-okhttp already depends on grpc-util and probably would be fine
without round_robin on Android.

There's not actually a circular dependency, but some tools can't handle
the compile vs runtime distinction. Such tools are broken, but fixes
have been slow and this approach works with no real downfalls.

Works around #10576 #10701
2024-02-16 12:31:54 -08:00
Alex Panchenko 257d1c2db4 servlet: introduce ServletServerBuilder.buildServlet() 2024-02-16 12:21:42 -08:00
Benjamin Peterson a68399a9b6
netty: improve server handling of writes to reset streams (#10258)
* netty: improve server handling of writes to reset streams

A server stream can be reset by the client while server writes are still queued. After the stream is reset, the netty connection will forget the stream object. The `NettyServerHandler` must deal with that situation. `sendResponseHandlers` already had some code to do that. This change standardizes that code and adds it to `sendGrpcFrame`. This fixes a potential bug where a `SendGrpcFrameCommand` with `endOfStream=true` would raise an `AssertionError` if written to a reset stream. (This bug is not currently reachable because `endOfStream=false` for all server `SendGrpcFrameCommand` objects.)

* Do not call into the encoder when we know the stream is gone.
2024-02-16 11:08:09 -08:00
Alex Panchenko 2f025b2b24
servlet: missing @Rule annotation in GrpcServletSmokeTest (#10922) 2024-02-16 10:29:45 -08:00
Larry Safran 67dfe49ae6 Fix compile failure 2024-02-15 21:28:37 -08:00
Terry Wilson eba699ad16
rls: Adding extra debug logs (#10902) 2024-02-15 15:23:36 -08:00
Larry Safran 044749706a
util:MultiChildLoadBalancer cleanup (#10780)
* add final, change method permissions, add javadoc, cleanup unneeded, move updateOverallBalancingState to ClusterManagerLB and make it abstract

* Restructure to eliminate the flags as protected methods

* Move methods around so that the candidates for override are near the top.

* Reorder picker methods lower
2024-02-15 14:12:40 -08:00
John Cormie f20c853c40
Factor out MainThreadScheduledExecutorService. (#10916)
So it can be used by other robolectric tests.
2024-02-15 11:11:43 -08:00
Terry Wilson 08f8ed1bc3 util: MultiChildLoadBalance.shutdown() log to FINE
The log level of this method is set to INFO, which is too spammy. Bring
it down to the FINE level.
2024-02-13 14:12:33 -08:00
Eric Anderson 7787673992 xds: Replace isEquivalentTo with equals in LeastRequest
This is similar to the changes to round robin in dca89b25.
2024-02-13 08:29:31 -08:00
Eric Anderson 7e72413233 okhttp: Move HostnameVerifier tests to TlsTest
Http2OkHttp is now unnecessary, as Http2Test tests OkHttp client to
Netty server. receivedDataForFinishedStream() was the only remaining
unique test and it seems already covered by AbstractInteropTest these
days.
2024-02-12 19:40:33 -08:00
Eric Anderson 92463f62bf interop-testing: Avoid Netty-specific APIs to creating channels
We prefer to test using the stable APIs, as they are what our users
should be using. Http2Client continues using NettyChannelBuilder because
it is intended to test grpc-netty.
2024-02-12 07:52:31 -08:00
Eric Anderson 372a535615 auth: Add support for Retryable interface
Retryable was added in google-auth-library 1.5.3 to make clear the
situations that deserve a retry of the RPC. Upgrading to that caused
problems because of transitive dependency issues syncing into Google so
it was reverted in 369f87be. google-auth-library 1.11.0 changed the
approach to avoid the transitive dependency updates. cl/601545581
upgraded to 1.22.0 inside Google. Bump to that version and swap away
from the imprecise IOException heuristic. go/auth-correct-retry

Fixes #6808
2024-02-09 15:15:18 -08:00
yifeizhuang ff34d51c73
api: stabilize server.getListenerSockets (#10910) 2024-02-09 14:40:54 -08:00
Anirudh Ramachandra 608bb8499c
Mark couple of helper functions in XdsClient as public. (#10871) 2024-02-09 12:53:03 -08:00
Anirudh Ramachandra 52b11c1d08
Expose the getOrCreate method via the InternalSharedXdsClientPoolProvider. This is needed for internal users to both set the bootstrap and interact with the XdsClient via the shared object pool (#10872) 2024-02-09 12:50:26 -08:00
Eric Anderson d568a8dc19 kokoro: Avoid using Kokoro's env_vars setting
It makes it hard to run ephemeral jobs, and we don't actually need it
here because we aren't changing the setting with manual runs.
2024-02-09 12:47:16 -08:00
Larry Safran 6847219657
Start 1.63.0 development cycle (#10906) 2024-02-08 13:11:38 -08:00
Eric Anderson 95b847e799 interop-testing: Use separate event loops in RetryTest
The RetryTest was flaky, and it seems to have been caused by the client
and server getting assigned to the same event loop. Separating the two
reduces the flake rate from ~3% to less than 0.1% (no flakes in a 1000).

While I was here fixing the executors, I reduced the number of threads
created and shut down the threads after they are no longer used. This
had no impact to the flake rate (no flakes in 1000).
2024-02-08 07:26:31 -08:00
Larry Safran 7ba0718bb9
Change log level from WARNING to FINER for expected exception (#10899)
* Change log level from WARNING to FINER.  Fixes #10816
2024-02-07 17:31:27 -08:00
Larry Safran 5faab7f47e
Revert "Fix retries that timeout hanging forever. (#10855)" (#10901)
This reverts commit c0a9d315d3.
2024-02-07 15:05:21 -08:00
yifeizhuang 03decafa1f
XdsClient is experimental (#10876) 2024-02-06 17:43:30 -08:00
yifeizhuang f6d9221b65
xds: hide TlsContextManager in XdsResourceType.Args (#10894) 2024-02-06 15:41:24 -08:00
Sergii Tkachenko 4f7ec131ec
xds: Googleapis proto sync to 2023-01-10 (#10896)
Sync googleapis protos to
googleapis/googleapis@114a745b28 for
consistency with Envoy and cncf/xds.

The same version is used in [envoy]
(62e7c59374/api/bazel/repository_locations.bzl (L69))
and [cncf/xds]
(0fa0005c9c/bazel/repository_locations.bzl (L23))
since Jan 2023.

Function-wise, this is a noop.
2024-02-05 18:52:55 -08:00
Sergii Tkachenko 68334a019a
xds: Envoy proto sync to 2024-01-24 (#10895)
`envoyproxy/envoy`: Sync protos to the latest imported version
147e6b9523
(commit 2024-01-24, cl/604403196).

Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler.
2024-02-05 17:21:51 -08:00
Larry Safran 374dbe9461
Fix flaky retry tests (#10887)
* Reorder tracing and actually closing listener to eliminate test flakiness
* Use real value rather than mock for flaky test
2024-02-05 10:54:55 -08:00
Anirudh Ramachandra 3202370684
Allow users to start watching xDS resources (#10864) 2024-02-01 09:21:00 -08:00
yifeizhuang a97f21b61e
xds: fix NPE in wrr in TF state (#10868) 2024-02-01 09:06:46 -08:00
Larry Safran c0a9d315d3
Fix retries that timeout hanging forever. (#10855)
Fixes #10336
2024-01-31 13:15:35 -08:00
yifeizhuang 8d280c97e3
xds: move filterRegistry and loadBalancerRegistry out of XdsResourceType.Args (#10843) 2024-01-31 09:45:37 -08:00
yifeizhuang 20abea47bc
xds: move tlsContextManager (#10859)
Minor refactor to the tlsContextManager to not expose itself on the xdsClientImpl constructor.
This is to allow people who plugins xdsTransportFactory to use the API easily.
2024-01-30 12:59:33 -08:00
Mateus Azis c2a3792a04
binder: Create a new RobolectricBinderSecurityTest (#10835)
Exercises async security policies codepaths.
2024-01-30 11:36:53 -08:00
Larry Safran ac428c14fd
Relax some timeouts that have been flaky for the automated CI testing. (#10858)
* Relax some timeouts that have been flaky for the automated CI testing.
2024-01-29 17:43:21 -08:00
Larry Safran ae5fdd5faa
Add check for empty addresses and if not there disable ipV6. (#10848)
* Add check for empty addresses and if not there dissable ipV6.
2024-01-29 17:20:11 -08:00
Larry Safran 7f4c16e068
internal:Happy eyeballs (#10731)
* implement happy eyeballs

---------

Co-authored-by: tonyjongyoonan <tonyjan@google.com>
2024-01-29 12:50:58 -08:00
yifeizhuang 8e1cc943b0
xds: change controlPlaneClient and loadReportClient to use xdsTransportFactory (#10829) 2024-01-25 17:05:12 -08:00
Anirudh Ramachandra 6d96e6588e
Allow users outside of io.grpc.xds package to create custom xDS resources (#10834)
Currently few of the interfaces needed to define and start a watch for a xDS resource are package private, which can't be used externally outside of io.grpc.xds. Exposing them outside allows users to define their own custom resources and start a watch along with the default supported resources.

Also as part of this change, move an Exception defined in the XdsClientImpl into XdsResourceType. As XdsClientImpl is an implementation package, it makes more sense to expose it via the XdsResourceType class.
2024-01-24 19:48:52 -08:00
Mateus Azis 3e8e56feea
Make AsyncSecurityPolicy.checkAuthorizationAsync public. (#10836)
I initially omitted the visibility modifier because this class began as an interface. Since it moved to an abstract class, we must make it public so it can be overriden by subclasses in the integrator's packages.

Part of #10566.
2024-01-22 15:54:48 -08:00
Mateus Azis 9b9e53359f
Add unit test coverage for PendingAuthListener. (#10806)
Part of #10566.
2024-01-22 15:06:16 -08:00
John Cormie 09acf2f6b7
transports can send a msg larger than their flow control window (#10842) 2024-01-22 10:30:32 -08:00
Terry Wilson c9db8fa4fe
buildscripts: Break up android-interop gradle builds (#10839)
Splits the :grpc-android-interop-testing:assembleDebug and
:grpc-android-interop-testing:assembleDebugAndroidTest build
targets with hopes of avoiding OOMs.
2024-01-22 09:05:46 -08:00
John Cormie fdd6f5001f
binder: fix up BinderClientTransportTest (#10822)
- Multiple test cases assumed all messages would arrive on a single MessageProducer but this isn't guaranteed by the API contract.
- testBadTransactionStreamThroughput_b163053382 was writing `serverCallsCompleted` on one thread and reading it on another without synchronization. A deeper problem was that waiting for the call to complete on the server doesn't mean messages are  immediately available on the client.
- Replaced 100ms polling loops with wait()/notifyAll()
- Close() InputStreams that we read as required by the MessageProducer#next contract.

Passes with --runs_per_test=1000
2024-01-19 17:47:08 -08:00
Eric Anderson 65a6b3bc21 java_grpc_library.bzl: Support runfiles for protoc and the plugin
To support runfiles, the rule has to track more than just the
executable. `files_to_run` has both the runfile and executable
information (as separate fields), as does `files`, (combined as depset).
So using those when able is inherently "safe." `files_to_run.executable`
is only the executable, so does not propagate dependency information,
so we make sure to pass `files` to the rule in addition.
(`files_to_run.executable` is formatted into a string, so it wouldn't
carry depset information anyway.)

As originally noticed in cl/597962426
2024-01-19 17:02:55 -08:00
Sergii Tkachenko 75492c8b36
services: Remove deprecated `io.grpc.services.BinaryLogs` (#10832)
* services: Remove deprecated `io.grpc.services.BinaryLogs`
* services: Remove //services:binarylog bazel target

Was deprecated more than 2 years ago in bab1fe3.
`io.grpc.protobuf.services.BinaryLogs` should be used instead.

Experimental tracking ticket:
https://github.com/grpc/grpc-java/issues/4017
2024-01-18 11:29:19 -08:00
yifeizhuang 005f8c006d
xds: add xdsTransportFactory interface (#10827)
This interface will be used to replace XdsChannelTransport.
2024-01-16 17:41:24 -08:00
Larry Safran a151099820
change since javadoc in ClientTransportFilter.java (#10817)
Change from 1.61 to 1.62 since it wasn't released in 1.61
2024-01-16 09:54:56 -08:00
Sergii Tkachenko be74bbdfe0
Bump gradle from 8.3 to 8.5 (#10791)
`./gradlew wrapper --gradle-version=8.5`

Changelog:
- [Gradle 8.4 Release Notes](https://docs.gradle.org/8.4/release-notes.html)
- [Gradle 8.5 Release Notes](https://docs.gradle.org/8.5/release-notes.html)
2024-01-12 16:01:48 -08:00
Sergii Tkachenko 1e69c712f7
Update README etc to reference 1.61.0 (#10821)
* Update README etc to reference 1.61.0
* Update README protoc references to 3.25.1
2024-01-12 11:48:14 -08:00