Commit Graph

6308 Commits

Author SHA1 Message Date
Terry Wilson ce2adcca93
binder: Add missing Android API annotations (#10841) 2024-02-28 08:02:58 -08:00
Eric Anderson 935f8f0afc servlet: Move deps to version catalog
This allows the checkForUpdates task to notice the dependencies and
suggest updates.

I plan to upgrade some of the servers after this change in hopes it
reduces test flakiness.
2024-02-27 21:48:19 -08:00
John Cormie a654d2ebb3
binder: Add fault injection hooksunit tests for BinderClientTransport (#10928) 2024-02-27 14:28:50 -08:00
Eric Anderson af117e9764 core: Don't use real time when comparing deadline in ClientCallImplTest
We can just compare the Deadline instances instead of asserting that
very little time has passed during the test. Real time probably still
matters in the test, but only insofar that the deadline is not expired
by the time ClientCallImpl sees it.

This fixes a test failure seen in the emulated aarch64 CI. Note that the
message says "ns" when it should say "ms", but this change deletes the
code with that typo.

```
java.lang.AssertionError: timeout: 548 ns
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at io.grpc.internal.ClientCallImplTest.assertTimeoutBetween(ClientCallImplTest.java:1102)
	at io.grpc.internal.ClientCallImplTest.contextDeadlineShouldBePropagatedToStream(ClientCallImplTest.java:828)
```
2024-02-27 12:54:58 -08:00
Larry Safran 2c6d6e62dc
Update README to v1.62.2 (#10965) 2024-02-27 12:52:59 -08:00
Eric Anderson 9b53bcaf66 xds: Increase timeouts in XdsClientFederationTest
`isolatedResourceDeletions()` has failed with a timeout waiting on
onChanged when running under TSAN. TSAN can slow things down, so let's
increase the timeout to ensure it isn't just timeout flake.
2024-02-27 11:28:24 -08:00
yifeizhuang c61fe69803
util, dual stack: change address based outlier detection to endpoint based (#10939) 2024-02-27 10:35:59 -08:00
Larry Safran 8087977c0b
Move xds classes for Stubby to xds.client package (#10912)
* Move bootstrap, XdsClient, load reporting, XdsLogger and XdsResourceType to xds.client package.
2024-02-26 16:41:16 -08:00
yifeizhuang 78b3972ff3
xds: fix xdsNameResolver virtual host lookup authority, use service authority instead of ldsResourceName (#10960) 2024-02-26 16:33:20 -08:00
Eric Anderson 0ffcd40ab6 buildscripts: Remove unused Kokoro files for Bazel
Bazel testing has been using GitHub Actions since 445efe74.
2024-02-26 10:03:35 -08:00
Eric Anderson 89d8a13d10 compiler: Support up-to-date checking in golden tests 2024-02-26 10:03:13 -08:00
Eric Anderson 5f8958f65c Use Javadoc's -linkoffline instead of -link
`-link` does I/O to download the package list, for every javadoc
invocation. There is no caching, so this happens many times per build.
Swap to offline mode to avoid spamming the servers, and avoid build
failures if the servers aren't entirely healthy.
2024-02-23 17:22:48 -08:00
Eric Anderson 871aaac0f0 compiler: actually add dependency on golden tests
We had 'test.dependsOn', but it is only run if the golden tasks are
configured, which they generally won't be because nothing depends on
them. This prevented the test from actually running. This bug was
introduced in 0ff9f37b because previously the golden tasks were eagerly
constructed.

Remove the "extraPackage" argument because it is a constant and it
confused me for a bit wondering when it was necessary.
2024-02-23 16:19:35 -08:00
Eric Anderson f768c4222b Remove build usages of Jetty ALPN
It wasn't actually being used. Since Java 8u252 in early 2020 we've been
using ALPN from the JDK. The Jetty ALPN Agent has been a noop.

We do keep the Jetty ALPN support in the code and tests, but we don't
have the infrastructure to actually run it.
2024-02-23 15:27:33 -08:00
Eric Anderson 569956e022 xds: Pre-add fallback to xds client pool accessor
When we implement A71, we're no longer going to have a single xds
client, but instead one per channel target. Add that parameter now, even
though it is unused, to avoid managing the (internal) API breakage when
we implement fallback.
2024-02-23 14:12:46 -08:00
Eric Anderson c7e87d2180 buildscripts: Use Java 11 on Windows CI
This version of Java is old, but not as old as the Java 8 version.
2024-02-23 14:12:11 -08:00
Eric Anderson bfc0f959cd xds: Avoid nonexistent DNS resolution in XdsClientImplV3Test
The DNS lookups are taking considerable time on the Windows CI (~11s),
which causes the test to time out:

```
Wanted but not invoked:
ldsResourceWatcher.onError(<any>);
-> at io.grpc.xds.XdsClientImplTestBase.sendToNonexistentHost(XdsClientImplTestBase.java:3733)
Actually, there were zero interactions with this mock.

	at io.grpc.xds.XdsClientImplTestBase.sendToNonexistentHost(XdsClientImplTestBase.java:3733)
```

The ARM build, which uses an emulator, has had this test succeed, so the
failure seems unrelated to CPU usage. We want to avoid external I/O
anyway during tests, so removing the DNS lookup is good.

The TSAN comment referenced XdsClientImplTestBase.sendToNonexistentHost,
but the test no longer calls fakeClock.forwardTime so the comment was
out-of-date. Change the comment to make clear the race involved.
2024-02-23 11:44:06 -08:00
Eric Anderson d7628a3aba
xds: Fix flow control data race in ControlPlaneClient
As discovered by TSAN, the adsStream field is not synchronized.
```
WARNING: ThreadSanitizer: data race (pid=1625)
  Read of size 4 at 0x00009b66fc88 by thread T23 (mutexes: write M0):
    #0 io.grpc.xds.ControlPlaneClient.isReady()Z ControlPlaneClient.java:203
    #1 io.grpc.xds.ControlPlaneClient.readyHandler()V ControlPlaneClient.java:211
    #2 io.grpc.xds.ControlPlaneClient$AdsStream.onReady()V ControlPlaneClient.java:328
    #3 io.grpc.xds.GrpcXdsTransportFactory$EventHandlerToCallListenerAdapter.onReady()V GrpcXdsTransportFactory.java:145
    #4 io.grpc.PartialForwardingClientCallListener.onReady()V PartialForwardingClientCallListener.java:44
    #5 io.grpc.ForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:23
    #6 io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:40
    #7 io.grpc.PartialForwardingClientCallListener.onReady()V PartialForwardingClientCallListener.java:44
    #8 io.grpc.ForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:23
    #9 io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onReady()V ForwardingClientCallListener.java:40
    #10 io.grpc.internal.DelayedClientCall$DelayedListener.onReady()V DelayedClientCall.java:497
    #11 io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamOnReady.runInternal()V ClientCallImpl.java:781
    #12 io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamOnReady.runInContext()V ClientCallImpl.java:772
    #13 io.grpc.internal.ContextRunnable.run()V ContextRunnable.java:37
    #14 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133
    #15 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V ThreadPoolExecutor.java:1130
    #16 java.util.concurrent.ThreadPoolExecutor$Worker.run()V ThreadPoolExecutor.java:630
    #17 java.lang.Thread.run()V Thread.java:830
    #18 (Generated Stub) <null>

  Previous write of size 4 at 0x00009b66fc88 by thread T4 (mutexes: write M1, write M2, write M3, write M4, write M5):
    #0 io.grpc.xds.ControlPlaneClient$AdsStream.cleanUp()V ControlPlaneClient.java:424
    #1 io.grpc.xds.ControlPlaneClient$AdsStream.close(Ljava/lang/Exception;)V ControlPlaneClient.java:418
    #2 io.grpc.xds.ControlPlaneClient$1.run()V ControlPlaneClient.java:130
    #3 io.grpc.SynchronizationContext.drain()V SynchronizationContext.java:94
    #4 io.grpc.SynchronizationContext.execute(Ljava/lang/Runnable;)V SynchronizationContext.java:126
    #5 io.grpc.xds.XdsClientImpl.shutdown()V XdsClientImpl.java:207
    #6 io.grpc.xds.SharedXdsClientPoolProvider$RefCountedXdsClientObjectPool.returnObject(Ljava/lang/Object;)Lio/grpc/xds/XdsClient; SharedXdsClientPoolProvider.java:144
    #7 io.grpc.xds.SharedXdsClientPoolProvider$RefCountedXdsClientObjectPool.returnObject(Ljava/lang/Object;)Ljava/lang/Object; SharedXdsClientPoolProvider.java:102
    #8 io.grpc.xds.XdsClientFederationTest.cleanUp()V XdsClientFederationTest.java:86
```
2024-02-22 14:43:29 -08:00
Sergii Tkachenko de7e649e04
(minor) Fix a typo in ReflectableMarshaller docstring (#10942)
`it's` -> `its`
2024-02-21 13:29:51 -08:00
Eric Anderson e19f1f15a9 examples: Fix file paths in debug example README 2024-02-21 08:34:19 -08:00
Eric Anderson 071a784795
Change GAE interop tests to use java11 runtime (#10933)
The Java 8 runtime is end of support. Leaving this a gae-jdk8 for now.
The gae-jdk8 was because AppEngine changed dramatically from Java 7 to
Java 8. Nowadays the versions are more in line with OpenJDK and not very
different from each other.

Fixes #10925
2024-02-20 10:52:22 -08:00
David Ankin 0d39c2c701
implement ability to skip generation of javax annotation (#10927)
* commit to implement feedback from #10786 to partially fix #9179
2024-02-16 17:49:20 -08:00
Eric Anderson 3994b15fea buildscripts: Avoid vswhere crashing Gradle
I'm trying to upgrade to a newer Windows Kokoro image, but the new one
has an old vswhere installed that breaks Gradle. Our old image doesn't
have vswhere at all. If vswhere isn't found, this rename prints some
errors, but the bat script continues executing. So this change is
compatible with both the older and newer image.
2024-02-16 15:12:14 -08:00
Eric Anderson f4cc166f18 xds: Copy data in least request to avoid picker data race
In 0d39bf50 the ReadyPicker was changed holding List<Subchannel> to
List<ChildLbState>, but ChildLbState mutates over time and is not
synchronized. We want the picker to have a snapshot of the data, so copy
the data from ChildLbState instead of using it directly.

Unfortunately the tests depended on the ChildLbState a bit, so we need
to save the EAG only to use it in tests. That's okay for now, but in the
future we'll probably want to remove that unnecessary memory usage.
2024-02-16 13:44:25 -08:00
Eric Anderson a231d80756 Remove semi-circular dependency between core and util
Add the 'fake' dependency to grpc-netty instead of grpc-core.
grpc-okhttp already depends on grpc-util and probably would be fine
without round_robin on Android.

There's not actually a circular dependency, but some tools can't handle
the compile vs runtime distinction. Such tools are broken, but fixes
have been slow and this approach works with no real downfalls.

Works around #10576 #10701
2024-02-16 12:31:54 -08:00
Alex Panchenko 257d1c2db4 servlet: introduce ServletServerBuilder.buildServlet() 2024-02-16 12:21:42 -08:00
Benjamin Peterson a68399a9b6
netty: improve server handling of writes to reset streams (#10258)
* netty: improve server handling of writes to reset streams

A server stream can be reset by the client while server writes are still queued. After the stream is reset, the netty connection will forget the stream object. The `NettyServerHandler` must deal with that situation. `sendResponseHandlers` already had some code to do that. This change standardizes that code and adds it to `sendGrpcFrame`. This fixes a potential bug where a `SendGrpcFrameCommand` with `endOfStream=true` would raise an `AssertionError` if written to a reset stream. (This bug is not currently reachable because `endOfStream=false` for all server `SendGrpcFrameCommand` objects.)

* Do not call into the encoder when we know the stream is gone.
2024-02-16 11:08:09 -08:00
Alex Panchenko 2f025b2b24
servlet: missing @Rule annotation in GrpcServletSmokeTest (#10922) 2024-02-16 10:29:45 -08:00
Larry Safran 67dfe49ae6 Fix compile failure 2024-02-15 21:28:37 -08:00
Terry Wilson eba699ad16
rls: Adding extra debug logs (#10902) 2024-02-15 15:23:36 -08:00
Larry Safran 044749706a
util:MultiChildLoadBalancer cleanup (#10780)
* add final, change method permissions, add javadoc, cleanup unneeded, move updateOverallBalancingState to ClusterManagerLB and make it abstract

* Restructure to eliminate the flags as protected methods

* Move methods around so that the candidates for override are near the top.

* Reorder picker methods lower
2024-02-15 14:12:40 -08:00
John Cormie f20c853c40
Factor out MainThreadScheduledExecutorService. (#10916)
So it can be used by other robolectric tests.
2024-02-15 11:11:43 -08:00
Terry Wilson 08f8ed1bc3 util: MultiChildLoadBalance.shutdown() log to FINE
The log level of this method is set to INFO, which is too spammy. Bring
it down to the FINE level.
2024-02-13 14:12:33 -08:00
Eric Anderson 7787673992 xds: Replace isEquivalentTo with equals in LeastRequest
This is similar to the changes to round robin in dca89b25.
2024-02-13 08:29:31 -08:00
Eric Anderson 7e72413233 okhttp: Move HostnameVerifier tests to TlsTest
Http2OkHttp is now unnecessary, as Http2Test tests OkHttp client to
Netty server. receivedDataForFinishedStream() was the only remaining
unique test and it seems already covered by AbstractInteropTest these
days.
2024-02-12 19:40:33 -08:00
Eric Anderson 92463f62bf interop-testing: Avoid Netty-specific APIs to creating channels
We prefer to test using the stable APIs, as they are what our users
should be using. Http2Client continues using NettyChannelBuilder because
it is intended to test grpc-netty.
2024-02-12 07:52:31 -08:00
Eric Anderson 372a535615 auth: Add support for Retryable interface
Retryable was added in google-auth-library 1.5.3 to make clear the
situations that deserve a retry of the RPC. Upgrading to that caused
problems because of transitive dependency issues syncing into Google so
it was reverted in 369f87be. google-auth-library 1.11.0 changed the
approach to avoid the transitive dependency updates. cl/601545581
upgraded to 1.22.0 inside Google. Bump to that version and swap away
from the imprecise IOException heuristic. go/auth-correct-retry

Fixes #6808
2024-02-09 15:15:18 -08:00
yifeizhuang ff34d51c73
api: stabilize server.getListenerSockets (#10910) 2024-02-09 14:40:54 -08:00
Anirudh Ramachandra 608bb8499c
Mark couple of helper functions in XdsClient as public. (#10871) 2024-02-09 12:53:03 -08:00
Anirudh Ramachandra 52b11c1d08
Expose the getOrCreate method via the InternalSharedXdsClientPoolProvider. This is needed for internal users to both set the bootstrap and interact with the XdsClient via the shared object pool (#10872) 2024-02-09 12:50:26 -08:00
Eric Anderson d568a8dc19 kokoro: Avoid using Kokoro's env_vars setting
It makes it hard to run ephemeral jobs, and we don't actually need it
here because we aren't changing the setting with manual runs.
2024-02-09 12:47:16 -08:00
Larry Safran 6847219657
Start 1.63.0 development cycle (#10906) 2024-02-08 13:11:38 -08:00
Eric Anderson 95b847e799 interop-testing: Use separate event loops in RetryTest
The RetryTest was flaky, and it seems to have been caused by the client
and server getting assigned to the same event loop. Separating the two
reduces the flake rate from ~3% to less than 0.1% (no flakes in a 1000).

While I was here fixing the executors, I reduced the number of threads
created and shut down the threads after they are no longer used. This
had no impact to the flake rate (no flakes in 1000).
2024-02-08 07:26:31 -08:00
Larry Safran 7ba0718bb9
Change log level from WARNING to FINER for expected exception (#10899)
* Change log level from WARNING to FINER.  Fixes #10816
2024-02-07 17:31:27 -08:00
Larry Safran 5faab7f47e
Revert "Fix retries that timeout hanging forever. (#10855)" (#10901)
This reverts commit c0a9d315d3.
2024-02-07 15:05:21 -08:00
yifeizhuang 03decafa1f
XdsClient is experimental (#10876) 2024-02-06 17:43:30 -08:00
yifeizhuang f6d9221b65
xds: hide TlsContextManager in XdsResourceType.Args (#10894) 2024-02-06 15:41:24 -08:00
Sergii Tkachenko 4f7ec131ec
xds: Googleapis proto sync to 2023-01-10 (#10896)
Sync googleapis protos to
googleapis/googleapis@114a745b28 for
consistency with Envoy and cncf/xds.

The same version is used in [envoy]
(62e7c59374/api/bazel/repository_locations.bzl (L69))
and [cncf/xds]
(0fa0005c9c/bazel/repository_locations.bzl (L23))
since Jan 2023.

Function-wise, this is a noop.
2024-02-05 18:52:55 -08:00
Sergii Tkachenko 68334a019a
xds: Envoy proto sync to 2024-01-24 (#10895)
`envoyproxy/envoy`: Sync protos to the latest imported version
147e6b9523
(commit 2024-01-24, cl/604403196).

Should be a noop, just a routine xDS proto update to make upcoming
RLQS-related imports simpler.
2024-02-05 17:21:51 -08:00
Larry Safran 374dbe9461
Fix flaky retry tests (#10887)
* Reorder tracing and actually closing listener to eliminate test flakiness
* Use real value rather than mock for flaky test
2024-02-05 10:54:55 -08:00