Commit Graph

5906 Commits

Author SHA1 Message Date
Eric Anderson ac8908ec7f xds: Plumb thirdparty classes through sourceSet instead of jar
Plumbing through sourceSet lets cross-project dependencies work the same
way as artifacts published to Maven. This fixes an issue for
interop-testing where build/install would include all the raw files from
thirdparty in addition to the grpc-xds.jar. For example:
build/install/grpc-interop-testing/lib/com/github/xds/data/orca/v3/OrcaLoadReport$1.class

b/288577812
2023-06-23 13:27:29 -07:00
Eric Anderson f46793e82d Remove unneeded dependency excludes
Since 44847bf4e, when we upgraded our JUnit version, the JUnit
exclusions have probably not been necessary. e0ac97c4f upgraded
Robolectric to a version that had the auto.service problem fixed.
2023-06-23 11:18:10 -07:00
Eric Anderson 0e6c01837c buildscripts: Fix prepare_qemu to not assume system already has emulation
Apparently our Kokoro image has this done already, and my laptop as
well. But the newer Kokoro image and other computers like my desktop
don't have it already.
2023-06-23 10:24:25 -07:00
Xuan Wang 1c9ce5d76b
[PSM interop] Don't fail url_map target if sub-target already failed (#10295) 2023-06-22 15:42:21 -07:00
Tony An 2effae6249
okhttp: tsan socket data race bug fix (#10279)
replaced use of bareSocket with a synchronized socket, added additional lock to synchronize initialization with shutdown() to fix a Java bug
2023-06-22 10:46:34 -07:00
Larry Safran f1de820c19
Change delay for hedging retry after a non-fatal error to be 0 to match the gRFC A6. (#10293)
Fixes #10145
2023-06-21 14:53:02 -07:00
Terry Wilson 31321562ee
xds: Use Rule order instead of RuleChain (#10289)
This is now the preferred way to specify rule execution order.
2023-06-16 13:39:47 -07:00
Terry Wilson 48192db68e
netty: Nix InternalNettyServerBuilder unused methods (#10287) 2023-06-15 15:03:05 -07:00
Eric Anderson d28e5f0fc5 buildscripts: Fail sonatype-upload for non-200 HTTP status code 2023-06-15 13:35:00 -07:00
Larry Safran 99392004a0
Handle Aggregate cluster loops and dups (#10274)
* Suppress duplicate children and NACK if detect loops (child is an ancestors of the current CDS aggregate).  

* Handle diamond shaped aggregations (same cluster appears under 2 distinct parents that doesn't create a loop).
2023-06-15 10:24:23 -07:00
Eric Anderson d4e26cc689 okhttp: Use a real socket during server transport testing
The PipeSocket was convenient and avoided real I/O, but the
shutdown/close while connecting/handshaking tests were triggering a
Socket bug in Java (https://bugs.openjdk.org/browse/JDK-8278326). Using
a real socket doesn't trigger the bug because the test stops sharing
state with the code under test.

Fixes #10228

```
Details
==================
WARNING: ThreadSanitizer: data race (pid=4528)
  Write of size 1 at 0x0000cfb9d5f4 by thread T36 (mutexes: write M0):
    #0 java.net.Socket.setCreated()V Socket.java:687
    #1 java.net.AbstractPlainSocketImpl.create(Z)V AbstractPlainSocketImpl.java:149
    #2 java.net.Socket.createImpl(Z)V Socket.java:477
    #3 java.net.Socket.getImpl()Ljava/net/SocketImpl; Socket.java:540
    #4 java.net.Socket.setTcpNoDelay(Z)V Socket.java:998
    #5 io.grpc.okhttp.OkHttpServerTransport.startIo(Lio/grpc/internal/SerializingExecutor;)V OkHttpServerTransport.java:164
    #6 io.grpc.okhttp.OkHttpServerTransport.lambda$start$0(Lio/grpc/internal/SerializingExecutor;)V OkHttpServerTransport.java:159
    #7 io.grpc.okhttp.OkHttpServerTransport$$Lambda$56.run()V ??
    #8 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133
    #9 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V ThreadPoolExecutor.java:1130
    #10 java.util.concurrent.ThreadPoolExecutor$Worker.run()V ThreadPoolExecutor.java:630
    #11 java.lang.Thread.run()V Thread.java:830
    #12 (Generated Stub) <null>

  Previous read of size 1 at 0x0000cfb9d5f4 by thread T35 (mutexes: write M1, write M2):
    #0 java.net.Socket.close()V Socket.java:1512
    #1 io.grpc.okhttp.OkHttpServerTransportTest$PipeSocket.close()V OkHttpServerTransportTest.java:1384
    #2 io.grpc.okhttp.OkHttpServerTransportTest.clientCloseDuringHandshake()V OkHttpServerTransportTest.java:290
```
2023-06-15 09:59:05 -07:00
Sergii Tkachenko 4d3a29b2af
Update README etc to reference 1.56.0 (#10283)
com.google.protobuf:protoc updated from 3.21.7 to 3.22.3.
2023-06-14 16:18:55 -07:00
Benjamin Peterson 9148095df4 netty: simplify SendGrpcFrameCommand
DefaultByteBufHolder has many perfectly good implementations. It should be enought to implement the replace method.
2023-06-14 08:45:31 -07:00
Mark S. Lewis ae59afb5bf
Update guava dependency to address CVE-2023-2976 (#10249)
Explicit dependencies to keep versions in step with newer Guava
2023-06-12 12:43:35 -07:00
Eric Anderson 5754518914 Upgrade netty-tcnative to 2.0.61.Final
This updates the version of boringssl and removes the dependency on APR.
netty-tcnative 2.0.56.Final uses APR 1.7.0, so is in scope for
CVE-2021-35940, CVE-2022-28331, and CVE-2022-24963. netty-tcnative is
not actually vulnerable. The binary does not include apr_socket_sendv(),
apr_encode_*(), apr_pencode_*(), apr_decode_*(), apr_pdecode_*(). The
binary does include apr_time_exp_*() but it is unused code.
Unfortunately --gc-sections wasn't used during compilation.
apr_time_now() is used, but that just calls gettimeofday() and is not
vulnerable.

There's no panic here, but this updates netty-tcnative just a few weeks
before we would have ordinarily done so. Bumping the version makes life
easier for everyone.
2023-06-12 11:01:06 -07:00
Eric Anderson 0f2c43a8fd Download Maven from Maven Central
Maven deleted older binaries from their CDN, so the download was
failing. Maven Central seems it'll be more stable.
2023-06-09 10:32:12 -07:00
Daniel Liu 1daf0ff720
services, xds, orca: use application_utilization and fallback to cpu_utilization if unset in WRR (#10256)
Implements updates to [A51][] and [A58][].

Imported cncf/xds using import.sh script.

A51: https://github.com/grpc/proposal/pull/374
A58: https://github.com/grpc/proposal/pull/373
2023-06-08 16:03:50 -07:00
Eric Anderson 33a402d924 Avoid NettyServerBuilder when unnecessary
These code locations just needed a generic gRPC server, without any
transport-specific configuration. Use vanilla ManagedServerBuilder
instead of hard-coding to Netty, as we would suggest to our users.
2023-06-05 17:11:16 -07:00
Larry Safran 217103cd5e
Flip default for RLS on XDS being enabled to true. (#10248) 2023-06-05 15:49:30 -07:00
sanjaypujare 6949b8790a
api: stabilize ManagedChannelBuilder.useTransportSecurity (#10244)
* api: stabilize ManagedChannelBuilder.useTransportSecurity and add a note in Java doc
2023-06-02 15:08:25 -07:00
sanjaypujare 639fd8edf2
xds: remove env var for security enable/disable (#10243) 2023-06-02 09:24:51 -07:00
Xuan Wang 02ac6f4393
buildscripts, psm interop: Don't fail target if sub-target already failed
We configured TestGrid to file bug separately for each 
failed sub-target, if we still fail the main target, 
TestGrid will fail duplicate bugs.

The same change in core: 
https://github.com/grpc/grpc/pull/33222.
2023-05-31 17:37:46 -07:00
Sergii Tkachenko 17b2df78ec
Start 1.57.0 development cycle (#10229) 2023-05-30 18:02:40 -07:00
kotlaja e6d9b41895
bazel: Add java toolchain type to all rules which are using java_common (#10225) 2023-05-30 11:48:43 -07:00
Stanley Cheung 4277867a7d
gcp observability: Update Docker images to eclipse-temurin (#10218)
Perform software update so that we install patches for latest
vulnerabilities. Similar to the fix in #10191
2023-05-30 08:03:36 -07:00
Larry Safran df85a45742
Add a java specific description of manual flow control to the example. (#10223)
* Add a java specific description of manual flow control to the example.
2023-05-26 18:36:40 -07:00
Daniel Liu 5a27e3ead2
services, xds, orca: support EPS in client-side WRR (#10177) 2023-05-26 12:52:42 -07:00
Michael Vorburger ⛑️ 6aa72b786d
docs: Document use with Bazel, like Maven & Gradle (fixes #10215) (#10217)
* docs: Document use with Bazel, like Maven & Gradle (fixes #10215)
2023-05-26 12:42:50 -07:00
apolcyn b77f0a022f
interop-testing: introduce --additional_metadata flag to let client optionally send extra metadata (#10201)
The flag was documented in https://github.com/grpc/grpc/pull/33180 .
This flag will be useful for some RLS integration tests where we need
to have the client set certain headers for routing purposes.
2023-05-26 06:45:50 -07:00
Eric Anderson f75c7c72f2 repositories.bzl: In doc, use | to combine dicts
The previous syntax for just adding your own keys doesn't seem to work,
but was similar to the approach of using `dict(d, foo=bar)`. You can't
have '.' and ':' in a key that way though. The doc was written before
Bazel 1.0 and in newer Bazel versions you can just use | to concatenate.

Fixes #10203
2023-05-25 15:26:43 -07:00
Eric Anderson 7217baa777 Remove old Java 7/8 TLS testing workarounds
Ciphers have been "fast enough" for testing since early Java 8 updates;
we haven't needed to override ciphers since we dropped Java 7 support.
Java 8u252 had ALPN, so Conscrypt or Jetty ALPN hasn't been necessary
for basic testing for a while. We still want specialized testing for
Conscrypt, but only tests testing Conscrypt need to care.
2023-05-25 14:04:01 -07:00
Eric Anderson cb42405217 testing: Move some internal classes to testFixtures
These two classes are precisely the purpose of test fixtures. Move them
close to the classes they make easier to use in tests.
2023-05-25 14:03:42 -07:00
sanjaypujare 3b2cba09c1
api: remove experimentalApi setting from proxyDetector and related classes (#10216) 2023-05-25 13:46:24 -07:00
sanjaypujare e875d1b01c
xds: remove remaining occurences of SDS in the security code (#10219) 2023-05-25 10:28:37 -07:00
Terry Wilson e172ea7efc
api: Experimental CallCredentials.thisUsesUnstableApi() (#10211)
The thisUsesUnstableApi() method was earlier deprecated and the
@ExperimentalApi annotation removed. Adding @ExperimentalApi back
to make it clear that this method can (and will) later be removed.
2023-05-23 08:32:33 -07:00
Terry Wilson dd2dc21de2
api: Stabilize io.grpc.CallCredentials (#10208) 2023-05-22 09:59:53 -07:00
Liam Miller-Cushon 478f30e645 testing-proto: Define supported SourceVersion in annotation test
This fixes the warning during the test run:
```
warning: No SupportedSourceVersion annotation found on io.grpc.testing.protobuf.SimpleServiceTest$AnnotationProcessor, returning RELEASE_6.
warning: Supported source version 'RELEASE_6' from annotation processor 'io.grpc.testing.protobuf.SimpleServiceTest$AnnotationProcessor' less than -source '1.8'
```
2023-05-18 11:01:39 -07:00
Kun Zhang 6fd93486cc
test: stop mocking Executor. (#10199)
It is forbidden internally. Error message:

Mocking types which have complex contracts or are easy to construct by
other means is forbidden:

'executor' is mocking 'interface java.util.concurrent.Executor'. Use a
real executor. Mocks of Executor don't execute submitted tasks at all,
which leads to suppressed errors, deadlocks, and brittle tests..
2023-05-17 16:40:32 -07:00
Terry Wilson 5d40396aa2
A note to include backports in release notes (#10198) 2023-05-17 11:36:01 -07:00
Terry Wilson 810d8cab09
xds: pick_first LB configuration (#10181)
Configure pick_first load balancing if the xDS PickFirst LB config
message is used in the Cluster load balancing config.
2023-05-17 10:19:29 -07:00
Eric Anderson e0ac97c4fd
Upgrade dependencies (#10178)
Required bumping android compile/targetSdkVersion to 33
2023-05-17 08:32:49 -07:00
Eric Anderson e38d928f75 examples: Fix hostname binary name in README
The command has been wrong since it was introduced. The hostname-server
comes from the applicationName, which hasn't changed.
2023-05-16 15:44:09 -07:00
Sergii Tkachenko 16b84924a1
xds interop: Update Docker images to `eclipse-temurin:11-jre` (#10172)
- Update to xDS Test Client and xDS test server Docker images to `eclipse-temurin:11-jre`. 
- Perform software update so that we install patches for latest vulnerabilities.
2023-05-16 15:20:55 -07:00
Eric Anderson 29b8483fd6
Use test fixtures instead of sourceSets.test.output
This avoids the (often missing) evaluationDependsOn and fixes using
results from other projects without propagating those through
Configuration. It also reduces the number of useless classes pulled in
by down-stream tests, reducing the probability of rebuilds.

The expectation of fixtures is they help testing down-stream code that
use the classes in main. That applies to all the classes here except for
FakeClock and StaticTestingClassLoader. It would also apply to many
internal classes in grpc-testing, but let's consider cleaning that up
future work.
2023-05-16 12:10:13 -07:00
Eric Anderson cd7b81c6d2 gradle: Assocate an artifact to every version
This allows the checkForUpdates task to look for newer versions.
2023-05-15 14:01:41 -07:00
Eric Anderson 2cc24a9bda gradle: Remove version pinning from POMs
The pinning is unreliable in Maven and ignored by Gradle. I'm not at all
convinced that we are pinning/not pinning in appropriate projects. The
pinning also serves less of a purpose since we started encouraging the
BOM and grpc-netty-shaded. Netty's HTTP/2 API has also become somewhat
stable compared to its earlier history. If we notice an up-tick in
version skew, we can reinstate it.

The pinning is annoying in the build.gradle code and causes Maven/Gradle
to download the version list once a day, which can be troublesome to
users unaware of how to tell the tools to work offline.

It also opens our users to platform issues like seen in #10043
and #10086 where Maven Central's version list was incorrectly generated.
Or like #9664 where Gradle Plugin's repository caches packages from
JCenter but the version list is not as cachable so exposed us to JCenter
instability.

This fixes #8357, by way of "we think we won't worry any more." See
90db93b9 when it was originally introduced. And issues
like #8337, #3634.
2023-05-15 13:08:40 -07:00
Terry Wilson 793fde840a
xds: Import latest Envoy protos (#10167) 2023-05-11 09:44:43 -07:00
Eric Anderson c16972c54a build.gradle: Add checkForUpdates task 2023-05-10 12:44:35 -07:00
Eric Anderson 3f0e90d54e build.gradle: Use proto task output instead of hard-coding path
This removes the need to specify dependsOn explicitly, because the task
input carries the dependency.
2023-05-10 12:44:10 -07:00
Daniel Liu 6e486cba48
import cncf/xds using import.sh script (#10160) 2023-05-10 12:42:26 -07:00