Commit Graph

6141 Commits

Author SHA1 Message Date
sanjaypujare dea7fa09ff
core, services, util: export util from core and move gson dep to services for the checkUpperBoundDeps error (#10390) 2023-07-20 10:20:10 -07:00
Eric Anderson 3f60746a98
servlet: Remove unused dependency on grpc-util in tests
Also convert to testFixtures, like we did elsewhere.

grpc-testing had previously been listed, but because of the missing
comma, it was not an actual dependency.
2023-07-20 07:57:11 -07:00
Eric Anderson b526a68622 Restrict checkUpperBoundDeps to library dependencies
Guava 32.1.0 added Gradle Module Metadata to their artifact (they still
use Maven for their build). Because of that, we're now seeing less
normal dependencies that were confusing checkUpperBoundDeps.  Obviously
'null' is not the version we were looking for.

We aren't upgrading to Guava 32.1.x yet as there's still other problems.

```
Execution failed for task ':grpc-census:checkUpperBoundDeps'.
> Maven version skew: com.google.guava:guava-parent (32.1.1-jre != null) Bad version dependency path: [project ':grpc-census', com.google.guava:guava:32.1.1-android] Run './gradlew :grpc-census:dependencies --configuration runtimeClasspath' to diagnose
```
2023-07-19 17:17:05 -07:00
Sergii Tkachenko e3016cf7f0
Stabilize setExtensionRegistry() of ProtoLiteUtils and ProtoUtils (#10392)
Closes #1787
2023-07-19 15:38:42 -07:00
Sergii Tkachenko 313c343729
Stabilize setMessageCompression() (#10393)
Closes #1703
2023-07-19 15:38:32 -07:00
Eric Anderson fefbd12c7e Remove unnecessary thisUsesUnstableApi from CallCredentials impls
This method has been unnecessary/deprecated since dd2dc21d.
2023-07-18 13:38:24 -07:00
Eric Anderson e8f36fa522 all: Remove grpc-context from grpc-all
grpc-context is empty, so isn't needed. Including it also causes the
warning:
```
> Task :grpc-all:javadoc
Resolution of the configuration :grpc-context:compileClasspath was attempted from a context different than the project context. Have a look at the documentation to understand why this is a problem and how it can be resolved. This behaviour has been deprecated and is scheduled to be removed in Gradle 8.0. See https://docs.gradle.org/7.6/userguide/viewing_debugging_dependencies.html#sub:resolving-unsafe-configuration-resolution-errors for more details.
```
2023-07-18 11:20:18 -07:00
sanjaypujare 0f5f07f876
core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util (#10362)
* core, inprocess, util: move inprocess and util code into their own new artifacts grpc-inprocess and grpc-util
2023-07-17 11:45:31 -07:00
Anirudh Ramachandra ac35ab67f2
xds: Encode the service authority in XdsNameResolver (#10207)
Encode the service authority before passing it into gRPC util in the xDS name resolver to handle xDS requests which might contain multiple slashes. Example: xds:///path/to/service:port.

As currently the underlying Java URI library does not break the encoded authority into host/port correctly simplify the check to just look for '@' as we are only interested in checking for user info to validate the authority for HTTP.

This change also leads to few changes in unit tests that relied on this check for invalid authorities which now will be considered valid.

Just like #9376, depending on Guava packages such as URLEscapers or PercentEscapers leads to internal failures(Ex: Unresolvable reference to com.google.common.escape.Escaper from io.grpc.internal.GrpcUtil). To avoid these issues create an in house version that is heavily inspired by grpc-go/grpc.
2023-07-14 12:54:18 -07:00
Larry Safran 0aaa2e0434
Bump version (#10378)
* Start 1.58.0 development cycle
2023-07-13 13:06:21 -07:00
Tony An 37351c690c
fixed StaticStrideScheduler looping too many times (#10370) 2023-07-13 10:24:07 -07:00
Terry Wilson 78cf1c39cf
core: Apply RetryingNameResolver in ManagedChannelImpl (#10371)
Wrapping the DnsNameResolver in DnsNameResolverProvider can cause
problems to external name resolvers that delegate to a DnsResolver
already wrapped in RetryingNameResolver. ManagedChannelImpl would
end up wrapping these name resolvers again, causing an exception
later from a RetryingNameResolver safeguard that checks for double
wrapping.
2023-07-12 10:14:50 -07:00
Daniel Liu 4fa2814d65
services, xds, orca: LRS named metrics support (#10282)
Implements gRFC A64: xDS LRS Custom Metrics Support
2023-07-11 13:29:31 -07:00
Larry Safran 8dbd47ceb5
Sort the policies in a rule by policy name when parsing from proto. (#10334)
* Sort the policies in a rule by policy name when parsing from proto.  This fixes the server sending a GOAWAY when an LDS update with no changes other than ordering is received.

* Remove use of deprecated method setSourceIp

* Fix style issues

* Update RbacFilterTest.java
2023-07-10 11:07:54 -07:00
Masakuni Oishi 68e29922cf
CheckedForwardingClientCall should pass trailers from the caught exception (#10356)
`StatusException` thrown from `checkedStart()` may have `trailers`. Therefore, `CheckedForwardingClientCall` should pass the `trailers` to `responseListener.onClose()`.
2023-07-10 10:00:10 -07:00
Eric Anderson 1af6b24a7c Add commit convenience
Co-authored-by: Sergii Tkachenko <hi@sergii.org>
2023-07-07 16:01:21 -07:00
Eric Anderson 13ca4d5f77 RELEASING.md: Reorder contents to make it a bit more streamlined
I moved items that could be done immediately after the release up into
the main release flow. I also stripped some outdated or unnecessary text
to make it quicker to follow.
2023-07-07 16:01:21 -07:00
Eric Anderson 75c9daf4a1 build.gradle: Avoid assignment to japicmp configuration
Instead of assignment, it now seems to be a FileCollection that you
modify. This has probably been broken since f458f22.

```
> Could not create task ':grpc-auth:japicmp'.
   > Cannot set the value of read-only property 'oldClasspath' for task ':grpc-auth:japicmp' of type me.champeau.gradle.japicmp.JapicmpTask.
```

The change works with or without the files(), but the conversion is just
not needed anymore as it is handled by FileCollection.
2023-07-07 14:34:20 -07:00
Stephane Landelle 1e30cb6601
core,grpclb: Resolve isAndroid only once on class loading (#10345)
Motivation:

When multiple NameResolvers are created, the Classloader is scanned every time trying to figure out if the Platform is Android. This expensive work could be done only once.

Modification:

Cache isAndroid resolution in a constant.

Result:

Less expensive multiple NameResolvers instantiation.
2023-07-06 13:42:43 -07:00
Tony An 0b53dd7304
implemented and tested static stride scheduler for weighted round robin load balancing policy (#10272) 2023-07-06 10:03:08 -07:00
Eric Anderson 361616ae7c Upgrade Netty to 4.1.94-Final 2023-07-06 08:46:45 -07:00
sanjaypujare 2be568d70b
core: stabilize MutableHandlerRegistry by removing ExperimentalApi annotation (#10348) 2023-07-05 16:07:37 -07:00
Larry Safran ed4cc896fc
require EDS service name in CDS resources with xdstp name (#10329)
* require EDS service name in CDS resources with xdstp name
2023-06-30 13:51:46 -07:00
sanjaypujare 1a9c2a0dac
api, context: fix bazel build: export api from context and remove context dep from api (#10326) 2023-06-29 13:51:13 -07:00
Terry Wilson 8192ff6103
core: ManagedChannelImpl to always use RetryingNameResolver (#10328)
ManagedCahnnelImpl did not make sure to use a RetryingNameResolver if
authority was not overriden. This was not a problem for DNS name
resolution as the DNS name resolver factory explicitly returns a
RetryingNameResolver. For polling name resolvers that do not do this in
their factories (like the grpclb name resolver) this meant not having retry
at all.
2023-06-29 13:47:23 -07:00
Eric Anderson 443a0502ba Remove old Protobuf Gradle Plugin caching workaround
This has been fixed in the Protobuf Gradle Plugin since probably 0.8.13.
2023-06-29 12:50:01 -07:00
Eric Anderson 5a0fccfae0
Rebuild Android generated code for java.lang.String (#10327)
3808e707 fixed references of String to be java.lang.String, but failed
to update the Android build.
2023-06-29 12:47:44 -07:00
Terry Wilson 3b40b22348
api: Stabilize two io.grpc.Status methods (#10322)
This stabilizes:
 - Status.asRuntimeException(Metadata)
 - Status.trailersFromThrowable(Throwable)
2023-06-29 11:58:15 -07:00
Philip K. Warren 3808e707f9
compiler: Use fully qualified String in codegen (#10321)
Currently, the gRPC compiler isn't properly using the fully qualified
string name `java.lang.String` instead of `String`. Update the generator
to use the `$String$` alias to avoid compile issues with protobuf
messages called String.

Fixes #10316.
2023-06-29 10:50:13 -07:00
Eric Anderson f902e16d3d compiler: Don't mention C++ Protobuf version in build message
There's been minor version skew between Java and C++ many times because
certain releases are one-language-only. And now we have more severe
skew, where we can't readily upgrade to newer C++ Protobuf versions
because of build complexity. Let's just remove the version, and have the
canonical C++ Protobuf version live in COMPILING.md.

See #10317
2023-06-29 06:15:22 -07:00
wwtbuaa01 5799febde0
Add UserHandle and BinderChannelCredentials to BinderChannelBuilder to support cross-user communication through OnDeviceServer. (#10197)
Add UserHandle and BinderChannelCredentials to BinderChannelBuilder to support cross-user ondevice server.
2023-06-29 13:01:13 +02:00
sanjaypujare 4d2c3aac0e
context, all: move Context classes to grpc-api (#10313)
* context, all: move Context classes to grpc-api
clean up grpc-context since it has no source code: only add dep on grpc-api
add exclusion for all transitive deps of grpc-api - only guava
exclude grpc-context as a dependency from grpc-alts because all context code is in grpc-api now
api: 1.7 as target Java version for Context source-set of grpc-api

* core, census: fix the issues with android project pulling in old grpc-context version


* api,context: make changes to bazel build files to account for context code moving from context to api
2023-06-28 18:28:30 -07:00
Eric Anderson d654707838
buildscripts: Build Android with main build in linux_artifacts
Note that this changes the JDK used to compile releases to Java 11. That
should only impact the appearance of the Javadoc.

This adds the Android SDK to the build container, removing the
dependency on the Android SDK being available on the CI host. This
allows running on newer Kokoro images. 'Android' and 'Android interop'
CIs still depend on the Android SDK being available on the host, but
since they aren't used as part of the release process, they can more
easily migrate off Kokoro as part of future work.

This also causes Android components to now be built with -Werror, as we
use -PfailOnWarnings=true in unix.sh but were missing it from the
Android build invocations.

Gradle will auto-download the necessary version of build-tools. We don't
want to download it ourselves because the version we specify might not
even be used. Looking at logs, we were previously downloading a version
that was unused.

We now fork javac to avoid OOM. The build fails 2/3 times before the
forking, and 0/3 after.
2023-06-28 16:24:21 -07:00
Eric Anderson b2327238c5 xds: Include third_party sources is sources jar 2023-06-27 09:48:54 -07:00
Terry Wilson f3399e3d0e
android-interop-testing: Use java.util.concurrent (#10312)
Instead of the deprecated Android AsyncTask, let's use the standard
Java concurrency library with a Callable and an ExecutorService.
2023-06-26 13:32:05 -07:00
Eric Anderson ac8908ec7f xds: Plumb thirdparty classes through sourceSet instead of jar
Plumbing through sourceSet lets cross-project dependencies work the same
way as artifacts published to Maven. This fixes an issue for
interop-testing where build/install would include all the raw files from
thirdparty in addition to the grpc-xds.jar. For example:
build/install/grpc-interop-testing/lib/com/github/xds/data/orca/v3/OrcaLoadReport$1.class

b/288577812
2023-06-23 13:27:29 -07:00
Eric Anderson f46793e82d Remove unneeded dependency excludes
Since 44847bf4e, when we upgraded our JUnit version, the JUnit
exclusions have probably not been necessary. e0ac97c4f upgraded
Robolectric to a version that had the auto.service problem fixed.
2023-06-23 11:18:10 -07:00
Eric Anderson 0e6c01837c buildscripts: Fix prepare_qemu to not assume system already has emulation
Apparently our Kokoro image has this done already, and my laptop as
well. But the newer Kokoro image and other computers like my desktop
don't have it already.
2023-06-23 10:24:25 -07:00
Xuan Wang 1c9ce5d76b
[PSM interop] Don't fail url_map target if sub-target already failed (#10295) 2023-06-22 15:42:21 -07:00
Tony An 2effae6249
okhttp: tsan socket data race bug fix (#10279)
replaced use of bareSocket with a synchronized socket, added additional lock to synchronize initialization with shutdown() to fix a Java bug
2023-06-22 10:46:34 -07:00
Larry Safran f1de820c19
Change delay for hedging retry after a non-fatal error to be 0 to match the gRFC A6. (#10293)
Fixes #10145
2023-06-21 14:53:02 -07:00
Terry Wilson 31321562ee
xds: Use Rule order instead of RuleChain (#10289)
This is now the preferred way to specify rule execution order.
2023-06-16 13:39:47 -07:00
Terry Wilson 48192db68e
netty: Nix InternalNettyServerBuilder unused methods (#10287) 2023-06-15 15:03:05 -07:00
Eric Anderson d28e5f0fc5 buildscripts: Fail sonatype-upload for non-200 HTTP status code 2023-06-15 13:35:00 -07:00
Larry Safran 99392004a0
Handle Aggregate cluster loops and dups (#10274)
* Suppress duplicate children and NACK if detect loops (child is an ancestors of the current CDS aggregate).  

* Handle diamond shaped aggregations (same cluster appears under 2 distinct parents that doesn't create a loop).
2023-06-15 10:24:23 -07:00
Eric Anderson d4e26cc689 okhttp: Use a real socket during server transport testing
The PipeSocket was convenient and avoided real I/O, but the
shutdown/close while connecting/handshaking tests were triggering a
Socket bug in Java (https://bugs.openjdk.org/browse/JDK-8278326). Using
a real socket doesn't trigger the bug because the test stops sharing
state with the code under test.

Fixes #10228

```
Details
==================
WARNING: ThreadSanitizer: data race (pid=4528)
  Write of size 1 at 0x0000cfb9d5f4 by thread T36 (mutexes: write M0):
    #0 java.net.Socket.setCreated()V Socket.java:687
    #1 java.net.AbstractPlainSocketImpl.create(Z)V AbstractPlainSocketImpl.java:149
    #2 java.net.Socket.createImpl(Z)V Socket.java:477
    #3 java.net.Socket.getImpl()Ljava/net/SocketImpl; Socket.java:540
    #4 java.net.Socket.setTcpNoDelay(Z)V Socket.java:998
    #5 io.grpc.okhttp.OkHttpServerTransport.startIo(Lio/grpc/internal/SerializingExecutor;)V OkHttpServerTransport.java:164
    #6 io.grpc.okhttp.OkHttpServerTransport.lambda$start$0(Lio/grpc/internal/SerializingExecutor;)V OkHttpServerTransport.java:159
    #7 io.grpc.okhttp.OkHttpServerTransport$$Lambda$56.run()V ??
    #8 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133
    #9 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V ThreadPoolExecutor.java:1130
    #10 java.util.concurrent.ThreadPoolExecutor$Worker.run()V ThreadPoolExecutor.java:630
    #11 java.lang.Thread.run()V Thread.java:830
    #12 (Generated Stub) <null>

  Previous read of size 1 at 0x0000cfb9d5f4 by thread T35 (mutexes: write M1, write M2):
    #0 java.net.Socket.close()V Socket.java:1512
    #1 io.grpc.okhttp.OkHttpServerTransportTest$PipeSocket.close()V OkHttpServerTransportTest.java:1384
    #2 io.grpc.okhttp.OkHttpServerTransportTest.clientCloseDuringHandshake()V OkHttpServerTransportTest.java:290
```
2023-06-15 09:59:05 -07:00
Sergii Tkachenko 4d3a29b2af
Update README etc to reference 1.56.0 (#10283)
com.google.protobuf:protoc updated from 3.21.7 to 3.22.3.
2023-06-14 16:18:55 -07:00
Benjamin Peterson 9148095df4 netty: simplify SendGrpcFrameCommand
DefaultByteBufHolder has many perfectly good implementations. It should be enought to implement the replace method.
2023-06-14 08:45:31 -07:00
Mark S. Lewis ae59afb5bf
Update guava dependency to address CVE-2023-2976 (#10249)
Explicit dependencies to keep versions in step with newer Guava
2023-06-12 12:43:35 -07:00
Eric Anderson 5754518914 Upgrade netty-tcnative to 2.0.61.Final
This updates the version of boringssl and removes the dependency on APR.
netty-tcnative 2.0.56.Final uses APR 1.7.0, so is in scope for
CVE-2021-35940, CVE-2022-28331, and CVE-2022-24963. netty-tcnative is
not actually vulnerable. The binary does not include apr_socket_sendv(),
apr_encode_*(), apr_pencode_*(), apr_decode_*(), apr_pdecode_*(). The
binary does include apr_time_exp_*() but it is unused code.
Unfortunately --gc-sections wasn't used during compilation.
apr_time_now() is used, but that just calls gettimeofday() and is not
vulnerable.

There's no panic here, but this updates netty-tcnative just a few weeks
before we would have ordinarily done so. Bumping the version makes life
easier for everyone.
2023-06-12 11:01:06 -07:00