Commit Graph

5922 Commits

Author SHA1 Message Date
Tony An 0b53dd7304
implemented and tested static stride scheduler for weighted round robin load balancing policy (#10272) 2023-07-06 10:03:08 -07:00
Eric Anderson 361616ae7c Upgrade Netty to 4.1.94-Final 2023-07-06 08:46:45 -07:00
sanjaypujare 2be568d70b
core: stabilize MutableHandlerRegistry by removing ExperimentalApi annotation (#10348) 2023-07-05 16:07:37 -07:00
Larry Safran ed4cc896fc
require EDS service name in CDS resources with xdstp name (#10329)
* require EDS service name in CDS resources with xdstp name
2023-06-30 13:51:46 -07:00
sanjaypujare 1a9c2a0dac
api, context: fix bazel build: export api from context and remove context dep from api (#10326) 2023-06-29 13:51:13 -07:00
Terry Wilson 8192ff6103
core: ManagedChannelImpl to always use RetryingNameResolver (#10328)
ManagedCahnnelImpl did not make sure to use a RetryingNameResolver if
authority was not overriden. This was not a problem for DNS name
resolution as the DNS name resolver factory explicitly returns a
RetryingNameResolver. For polling name resolvers that do not do this in
their factories (like the grpclb name resolver) this meant not having retry
at all.
2023-06-29 13:47:23 -07:00
Eric Anderson 443a0502ba Remove old Protobuf Gradle Plugin caching workaround
This has been fixed in the Protobuf Gradle Plugin since probably 0.8.13.
2023-06-29 12:50:01 -07:00
Eric Anderson 5a0fccfae0
Rebuild Android generated code for java.lang.String (#10327)
3808e707 fixed references of String to be java.lang.String, but failed
to update the Android build.
2023-06-29 12:47:44 -07:00
Terry Wilson 3b40b22348
api: Stabilize two io.grpc.Status methods (#10322)
This stabilizes:
 - Status.asRuntimeException(Metadata)
 - Status.trailersFromThrowable(Throwable)
2023-06-29 11:58:15 -07:00
Philip K. Warren 3808e707f9
compiler: Use fully qualified String in codegen (#10321)
Currently, the gRPC compiler isn't properly using the fully qualified
string name `java.lang.String` instead of `String`. Update the generator
to use the `$String$` alias to avoid compile issues with protobuf
messages called String.

Fixes #10316.
2023-06-29 10:50:13 -07:00
Eric Anderson f902e16d3d compiler: Don't mention C++ Protobuf version in build message
There's been minor version skew between Java and C++ many times because
certain releases are one-language-only. And now we have more severe
skew, where we can't readily upgrade to newer C++ Protobuf versions
because of build complexity. Let's just remove the version, and have the
canonical C++ Protobuf version live in COMPILING.md.

See #10317
2023-06-29 06:15:22 -07:00
wwtbuaa01 5799febde0
Add UserHandle and BinderChannelCredentials to BinderChannelBuilder to support cross-user communication through OnDeviceServer. (#10197)
Add UserHandle and BinderChannelCredentials to BinderChannelBuilder to support cross-user ondevice server.
2023-06-29 13:01:13 +02:00
sanjaypujare 4d2c3aac0e
context, all: move Context classes to grpc-api (#10313)
* context, all: move Context classes to grpc-api
clean up grpc-context since it has no source code: only add dep on grpc-api
add exclusion for all transitive deps of grpc-api - only guava
exclude grpc-context as a dependency from grpc-alts because all context code is in grpc-api now
api: 1.7 as target Java version for Context source-set of grpc-api

* core, census: fix the issues with android project pulling in old grpc-context version


* api,context: make changes to bazel build files to account for context code moving from context to api
2023-06-28 18:28:30 -07:00
Eric Anderson d654707838
buildscripts: Build Android with main build in linux_artifacts
Note that this changes the JDK used to compile releases to Java 11. That
should only impact the appearance of the Javadoc.

This adds the Android SDK to the build container, removing the
dependency on the Android SDK being available on the CI host. This
allows running on newer Kokoro images. 'Android' and 'Android interop'
CIs still depend on the Android SDK being available on the host, but
since they aren't used as part of the release process, they can more
easily migrate off Kokoro as part of future work.

This also causes Android components to now be built with -Werror, as we
use -PfailOnWarnings=true in unix.sh but were missing it from the
Android build invocations.

Gradle will auto-download the necessary version of build-tools. We don't
want to download it ourselves because the version we specify might not
even be used. Looking at logs, we were previously downloading a version
that was unused.

We now fork javac to avoid OOM. The build fails 2/3 times before the
forking, and 0/3 after.
2023-06-28 16:24:21 -07:00
Eric Anderson b2327238c5 xds: Include third_party sources is sources jar 2023-06-27 09:48:54 -07:00
Terry Wilson f3399e3d0e
android-interop-testing: Use java.util.concurrent (#10312)
Instead of the deprecated Android AsyncTask, let's use the standard
Java concurrency library with a Callable and an ExecutorService.
2023-06-26 13:32:05 -07:00
Eric Anderson ac8908ec7f xds: Plumb thirdparty classes through sourceSet instead of jar
Plumbing through sourceSet lets cross-project dependencies work the same
way as artifacts published to Maven. This fixes an issue for
interop-testing where build/install would include all the raw files from
thirdparty in addition to the grpc-xds.jar. For example:
build/install/grpc-interop-testing/lib/com/github/xds/data/orca/v3/OrcaLoadReport$1.class

b/288577812
2023-06-23 13:27:29 -07:00
Eric Anderson f46793e82d Remove unneeded dependency excludes
Since 44847bf4e, when we upgraded our JUnit version, the JUnit
exclusions have probably not been necessary. e0ac97c4f upgraded
Robolectric to a version that had the auto.service problem fixed.
2023-06-23 11:18:10 -07:00
Eric Anderson 0e6c01837c buildscripts: Fix prepare_qemu to not assume system already has emulation
Apparently our Kokoro image has this done already, and my laptop as
well. But the newer Kokoro image and other computers like my desktop
don't have it already.
2023-06-23 10:24:25 -07:00
Xuan Wang 1c9ce5d76b
[PSM interop] Don't fail url_map target if sub-target already failed (#10295) 2023-06-22 15:42:21 -07:00
Tony An 2effae6249
okhttp: tsan socket data race bug fix (#10279)
replaced use of bareSocket with a synchronized socket, added additional lock to synchronize initialization with shutdown() to fix a Java bug
2023-06-22 10:46:34 -07:00
Larry Safran f1de820c19
Change delay for hedging retry after a non-fatal error to be 0 to match the gRFC A6. (#10293)
Fixes #10145
2023-06-21 14:53:02 -07:00
Terry Wilson 31321562ee
xds: Use Rule order instead of RuleChain (#10289)
This is now the preferred way to specify rule execution order.
2023-06-16 13:39:47 -07:00
Terry Wilson 48192db68e
netty: Nix InternalNettyServerBuilder unused methods (#10287) 2023-06-15 15:03:05 -07:00
Eric Anderson d28e5f0fc5 buildscripts: Fail sonatype-upload for non-200 HTTP status code 2023-06-15 13:35:00 -07:00
Larry Safran 99392004a0
Handle Aggregate cluster loops and dups (#10274)
* Suppress duplicate children and NACK if detect loops (child is an ancestors of the current CDS aggregate).  

* Handle diamond shaped aggregations (same cluster appears under 2 distinct parents that doesn't create a loop).
2023-06-15 10:24:23 -07:00
Eric Anderson d4e26cc689 okhttp: Use a real socket during server transport testing
The PipeSocket was convenient and avoided real I/O, but the
shutdown/close while connecting/handshaking tests were triggering a
Socket bug in Java (https://bugs.openjdk.org/browse/JDK-8278326). Using
a real socket doesn't trigger the bug because the test stops sharing
state with the code under test.

Fixes #10228

```
Details
==================
WARNING: ThreadSanitizer: data race (pid=4528)
  Write of size 1 at 0x0000cfb9d5f4 by thread T36 (mutexes: write M0):
    #0 java.net.Socket.setCreated()V Socket.java:687
    #1 java.net.AbstractPlainSocketImpl.create(Z)V AbstractPlainSocketImpl.java:149
    #2 java.net.Socket.createImpl(Z)V Socket.java:477
    #3 java.net.Socket.getImpl()Ljava/net/SocketImpl; Socket.java:540
    #4 java.net.Socket.setTcpNoDelay(Z)V Socket.java:998
    #5 io.grpc.okhttp.OkHttpServerTransport.startIo(Lio/grpc/internal/SerializingExecutor;)V OkHttpServerTransport.java:164
    #6 io.grpc.okhttp.OkHttpServerTransport.lambda$start$0(Lio/grpc/internal/SerializingExecutor;)V OkHttpServerTransport.java:159
    #7 io.grpc.okhttp.OkHttpServerTransport$$Lambda$56.run()V ??
    #8 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133
    #9 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V ThreadPoolExecutor.java:1130
    #10 java.util.concurrent.ThreadPoolExecutor$Worker.run()V ThreadPoolExecutor.java:630
    #11 java.lang.Thread.run()V Thread.java:830
    #12 (Generated Stub) <null>

  Previous read of size 1 at 0x0000cfb9d5f4 by thread T35 (mutexes: write M1, write M2):
    #0 java.net.Socket.close()V Socket.java:1512
    #1 io.grpc.okhttp.OkHttpServerTransportTest$PipeSocket.close()V OkHttpServerTransportTest.java:1384
    #2 io.grpc.okhttp.OkHttpServerTransportTest.clientCloseDuringHandshake()V OkHttpServerTransportTest.java:290
```
2023-06-15 09:59:05 -07:00
Sergii Tkachenko 4d3a29b2af
Update README etc to reference 1.56.0 (#10283)
com.google.protobuf:protoc updated from 3.21.7 to 3.22.3.
2023-06-14 16:18:55 -07:00
Benjamin Peterson 9148095df4 netty: simplify SendGrpcFrameCommand
DefaultByteBufHolder has many perfectly good implementations. It should be enought to implement the replace method.
2023-06-14 08:45:31 -07:00
Mark S. Lewis ae59afb5bf
Update guava dependency to address CVE-2023-2976 (#10249)
Explicit dependencies to keep versions in step with newer Guava
2023-06-12 12:43:35 -07:00
Eric Anderson 5754518914 Upgrade netty-tcnative to 2.0.61.Final
This updates the version of boringssl and removes the dependency on APR.
netty-tcnative 2.0.56.Final uses APR 1.7.0, so is in scope for
CVE-2021-35940, CVE-2022-28331, and CVE-2022-24963. netty-tcnative is
not actually vulnerable. The binary does not include apr_socket_sendv(),
apr_encode_*(), apr_pencode_*(), apr_decode_*(), apr_pdecode_*(). The
binary does include apr_time_exp_*() but it is unused code.
Unfortunately --gc-sections wasn't used during compilation.
apr_time_now() is used, but that just calls gettimeofday() and is not
vulnerable.

There's no panic here, but this updates netty-tcnative just a few weeks
before we would have ordinarily done so. Bumping the version makes life
easier for everyone.
2023-06-12 11:01:06 -07:00
Eric Anderson 0f2c43a8fd Download Maven from Maven Central
Maven deleted older binaries from their CDN, so the download was
failing. Maven Central seems it'll be more stable.
2023-06-09 10:32:12 -07:00
Daniel Liu 1daf0ff720
services, xds, orca: use application_utilization and fallback to cpu_utilization if unset in WRR (#10256)
Implements updates to [A51][] and [A58][].

Imported cncf/xds using import.sh script.

A51: https://github.com/grpc/proposal/pull/374
A58: https://github.com/grpc/proposal/pull/373
2023-06-08 16:03:50 -07:00
Eric Anderson 33a402d924 Avoid NettyServerBuilder when unnecessary
These code locations just needed a generic gRPC server, without any
transport-specific configuration. Use vanilla ManagedServerBuilder
instead of hard-coding to Netty, as we would suggest to our users.
2023-06-05 17:11:16 -07:00
Larry Safran 217103cd5e
Flip default for RLS on XDS being enabled to true. (#10248) 2023-06-05 15:49:30 -07:00
sanjaypujare 6949b8790a
api: stabilize ManagedChannelBuilder.useTransportSecurity (#10244)
* api: stabilize ManagedChannelBuilder.useTransportSecurity and add a note in Java doc
2023-06-02 15:08:25 -07:00
sanjaypujare 639fd8edf2
xds: remove env var for security enable/disable (#10243) 2023-06-02 09:24:51 -07:00
Xuan Wang 02ac6f4393
buildscripts, psm interop: Don't fail target if sub-target already failed
We configured TestGrid to file bug separately for each 
failed sub-target, if we still fail the main target, 
TestGrid will fail duplicate bugs.

The same change in core: 
https://github.com/grpc/grpc/pull/33222.
2023-05-31 17:37:46 -07:00
Sergii Tkachenko 17b2df78ec
Start 1.57.0 development cycle (#10229) 2023-05-30 18:02:40 -07:00
kotlaja e6d9b41895
bazel: Add java toolchain type to all rules which are using java_common (#10225) 2023-05-30 11:48:43 -07:00
Stanley Cheung 4277867a7d
gcp observability: Update Docker images to eclipse-temurin (#10218)
Perform software update so that we install patches for latest
vulnerabilities. Similar to the fix in #10191
2023-05-30 08:03:36 -07:00
Larry Safran df85a45742
Add a java specific description of manual flow control to the example. (#10223)
* Add a java specific description of manual flow control to the example.
2023-05-26 18:36:40 -07:00
Daniel Liu 5a27e3ead2
services, xds, orca: support EPS in client-side WRR (#10177) 2023-05-26 12:52:42 -07:00
Michael Vorburger ⛑️ 6aa72b786d
docs: Document use with Bazel, like Maven & Gradle (fixes #10215) (#10217)
* docs: Document use with Bazel, like Maven & Gradle (fixes #10215)
2023-05-26 12:42:50 -07:00
apolcyn b77f0a022f
interop-testing: introduce --additional_metadata flag to let client optionally send extra metadata (#10201)
The flag was documented in https://github.com/grpc/grpc/pull/33180 .
This flag will be useful for some RLS integration tests where we need
to have the client set certain headers for routing purposes.
2023-05-26 06:45:50 -07:00
Eric Anderson f75c7c72f2 repositories.bzl: In doc, use | to combine dicts
The previous syntax for just adding your own keys doesn't seem to work,
but was similar to the approach of using `dict(d, foo=bar)`. You can't
have '.' and ':' in a key that way though. The doc was written before
Bazel 1.0 and in newer Bazel versions you can just use | to concatenate.

Fixes #10203
2023-05-25 15:26:43 -07:00
Eric Anderson 7217baa777 Remove old Java 7/8 TLS testing workarounds
Ciphers have been "fast enough" for testing since early Java 8 updates;
we haven't needed to override ciphers since we dropped Java 7 support.
Java 8u252 had ALPN, so Conscrypt or Jetty ALPN hasn't been necessary
for basic testing for a while. We still want specialized testing for
Conscrypt, but only tests testing Conscrypt need to care.
2023-05-25 14:04:01 -07:00
Eric Anderson cb42405217 testing: Move some internal classes to testFixtures
These two classes are precisely the purpose of test fixtures. Move them
close to the classes they make easier to use in tests.
2023-05-25 14:03:42 -07:00
sanjaypujare 3b2cba09c1
api: remove experimentalApi setting from proxyDetector and related classes (#10216) 2023-05-25 13:46:24 -07:00
sanjaypujare e875d1b01c
xds: remove remaining occurences of SDS in the security code (#10219) 2023-05-25 10:28:37 -07:00