Commit Graph

6268 Commits

Author SHA1 Message Date
Vladimir Gordiychuk 0f21574bb1
core: reduce CompositeReadableBuffer allocation (#3279)
* core: reduce CompositeReadableBuffer allocation

Add ability for CompositeReadableBuffer read specified length bytes
to another CompositeReadableBuffer instead of create temp
CompositeReadableBuffer.

fixes #3278
---------

Co-authored-by: Larry Safran <107004254+larry-safran@users.noreply.github.com>
2023-12-28 14:12:52 -08:00
yifeizhuang 846e008399
xds: Revert xds flow control change. (#10784)
* Revert "xds: fix flow control test failure (#10773)"

This reverts commit f67ec2ecd9.

* Revert "xDS: implement ADS stream flow control mechanism (#10674)"

This reverts commit 0a704a52ee.
2023-12-28 10:59:53 -08:00
yifeizhuang f67ec2ecd9
xds: fix flow control test failure (#10773) 2023-12-19 19:02:52 -08:00
Larry Safran 7be2b6c08d
Add increment as identified in #10768 (#10769) 2023-12-19 10:11:11 -08:00
James Roper 67b67f8382
Fix outbound message size checking (#10739)
This fixes two bugs in outbound message size checking:

* When thet checke failed, the thrown StatusRuntimeException with a status code
  of RESOURCE_EXHAUSTED was been caught and rewrapped in another
  StatusRuntimeException but this time with status code UNKNOWN.
* This applies the max outbound message size check to messages prior to, and
  after compression, since compression of a message that is smaller than the
  maximum send size can result in a larger message that exceeds the maximum
  send size.
2023-12-18 11:02:51 -08:00
yifeizhuang 0a704a52ee
xDS: implement ADS stream flow control mechanism (#10674) 2023-12-16 11:17:02 -08:00
Sergii Tkachenko 1680ada2fd
buildscripts: Use the Kokoro shared install lib from the new repo (#10757)
Source: 4f7ead2c7b/tools/internal_ci/linux/grpc_xds_k8s_install_test_driver.sh
New repo: https://github.com/grpc/psm-interop
New path: `.kokoro/psm_interop_kokoro_lib.sh`
2023-12-15 15:27:58 -08:00
Eric Anderson d6830d7f99
Change many api deps to implementation deps
These look pretty fair now, mostly only exposing grpc-api and
annotations as api dependencies.
2023-12-15 15:14:29 -08:00
Terry Wilson 8c4f4e08d9
Move Noop{Client,Server}Call from testing to api (#10753)
To help reduce cross dependencies, we move these test classes to the test
fixtures of grpc-api.
2023-12-14 23:00:30 -08:00
Eric Anderson a5f3483653 Upgrade distroless base images
gcr.io/distroless/java:8 is no longer being updated. Java 8 isn't a
distroless option any more. Java 11 and 17 are options, but only Java 17
with Debian 12. The main alternative is to stick with Java 8 and use
something like docker.io/library/eclipse-temurin:8-jre . But there
doesn't seem to be much need to use an old JDK for these containers.

jib needed updating to support the oci manifest format used in the
updated image.
2023-12-14 16:58:31 -08:00
Eric Anderson 225a7d30c9 core: Remove test dependency on grpclb
It was just unnecessary, and is isn't great having lower-level tests
depend on higher-level packages.
2023-12-14 09:26:58 -08:00
Eric Anderson 682d84c427 api: Remove unused grpc-core jmh dependency 2023-12-14 08:35:13 -08:00
Eric Anderson cf19263595 api: Remove test dependency on grpclb
It was just unnecessary, and is isn't great having lower-level tests
depend on higher-level packages.
2023-12-14 08:07:34 -08:00
Eric Anderson 201893f5e7 core: Improve language for leaked channel error
Originally you had to confirm that awaitTermination() returned true, but
that was annoying and useless, especially after calling shutdownNow().
The behavior was changed in ce2ae1fb because the awaitTermination()
detection logic could prevent the channel from getting garbage
collected.

Fixes #10732
2023-12-12 16:25:40 -08:00
Larry Safran e562a0558b
Remove Unused experimental API ManagedChannelBuilder.enableFullStreamDecompression. (#10744) 2023-12-12 14:56:22 -08:00
Ulf Adams 692696e2ed NettyServerBuilder: remove dead code
The KeepAliveManager clamps the keepalive and keepalive time values such that the result is larger than the minimum values specified here. Therefore, a second check here is unnecessary.
2023-12-12 14:02:00 -08:00
Shohei Kamimori 60c3133f4a README.md: Update the build badge
Travis is no longer used, the badge should be replaced by GitHub
Actions.
2023-12-12 13:01:40 -08:00
Eric Anderson c985797d90
Upgrade dependencies
All the changes outside libs.versions.toml and examples were
because of ErrorProne. It didn't actually find anything to fix; signal
vs noise has gotten pretty bad with the newer checks.

Status was changed for ErrorProne's SuperCallToObjectMethod. With the
old code it didn't notice the trivial implementation. The fail-for-test
code wasn't used, so it was easiest to just remove it.

Some of the libs had their versions inlined; now that we have
:checkForUpdates it isn't much of a risk for versions to diverge when
there's only a few artifacts sharing a version. If we need 4+ artifacts
to have the same version, then it makes sense to still use a shared
version.

Dependencies not upgraded: google-auth-libray, mockito, netty, cronet
2023-12-12 12:40:20 -08:00
Eric Anderson 68b14aaa49 RELEASING.md: Remove $ before commands
Removing the $ prompt makes it easier to copy+paste. At no point are we
running as root, so there's no # vs $ distinction, not that many people
would even notice the difference.

There is a risk here that not all commands end up getting run. When
pasting multiple commands at once, gradle or another tool might read in
stdin and discard them. But it's probably not worth continuing the
copy-each-command-separately-and-avoiding-the-$.
2023-12-12 08:57:20 -08:00
yifeizhuang 7567894e67
dual stack: generic health checking (#10623)
This change has health checking consumer (new pick first) to install a listener through and health checking producer (outlier detection and client health checking) producing health checks. Health notification chain is built reusing the previous connectivity state chain.
Pickfirst installs the health listener, and is capable of detecting when no health checking producer is installed in the system. In that case, it sets health status to be READY so that health system is no-op.
2023-12-11 17:49:56 -08:00
Eli Lipsitz 3e3ba56838 services: make BinlogHelper compatible with lite runtime
This commit makes a small change to BinlogHelper to make it compatible with the Protobuf Java Lite runtime.

In the Lite runtime, the `addXBuilder` for repeated fields is not available. Instead, the `addX` method must be used with a manually-constructed Builder.
2023-12-07 11:43:54 -08:00
Eric Anderson 55a9c012c3 Drop support for Bazel 5
Protobuf 25.0 drops support for Bazel 5. Bazel 7 is coming out very
soon, at which point we'd drop Bazel 5 anyway.
2023-12-07 10:06:42 -08:00
yifeizhuang 1a0c97a7a4
util: attributs is petiole policy (#10729) 2023-12-06 13:14:26 -08:00
Eric Anderson 8ac43dd816 util: Disable publishing of fixtures
We do the same in grpc-api and grpc-core
2023-12-06 10:07:21 -08:00
Mateus Azis a053889869
Handle slow security policies without blocking gRPC threads. (#10633)
* Handle slow security policies without blocking gRPC threads.

- Introduce PendingAuthListener to handle a ListenableFuture<Status>, progressing the gRPC through each stage in sequence once the future completes and is OK.
- Move unit tests away from `checkAuthorizationForService` and into `checkAuthorizationForServiceAsync` since that should be the only method called in production now.
- Some tests in `ServerSecurityPolicyTest` had their expectations updated; they previously called synchornous APIs that transformed failed `ListenableFuture<Status>` into one or another status. Now, we call the sync API, so those transformations do not happen anymore, thus the test needs to deal with failed futures directly.
- I couldn't figure out if this PR needs extra tests. AFAICT `BinderSecurityTest` should already cover the new codepaths, but please let me know otherwise.
2023-12-06 11:13:28 +01:00
Eric Anderson 4477269e2f Update README etc to reference 1.60.0 2023-12-05 13:40:06 -08:00
Eric Anderson d62334ea85
RELEASING.md: Include commit hash in shortlog (#10716)
This removes the benefit of including the PR number in the title without
also requiring using github APIs to query the PR number. It still
provides the same details about the change, and indirectly links to the
PR if the user wants to see the review.
2023-11-29 16:11:59 -08:00
Eric Anderson 56c5a0fe76 Update README etc to reference 1.59.1 2023-11-28 10:00:49 -08:00
Eric Anderson 7eb24d6ff0 Make custom tasks compatible with configuration cache
Gradle is forcing a move away from using 'project' during task excution
and because of some interactions there, this is easiest by making them
real classes. That makes them start looking quite strange in the build
file, so they are now moved to buildSrc/. We could have continued using
Groovy, but it is weird in some ways that are more apparent when making
classes and not just scripting. Instead, they were converted to Java.

They are compatible with delayed configuration resolution as well.
2023-11-27 14:28:44 -08:00
Eric Anderson dca89b25bf
util: Pass an AtomicInteger to RR's ReadyPicker
We already do this for WRR. Notably, we are no longer trying to avoid
the modulus each pick. It was of questionable value, and removing it is
necessary to continue sharing the same integer when the list size
changes.

The change means we can implement a stronger isEquivalentTo() by
comparing the AtomicInteger references. It is strong enough that the
operation aligns with normal equals(). Using equals() instead of
isEquivalentTo() also made more obvious an equals() optimization that
uses the hashCode() before the more expensive HashSet creation; equals()
should now be very fast except when they are (very likely) equal.
2023-11-27 14:27:42 -08:00
Eric Anderson 43e06372ec kokoro: Build android-interop-testing and binder separately
This avoids an OOM. We could probably configure things to run them
together, but that generally takes trial-and-error.
2023-11-27 10:15:42 -08:00
Eric Anderson a70dddd3dc kokoro: Use Firebase's arm virtual Android devices
They are a lot faster. Instead of 1-3 minutes of test execution, I now
see 2-22 seconds. There still may be 3 minutes of overhead for the
gcloud command to complete, but the reduction is noticable in the total
execution time. And it seems the tests are actually being run, as there
is some flakiness. The flakiness appears to be at a lower rate.

The script was slightly reorganized to make it easier to copy commands
to run locally.
2023-11-27 08:22:52 -08:00
Eric Anderson e65b4eb5a2 all: Add grpc-inprocess
This should have been added when inprocess was moved out of core. This
includes inprocess in the Javadoc and code coverage reporting.
2023-11-25 08:01:18 -08:00
Eric Anderson 0567ab5773 android-interop: Add testing for API levels 28-30
Note that this uses Pixel2 and Pixel3. Also swap 26-27 from Nexus6P to
Pixel2. We tend to prefer the latest (virtual) device for each API
level.

The current models and their supported API levels are available via:
```
gcloud firebase test android models list --filter=form=virtual
```

Pixel2.arm supports 31-32, but is beta, so I didn't swap to it. It also
supports the preview 33.
2023-11-22 14:49:19 -08:00
Eric Anderson 57dae6e20c inprocess: Add missing anonymous address as supported 2023-11-22 13:55:10 -08:00
Eric Anderson 860b5cb1f0 api: Deprecate LoadBalancer.EMPTY_PICKER
FixedResultPicker is our preferred approach.
2023-11-21 12:57:06 -08:00
Eric Anderson 3b92333890 core: Detect NameResolverProviders passed as Factories
This may help some to move closer to Providers. It especially helps
cases where `NameResolverFactory`s aren't returning `InetSocketAddress`,
as it allows them to override `getProducedSocketAddressTypes()`, which
will now fail starting in 15fc70be.
2023-11-21 10:56:55 -08:00
Eric Anderson c6d448e43f Remove getSubjectDN(), which is deprecated in Java 17 2023-11-20 10:06:19 -08:00
Eric Anderson 0987dc401c
netty: Add option to limit RST_STREAM rate
The behavior purposefully mirrors that of Netty's
AbstractHttp2ConnectionHandlerBuilder.decoderEnforceMaxRstFramesPerWindow().
That API is not available to our code as we extend the
Http2ConnectionHandler, but we want our API to be able to delegate to
Netty's in the future if that ever becomes possible.
2023-11-17 15:10:52 -08:00
Terry Wilson da4618ace1
util: Remove shutdown subchannels from OD tracking (#10683)
An OutlierDetectionLoadBalancer child load balancer might decided to
shut down any subchannel it is tracking. We need to make sure that those
subchannels are removed from the outlier detection tracker map to avoid
a memory leak.
2023-11-17 15:04:45 -08:00
John Cormie e89389a50a
api: Clarify that NameResolver can only be started once (#10681) 2023-11-17 07:44:24 -08:00
John Cormie bdb0c72d45
binder: Beef up hasPermissions security policy Javadoc (#10650)
Try to manage the fact that runtime permissions could be granted externally by the user after a hasPermissions() SecurityPolicy check has already been made on a transport.
2023-11-15 10:05:31 -08:00
Eric Anderson 32d85d8309 Start 1.61.0 development cycle 2023-11-15 07:41:12 -08:00
Benjamin Fedorka 2b65e660c0
netty: disable huffman coding in headers (#10563)
Huffman in the datacenter doesn't add much value in the common cases.  It could be useful to turn on huffman based on the connection latency (say, >10ms means "assume cross-datacenter") but the Netty API doesn't lend it to that. The savings here aren't huge and it is expensive; the table provides the biggest savings.
2023-11-14 11:09:38 -08:00
Patrick Koenig 90e76a1b4a
Implement Http2Headers.isEmpty (#10663) 2023-11-14 10:39:16 -08:00
Eric Anderson 0299788807 util: Make grpc-core an implementation dependency
This prevents grpc-core from being exposed on the classpath when
compiling code using grpc-util.
2023-11-13 16:52:42 -08:00
Eric Anderson 6257c59566
benchmarks: Remove usage of deprecated StreamObservers 2023-11-13 16:29:38 -08:00
yifeizhuang ae62785e0b
xds: fix ring hash childLB acceptResolvedAddress not in syncContext (#10664) 2023-11-13 14:58:30 -08:00
Vindhya Ningegowda d5544bbb02
opentelemetry : Implement metrics (#10593)
Adds a new module grpc-opentelemetry that integrates OpenTelemetry and focuses on metrics.

OpenTelemetry APIs are used for instrumenting metrics collection. Users are expected to provide SDK with implementations.

If no SDK is passed, by default gRPC uses OpenTelemetry.noop().
2023-11-13 10:58:16 -08:00
Eric Anderson 84baad12fc inprocess: Remove InProcessNameResolver
It was introduced in 15fc70be but unused. It could be "used" from
inprocess: targets, but the in-process transport wasn't registered, so
would fail.

We do want an in-process name resolver, but we need to agree no the URI
format cross-language before we introduce it.
2023-11-10 15:48:30 -08:00