Commit Graph

5141 Commits

Author SHA1 Message Date
Eric Anderson a282019902
netty: Use charset=utf-8, not encoding=utf-8 for error pages
This has been wrong since the introduction of the code in df357cb8.
Noticed as part of https://github.com/grpc/grpc-go/pull/4491 . The error
text is generally ASCII, so this probably doesn't matter much.
2021-07-19 13:32:11 -07:00
Robert Turner b9becb5c8e
core: correcting a minor resource releasing issue
This change adds a traditional try/finally block around readers and
streams to control the closing of those objects when the method has
completed rather than relying on the GC to deal with them.

This issue was flagged by an analysis tool via binary analysis of the
grpc-core package as part of a dependency from another project.
2021-07-15 17:08:00 -07:00
ZHANG Dapeng 9ed444ea2a
xds: add hint of fault injection to injected failures (#8326) 2021-07-14 19:37:25 -07:00
markb74 5e18ff208a
Run binderchannel android tests. (#8306) 2021-07-14 13:10:28 +02:00
markb74 eb802cf6b1
Add an asAndroidAppUri() method to AndroidComponentAddress. (#8312)
This returns the address as an "android-app://" uri, which can be
used with AndroidAppNameResolverProvider.
2021-07-14 11:01:07 +02:00
ZHANG Dapeng 4429ec2b78
core: add perAttemptRecvTimeout to retry policy (#8301) 2021-07-12 14:53:48 -07:00
Eric Anderson 9b55aed12e testing: Make more obvious that GrpcServerRule has been replaced
The previous text did explain to use GrpcCleanupRule, but a user just
skimming might not read the long paragraph. Now we have a TL;DR.
2021-07-12 14:46:12 -05:00
sanjaypujare 629748da61
xds: fix the race condition in SslContextProviderSupplier's updateSslContext and close (#8294) 2021-07-09 10:48:18 -07:00
sanjaypujare 3965315039
xds: implement filter-chain uniqueness check as per grfc A36 (#8295) 2021-07-08 17:22:43 -07:00
John Cormie d95eebe1a8
Document the BinderChannel status code mapping. (#8291)
Document the BinderChannel status code mapping.
2021-07-08 16:38:29 -07:00
yifeizhuang 6857d1b482
fix ServerBuilder callExecutor javadoc about direct executor (#8305) 2021-07-08 13:57:14 -07:00
ZhouyihaiDing 22aa8fcef1 Add Info log for gRPC LB's init response 2021-07-08 13:41:49 -05:00
Eric Anderson dd2a422193 binder: Remove unused import 2021-07-08 09:48:43 -05:00
markb74 d4ecc7cdb4
More exposed APIs to support internal transition. (#8303)
All maked with Internal to make it clear these APIs shouldn't
be used normally.
2021-07-05 13:12:49 +02:00
Eric Anderson 0cabf5672a compiler: Add GrpcGenerated annotation to generated class
This can be used by annotation processors to avoid processing the
gRPC-generated code. The normal Generated annotation only has SOURCE
retention, so isn't available to annotation processors.

I don't include the service name within the annotation as that assumes
we'll never have need for any other type of generated class. If there's
a request for exposing service name via an annotation in the future, we
can make an RpcService annotation or the like.

Fixes #8158
2021-07-02 22:11:40 -07:00
Eric Anderson 603135791a gitattributes: Show diffs for golden generated code
'binary' in gitattributes is a macro for '-text -diff' where -text
disables EOL modification and -diff disables textual diffs. However,
these are text files, so we'd actually rather like to have diff.
2021-07-02 22:11:40 -07:00
John Cormie 380f26fd8d
Inform the server of the client's deadline using the standard header. (#8286) 2021-07-02 18:31:50 +02:00
markb74 06d34925f9
binder: Updates to support migrations to this API from internal versions. (#8299) 2021-07-02 12:53:47 +02:00
ZHANG Dapeng f86dc6c89f
Update README etc to reference 1.39.0 (#8298) 2021-06-30 09:23:15 -07:00
ZHANG Dapeng 4f09073e0f
all: remove 2-arg ClientStreamListener.closed()
We used to have two ClientStreamListener.closed() methods. One is simply calling the other with default arg. This doubles debugging (e.g. #7921) and sometimes unit testing work. Deleting the 2-arg method to cleanup.

This PR is purely refactoring.
2021-06-29 10:27:03 -07:00
Eric Anderson f93cfe5add xds: Delete unused ScheduledExecutorService management code
In 02ff64fa2 the SharedResourceHolder.get() was removed and it became
dead code.
2021-06-29 11:33:19 -05:00
Eric Anderson 4814d975a5
xds: Avoid NPE for no filter chain match on server-side 2021-06-29 09:32:37 -07:00
yifeizhuang 7644350ecb
core: fix jumpstream listener not set when closing stream inline (#8290)
* Revert "core: rollback executor supplier, needs investigation (#8289)"

This reverts commit 1b57d48ac1.

* set jumpStream listener before close stream
2021-06-28 08:50:20 -07:00
John Cormie 66faf105dd
Fix BinderSecurityTests that would always pass due to empty 'methods' map (#8287) 2021-06-25 22:15:08 +02:00
yifeizhuang 1b57d48ac1
core: rollback executor supplier, needs investigation (#8289)
* Revert "core: fix boq conformance failures (#8281)"

This reverts commit c1ad5de135.

* Revert "core: allow per-service/method executor (#8266)"

This reverts commit c540229d79.
2021-06-25 13:10:45 -07:00
yifeizhuang 3aa871b7de
xds: remove cell based rbac engine (#8277) 2021-06-25 11:20:11 -07:00
yifeizhuang c1ad5de135
core: fix boq conformance failures (#8281)
Fix false alarm to please boq conformance verifications, ref. go/boq-conformance-violations/BLOCKING#what-should-i-do-instead.
2021-06-24 08:07:06 -07:00
John Cormie c092d94255
Update AndroidComponentAddress to wrap a binding Intent. (#8275)
By considering this Intent's action, data, type, identity and categories
we align gRPC/Binder's addressing with Android's natural equivalence
relation for "cached" IBinders.
2021-06-24 13:38:06 +02:00
sanjaypujare b118a590c8
xds: remove unused SDS code (#8282) 2021-06-23 20:58:22 -07:00
sanjaypujare e4ab8287d0
xds: get rid of legacy SDS and file watching code (#8276) 2021-06-23 11:13:19 -07:00
yifeizhuang c540229d79
core: allow per-service/method executor (#8266)
Add ServerCallExecutorSupplier interface in serverBuilder to allow defining which executor to to handle the server call.
Split StreamCreated() contextRunnable into two to support this new feature: one for method lookup, the other for server call handling. methodLookup() runs on default executor, handleServerCall() may run on the executorSupplier executor.
callbacks are queued after methodLookup() and handleServerCall() on serializing executor to ensure stream listener is set when callbacks starts running.
Make executor settable in serializing executor to allow switching executor for the server call handling runnable as a result of the outcome of the method lookup runnable.
2021-06-21 11:17:36 -07:00
Chengyuan Zhang 9a8bc10f51
xds: unify client and server handling HttpConnectionManager (#8228)
Enables parsing HttpConnectionManager filter for the server side TCP listener, with the same codepath for handling it on the client side. Major changes include:

- Remodeled LdsUpdate with HttpConnectionManager. Now LdsUpdate is an oneof of HttpConnectionManager (for client side) or Listener (for server side). Each of Listener's FiliterChain contains an HttpConnectionManager (required).
Refactored code for validating and parsing the TCP Listener (for server side), put it into ClientXdsClient. The common part of validating/parsing HttpConnectionManager is reused/shared for client side.
- Included the name of FilterChain in the parsed form. As specified by the API, each FilterChain has a unique name. If the name is not provided by the control plane, a UUID is used. FilterChain names can be used for bookkeeping a set of FilterChain easily (e.g., used as map key).
- Added methods isSupportedOnClients() and isSupportedOnServers() to the Filter interface. Parsing the top-level HttpFilter requires knowing if the HttpFilter implementation is supported for the target usage (client-side or server-side). Note, parsing override HttpFilter configs does not need to know whether the config is used for an HttpFilter that is only supported for the client-side or server side.
- Added a new kind of Route: Route with non-forwarding action. Updated the XdsNameResolver being able to handle Route with non-forwarding action: if such a Route is matched to an RPC, that RPC is failed. Note, it is possible that XdsNameResolver receives xDS updates with all Routes with non-forwarding action. That is, the service config will not reference any cluster. Such case can be handled by cluster_manager LB policy's LB config parser: the parser returns the error to Channel and the Channel will handle it as error service config.
2021-06-18 11:57:36 -07:00
Eric Anderson 2258d2e3e9 netty-shaded: Use compile instead of runtime dep in pom for core
This is essentially re-adding #5901 after its behavior was "reverted"
in #8242. At the time grpc-core was the only dependency, so this change
only applies to grpc-core to make it a tiny bit easier to swap to
runtime scope in the future.
2021-06-18 07:48:04 -07:00
Eric Anderson 79e75bace4
Port from Travis-CI to GitHub Actions
Travis-CI no longer has a free tier (only a free trial). That was a
major reason we used Travis-CI, so that external contributors would be
able to run the CI on their forks. Iterating on a Travis config in a
personal repo was also quite convenient. The other reason was that
Travis-CI was safe to run even with untrusted code.

Since the introduction of the permissions field in workflows, GitHub
Actions appears safe to run untrusted code and has a free tier for
external contributors. GitHub Actions and Google Cloud Build are the
main contenders for a Kokoro replacement, but Cloud Build isn't safe for
untrusted code. Instead of migrating to Travis-CI.com from
Travis-CI.org, let's migrate to GitHub Actions and gain some familiarity.

I've really appreciated Travis-CI.org and have wanted to pay for it for
years but wasn't about to give it write permission to the repo. I'm
disappointed to migrate off it, now that the permissions issues have
been sorted out.
2021-06-17 13:32:56 -07:00
Eric Anderson 6201db2d96 Fix aarch64 emulation-triggered flakes
Because we are emulating aarch64 during testing the tests run slower
than normal. Bumping the timeout seems easy and obvious for
StressTestClientTest. The warmup for BinlogHelperTest mirrors what we've
done with timing-based tests in the past. Hopefully that is enough to
lower variance so that it passes consistently; seems likely. I'd rather
not increase the timing fuzziness if I can avoid it, as that reduces the
test's ability to detect issues.

Fixes #8135 (mostly+hopefully)
2021-06-17 13:32:11 -07:00
Sergii Tkachenko 814655cdde buildscripts: add option to use xds-k8s test driver from a fork 2021-06-16 19:09:04 -04:00
ZHANG Dapeng e980ebd118
Start 1.40.0 development cycle (#8263) 2021-06-16 08:04:43 -07:00
Daniel Zou dc74a31be1
Modify netty-shaded resources to reference shaded class names (#8258) 2021-06-16 07:56:48 -07:00
yifeizhuang 84eb285742
xds: add override rbacfilter type url RbacPerProto (#8262) 2021-06-15 16:50:50 -07:00
Chengyuan Zhang 25122f9e83
api: clean up duplicated method in InternalServerInterceptors (#8261) 2021-06-15 15:11:56 -07:00
yifeizhuang c8ba601529
xds: add rbac http filter (#8251) 2021-06-14 12:54:07 -07:00
Chengyuan Zhang 2cbc7fc3a5
grpclb: skip fallback if the LB is already in fallback mode (#8253)
Manually checks if the gRPCLB policy is already in fallback mode when trying to fallback due to receiving address update without LB addresses. 

Commit b956f8852d added an invariant check in the FallbackModeTask runnable to ensure the task is fired only when the LB is not already in fallback mode. However, that commit missed the case that receiving address updates without LB addresses can trigger the run of FallbackModeTask runnable, because the existing implementation chose to reuse the code in FallbackModeTask. In such case, running FallbackModeTask could break the invariant check as the LB policy may already in fallback mode.

This change eliminates the reuse of FallbackModeTask for handling address update without LB address. That is, every time receiving address update, we manually check if it is already in fallback instead of reusing to FallbackModeTask perform the check.

Note there was a discussion brought up whether we should force entering fallback (shutdown existing subchannels) or we should still keep the balancer connection. Different languages have already diverged on this. Go shuts down the balancer connection and all subchannel connections to force using fallback addresses. C-core keep the balancer connection working and does not shutdown subchannels, only let fallback happens after the existing balancer connection and subchannel connections become broken. Java shuts down the balancer connection but not subchannels. This change does not try to change the existing behavior, but only fixes the invariant check breakage.

-------------------
See bug reported in  b/190700476
2021-06-11 14:53:18 -07:00
Eric Anderson 5642e01243
Replace failOnVersionConflict() with custom requireUpperBoundDeps
failOnVersionConflict has never been good for us. It is equivalent to
Maven dependencyConvergence which we discourage our users to use because
it is too tempermental and _creates_ version skew issues over time.
However, we had no real alternative for determining if our deps would be
misinterpeted by Maven.

failOnVersionConflict has been a constant drain and makes it really hard
to do seemingly-trivial upgrades. As evidenced by protobuf/build.gradle
in this change, it also caused _us_ to introduce a version downgrade.

This introduces our own custom requireUpperBoundDeps implementation so
that we can get back to simple dependency upgrades _and_ increase our
confidence in a consistent dependency tree.
2021-06-11 14:01:18 -07:00
Penn (Dapeng) Zhang aa18b2c228 grpclb: update load_balancer.proto 2021-06-11 13:28:48 -07:00
Penn (Dapeng) Zhang 11c0d1d81e rls: update rls proto 2021-06-11 13:28:48 -07:00
markb74 b846ae79bb
Allow SecurityPolicy definitions. (#8254)
Internally this was package visible to retain strict control
over the available policies. However, that kind of strict control
doesn't work with the open-source version, since users will want
to create their own policies. There's at least google-specific
policy internally.
2021-06-11 18:02:36 +02:00
Eric Anderson 8360e60330 netty/shaded: Leverage shadow plugin's publishing support
https://github.com/johnrengelman/shadow/pull/543 fixed project support,
which means the workaround is no longer necessary.
2021-06-11 08:32:11 -07:00
Chengyuan Zhang 91948b2606
xds: fix lint (#8248) 2021-06-09 14:57:26 -07:00
Chengyuan Zhang d41094944c
xds: equally weight endpoints within locality if endpoint-level weight unspecified (#8245)
Use a multiplier of 1 for endpoints with endpoint-level load balancing weight unspecified when computing weights for mixing-locality load balancing. Therefore, if a locality has endpoints without endpoint-level load balancing weight, they are weighted equally within the locality.
2021-06-09 12:04:17 -07:00
yifeizhuang b7f3fddc76
xds, rbac: implement rbac engine (#8168) 2021-06-08 14:45:11 -07:00