Commit Graph

205 Commits

Author SHA1 Message Date
Sergii Tkachenko 88770009fb
Increase memory in Linux aarch64 (emulated) builds (#9111)
Fix the issue with `Linux aarch64 (emulated)` builds failing with 

```
Expiring Daemon because JVM heap space is exhausted
Daemon will be stopped at the end of the build after running out of JVM memory
```

This fixes the build itself, however certain tests still fail.
2022-04-27 16:02:05 -07:00
Lidi Zheng d196e588a2
Add more xds_lb test cases (#8998)
This test suite is only fully enabled for C++ and Python at this
moment. This commit enables the rest of the test suite for grpc-java.

* change_backend_service_test
* failover_test
* remove_neg_test
* round_robin_test
* affinity_test
2022-03-21 12:17:26 -07:00
Lidi Zheng d71bef5afe
Redirect xDS tests to use grpc/grpc's master branch (#8982)
We are solving the issue of grpc/grpc delaying the release, causing GCE
tests to fail. Updating the master branch prevents similar cases from
happening.

Tested: [prod:grpc/java/master/branch/xds_v3](http://sponge/010f5353-e65e-4b1a-b6d3-f5a84e31546b)
2022-03-17 09:55:41 -07:00
yifeizhuang 467985e958
add kokoro config for basic tests xds k8s (#8888) 2022-02-04 11:28:13 -08:00
Eric Anderson 39cc44e38c
kokoro: Pretty test results
Previously, only Windows had the plumbing to rename test results for
the Kokoro result viewers to pretty-print.

macos.cfg was the only CI that lacked a corresponding .sh, which maked
unix.sh harder to reason about. Created macos.sh so that unix.sh is now
just a helper script and will not be called directly by Kokoro.

We now avoid "gradle clean" to avoid wiping results. Still clean compiler
since we do re-run the build multiple times with varying platforms.
Shouldn't be necessary, but "just in case" since I want this commit to
be low risk. This improves Windows to produce detailed results even
if the CI was successful.
2022-01-12 15:44:01 -08:00
sanjaypujare d7f951a9d8
all: update netty to 4.1.72.Final and tcnative to 2.0.46.Final (#8780) 2022-01-07 15:34:59 -08:00
ZHANG Dapeng 5ae6f9ca25
all: Upgrade Android plugin to 4.2.0
- bump android plugin version to 4.2.0
- migrate deprecated android.support dependencies to androidx dependencies
- bump `targetSdkVersion` to 29
- temporarily ignore lint error for 'MissingClass' due to #8799
- run android CIs with `-Pandroid.useAndroidX=true -Pandroid.enableJetifier=true` flags
- android examples are still using android.support dependencies, will not be updated in this PR.
2022-01-07 09:52:26 -08:00
Eric Anderson 80c3be0f80
Bump protobuf to 3.19.2 2022-01-06 09:08:50 -08:00
sanjaypujare e474e3dc7e
buildscripts: always make sure we are using jdk8 (#8800) 2022-01-04 13:01:02 -08:00
Lidi Zheng 80f1be03f0
Use xds-test-server-5 as the GCE interop server (#8777) 2021-12-21 16:39:31 -08:00
Eric Anderson efd968bcbb Upgrade Protobuf to 3.19.1 and Guava to 30.1.1
Protobuf uses Guava 30.1.1, so I upgrade it at the same time. It also
caused an update to rules_jvm_external and reworking the Bazel build.
Protobuf no longer requires bind() so they were dropped. Although
Protobuf's protobuf_deps() brings in rules_jvm_external, and so we don't
need to define it ourselves, it seems better to define it directly and
not depend on transitive deps since we use it directly.

Protobuf now has support for maven_install() by exposing
PROTOBUF_MAVEN_ARTIFACTS, which required reorganizing the WORKSPACE to
use maven_install() after loading protobuf. Protobuf still doesn't
define target overrides for itself so we still maintain those. When
reorganizing the WORKSPACE I noticed http_archive should ideally be
above io_grpc_grpc_java as most users will need it there, so I fixed
that since there were lots of other load()-reordering already.
2021-12-09 10:35:39 -08:00
markb74 ee581bfdfa
buildscripts: add config for building grpc-binder artifact (#8722) 2021-11-23 18:11:54 +01:00
sanjaypujare b746bab97b
buildscripts: rename xds-k8s to psm-security as part of tech-debt cleanup and name clarity (#8695) 2021-11-12 17:40:07 -08:00
Kevin Wooten 6518d7bd6d
Copy macOS x86 artifacts to aarch during upload (#8680) 2021-11-09 10:36:04 -08:00
Eric Anderson fe9026ed8a kokoro: Increase xds-k8s timeout to 3 hours
The addition of the authz tests in 0d345721 is causing the tests to
exceed their timeout. By itself, the authz test takes about an hour in
this environment. Before the authz tests, xds-k8s was taking an hour
and a half.
2021-11-05 10:04:48 -07:00
Eric Anderson 0d34572149 kokoro: Enable xds authz_test
It is successfully passing against prod:
https://source.cloud.google.com/results/invocations/e2be0996-ed4d-4a4c-90ad-20bc706f9f70/targets
2021-11-04 16:31:16 -07:00
Ivo List bb51bb6dfa
java_grpc_library.bzl: Fix parameters of java_common.compile (#7598)
Parameter host_javabase is removed.

This is preparation for flipping incompatible_java_common_parameters in
Bazel 5. See https://github.com/bazelbuild/bazel/issues/12373

Bazel versions prior to 4 require host_javabase, so are no longer supported.
2021-10-08 13:48:22 -07:00
Terry Wilson 83d36104e1
Source k8s test driver install script from core repo (#8573)
The test driver install script is read directly from the core repo master branch and the copy in the Java repo is deleted.
2021-10-07 13:05:03 -07:00
Lidi Zheng 8ac9a4e7bd [xDS interop] add Docker tagging logic to the xds_url_map job 2021-10-07 09:08:23 -07:00
markb74 f57de6bd03
Make binder instrumentation tests run on kokoro. (#8563)
The tests run as part of the existing android-interop-testing job.

We needed to modify the manifest of the apk built under android-interop-testing to declare Android Services used by the binder tests.
2021-09-29 11:28:14 +02:00
Lidi Zheng 0287d83182 Add testing_version flag 2021-09-27 15:19:41 -07:00
Jan Tattermusch 3049c2c147 reenable previously disabled aarch64 tests 2021-09-22 10:53:10 -07:00
Lidi Zheng 40f70ca3c1
Change to a non-workload-identity GKE cluster (#8461)
Part of grpc/grpc#27189 and b/198291728.

By disabling the workload identity, we should be able to run tests faster and avoid future IAM policy size issue.

Kokoro run: https://fusion2.corp.google.com/invocations/b52b1684-47de-406d-a9f6-644909755f34/targets
2021-08-31 10:35:51 -07:00
Lidi Zheng c54fcba2ee
Extend the xds_url_map job's timeout to 90 minutes (#8429)
As title. We recently had one flake caused by the Kokoro job timeout.
2021-08-20 12:12:54 -07:00
Eric Anderson 3e9488be25 buildscripts: Increase memory for Gradle in Android CI
We've still been seeing random memory-related failures with the Android
CI, but it is nowhere near as severe as it was. But even when running
locally with "-Xmx512m -XX:MaxMetaspaceSize=512m" I get failures. Our CI
environment has lots of RAM; let's use it.
2021-08-17 12:31:49 -07:00
Lidi Zheng 6a6a5279c0
Add a branch name in xds_url_map's CloudBuild (#8405) 2021-08-12 13:45:19 -07:00
Lidi Zheng 2a636420ef
Update xDS client/server image per-branch tag after build (#8400) 2021-08-11 14:01:21 -07:00
Eric Anderson 57bd087cdf buildscripts: Build android instrumentation tests in android CI
Binder's :build was missing. Cronet build failed without specifying
Java 8 because of the transitive Guava dependency.
2021-08-02 16:52:30 -07:00
Eric Anderson f781d24ddd Mostly revert "Run binderchannel android tests. (#8306)"
This partilaly reverts commit 5e18ff208a.
It leaves the compilation fix that was made to
BinderClientTransportTest.

Running instrumentation tests via firebase requires a `--app` argument.
However, we don't have such an app and it isn't immediately clear how
we'll go about making one. Revert the change to let android-testing to
start passing again.

This problem wasn't noticed before merging the original commit because
android-testing is a post-commit CI.
2021-08-02 16:13:33 -07:00
Lidi Zheng 39b7415bc0
Fix a typo in xds_url_map.cfg (#8344) 2021-07-26 16:41:34 -07:00
Lidi Zheng b3a26b732e
Add xDS k8s url-map Kokoro job (#8340)
Related PR grpc/grpc#26764
Related CL cl/386366746

This test suite will be run every 6 hours, and takes around 30 minutes. The script is copied from xds-k8s.sh, and removed the generation of server image.
2021-07-26 10:14:27 -07:00
markb74 5e18ff208a
Run binderchannel android tests. (#8306) 2021-07-14 13:10:28 +02:00
Eric Anderson 79e75bace4
Port from Travis-CI to GitHub Actions
Travis-CI no longer has a free tier (only a free trial). That was a
major reason we used Travis-CI, so that external contributors would be
able to run the CI on their forks. Iterating on a Travis config in a
personal repo was also quite convenient. The other reason was that
Travis-CI was safe to run even with untrusted code.

Since the introduction of the permissions field in workflows, GitHub
Actions appears safe to run untrusted code and has a free tier for
external contributors. GitHub Actions and Google Cloud Build are the
main contenders for a Kokoro replacement, but Cloud Build isn't safe for
untrusted code. Instead of migrating to Travis-CI.com from
Travis-CI.org, let's migrate to GitHub Actions and gain some familiarity.

I've really appreciated Travis-CI.org and have wanted to pay for it for
years but wasn't about to give it write permission to the repo. I'm
disappointed to migrate off it, now that the permissions issues have
been sorted out.
2021-06-17 13:32:56 -07:00
Sergii Tkachenko 814655cdde buildscripts: add option to use xds-k8s test driver from a fork 2021-06-16 19:09:04 -04:00
Eric Anderson 5642e01243
Replace failOnVersionConflict() with custom requireUpperBoundDeps
failOnVersionConflict has never been good for us. It is equivalent to
Maven dependencyConvergence which we discourage our users to use because
it is too tempermental and _creates_ version skew issues over time.
However, we had no real alternative for determining if our deps would be
misinterpeted by Maven.

failOnVersionConflict has been a constant drain and makes it really hard
to do seemingly-trivial upgrades. As evidenced by protobuf/build.gradle
in this change, it also caused _us_ to introduce a version downgrade.

This introduces our own custom requireUpperBoundDeps implementation so
that we can get back to simple dependency upgrades _and_ increase our
confidence in a consistent dependency tree.
2021-06-11 14:01:18 -07:00
Eric Anderson 29618a6bb6 Bump Protobuf to 3.17.2
Removed some references to the protobuf version from COMPILING.md and
compiler/README.md to make updates a tiny bit easier.
2021-06-07 11:20:48 -07:00
Menghan Li d4e90a78fd
buildscript: fold header/path matching tests to all (#8054) 2021-05-27 09:47:50 -07:00
Sergii Tkachenko 976daf2dd0 buildscripts: switch xds-k8s cluster to 1.20.x 2021-05-05 20:01:15 -04:00
Lidi Zheng d2160ea703
Extend the xDS interop tests timeout to 360 mins (#8133) 2021-05-05 10:18:44 -07:00
Jan Tattermusch 72527708f5
skip flaky :grpc-xds:test in linux aarch64 tests (#8119)
Due to #8118 the :grpc-xds:test currently doesn't provide useful signal.
2021-04-27 10:18:39 -07:00
Sergii Tkachenko fcbc1abc44 buildscripts: xds-k8s pin pip to 21.0.1
pip 21.1 released on Apr 24 introduced a regression for python 3.6.1.
The regression was identified on Apr 24, the fix merged on Apr 25.
The fix is expected to be delivered in the 21.1.1 patch.

There's no clear date, when 21.1.1 will be released.
Until then, pin is temporarily pinned to the previous release, 21.0.1.
2021-04-26 18:15:18 -04:00
Jan Tattermusch b436d0dfb7
Improve emulated linux aarch64 tests, without protoc artifact build
Based on my testing, the tests are currently flaky, so it would be better to first merge it and start running it continuously to asses the situation.
2021-04-26 10:54:28 -07:00
Lidi Zheng 134e9cbc42
buildscripts: enable CSDS test 2021-04-21 20:48:34 -04:00
Sergii Tkachenko 278a336d1f buildscript: xds-k8s increase build timeout 2021-04-08 22:05:29 -04:00
Sergii Tkachenko c113ba1030 buildscript: add xds-k8s cluster endpoint override
Missed this in 1b86618ce9
2021-04-08 21:14:08 -04:00
Sergii Tkachenko 1b86618ce9 buildscript: use different xds-k8s cluster
In preparation to the Public Preview.
2021-04-08 19:00:34 -04:00
ZHANG Dapeng e4ccf0eeac
buildscript: enable fault injection interop test 2021-03-17 17:41:58 -07:00
Doug Fawley 3752b9e365
add timeout test case to xds interop tests (#7888) 2021-02-12 14:25:20 -08:00
Eric Gribkoff 0ffd942fd2
buildscripts: increase xds job timeouts (#7831) 2021-01-25 04:33:01 -08:00
ZHANG Dapeng 150f994315
buildscript: fix execution path of xds.sh in xds_v3.sh 2021-01-20 09:21:36 -08:00