CentOS 7 became end-of-life on July 1st and is no longer working. We now
dynamically link against libstdc++, as RHEL 8 doesn't support static
linking: https://access.redhat.com/articles/rhel8-abi-compatibility
We now use objdump in check-artifact for all linux architectures. This
avoids using a mix of objdump and ldd. ldd shows transitive
dependencies, which is less convenient.
Co-authored-by: Eric Anderson <ejona@google.com>
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73.
Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh):
1. The list of tests in the each test suite
2. Per-test-suite flag customization
3. `run_test` methods
4. `build_docker_images_if_needed` methods
5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow.
Now all PSM Interop jobs share the same buildscripts by all test suites:
1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg)
2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`)
`psm-interop-test-{language}.sh`:
1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables.
2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo.
3. Sources `psm-interop-build-{language}.sh`
4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file).
`psm-interop-build-{language}.sh`:
1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`.
2. Invokes any repo-specific logic.
3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method.
References:
- b/288578634
- See the full list of the new features at grpc/psm-interop#73.
- Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
I'm trying to upgrade to a newer Windows Kokoro image, but the new one
has an old vswhere installed that breaks Gradle. Our old image doesn't
have vswhere at all. If vswhere isn't found, this rename prints some
errors, but the bat script continues executing. So this change is
compatible with both the older and newer image.
Splits the :grpc-android-interop-testing:assembleDebug and
:grpc-android-interop-testing:assembleDebugAndroidTest build
targets with hopes of avoiding OOMs.
They are a lot faster. Instead of 1-3 minutes of test execution, I now
see 2-22 seconds. There still may be 3 minutes of overhead for the
gcloud command to complete, but the reduction is noticable in the total
execution time. And it seems the tests are actually being run, as there
is some flakiness. The flakiness appears to be at a lower rate.
The script was slightly reorganized to make it easier to copy commands
to run locally.
Note that this uses Pixel2 and Pixel3. Also swap 26-27 from Nexus6P to
Pixel2. We tend to prefer the latest (virtual) device for each API
level.
The current models and their supported API levels are available via:
```
gcloud firebase test android models list --filter=form=virtual
```
Pixel2.arm supports 31-32, but is beta, so I didn't swap to it. It also
supports the preview 33.
Note that this changes the JDK used to compile releases to Java 11. That
should only impact the appearance of the Javadoc.
This adds the Android SDK to the build container, removing the
dependency on the Android SDK being available on the CI host. This
allows running on newer Kokoro images. 'Android' and 'Android interop'
CIs still depend on the Android SDK being available on the host, but
since they aren't used as part of the release process, they can more
easily migrate off Kokoro as part of future work.
This also causes Android components to now be built with -Werror, as we
use -PfailOnWarnings=true in unix.sh but were missing it from the
Android build invocations.
Gradle will auto-download the necessary version of build-tools. We don't
want to download it ourselves because the version we specify might not
even be used. Looking at logs, we were previously downloading a version
that was unused.
We now fork javac to avoid OOM. The build fails 2/3 times before the
forking, and 0/3 after.
Apparently our Kokoro image has this done already, and my laptop as
well. But the newer Kokoro image and other computers like my desktop
don't have it already.
We configured TestGrid to file bug separately for each
failed sub-target, if we still fail the main target,
TestGrid will fail duplicate bugs.
The same change in core:
https://github.com/grpc/grpc/pull/33222.
- Update to xDS Test Client and xDS test server Docker images to `eclipse-temurin:11-jre`.
- Perform software update so that we install patches for latest vulnerabilities.
Previously builds were done with Ubuntu 16.04, and now we are using
18.04. Thus the generated binaries will no longer work for
Ubuntu 16.04 and Debian 9 users, both of which are outside of their
support window and aren't supported by Abseil. RHEL users are
unaffected, as the binaries already didn't work on RHEL 7 and they will
remain working with RHEL 8. FWIW, Ubuntu 18.04 will leave its support
window in June.
* Added s390x platform support
* Adapt to existing platform naming scheme
* Updated s390_64 library whitelist
* Use g++ compiler version 8.x for s390x
* Introduced dedicated Docker container for building s390x artifacts Minor fix
---------
Signed-off-by: Dirk Haubenreisser <haubenr@de.ibm.com>
Co-authored-by: Eric Anderson <ejona@google.com>
This is the latest version of the plugin supported by the Gradle version
in use at the moment (7.6).
Note that this also upgrades the R8 optimizer to a version (4.0.48) that
now uses "full mode" optimization by default.
This also splits off Android projects to run under Java 11 (Gradle
plugin requirement) while the other projects continue to run under Java
8.
gcp-observability has many dependencies so is a bit annoying in some
build systems to get working... just for it not to be used in
non-observability scenarios.
grpc-go and c core are using separate binaries for gcp-observability
interop testing, so do the same in Java, which makes interop-testing a
bit lighter.
Maven seems to have improved their download management and instead of
having their webpage choose a host they now have a CDN domain.
apache.cs.utah.edu is slow and is failing to finish the downloading.
Apparently there's a difference between bash 3 and bash 4.
OSX comes with bash 3 out-of-box, so for whoever wrote this logic
it "worked on my machine".
The `((` construct returns a 0 exit code if the value is non-zero.
Since the value starts at 0 and we do a post-increment,
it will always fail the first time.
Changing it to a pre-increment should fix it.
Apparently there's a difference between bash 3 and bash 4.
OSX comes with bash 3 out-of-box, so for whoever wrote this logic
it "worked on my machine".
Trying to upgrade Gradle to 7.6 improved the checkstyle plugin such that
it appears to have been running in new occasions. That in turn exposed
us to https://github.com/checkstyle/checkstyle/issues/5088. That bug was
fixed in 8.28, which also fixed lots of other bugs. So now we have
better checking and some existing volations needed fixing. Since the
code style fixes generated a lot of noise, this is a pre-fix to reduce
the size of a Gradle upgrade.
I did not upgrade past 8.28 because at some point some other bugs were
introduced, in particular with the Indentation module. I chose the
oldest version that had the particular bug impacting me fixed. Upgrading
to this old-but-newer version still makes it easier to upgrade to a
newer version in the future.
This aligns the C++ version we're using for gRPC-generated code with the
Java version. This should have no real impact to our users, as there
were no features added to .proto files or the like that would be visible
to users.
- Enables pod log collection in all PSM interop jobs implemented
in https://github.com/grpc/grpc/pull/30594.
- Associate test suite runs with their own log file, so it's displayed
on the "Target Log" tab
- Updates security job to not stop after a failed suite, so that
authz_test run even if security_test failed
- Fix run_test not returning correct exit status, causing false
positives in some cases. See https://github.com/grpc/grpc/pull/30768
- The primary should've been `GKE_CLUSTER_PSM_LB`
- The secondary cluster was not activated for LB tests. This resulted
in the failover test failing, as it relies on workloads running in
different zones.
Secondary cluster was not activated for LB tests. This resulted in the failover test failing, as it relies on workloads running in different zones.
ref b/238226704
Larry Safran